The Winner’s Curse

If you have done your job right you have cultivated several offerors and you have created good competition among them. Going back to Chapter 2, you have written the work in such a way that you are going to get apples and more apples which will help to make a comparison among the offerors. You might even have sequences projects to create this hyper competition. The problem begins when the competition takes off and the bids go lower than the cost of the work.

Let’s say you receive bids from four companies, but you will only make an award to one of them. They all attended the pre-solicitation conferences, so they all know about each other. This is new work, so there is no incumbent contractor. The advanced acquisition forecast lists a really big solicitation coming down the pike nine to 12 months from now. This is the best situation to get really good pricing but it is also the best solicitation to realize the winner’s curse.

Each of these offerors will try to win on quality. They will tell you that they are the best and go to great lengths to extol their virtues. But when it comes down to it, they know that the award will be made mostly by price. They will be looking for cues that the government gives away to try to figure out what the competitive range is, and they will be working to be the lowest price in that range. The winner’s curse comes into play when the government makes an award for a price that is too low for the offeror to actually deliver. As such, they won the competition, but they will be out of business because they aren’t making any profit.

Theoretically the government protects the contractors in this situation because we have to determine if the prices we are paying are “fair and reasonable”⁶. But keep in mind that there can be a wide range in what is considered fair and reasonable. In the Report, 15-549, GAO really looked at this issue in terrific detail and found that there is a ton of variability in the prices for the same services.

I’m not saying that each of the lowest bidders in the following table is suffering from the winner’s curse, but some of the low-end rates are scary, and should be considered just as scary as the high-end rates. I think DoD and DHS should be slammed for allowing this level of volatility. My goal here is to introduce you to it so that you don’t make their same mistakes.

Independent Government Cost Estimate

One of the most important things you will work on in an acquisition is the Independent Government Cost Estimate (IGCE). If you went out and surveyed a bunch of 1102s (that is the career series for contracting specialists and related disciplines), they probably wouldn’t list the IGCE as one of the most important things in an acquisition. They would probably identify the work statement as the most important thing. But remember from Chapter 2, we are buying the process instead of the product. When we do that, we turn the work statement into a commodity that can probably be used on most development and operations projects. As such you use the one from another recent project and tailor it to your specific needs. No, making a defensible IGCE is way more important.

The first thing I would recommend is generating a model. Here is the bottom line up front:

Once you build the model, you are controlling the situation. All of my cells are linked. Notice that the rates all increase by 5 percent over time. The model allows you to quickly and easily run different simulations and see the cost. In this model I have a base year that will have one major release. It will include six Web-based training (YouTube style) videos telling people how to use the new system. I also have one small maintenance release and one Production Database Change Request (PDCR) to fix data anomalies. In option year one and each option year thereafter I am looking for two major releases, each half as large as the initial re-engineering effort. I will also have four maintenance releases, three PDCRs and get six new or improved videos per year. Laid out on a timeline, it should look something like this:

I intentionally follow each major release with a maintenance release. This serves as my shadow project to clean up and fix any outstanding issues or unintended consequences of the major release. If you aren’t planning down to this level of detail for your dev/ops projects, you need to.

The IGCE model will become a part of my solicitation package. I will erase the rates for each labor category and the offerors will fill that in. My instructions will also tell them that if they need more or fewer hours for specific areas, or different labor categories, they can make those alterations. But I must still get the data in a tabular form so that I can see the escalation of rates, and the fixed price of a single iteration. That is the biggest issue. In my model, four-week iterations cost a little more than $110,000 and escalate to a little more than $134,000. As such, if I need more of those there is very little negotiation, we have basically agreed on the price.

As I said in Chapter 2, you can have four-week iterations, two-week iterations and everything in between. You have to understand the cadence that works best for your organization. Once you figure out the cadence, work backward to plug the right effort into the IGCE model. Then you figure out how many iterations you anticipate per major release. My experience has been that releasing big new features twice a year gives people a good amount of change over time without feeling like we’re bouncing them around like a game of pinball. But you may have a user community that needs more change at a faster pace.

Remember, agile development teams are supposed to be 7+/- 2 people. If you are looking to build change faster than what we have here then you are in the ballpark of running multiple development teams on the same application. This can be done, but you can’t do it without experience and the right controls in place. You will need a layer of portfolio management and configuration management sitting on top of the teams to ensure that the iterations integrate well.

Back to the IGCE, you give them the model and they fill in the rates, adjust the hours as they deem appropriate and the totals are calculated for you. They include this as their cost proposal. You will evaluate this after you have considered their technical proposal. It is really important that you get an IGCE that you are comfortable with because it will play a big part in deciding who wins the forthcoming competition.

Below is a Technical Evaluation Report and Best Value analysis that I wrote as the Chair of a Technical Evaluation Panel (TEP). I washed out the details of the offerors, but I am 100 percent confident in the outcome. It was different than most projects because we had 20 offerors, which is quite a lot. I am including little call-out boxes in the report to draw your attention to important details of the report.

Technical Evaluation Report

The Application had a very well defined Combined Synopsis and Solicitation. The combined package included the Statement of Work (SOW) as well as many SOW attachments including a Microsoft Project Plan which included a Work Breakdown Structure as well as a preliminary cost estimate, a Requirements Specification, a Project Management Plan, a Responsibility Assignment Matrix, a Concept of Operations, a Requirements Traceability Matrix, and a Requirements Strategy. This combined Synopsis and Solicitation had so much detail because this acquisition is dedicated to build upon the preliminary requirements work initiated under a separate solicitation.

The industry responded to this level of detail with a larger than average number of proposals for an acquisition of this magnitude, 20. With the large number of proposal there was a corresponding wide range of technical capabilities proposed to meet this requirement.

Determination for the award of this work will be made, in accordance with Section 2.6 of the Solicitation, best value given price and nonprice factors. Factor weights are:

• 65% – Technical
• 35% – Price