The MVP
On page 24 of the IG report there is a story about the creation of the functionality to sign up for an account. I think this is a real nugget. Remember that they are supposed to be using an Agile development process. For me, I always like to use that first cycle to build the easiest, most fundamental capabilities of the system. I call this sprint 0 (sprint zero) because this is essentially the “hello world” screen for the project. My projects always see the account creation process and what I think the roles will be built first because I need this capability when I’m testing more complicated features later. So we want the user roles, levels of authentication, and all that goes with it settled early so that we are developing use cases or story points and testing them with real-world scenarios. That they didn’t even think of including this as work until May 2013 is incredible.
They were supposedly building a minimally viable product (MVP). We may disagree about what features and capabilities must be included in that MVP, but one thing we would never disagree on is that there would be different user roles and that people would need to authenticate to the system. So how can we build a MVP without this foundational capability? This is the aspect that makes me question the leadership. Somebody was building capabilities in sprints but I don’t know how you could have been testing those features without the ability to authenticate.
The Surge
Then on page 25 we see the surge of people. I already talked about this in a previous chapter. Surge of troops may be good for fighting a ground war, but it is generally bad when building an IT system. They went from 51 people working on the project to 111 people. This means that they went from 1,275 lines of communication to 6,105 lines of communication. The formula for lines of communication is n*(n-1) /2. Keep in mind that they were struggling to communicate effectively before the surge. You add in that many more people; you are adding so much more complexity and confusion. It would have been interesting to examine the impact of reducing the number of people on the project. If that number went from 51 to 40, what would have happened? What would have happened if instead of diluting authority and decision-making, if we instead increased it? That would be a very interesting discussion.
Big Bang
While I think the report is very good, it sort of dances around what I consider to be a core issue of the project; the big bang approach. The notion that the world will go to bed one night, and that it will wake up the next morning and that during the night the development team flipped a light switch and the new system popped into existence is about as realistic as your alarm clock being replaced by a unicorn and waking up to a unicorn licking your face.
But we do it. Even today we make these plans for this big bang deployment like the universe suddenly just popping into existence. I mean it has happened, but it hasn’t happened again in 14 billion years, so there is a chance, and we are basing our projects on this very slim chance. I call this maximizing our risk. We are essentially trying to get all our users to use the new system at virtually the same. What could possibly go wrong?
But people continue to plan like this because they don’t understand what the alternative looks like. I often say that people don’t make bad decisions, instead they make uninformed decisions. What if for HealthCare.gov, instead of launching the whole thing, what if they just launched it for a small community? Pick a town, Springfield is a good town. Instead of trying to set the universe of expectations, start small and learn from that experience. If they would have done this they would have said, “well it wasn’t a successful test, but we are going to learn from this result and improve the product.” Then, the only people who are pissed off are the people who live in Springfield and Homer and Bart Simpson make fun of it on Sunday night, but that is about it.
Alternatively, what if they only launch the “register” capability? What if you could just sign up; get a user name and password. Then, as new functionality progresses, you promote that functionality to the production environment.
There is no project that can’t be rolled out in a small-scale way. You limit the project by geography, scope, impact, whatever. They key is that you take the time to set peoples’ expectations. If you fail to do that then people will always use your service expecting it to be perfect. It is never perfect, especially at the beginning. Call it Beta, call it draft, roll out a prototype or pilot. Whatever you do, take the time to set peoples’ expectations, under-promise and over-deliver. Never roll out in a big bang.
One Neck to Hang
The CIO at HHS at the time was Frank Baitman. He is a smart guy who really does understand IT development. It was a big deal that the HealthCare.gov investments were rated “Green” on the IT Dashboard. I understand why it was rated green. Frank and his team had zero visibility into the investment and were required to make a rating. So the system that they devised was a compliance exercise. Did the team turn in their documentation? Did they turn it in on time? If the answer to these questions is “yes” then they get a green rating.
This is what Congress was reacting to when they passed the FITARA legislation in the back of the NDAA in 2014. There was a CHIEF Information Officer who didn’t have any visibility into the most important IT investment in the department. How can the CIO be the chief when he didn’t have visibility into this program? We would love for this problem to be isolated to just HHS, but it isn’t. In December 2013 GAO released a report that identified that some CIOs across the government are struggling with this issue. DOT, Treasury, Justice, and SSA are pretty good on this issue, but Commerce and Energy have some work to do. USDA has a lot of work to do, and VA, I don’t know what you are doing at all.
GAO followed up on that report just this year with a new report that more clearly identifies the magnitude of the problem. Remember, investments today are categorized as Green, Yellow, and Red to indicate low, moderate, and high risk. In this new report, GAO-16-494, GAO took the additional step of looking at the universe of green investments and re-categorizing them based on the submitted data. They then did the same thing for the yellow and red investments.
