Why CMS's State-Level Projections are Silly, and an Alternative Measure of Success

I've written about this in passing before, but after repeated requests about the subject, I've decided to devote a full blog entry about the serious problems inherent in the CMS's state-level Private QHP projection numbers.

As far as I can tell, while the national "7 million" private enrollment projection figure issued by the CBO was based on solid analysis of the demographic situation at the time, the state-level CMS projection breakdown is, in many cases, based on little more than educated guesswork.

As the CMS put it in their own report: 

 In general, the enrollment targets provided by state- based Marketplaces are more ambitious than the initial Department enrollment targets for those specific states. As a result, using the publicly available SBM targets without adjusting other states would have raised the projected number of 2014 Marketplace enrollees by 1.4 million; therefore, the Marketplace ramp-up rates for other states were revised downward accordingly to maintain the 7 million total. 

Basically, they lowered the estimates for states that didn't provide their own estimates in order to shoehorn all the state target numbers into that overall 7 million target. As a result, you get some pretty strange projections: Connecticut's CMS target was only 33,000, while Kentucky, whose uninsured population is only 1.8x as high, has a target 6.6x higher. Vermont and Utah, with a 4.5x difference in population and wildly different socioeconomic conditions, have IDENTICAL projection numbers, not just for the final tally but for every individual month in between.

10 states did issue their own projection numbers: CA, DC, KY, MD, MA, NV, NM, OR, VT and WA. The projections for the other 41 states are basically just numbers that the CMS sliced up to "match" the 7 million total...and as far as I can tell, the CBO didn't utilize the projections by those 10 states when they came up with the original 7M number anyway.

in other words, while the CMS projection numbers may have been baked into the public consciousness to the point that they're touting their success when hitting them, I also wouldn't get too concerned about states which don't meet these "targets" either, since the target numbers don't necessarily bear a whole lot of connection to the population or demographics of that particular state anyway.

So, if using the CMS projections (which allow Connecticut to declare "100% achieved!" when they only reached 33,000 enrollees out of 359,000 uninsured residents, while Kentucky is "stuck" at only 19% of their "goal" of 220K enrollees even though their total uninsured is only 83% higher) doesn't make sense, what does?

Well, commentor Brutus56 at Daily Kos has long been pointing out that 7.066 million is roughly 14% of the 50 million people uninsured nationally. If you then divide that 7 million proportionally by each state's uninsured population, you get a much more reasonable indicator of how to measure success on a state-by-state basis.

For instance: Kentucky has around 657,000 uninsured. 14% of this is 91,980 people, a much more reasonable goal than expecting KY to enroll a full 1/3 of their uninsured population in the first year while a state like Connecticut is only expected to enroll 10% of theirs.

Using this measure, Connecticut is still doing very well (87% of their 14% goal), but Kentucky is doing much better (about 46% of theirs, vs. only 19% of the CMS number).

Now, don't get me wrong; the projections reached by the 10 states which did issue their own goals don't always match this '14%" figure, and I'm sure there are legitimate state-specific demographic reasons for that. However, when you only have that reasoning for 10 out of 51 states (w/DC), the "14%" method sure sounds a lot more rational overall.

Therefore, I've added an extra column to the Private QHP spreadsheet representing this "Proportional % of 7M Total" yardstick. Unfortunately, you'll have to scroll horizontally on the spreadsheet to see it, since I've run out of space.

There's also an additional column by request: The official HHS Report through 12/28; this is only listed so you can compare updates since 12/28 for various states.