This Article Extract Reprint (PDF) Submit a response to this article Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Services E-mail this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to My Personal Archive Download to Citation Manager Reprints & Permissions Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Woolhandler, S. Articles by Born, P. H. Search for Related Content PubMed PubMed Citation Articles by Woolhandler, S. Articles by Born, P. H.

Letters

HMO Profits And Quality

Patricia Born and Carol Simon elaborately massaged a virtually useless data set in an effort to defend HMO profit making (Mar/Apr 01). The 1997 National Committee for Quality Assurance (NCQA) data they analyzed are in-appropriate for the kind of research they undertook. That year 155 plans (35 percent of all plans submitting data to the NCQA) refused to allow disclosure of their Health Plan Employer Data and Information Set (HEDIS) scores.¹ Moreover, the plans that allowed disclosure (Born and Simon’s data set) represent a biased sample. As the NCQA stressed, poor-quality plans were more likely to refuse release of their scores.² Two of the largest for-profit HMO firms that scored poorly in 1996 (CIGNA and Prudential) refused to release scores for any of their seventy-five plans in 1997. It appears that executives at poor-quality HMOs selectively culled their scores from the NCQA’s data. No amount of statistical manipulation can compensate for such market-driven efforts to thwart honest evaluation.

Our earlier analysis demonstrating that for-profit HMOs had strikingly worse quality scores than did nonprofit plans was based on 1996 HEDIS data for 329 HMOs.³ In that year only 11 percent of the 370 plans submitting data to the NCQA refused to allow public release of their HEDIS scores. The nonreleasing HMOs represented mainly a scattering of small plans; virtually all of the major HMO chains allowed release of their data.

The finding of Bruce Landon and colleagues (Mar/Apr 01) that patients are dissatisfied with for-profit HMOs is also based on far more robust data than those used by Born and Simon. Plans participating in Medicare must participate in the satisfaction survey Landon analyzed, and Medicare releases all scores.

Born and Simon’s use of poor data is compounded by their highly suspect analytic strategy. In univariate analysis, for-profit plans scored lower on quality. Only by throwing a large number of questionable variables into a multivariate model could they reverse this finding. Their multivariate model cut sample size to as few as 140 HMOs (of the 441 that submitted scores to the NCQA).

They apparently forced sixteen or seventeen predictor variables into their multivariate models, whether or not these variables predicted HMO quality scores in univariate analysis. Although they justified this practice by positing that each of their variables is likely to confound the relationship between for-profit ownership and quality, they provided little empirical support for this claim. Forcing large numbers of nonsignificant "predictors" into a model often masks real effects with statistical noise.

Moreover, several of the variables are more properly viewed as path variables than as confounders. For instance, for-profit HMOs may try to cut costs by avoiding contracts with board-certified physicians, which may lower quality. Controlling for board certification ignores this possibility and assumes that board certification and for-profit ownership are unrelated. In fact, this and other predictor variables in the analysis reflect HMO executives’ decisions based on expected profitability and should not be analyzed as if they are independent confounders.

In sum, Born and Simon have tortured a biased subset of data to conclude that the more premium dollars diverted to profits, the better the care. Their conclusions are scientifically indefensible, and silly.

Steffie Woolhandler, David U. Himmelstein, Ida Hellander and Sidney M. Wolfe

Harvard Medical School, Cambridge, Massachusetts

NOTES

 

1. National Committee for Quality Assurance, NCQA’s State of Managed Care Quality Report, 1998, <www.ncqa.org> (10 May 2001).
2. Ibid.
3. D.U. Himmelstein et al., "Quality of Care in Investor-Owned vs. Not-for-Profit HMOs," Journal of the American Medical Association 282, no. 2 (1999): 159–163.

We appreciate the opportunity to respond to Woolhandler and colleagues. We are glad that they have read our work, and we have solid evidence that refutes their speculations.

There is no evidence that omissions drive our results. First, it is important to recognize that our 1997 data are far more complete than those they used. Their 1996 sample covers 56 percent of total HMO enrollment.¹ Our 1997 sample covers more than 75 percent. Indeed, there are 120 additional plans in 1997, most which existed in 1996 but did not report to HEDIS. The plans added in 1997 have scores that are significantly lower than the plans in these authors’ analyses. Any bias runs opposite of the direction suggested. At the same time, many plans that "disappeared" between 1996 and 1997 were acquired by other HMOs in the 1997 data—including many allegedly missing Aetna plans.

Firms that enter or exit a market are often different; HMOs are no exception. In our analyses we carefully compared plans that dropped out with those that entered. Both kinds of plans had somewhat lower quality scores (typically two to four points). However, on net, the expansion of coverage in 1997 pulled down scores by more than the exit of 1996 firms would have raised them. As an extreme test, we simulated what would happen if the CIGNA plans were included in our analysis, but with their 1996 reported scores. None of our findings were altered.²

Regarding specification, a large literature confirms that firm and market characteristics affect performance. We know of no theoretical framework that speaks to regional census designation. Multicollinearity does not bias econometric results, but analyses that omit variables are prone to bias.³

We were intrigued by the remarks on for-profit firms and did some work to look into this. There is no relationship between for-profit status and board certification (correlation is .02). Interestingly, for-profit HMOs have significantly lower turnover in primary care physicians, suggesting perhaps a higher-quality strategy. We appreciate the suggestion for extending our research.

Finally, we are most concerned about the misinterpretation of our results. We find no systematic relationship between for-profit status and HEDIS measures. This is neither an endorsement nor a condemnation of for-profit HMOs. In light of the variation that exists in health care quality, we believe that it is best to focus attention on forces that matter.

Carol J. Simon and Patricia H. Born

Boston University, Boston, Massachusetts

NOTES

1. D.U. Himmelstein et al., "Quality of Care in Investor-Owned vs. Not-for-Profit HMOs," Journal of the American Medical Association 282, no. 2 (1999): 159–163.
2. Authors will gladly provide results upon request.
3. W. Greene, Econometric Analysis, 3d ed. (Englewood Cliffs, N.J.: Prentice-Hall, 1997).

                      What's this?