This Article Abstract Reprint (PDF) Submit a response to this article Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Services E-mail this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to My Personal Archive Download to Citation Manager Reprints & Permissions Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Landon, B. E. Articles by Cleary, P. D. Search for Related Content PubMed PubMed Citation Articles by Landon, B. E. Articles by Cleary, P. D. Related Collections Consumer Issues Managed Care Quality Of Care

DataWatch

Health Plan Characteristics And Consumers’ Assessments Of Quality

Abstract

 
Many purchasers and consumers of health care have become concerned about the quality of care being delivered in managed care plans. Little is known, however, about the health plan characteristics that are associated with better performance. We used survey responses from 82,583 Medicare beneficiaries from 182 health plans to study the association of consumers’ assessments of care with health plan characteristics. For-profit and nationally affiliated health plans received much worse scores on the outcomes of interest, particularly for overall ratings of the health plan and composite measures of customer service and access to care. Health plans accredited by the National Committee for Quality Assurance did not receive higher scores.

These developments have heightened interest in standardized quality measures for managed care plans, including measures of clinical processes of care, outcomes, and patients’ reports of care. The Consumer Assessment of Health Plans (CAHPS) survey has quickly become the most widely accepted patient-based method for assessing of health plan quality.

In 1998 and 1999 the Health Care Financing Administration (HCFA) administered the CAHPS survey to samples of Medicare beneficiaries enrolled in all Medicare managed care plans. This provides an unprecedented opportunity to explore the relationships between health plan characteristics and the selected aspects of quality assessed by CAHPS. Such information could provide insights into features of plans that might affect quality of care.

Analyses of plan characteristics also could broaden our understanding of factors associated with accreditation. For example, if to become accredited by the National Committee for Quality Assurance (NCQA), health plans adopt policies and procedures that are related to customer service or access to care, accredited plans might perform better in some of these areas. In this paper we link information on plan characteristics from the InterStudy Competitive Edge database and the NCQA to Medicare managed care plans and assess the characteristics that are associated with performance on CAHPS and that might be reflected in disenrollment rates.⁴

Data And Methods

Top Data And Methods Study Results Discussion And Policy... NOTES

 
Sites and sample. The Medicare version of CAHPS was administered nationally by a single vendor, and all aspects of the survey, including sampling, coding, and analysis, were conducted uniformly. Therefore, it is not subject to variations in data quality that may affect other assessments of health care quality. Eligible plans included all 212 health plans or health plan units with separate Medicare risk contracts in effect on or before 1996 and in business for at least two years. For health plans with multiple contracts, each contract unit was studied separately, since each contract represents a distinct business relationship with HCFA and covers a separate geographic region. For each contract, HCFA drew a sample of enrollees who had been enrolled for at least twelve months. For most plans, a simple random sample of 600 patients was drawn, except for twelve plans with small enrollment, for which all eligible patients were surveyed. We excluded beneficiaries who left their plan before actually completing the survey, because the corresponding responses could not reliably be linked with a single plan. We also deleted one plan with only fifteen respondents and an additional thirteen plans that had ceased activity or been terminated. This left a total of 89,419 responses from198 risk contracts, for a response rate of 75 percent. Survey data collection took place from February to May 1998 and reflects care in calendar year 1997.

Measures. The CAHPS–Medicare Managed Care (MMC) survey includes four items that elicit overall ratings (of the plan, personal doctor, care received overall, and care received from specialists) and thirty-four that elicit reports of respondents’ experiences. We previously found that four factors explained most of the interplan variability in reports about specific experiences with care.⁵ The "delivery" composite includes sixteen items on care received primarily at the doctor’s office, and the "customer service" composite consists of five items on customers’ dealings with the plans. The "access" composite summarized eight reports on obtaining medical services and equipment from the plan, and the "advice" composite sums items on advice to quit smoking and about diet and exercise. These four composites and the four ratings of care have been shown to differentiate among plans within the same market as well as among plans in different metropolitan statistical areas (MSAs) and regions of the country.⁶ Three of the measures—the customer service and access composites and the plan rating—were shown to be most significantly determined by the health plan, as opposed to the market or region where the plan is located. We analyzed all eight of the above outcomes, but we focus on these three outcomes of interest because we thought they would be most likely to reflect the health plan’s behavior. Finally, we obtained rates of disenrollment from each plan in the prior year from HCFA.

InterStudy data. We obtained health plan characteristics from the InterStudy Competitive Edge 8.2 (1998) release containing annually updated information on U.S. health maintenance organizations (HMOs).⁷ We hypothesized that several variables in the InterStudy database would be related to CAHPS scores. These included the number of years the plan had been in operation, model type, profit status, national affiliation (defined by InterStudy as health plans operating in two or more states and with more than 10,000 total enrollees), several measures of health plan size, federal qualification , and whether the health plan enrolled Medicaid, point-of-service (POS), or preferred provider organization (PPO) enrollees.⁸ Because tax status and whether the plan was national or not were strongly correlated, we created separate categories for local and national not-for-profit and for-profit plans. Unaffiliated Blue Cross plans were classified as local because of the independent operations of the Blue Cross affiliates. Each health plan was assigned to the federal region where it had the largest enrollment; for this analysis, the contiguous Upper Midwest, East Midwest, and Mountain regions were combined into a single region because of the small numbers of plans in these regions.⁹

NCQA data. We obtained information from the NCQA on whether health plans were fully accredited, accredited for one year (including provisional accreditation), or unaccredited in 1998.¹⁰ The last category included six plans that had never applied for accreditation or had failed accreditation.

Matching the data. To match plans in the InterStudy database with Medicare contracts, we first matched health plan names and confirmed that all matches had overlapping service areas. We reviewed the list of unmatched plans and found and verified matches for plans with slightly different official names for the Medicare product and, in some cases, either called the health plan directly to confirm alternative names or confirmed a match using information from HCFA or other sources. We successfully linked 182 of 198 Medicare plans to the InterStudy database and to the NCQA accreditation data. After eliminating survey responses from the unmatched plans, we were left with 82,583 completed surveys out of 109,542 eligible sampled cases, for a response rate of 75.4 percent.

Analysis. We tested bivariate associations between plan characteristics and case-mix-adjusted CAHPS scores using t-tests for dichotomous variables and ANOVA for other categorical variables.¹¹ We then performed multiple linear regression to analyze the association of the CAHPS scores with features of health plans, controlling for regional effects, and we calculated regression adjusted means for the ownership/national affiliation categories. All candidate variables were included in the final regression models with several exceptions.¹² To check possible sensitivity to differences between plans in the Pacific region, which were largely for-profit, and the rest of the country, we repeated the analysis excluding plans from the Pacific region. Also, because there were relatively few national not-for-profit plans (twenty-nine health plan units, six national plans), and because many of these units came from one company, we reestimated the models including an indicator variable for the national not-for-profit plan with the most contracts. Finally, we repeated the analysis using a model including a dummy for each MSA.¹³

Study Results

Top Data And Methods Study Results Discussion And Policy... NOTES

 
Plan characteristics. Survey respondents were generally representative of the Medicare managed care population as a whole (Exhibit 1). Differences across regions are attributable to variation in the size of health plans among regions.¹⁴ Of the 182 health plans, there were at least thirteen plans in each region (Exhibit 2). Medicare beneficiaries, on average, comprised 13 percent of plan enrollment, although a few plans had a much higher proportion of Medicare enrollees, mostly larger for-profit health plans located in the Pacific, Southwest, and Southeast regions. Most plans were either network or independent practice association (IPA) plans; only 25 percent were staff- or group-model HMOs. Of the plans, 136 had either full or one-year accreditation from the NCQA, 113 were affiliated with national managed care companies, and 112 were for-profit.

Survey respondents tended to rate their care toward the upper end of an eleven-point scale. Likewise, mean responses for the experience composites were all above 3 on a scale of 1 to 4 (1 being "never," 4 being "always"), with a higher score indicating better experiences with the health plan. Bivariate analyses revealed important differences in performance by region, profit status, and national affiliation. Plan size, model type, and accreditation status were associated with better performance for a few endpoints but were generally not related to the three outcomes of primary interest.

Most of these relationships were still significant after multivariable adjustment.¹⁵ Results for the three primary outcomes are presented in Exhibit 3. Each number shows the estimated effect that the relevant variable had on the score or rating relative to the omitted reference category. For example, the –0.033 for access in local for-profit plans is significantly different (at the 99 percent level) from access in local not-for-profit plans (the reference group). The interpretation is that enrollees in local for-profit plans would be expected to rate their plans lower on access than enrollees in local not-for-profit plans would rate their plans on that measure.

Health plan type and accreditation. In bivariate analyses, group- and staff-model HMOs had lower scores on the delivery composite and lower ratings of specialists than IPA or network plans had (results not shown), but these results were not significant in the multivariate analyses. Small health plans tended also to have lower ratings of specialist care and of the plan in general; they also had lower scores on the advice composite than medium-size plans had.

With one exception, both federal qualification and NCQA accreditation were not significantly associated with plan performance. Full NCQA accreditation was associated with improved performance only on the access composite.

Tax status and national operations. The strongest predictors of performance were tax status and national affiliation (Exhibit 4). In unadjusted analyses, for-profit health plans had significantly worse scores on the customer service and access composites as well as on overall ratings of the health plan (all p < .01). In addition, national plans, whether for-profit or not-for-profit, had lower average scores on the customer service and delivery composites as well as on ratings of care, specialists, and the plan. Controlling for other variables, the plan ratings and customer service composite scores of national plans, regardless of tax status, were about one standard deviation lower than those of local not-for-profit plans, and the ratings of local for-profit plans were about one-half standard deviation lower. In addition, the scores on the customer service composite were more than one standard deviation worse for both national for-profits and national not-for-profits, and the scores on the access composite were about one-half standard deviation lower for both national and local for-profit plans. The scores of the groups overlap, however. For example, 12 percent of national for-profit health plans (ten contracts) had ratings that were above the median rating for local not-for- profit plans.

To explore whether results for for-profit plans were confounded with the effect of being in that region, where many of them are located, we repeated the analysis excluding the Pacific region. These results were generally consistent with the model based on the full sample. Some findings for local for-profit plans were no longer statistically significant, primarily because many of these plans were in the Pacific region. When we repeated the models including an indicator variable for the not-for-profit plan with the largest number of contracts, the coefficient for that plan was similar to that for the remaining national not-for-profits, and both remained significant.

Discussion And Policy Implications

Top Data And Methods Study Results Discussion And Policy... NOTES

 
This is the first study to report health plan characteristics associated with performance based on a uniform national administration of the CAHPS survey. We found that for-profit and national health plans scored lower for almost all of the outcomes we examined, even after region and other plan characteristics were controlled for. The fact that these effects remained even after controlling for MSA effects indicates that they are attributable to characteristics of individual plans rather than characteristics of the area in which plans operate. Although we do not know what accounts for these differences, our findings are consistent with prior studies.¹⁷

Other factors, including model type, federal qualification, and NCQA accreditation status, were only weakly related to plan performance. This finding is of particular policy relevance because many purchasers of care use NCQA accreditation as a proxy for quality. One could reasonably conclude that what NCQA accreditation measures, such as structural characteristics and financial stability, is not directly related to consumers’ experiences with care. Of course, accreditation status could be justified on other grounds.¹⁸

For-profit health plans had lower average scores on the composites and ratings that we hypothesized would be most related to health plan characteristics, but they performed less poorly on factors related to physician care. This suggests that patients have a more difficult time interacting with for-profit plans and gaining access to services, but they receive similar care from physicians. The results also support the notion that consumers can distinguish between care from the physician and care provided by the plan.

Although most for-profit plans are national, the findings related to national plans are distinct from those related to profit status, primarily because they also apply to national not-for-profit plans. National plans performed worse for ratings of care, the primary doctor and specialists, and the delivery composite. This suggests hat the effects related to national plans extend into physicians’ offices. There are several possible explanations for this. Because of more far-flung operations, national plans might have looser relationships with physicians. These plans, despite their large enrollments, might be less important for any individual physician’s practice. As a result, physicians’ offices might be less familiar with the administrative requirements of the plan or less willing to invest time in improving the working relationship, and physicians might feel less of a connection to them than to other plans.

Our findings are consistent with a growing literature demonstrating lower performance by for-profit health care. David Himmelstein and colleagues recently reported that for-profit health plans, after adjusting for regional effects, had lower performance on Health Plan Employer Data and Information Set (HEDIS) quality measures.¹⁹ That study, however, was based on a smaller sample of voluntarily reporting plans and relied on decentralized data collection. Bruce Landon and Arnold Epstein report that for-profit Medicaid health plans had more stringent utilization review rules than not-for- profit plans had.²⁰ In other sectors, such as the hospital industry, studies have demonstrated increased costs and utilization in forprofits.²¹ While these overall results mask substantial heterogeneity within these classifications, taken together, these findings represent a disturbing pattern relating for-profit status and the delivery of care. Minnesota currently requires that health plans be not-for-profit. Our data show that many for-profit plans outperform the average not-for-profit plans, so the evidence for such a step is mixed at best. However, the data also indicate a pressing need to understand the reasons for such differences.

We failed to find any significant associations between model type and consumer ratings, unlike several prior studies.²² In general, these studies showed improved satisfaction for IPA model plans. Our results might differ for several reasons. First, these studies use data that are more than ten years old and thus might not reflect fundamental changes in the health care market since that time.²³ Second, there are relatively few staff/group-model HMOs that provide Medicare services. Finally, with time, staff/group-model plans have become increasingly less relevant to the market, which clearly favors expansive plans with larger networks. In addition, previous differences might have also diminished because many large physician organizations have adopted the features of closed-panel health plans and thus might appear similar to consumers. Thus, from a policy perspective, distinctions by model type have diminishing relevance.

Limitations. Our study has several limitations. First, we were limited to analyzing a small set of health plan characteristics that we were able to obtain from InterStudy and the NCQA. We did not have sufficient information to characterize fully the diversity of health plans or to describe the heterogeneity that exists within classes of health plans. While tax status might affect an overall approach to the business of health care, not all for-profit plans operate similarly; we do not know the organizational mediators between tax status and plan performance. The limited availability of descriptive information on health plans, particularly given their ever changing nature and how they organize to manage and provide care, highlights the need for better and more comprehensive data on health plan characteristics so that changes in how health plans organize to manage care can be assessed more systematically.

Second, our data are cross-sectional and limited to the Medicare population. Consequently, our findings reflect the experiences of an elderly and relatively sicker population and might not generalize to the larger commercially insured population. However, because the elderly are more likely to have complex health care needs and many Medicare products are newer than commercial products, their experiences may be more sensitive to plan factors that affect quality. Finally, we did not specifically assess effects of benefit generosity or of local reimbursement rates that the health plans receive per enrollee. By controlling for market, however, we avoided confounding with these effects because reimbursement rates and levels of benefits tend to be similar within each market.

Although the data from CAHPS assess only selected areas of quality, they provide the most standardized, independent, and comprehensive measure of health care quality available for Medicare managed care plans. These measures are not related to either federal qualification or accreditation status. There are, however, systematic and significant differences in quality related to region, organizational form, and ownership. Understanding the reasons for these variations might provide important insights into reasons for differences in plan performance that would allow us to improve health care quality in all plans.

Editor's Notes

 
Bruce Landon is an instructor in health care policy at Harvard Medical School and an instructor in medicine at Beth Israel Deaconess Medical Center. Alan Zaslavsky is associate professor of statistics, Department of Health Care Policy, Harvard Medical School. Nancy Beaulieu is an assistant professor at the Harvard Business School. James Shaul is project director for the development of the Consumer Assessment of Health Plans (CAHPS) Behavioral Health Survey. Paul Cleary is professor of health care policy, Harvard Medical School and School of Public Health.

This work was supported by a grant from the Commonwealth Fund and a contract from the Health Care Financing Administration (500-95-007). We acknowledge programming assistance by Matthew Cioffi and Lin Ding. We thank Tom Reilly, Liz Goldstein, Terry Lied, and collaborators at HCFA, Barents, Westat, DRC, and the Picker Institute for their efforts in the survey implementation that generated the data on which this paper is based.

NOTES

Top Data And Methods Study Results Discussion And Policy... NOTES

 

1. InterStudy Competitive Edge 8.2. (St. Paul, Minn.: InterStudy, 1998); and Health Care Financing Administration, Medicaid Managed Care Enrollment Report, 1999; and HCFA Enrollment Data, <www.hcfa.gov>.
2. R.J. Blendon, M. Brodie, and J.M. Benson, "Understanding the Managed Care Backlash," Health Affairs (July/Aug 1998): 80–94; M. Brodie, L.A. Brady, and D.E. Altman, "Media Coverage of Managed Care: Is There a Negative Bias?" Health Affairs (Jan/Feb 1998): 9–25; K. Ignagni, "Covering a Breaking Revolution: The Media and Managed Care," Health Affairs (Jan/Feb 1998): 26–34; and K. Donelan et al., "The Cost of Health System Change: Public Discontent in Five Nations," Health Affairs (May/June 1999): 206–216.
3. R.H. Miller and H.S. Luft, "Managed Care Plan Performance since 1980: A Literature Analysis," Journal of the American Medical Association 271, no. 19 (1994): 1512–1519; R.H. Miller and H.S. Luft, "Does Managed Care Lead to Better or Worse Quality of Care?" Health Affairs (Sep/Oct 1997): 7–25; and R.S. Brown et al., The Medicare Risk Program for HMOs—Final Summary Report on Findings from the Evaluation (Princeton, N.J.: Mathematica Policy Research Inc., 1993).
4. National Committee for Quality Assurance, Health Plan Report Card, <www.ncqa.org>.
5. A.M. Zaslavsky et al., "Dimensions of Consumer-Assessed Quality of Medicare Managed Care Health Plans," Medical Care 38, no. 2 (2000): 162–174.
6. A.M. Zaslavsky et al., "Variation within and between Markets in Consumer Assessments of Medicare Managed Care," Inquiry 37, no. 2 (2000): 146–161.
7. InterStudy Competitive Edge 8.2.
8. Measures of health plan size included total enrollees, Medicare enrollees, number of hospital contracts, number of total enrollees per primary care physician, and number of enrollees per specialist physician.
9. We defined "small" plans as those with fewer than 75,000 total members and "large" plans as those with more than 400,000 total members. We defined plans in operation for five years or less as "new" plans. We included enrollment per physician as an indicator of the concentration of the plan’s network and hence the degree to which it might influence the practices of affiliated physicians. A dichotomous variable indicating plans with more than 25 percent of their enrollment drawn from the Medicare market was created to indicate commitment to serving Medicare.
10. NCQA, Health Plan Report Card, <www.ncqa.org>.
11. The adjustor variables were age, self-reported health status, education, and proxy response, as well as interactions of age and education with region. The "advised to quit smoking" variable was also adjusted for the respondent’s amount of smoking. Following procedures used by HCFA, plan means for each item or composite were case-mix-adjusted to remove the effects of differing distributions of member characteristics at various plans, without affecting means in each region. We created mean disenrollment rates by ownership/national affiliation categories.
12. Because bivariate relationships with the types of populations that the health plan served (such as Medicaid or commercial) were not significant and did not confound any additional relationships in the multivariate models, we excluded these from our final models.
13. We did so to determine whether the observed effects represented differences among competing plans or represented unobserved differences between different MSAs. Such differences, for instance, might reflect differing reimbursement rates and levels of benefits or differential patterns of selection into managed care versus traditional Medicare. T.D. McBride, "Disparities in Access of Medicare Managed Care Plans and Their Benefits," Health Affairs (Nov/Dec 1998): 170–180.
14. Zaslavsky, "Dimensions of Consumer-Assessed Quality."
15. When region alone was controlled for, the variance explained by the models ranged from 0.13 to 0.29. After including all other explanatory variables, the explained variance for the models increased to between 0.38 and 0.45. The "high Medicare" variable was highly correlated with certain regions of the country, national affiliation, and tax status, so we elected not to use it in our final analyses. Results were very similar when we included dummy variables for MSAs instead of regional effects.
16. L.F. Rossiter et al., "Patient Satisfaction among Elderly Enrollees and Disenrollees in Medicare Health Maintenance Organizations: Results from the National Medicare Competition Evaluation," Journal of the American Medical Association 262, no. 1 (1989): 57–63; and G.F. Riley, M.J. Ingber, and C.G. Tudor, "Disenrollment of Medicare Beneficiaries from HMOs," Health Affairs (Sep/Oct 1997): 117–124.
17. D.U. Himmelstein et al., "Quality of Care in Investor-Owned vs. Not-for-Profit HMOs," Journal of the American Medical Association 282, no. 2 (1999): 159–163.
18. Zaslavsky, "Variation within and between Markets."
19. Himmelstein et al., "Quality of Care."
20. B.E. Landon and A.M. Epstein, "Differences between For-Profit and Not-for- Profit Health Plans Participating in the Medicaid Program" (Unpublished paper, 2000).
21. E.M. Silverman, J.S. Skinner, and E.S. Fisher, "The Association between For- Profit Hospital Ownership and Increased Medicare Spending," New England Journal of Medicine 341, no. 6 (1999): 420–426.
22. L.R. Burns and D.R. Wholey, "Differences in Access and Quality of Care across HMO Types," Health Services Management Research 4, no. 1 (1991): 32–45; A.R. Davies and M.E. Ware, "Patients’ Evaluation of Competing Systems of Care: Preliminary Results from the Medical Outcomes Study" (Paper presented at the Fifth Annual Association for Health Services Research Meeting, Washington, D.C., 1988); and N. Boles and T.T.H. Wan, "Longitudinal Analysis of Patient Satisfaction among Medicare Beneficiaries in Different Models of Health Maintenance Organizations and Fee-for-Service Care," Health Services Management Research 5, no. 3 (1992): 198–206.
23. Landon, "A Conceptual Model."

                      What's this?

This article has been cited by other articles:

				D. P. Scanlon, S. Swaminathan, M. Chernew, and W. Lee Market and Plan Characteristics Related to HMO Quality and Improvement. Med Care Res Rev, December 1, 2006; 63(6 Suppl): 56S - 89S. [Abstract] [PDF]

				M. Schlesinger and B. H. Gray How Nonprofits Matter In American Medicine, And What To Do About It Health Aff., July 1, 2006; 25(4): W287 - W303. [Abstract] [Full Text] [PDF]

				E. C. Schneider, A. M. Zaslavsky, and A. M. Epstein Use of High-Cost Operative Procedures by Medicare Beneficiaries Enrolled in For-Profit and Not-for-Profit Health Plans N. Engl. J. Med., January 8, 2004; 350(2): 143 - 150. [Abstract] [Full Text] [PDF]

				W. S. Brewbaker III Will Physician Unions Improve Health System Performance? Journal of Health Politics Policy and Law, August 1, 2002; 27(4): 575 - 604. [Abstract] [PDF]

				B. H. GRAY and J. P. CLEMENT Databases for Research on Nonprofit Health Care Organizations: Opportunities and Limitations American Behavioral Scientist, June 1, 2002; 45(10): 1550 - 1591. [Abstract] [PDF]

				H. T. Tu and J. D. Reschovsky Assessments of Medical Care by Enrollees in For-Profit and Nonprofit Health Maintenance Organizations N. Engl. J. Med., April 25, 2002; 346(17): 1288 - 1293. [Abstract] [Full Text] [PDF]

				J. Needleman The Role of Nonprofits in Health Care Journal of Health Politics Policy and Law, October 1, 2001; 26(5): 1113 - 1130. [PDF]