|
Health Plan Characteristics And Consumers’ Assessments Of Quality
Many purchasers and consumers of health care have become concerned about the quality of care being delivered in managed care plans. Little is known, however, about the health plan characteristics that are associated with better performance. We used survey responses from 82,583 Medicare beneficiaries from 182 health plans to study the association of consumers’ assessments of care with health plan characteristics. For-profit and nationally affiliated health plans received much worse scores on the outcomes of interest, particularly for overall ratings of the health plan and composite measures of customer service and access to care. Health plans accredited by the National Committee for Quality Assurance did not receive higher scores.
Managed care is the most prevalent type of coverage for commercially insured persons and is widely used in Medicaid programs and Medicare.1 Consequently, purchasers and consumers of care have become increasingly interested in the quality of care provided by managed care plans. Although managed care is well established, there has been a "backlash" against it, including proposed patient protection legislation in both federal and state governments.2 This backlash, however, has been driven more by perceptions and anecdote than by objective information.3 These developments have heightened interest in standardized quality measures for managed care plans, including measures of clinical processes of care, outcomes, and patients’ reports of care. The Consumer Assessment of Health Plans (CAHPS) survey has quickly become the most widely accepted patient-based method for assessing of health plan quality. In 1998 and 1999 the Health Care Financing Administration (HCFA) administered the CAHPS survey to samples of Medicare beneficiaries enrolled in all Medicare managed care plans. This provides an unprecedented opportunity to explore the relationships between health plan characteristics and the selected aspects of quality assessed by CAHPS. Such information could provide insights into features of plans that might affect quality of care. Analyses of plan characteristics also could broaden our understanding of factors associated with accreditation. For example, if to become accredited by the National Committee for Quality Assurance (NCQA), health plans adopt policies and procedures that are related to customer service or access to care, accredited plans might perform better in some of these areas. In this paper we link information on plan characteristics from the InterStudy Competitive Edge database and the NCQA to Medicare managed care plans and assess the characteristics that are associated with performance on CAHPS and that might be reflected in disenrollment rates.4
Sites and sample. The Medicare version of CAHPS was administered nationally by a single vendor, and all aspects of the survey, including sampling, coding, and analysis, were conducted uniformly. Therefore, it is not subject to variations in data quality that may affect other assessments of health care quality. Eligible plans included all 212 health plans or health plan units with separate Medicare risk contracts in effect on or before 1996 and in business for at least two years. For health plans with multiple contracts, each contract unit was studied separately, since each contract represents a distinct business relationship with HCFA and covers a separate geographic region. For each contract, HCFA drew a sample of enrollees who had been enrolled for at least twelve months. For most plans, a simple random sample of 600 patients was drawn, except for twelve plans with small enrollment, for which all eligible patients were surveyed. We excluded beneficiaries who left their plan before actually completing the survey, because the corresponding responses could not reliably be linked with a single plan. We also deleted one plan with only fifteen respondents and an additional thirteen plans that had ceased activity or been terminated. This left a total of 89,419 responses from198 risk contracts, for a response rate of 75 percent. Survey data collection took place from February to May 1998 and reflects care in calendar year 1997. Measures. The CAHPS–Medicare Managed Care (MMC) survey includes four items that elicit overall ratings (of the plan, personal doctor, care received overall, and care received from specialists) and thirty-four that elicit reports of respondents’ experiences. We previously found that four factors explained most of the interplan variability in reports about specific experiences with care.5 The "delivery" composite includes sixteen items on care received primarily at the doctor’s office, and the "customer service" composite consists of five items on customers’ dealings with the plans. The "access" composite summarized eight reports on obtaining medical services and equipment from the plan, and the "advice" composite sums items on advice to quit smoking and about diet and exercise. These four composites and the four ratings of care have been shown to differentiate among plans within the same market as well as among plans in different metropolitan statistical areas (MSAs) and regions of the country.6 Three of the measures—the customer service and access composites and the plan rating—were shown to be most significantly determined by the health plan, as opposed to the market or region where the plan is located. We analyzed all eight of the above outcomes, but we focus on these three outcomes of interest because we thought they would be most likely to reflect the health plan’s behavior. Finally, we obtained rates of disenrollment from each plan in the prior year from HCFA. InterStudy data. We obtained health plan characteristics from the InterStudy Competitive Edge 8.2 (1998) release containing annually updated information on U.S. health maintenance organizations (HMOs).7 We hypothesized that several variables in the InterStudy database would be related to CAHPS scores. These included the number of years the plan had been in operation, model type, profit status, national affiliation (defined by InterStudy as health plans operating in two or more states and with more than 10,000 total enrollees), several measures of health plan size, federal qualification , and whether the health plan enrolled Medicaid, point-of-service (POS), or preferred provider organization (PPO) enrollees.8 Because tax status and whether the plan was national or not were strongly correlated, we created separate categories for local and national not-for-profit and for-profit plans. Unaffiliated Blue Cross plans were classified as local because of the independent operations of the Blue Cross affiliates. Each health plan was assigned to the federal region where it had the largest enrollment; for this analysis, the contiguous Upper Midwest, East Midwest, and Mountain regions were combined into a single region because of the small numbers of plans in these regions.9 NCQA data. We obtained information from the NCQA on whether health plans were fully accredited, accredited for one year (including provisional accreditation), or unaccredited in 1998.10 The last category included six plans that had never applied for accreditation or had failed accreditation. Matching the data. To match plans in the InterStudy database with Medicare contracts, we first matched health plan names and confirmed that all matches had overlapping service areas. We reviewed the list of unmatched plans and found and verified matches for plans with slightly different official names for the Medicare product and, in some cases, either called the health plan directly to confirm alternative names or confirmed a match using information from HCFA or other sources. We successfully linked 182 of 198 Medicare plans to the InterStudy database and to the NCQA accreditation data. After eliminating survey responses from the unmatched plans, we were left with 82,583 completed surveys out of 109,542 eligible sampled cases, for a response rate of 75.4 percent. Analysis. We tested bivariate associations between plan characteristics and case-mix-adjusted CAHPS scores using t-tests for dichotomous variables and ANOVA for other categorical variables.11 We then performed multiple linear regression to analyze the association of the CAHPS scores with features of health plans, controlling for regional effects, and we calculated regression adjusted means for the ownership/national affiliation categories. All candidate variables were included in the final regression models with several exceptions.12 To check possible sensitivity to differences between plans in the Pacific region, which were largely for-profit, and the rest of the country, we repeated the analysis excluding plans from the Pacific region. Also, because there were relatively few national not-for-profit plans (twenty-nine health plan units, six national plans), and because many of these units came from one company, we reestimated the models including an indicator variable for the national not-for-profit plan with the most contracts. Finally, we repeated the analysis using a model including a dummy for each MSA.13
Plan characteristics. Survey respondents were generally representative of the Medicare managed care population as a whole (Exhibit 1  ). Differences across regions are attributable to variation in the size of health plans among regions.14 Of the 182 health plans, there were at least thirteen plans in each region (Exhibit 2). Medicare beneficiaries, on average, comprised 13 percent of plan enrollment, although a few plans had a much higher proportion of Medicare enrollees, mostly larger for-profit health plans located in the Pacific, Southwest, and Southeast regions. Most plans were either network or independent practice association (IPA) plans; only 25 percent were staff- or group-model HMOs. Of the plans, 136 had either full or one-year accreditation from the NCQA, 113 were affiliated with national managed care companies, and 112 were for-profit.
Associations between CAHPS scores and plan characteristics. CAHPS includes both reports about members’ specific experiences and more general ratings, and it distinguishes between members’ evaluations of health plans and their experiences with physicians. For this analysis, we focused on the scores that we thought would be most likely to reflect the health plan’s behavior: plan rating and the access and customer service composites. Survey respondents tended to rate their care toward the upper end of an eleven-point scale. Likewise, mean responses for the experience composites were all above 3 on a scale of 1 to 4 (1 being "never," 4 being "always"), with a higher score indicating better experiences with the health plan. Bivariate analyses revealed important differences in performance by region, profit status, and national affiliation. Plan size, model type, and accreditation status were associated with better performance for a few endpoints but were generally not related to the three outcomes of primary interest.
Most of these relationships were still significant after multivariable adjustment.15 Results for the three primary outcomes are presented in Exhibit 3
Regional effects. Region was a strong predictor in each model. Mean scores were lowest in the Pacific region and highest in the Northeast and North Mid-Atlantic regions. For instance, the coefficient representing the difference in care rating between the Northeast and the Pacific regions was equal to almost three standard deviations (not shown), while the coefficient for plan rating was equal to about one standard deviation in the Northeast and two in the Southwest compared to the Pacific. A difference of one standard deviation is thought to represent a considerable difference in analyses such as these. Health plan type and accreditation. In bivariate analyses, group- and staff-model HMOs had lower scores on the delivery composite and lower ratings of specialists than IPA or network plans had (results not shown), but these results were not significant in the multivariate analyses. Small health plans tended also to have lower ratings of specialist care and of the plan in general; they also had lower scores on the advice composite than medium-size plans had. With one exception, both federal qualification and NCQA accreditation were not significantly associated with plan performance. Full NCQA accreditation was associated with improved performance only on the access composite.
Tax status and national operations.
The strongest predictors of performance were tax status and national affiliation (Exhibit 4
To further evaluate the importance of these results, we examined disenrollment rates for the prior year.16 National for-profit health plans had a disenrollment rate that was nearly twice that of national not-for-profits (14.7 percent versus 7.7 percent, p = .0002). Similarly, the disenrollment rate for local for-profit health plans was greater than that of local not-for-profits, although this difference is not statistically significant (16.3 percent versus 11.7 percent, p = .11). To explore whether results for for-profit plans were confounded with the effect of being in that region, where many of them are located, we repeated the analysis excluding the Pacific region. These results were generally consistent with the model based on the full sample. Some findings for local for-profit plans were no longer statistically significant, primarily because many of these plans were in the Pacific region. When we repeated the models including an indicator variable for the not-for-profit plan with the largest number of contracts, the coefficient for that plan was similar to that for the remaining national not-for-profits, and both remained significant.
This is the first study to report health plan characteristics associated with performance based on a uniform national administration of the CAHPS survey. We found that for-profit and national health plans scored lower for almost all of the outcomes we examined, even after region and other plan characteristics were controlled for. The fact that these effects remained even after controlling for MSA effects indicates that they are attributable to characteristics of individual plans rather than characteristics of the area in which plans operate. Although we do not know what accounts for these differences, our findings are consistent with prior studies.17 Other factors, including model type, federal qualification, and NCQA accreditation status, were only weakly related to plan performance. This finding is of particular policy relevance because many purchasers of care use NCQA accreditation as a proxy for quality. One could reasonably conclude that what NCQA accreditation measures, such as structural characteristics and financial stability, is not directly related to consumers’ experiences with care. Of course, accreditation status could be justified on other grounds.18 For-profit health plans had lower average scores on the composites and ratings that we hypothesized would be most related to health plan characteristics, but they performed less poorly on factors related to physician care. This suggests that patients have a more difficult time interacting with for-profit plans and gaining access to services, but they receive similar care from physicians. The results also support the notion that consumers can distinguish between care from the physician and care provided by the plan. Although most for-profit plans are national, the findings related to national plans are distinct from those related to profit status, primarily because they also apply to national not-for-profit plans. National plans performed worse for ratings of care, the primary doctor and specialists, and the delivery composite. This suggests hat the effects related to national plans extend into physicians’ offices. There are several possible explanations for this. Because of more far-flung operations, national plans might have looser relationships with physicians. These plans, despite their large enrollments, might be less important for any individual physician’s practice. As a result, physicians’ offices might be less familiar with the administrative requirements of the plan or less willing to invest time in improving the working relationship, and physicians might feel less of a connection to them than to other plans. Our findings are consistent with a growing literature demonstrating lower performance by for-profit health care. David Himmelstein and colleagues recently reported that for-profit health plans, after adjusting for regional effects, had lower performance on Health Plan Employer Data and Information Set (HEDIS) quality measures.19 That study, however, was based on a smaller sample of voluntarily reporting plans and relied on decentralized data collection. Bruce Landon and Arnold Epstein report that for-profit Medicaid health plans had more stringent utilization review rules than not-for- profit plans had.20 In other sectors, such as the hospital industry, studies have demonstrated increased costs and utilization in forprofits.21 While these overall results mask substantial heterogeneity within these classifications, taken together, these findings represent a disturbing pattern relating for-profit status and the delivery of care. Minnesota currently requires that health plans be not-for-profit. Our data show that many for-profit plans outperform the average not-for-profit plans, so the evidence for such a step is mixed at best. However, the data also indicate a pressing need to understand the reasons for such differences. We failed to find any significant associations between model type and consumer ratings, unlike several prior studies.22 In general, these studies showed improved satisfaction for IPA model plans. Our results might differ for several reasons. First, these studies use data that are more than ten years old and thus might not reflect fundamental changes in the health care market since that time.23 Second, there are relatively few staff/group-model HMOs that provide Medicare services. Finally, with time, staff/group-model plans have become increasingly less relevant to the market, which clearly favors expansive plans with larger networks. In addition, previous differences might have also diminished because many large physician organizations have adopted the features of closed-panel health plans and thus might appear similar to consumers. Thus, from a policy perspective, distinctions by model type have diminishing relevance. Limitations. Our study has several limitations. First, we were limited to analyzing a small set of health plan characteristics that we were able to obtain from InterStudy and the NCQA. We did not have sufficient information to characterize fully the diversity of health plans or to describe the heterogeneity that exists within classes of health plans. While tax status might affect an overall approach to the business of health care, not all for-profit plans operate similarly; we do not know the organizational mediators between tax status and plan performance. The limited availability of descriptive information on health plans, particularly given their ever changing nature and how they organize to manage and provide care, highlights the need for better and more comprehensive data on health plan characteristics so that changes in how health plans organize to manage care can be assessed more systematically. Second, our data are cross-sectional and limited to the Medicare population. Consequently, our findings reflect the experiences of an elderly and relatively sicker population and might not generalize to the larger commercially insured population. However, because the elderly are more likely to have complex health care needs and many Medicare products are newer than commercial products, their experiences may be more sensitive to plan factors that affect quality. Finally, we did not specifically assess effects of benefit generosity or of local reimbursement rates that the health plans receive per enrollee. By controlling for market, however, we avoided confounding with these effects because reimbursement rates and levels of benefits tend to be similar within each market. Although the data from CAHPS assess only selected areas of quality, they provide the most standardized, independent, and comprehensive measure of health care quality available for Medicare managed care plans. These measures are not related to either federal qualification or accreditation status. There are, however, systematic and significant differences in quality related to region, organizational form, and ownership. Understanding the reasons for these variations might provide important insights into reasons for differences in plan performance that would allow us to improve health care quality in all plans.
Bruce Landon is an instructor in health care policy at Harvard Medical School and an instructor in medicine at Beth Israel Deaconess Medical Center. Alan Zaslavsky is associate professor of statistics, Department of Health Care Policy, Harvard Medical School. Nancy Beaulieu is an assistant professor at the Harvard Business School. James Shaul is project director for the development of the Consumer Assessment of Health Plans (CAHPS) Behavioral Health Survey. Paul Cleary is professor of health care policy, Harvard Medical School and School of Public Health. This work was supported by a grant from the Commonwealth Fund and a contract from the Health Care Financing Administration (500-95-007). We acknowledge programming assistance by Matthew Cioffi and Lin Ding. We thank Tom Reilly, Liz Goldstein, Terry Lied, and collaborators at HCFA, Barents, Westat, DRC, and the Picker Institute for their efforts in the survey implementation that generated the data on which this paper is based.
This article has been cited by other articles:
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Top
Data And Methods
Study Results
