|
Medicare’s National Coverage Decisions, 1999–2003: Quality Of Evidence And Review Times
In 1998 Medicare amended its procedures for making national coverage decisions for new technologies in an attempt to make the process more transparent and evidence based. We examined the quality of evidence for sixty-nine technologies reviewed by Medicare since then. Determinations by the Centers for Medicare and Medicaid Services (CMS) have generally been consistent with the strength of evidence. Good clinical evidence from rigorous studies is usually lacking for the technologies Medicare considers, although in most cases the CMS covers with conditions if there is at least fair evidence that benefits outweigh harms. Decisions referred to the external Medicare Coverage Advisory Committee (MCAC) have averaged eight months longer than non-MCAC decisions.
Medicare coverage rules for new technologies can have profound implications for beneficiaries’ access to medical advances and for Medicare’s fiscal well-being. Many have criticized the coverage process over the years for its lack of consistency or openness.1 In 1998 Medicare amended its procedures for making national coverage decisions for new technologies, in an attempt to make the process more transparent, predictable, and evidence based. However, little empirical research has examined how the process is working. We investigated the quality of evidence available to Medicare decisionmakers for each technology it has considered for national coverage determinations since it created its new process. We also analyzed how evidence and other factors influence coverage decisions and review times at the Centers for Medicare and Medicaid Services.
Medicare’s authorizing legislation in 1965 established broad categories of coverage for hospital and physician services but prohibited payment for expenses incurred for "items and services that are not reasonable and necessary for the diagnosis or treatment of illness or injury or to improve the functioning of a malformed body member" (emphasis added).2 However, broad discretion was left to Medicare officials. In practice, most coverage decisions have been made by Medicare’s local contractors—the state- or regionwide health insurers (carriers and fiscal intermediaries) who pay claims for the program.3 The Medicare program has made national coverage decisions (NCDs) each year for some ten to fifteen technologies deemed particularly controversial or projected to have a major impact on the program.4 In terms of evidence, for many years the phrase "reasonable and necessary" was understood (although never clearly defined) to reflect the prevailing views of the physician community. However, the Medicare program did not have formal criteria to define this standard, nor did it formally define a process for reviewing technologies at the national level. Amid growing complaints that its coverage process was opaque, unpredictable, and often lengthy, in 1989 HCFA (now the CMS) published a proposed regulation stating that for purposes of coverage, a technology would have to be accepted by the medical community, safe, effective, noninvestigational, and appropriate.5 Adding cost-effectiveness as an explicit criterion for selected cases was also proposed. The proposal was criticized, especially for its cost-effectiveness provision, and was eventually withdrawn.6 Criticism of the process continued, however, and in the late 1990s the CMS decided to overhaul the national Medicare coverage process. It established the Medicare Coverage Advisory Committee (MCAC) to provide external assistance in judging whether evidence existed to establish the safety, efficacy, and clinical benefit of a medical service or product for national coverage decisions.7 Importantly, MCAC only proffers advice; the CMS retains control over final decisions. Moreover, only those national coverage decisions deemed in need of additional expertise go to MCAC. The CMS relies on its own medical experts and occasionally requests a formal health technology assessment (HTA) from the Agency for Healthcare Research and Quality (AHRQ). In 1999 the CMS also sought to streamline its NCD process.8 The CMS stipulated it would respond in writing to a requester within ninety days of accepting a complete request (and if the requester submitted additional information, the agency would respond within ninety days of receiving it). If a referral was made to MCAC, the CMS would make a decision within sixty days of the committee’s recommendation. In recent years the CMS has attempted to further clarify its process, although it has never published explicit criteria for judging evidence.9
We reviewed all complete Medicare national coverage decisions from 1998 through August 2003, based on publicly available decision memoranda posted on the CMS Web site.10 Of eighty-two decisions, several were excluded because they pertained to minor coding or language changes (n = 7), dealt with exceptional circumstances (for example, coverage of liver transplants at nonauthorized hospitals during a local flood) (n = 3), or had incomplete decision memoranda (n = 3), leaving a final sample of sixty-nine. Each memorandum was scrutinized independently by two trained reviewers who extracted data on numerous variables, including CMS review times (from the date the CMS accepted a request to the date it posted the final decision memorandum), the quality of evidence, and the final decision. Data were extracted with the aid of a data collection form, which was pilot-tested several times by our reviewers and amended for clarity and completeness before implementation. For each technology, we coded the strength of supporting evidence in its favor, as reported by the CMS in decision memoranda, based on criteria developed by the U.S. Preventive Services Task Force (USPSTF).11 Evidence was deemed "good" if it included consistent results from well-designed, well-conducted studies in representative populations; "fair" if evidence was sufficient to determine the effect on health outcomes but its strength was limited by the number, quality, or consistency of individual studies; or "poor" if evidence was insufficient to assess the effects on health outcomes because of the limited number or power of studies, flaws in their design or conduct, or lack of information on important health outcomes.
Following the USPSTF, we also considered a broader conception of evidence, taking into account "net benefit" (substantial, moderate, small, or zero/negative), based on trade-offs among benefits and harms.12 We classified technologies according to the approach employed by the USPSTF accounting for both quality of evidence and net benefit (Exhibit 1
We reviewed all of the background information and evidence provided by the CMS for each decision, and we reviewed all primary data provided on the CMS Web site for each decision, including the original technology assessments for every decision that had an assessment available. We then rated the quality of the evidence and net benefit, taking into account these data as well as the CMS analysis of the data. We also recorded evidence limitations reported by the CMS and any conditions placed on coverage decisions. All data were entered into a Microsoft Access database, and analyses were conducted in Excel and SAS version 8. Interrater reliability was determined, and a kappa score was calculated for each item (0.84 for the item on quality of evidence). A consensus meeting was held if the two readers disagreed. We hypothesized that CMS decisions would be consistent with the quality of the evidence: that is, technologies with stronger evidence would be more likely to receive favorable reviews. We also hypothesized that decisions involving MCAC participation or HTAs would take much longer than those without such involvement, even when the decision was adjusted for the quality of evidence.
The sixty-nine technologies considered for national coverage decisions between January 1999 and August 2003 applied to a wide range of diseases and conditions, and they pertained mostly to medical devices (41 percent) and surgical procedures (23 percent) (Exhibit 2  ). The technologies constitute a diverse group, from air-fluidized beds for pressure ulcers to implantable cardioverter defibrillators and positron emission tomography (PET) scanning for various indications (each of which requires a separate decision).14
During this time period, the CMS referred 22 percent of decisions to MCAC (Exhibit 2  ). In 20 percent of cases, a formal HTA was conducted. Overall, 16 percent of technologies had what the CMS considered "good" evidence, while 42 percent had "fair" and 33 percent "poor" evidence. Common limitations reported by the CMS included a limited number of supporting studies (68 percent), a limited number of patients in the studies (58 percent), and lack of controls (52 percent). In 61 percent of cases, the CMS’s final verdict was to "cover with conditions." In 46 percent of all cases, coverage was limited to patients with more severe forms of the disease or condition in question. Other common restrictions included the imposition of diagnostic test thresholds (30 percent) and limiting coverage to patients who failed first-line therapy of other treatments (23 percent) or to specific settings of care (for example, coverage of transplants only in facilities with demonstrated experience) (13 percent).
Technologies with good evidence were more likely than those with fair or poor evidence to be covered (relative risk = 1.46, = .004) (Exhibit 3
Involvement of MCAC or HTA, or both, was associated with longer review times than cases not involving MCAC or HTA: 14.1 months versus 6.0 months (p < .00001) (Exhibit 5 ). (Decisions with MCAC involvement alone compared with no MCAC involvement took 14.9 months and 7.1 months, respectively, p <.0001.) After we adjusted for the quality of evidence, we found that decisions with MCAC involvement still took much longer than those without it (15.3 months versus 7.8 months, p <.01 for decisions with fair or poor evidence) (Exhibit 6). Reviews were shortest for the ten decisions with no MCAC involvement and good evidence (6.5 months). We obtained similar results for decisions with HTA, controlling for evidence (data not shown).
Reviews begun during 2001–2003 took longer than reviews begun during 1999–2000, although differences were not significant (16.1 months in 2001–2003 versus 14.0 in 1999–2000, p = .45, for reviews with MCAC involvement; and 10.1 months versus 7.9 months, p = .22, for reviews without MCAC involvement) (as of 1 August 2004; data not shown).
Medicare and evidence-based medicine. The Medicare national coverage process has become more transparent and evidence based. The availability of decision memoranda and tracking sheets on the CMS Web site has made it much easier to follow these decisions and the rationales underlying them. CMS decisions have generally been consistent with the strength of supporting evidence. Technologies with stronger evidence are much more likely to receive a favorable review. However, our research underscores the fact that there are gaps in the evidence base and that the quality of evidence available to the CMS is usually less than optimal. Despite the call for evidence-based decision making, most of the technologies considered by the CMS had only fair or poor supporting evidence, with few or no data available from randomized controlled trials. Nevertheless, the CMS has generally found that there is at least fair evidence that the technology’s benefits outweigh its harms in some circumstances. Most frequently, the national process yields a favorable review, although Medicare places conditions on coverage. As others have noted in describing private health plans’ decision making, coverage policy involves much more than reviewing the rigor of the study design underlying the evidence and simply granting a yes/no verdict.15 Rather, it means sifting through the multifaceted evidence, balancing benefits and risks, and determining more finely when coverage is appropriate for which patients, under what conditions, and in what settings. The creation of MCAC has provided the CMS with external expertise and advice, as well as a public forum for deliberation. However, MCAC participation is associated with longer review times—about eight months longer on average than reviews without MCAC involvement. It is not clear to what extent MCAC causes delays: It is possible that MCAC receives the more difficult and time-consuming cases in the first place. However, even when the quality of evidence is adjusted for, MCAC decisions are associated with much longer time frames. In addition, the presence of a formal HTA is also associated with extended review times. The longer review times for MCAC and HTA-assisted reviews are not surprising, because the committee takes time to convene, and assessments require time to prepare. However, the data underscore trade-offs between the intensity of the review and timeliness of the decision. Moreover, the CMS has frequently failed to meet its own standards for prompt assessment, a finding consistent with other smaller studies on the subject.16 Several factors may cause elongated review times for national coverage decisions, including the complexity and controversy surrounding cases and the fact that on occasion the CMS considers additional materials provided by NCD requesters during the review process.17 However, even under the conservative assumption that all non-MCAC and non-HTA decisions involve additional information submitted to the CMS (in which case review deadlines increase from three months to six months), fourteen of the forty-four decisions (32 percent) exceeded the six-month window. Furthermore, review times have increased over the years, which could reflect the increased complexity of cases or more rigorous evaluations.18 The influence of costs and politics. Although our study did not directly examine the role of economic and political considerations, it seems probable that such factors play a role. Regarding costs, CMS officials have often denied that cost considerations play a role in coverage decisions, although they have also acknowledged that technologies with high budget impacts may receive a much more careful evidence review.19 CMS officials have also stated that because of the special challenges raised by using economic evaluation to inform decisions, they may consider cost outside the realm of traditional medical necessity determinations—for example, by limiting coverage to centers with evidence of unique expertise, as in the case of lung volume reduction surgery (LVRS) or left ventricular assist devices.20 It is also important to acknowledge that medical technology attracts bipartisan and public support.21 The national coverage process is emblematic of broader tensions in the Medicare program over how to expand access to beneficial services while ensuring judicious use of public funds. Stakeholders in the process tend to focus on potential benefits of treatments rather than potential health risks or costs to Medicare. Not surprisingly, coverage decisions for tests and treatments for diseases that are well known to the public, such as PET scanning for Alzheimer’s disease, tend to attract greater political attention. Such decisions may take longer when evidence is complex, when consequences for beneficiaries’ health are less certain, or when there are differing views on how to interpret or apply available evidence. Our group is assessing these links in ongoing work.22 Policy implications. The Medicare Prescription Drug, Improvement, and Modernization Act (MMA) of 2003 attempts to open and improve the coverage process further, by imposing stricter standards for CMS reviews, by requiring the CMS to make its rationale for decisions public, and by adding a public comment period before implementing NCDs (MMA Sec. 1013). It creates a council for technology and innovation within the CMS to coordinate coverage, coding, and payment for new technologies, including drugs. Congress and other federal officials could help matters further by addressing gaps in the evidence base—for example, by funding more "pragmatic" clinical trials to aid public- and private-sector decisionmakers.23 Section 1013 of MMA calls for research on outcomes, comparative clinical effectiveness, and appropriateness of health care and thus provides a potentially important vehicle for considering and prioritizing such studies. Medicare could also move to establish clear criteria for coverage and to engage MCAC at the earliest time possible.24 Medical innovators who desire reimbursement can help by bringing good evidence to the review process in the first place. Even with improvements, challenges will continue to confront policymakers, including the methods used for weighing evidence. The fact that the CMS is confronted with fair or poor evidence is not surprising and not necessarily alarming. As underscored by other papers in this issue of Health Affairs, in some cases the costs of obtaining clinical evidence from well-controlled trials can be prohibitively high, particularly for "small-ticket" medical devices.25 Also, treatments that get into the national process are, by definition, those for which there is complex or uncertain evidence about their effectiveness (if the evidence was a "slam dunk" in favor of or against use, a national coverage decision would not be needed). Although many practical issues remain, the CMS could shore up the methodological basis for making evidence determinations and judging net benefit, which today remain vague. Specifically, the CMS could make more use of formal methods to structure decision problems, to synthesize evidence and characterize the uncertainty surrounding a technology, and to determine when a decision should be postponed until additional information is collected.26 Another issue pertains to the role of cost-effectiveness analysis (CEA) in coverage decisions. History suggests that it will be extremely difficult for the CMS to use CEA explicitly, because of concerns about the validity and transparency of CEA methods, difficulties in applying results to individual patients, and political opposition to explicit limit-setting exercises.27 MMA is notably silent about cost-effectiveness, although CEA could be incorporated at a local level by competing private plans or pharmacy benefit managers (PBMs) administering the drug benefit.28 Despite the challenges, CEA offers a powerful technique to help improve the value of health spending, and Congress and administration officials should explore ways in which it might be used to help target resources.29 Study limitations. Our study has several limitations. First, as noted, there could be other factors influencing decisions that are not made public by the CMS or not easily codified, such as the budget impact or degree of political pressure associated with a technology. Moreover, our classifications of evidence into "good," "fair," and "poor" categories, and according to net benefit, are relatively crude and may not adequately capture the complexity involved. Similarly, our classification of decisions into the broad category "covered with conditions" does not capture the restrictiveness of the conditions. Other researchers might categorize variables differently, although our overall interrater reliability scores were high. It will be important in future work to consider more closely the quality of evidence and the nature of conditions placed on coverage. Second, judgments about the quality of evidence for each technology were based on assessments by CMS staff, as coded by our research team, and were not independently validated. On the one hand, it might not be surprising that CMS memos on evidence quality would correlate with CMS coverage decisions, because CMS staff would strive to achieve consistency between the two. On the other hand, CMS decision memoranda are generally detailed about the type of evidence available to them—for example, about the lack of well-conducted randomized clinical trials or controlled studies. Moreover, documenting the CMS’s own reflections of the poor quality of evidence is important in itself. Finally, we lack good information on why technologies were considered for national coverage decisions in the first place. Medicare’s national cover age p olicy continues to evolve toward becoming a more open and evidence-based process. CMS decisions from 1998 to 2003 were generally consistent with the strength of evidence. Good clinical evidence from rigorous studies was usually lacking, although in most cases the CMS covered with conditions on grounds that there was at least fair evidence in certain circumstances that benefits outweighed harms. Our findings also underscore the trade-offs between rigorous review and timely decision making. In the future it will be important to continue tracking the quality of evidence, as well as the timeliness and consistency of CMS decision making. The steady development of medical innovation, the aging of the baby boomers, and ongoing fiscal pressures will continue to challenge the CMS. Coverage policy can serve as a critical lever with which to help ensure that Medicare obtains good value for its spending.
Peter Neumann (pneumann{at}hsph.harvard.edu) is an associate professor of policy and decision sciences at the Harvard School of Public Health in Boston, Massachusetts. Nomita Divi is project coordinator, Cost-Effectiveness of Preventing AIDS Complications Project, Massachusetts General Hospital (MGH), in Boston. Molly Beinfeld is research associate, Foundation for Informed Medical Decision Making, in Boston. Bat-Sheva Levine is an assistant in medicine, Department of Endocrinology, Children’s Hospital, Boston, and an instructor in pediatrics at the Harvard Medical School. Patricia Keenan is a postdoctoral fellow in aging and health economics at the National Bureau of Economic Research in Cambridge, Massachusetts. Elkan Halpern is director of statistics for the Institute for Technology Assessment (ITA) at MGH and chief statistician for the MGH Department of Radiology. Scott Gazelle is founder and director of the ITA at MGH. This project was supported by Grant no. 046071 from the Robert Wood Johnson Foundation under its Changes in Health Care Financing and Organization (HCFO) initiative. Patricia Keenan gratefully acknowledges funding from the National Institute on Aging, Grant no. T32-AG00186.
This article has been cited by other articles:
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Top
The Evolution Of...
Study Data And Methods
