Health Affairs, 24, no. 1 (2005): 243-254 doi: 10.1377/hlthaff.24.1.243 © 2005 by Project HOPE	New Online   Obama's Success & Next Steps   3 Health Policy Options for President Obama   A Hybrid Proposal   Bipartisan Compromise

This Article Abstract Figures Only Reprint (PDF) Submit a response to this article Alert me when this article is cited Alert me when Comments are posted Alert me if a correction is posted Services E-mail this article to a friend Similar articles in this journal Similar articles in ISI Web of Science Similar articles in PubMed Alert me to new issues of the journal Add to My Personal Archive Download to Citation Manager Reprints & Permissions Citing Articles Citing Articles via ISI Web of Science (12) Citing Articles via Google Scholar Google Scholar Articles by Neumann, P. J. Articles by Gazelle, G. S. Search for Related Content PubMed PubMed Citation Articles by Neumann, P. J. Articles by Gazelle, G. S. Related Collections Medicare Quality Of Care

DataWatch

Medicare’s National Coverage Decisions, 1999–2003: Quality Of Evidence And Review Times

Abstract

 
In 1998 Medicare amended its procedures for making national coverage decisions for new technologies in an attempt to make the process more transparent and evidence based. We examined the quality of evidence for sixty-nine technologies reviewed by Medicare since then. Determinations by the Centers for Medicare and Medicaid Services (CMS) have generally been consistent with the strength of evidence. Good clinical evidence from rigorous studies is usually lacking for the technologies Medicare considers, although in most cases the CMS covers with conditions if there is at least fair evidence that benefits outweigh harms. Decisions referred to the external Medicare Coverage Advisory Committee (MCAC) have averaged eight months longer than non-MCAC decisions.

The Evolution Of Medicare’s National Coverage Process

Top The Evolution Of... Study Data And Methods Study Results Discussion And Policy... Editor's Notes NOTES

 
Medicare’s authorizing legislation in 1965 established broad categories of coverage for hospital and physician services but prohibited payment for expenses incurred for "items and services that are not reasonable and necessary for the diagnosis or treatment of illness or injury or to improve the functioning of a malformed body member" (emphasis added).² However, broad discretion was left to Medicare officials. In practice, most coverage decisions have been made by Medicare’s local contractors—the state- or regionwide health insurers (carriers and fiscal intermediaries) who pay claims for the program.³ The Medicare program has made national coverage decisions (NCDs) each year for some ten to fifteen technologies deemed particularly controversial or projected to have a major impact on the program.⁴

In terms of evidence, for many years the phrase "reasonable and necessary" was understood (although never clearly defined) to reflect the prevailing views of the physician community. However, the Medicare program did not have formal criteria to define this standard, nor did it formally define a process for reviewing technologies at the national level. Amid growing complaints that its coverage process was opaque, unpredictable, and often lengthy, in 1989 HCFA (now the CMS) published a proposed regulation stating that for purposes of coverage, a technology would have to be accepted by the medical community, safe, effective, noninvestigational, and appropriate.⁵ Adding cost-effectiveness as an explicit criterion for selected cases was also proposed. The proposal was criticized, especially for its cost-effectiveness provision, and was eventually withdrawn.⁶

Criticism of the process continued, however, and in the late 1990s the CMS decided to overhaul the national Medicare coverage process. It established the Medicare Coverage Advisory Committee (MCAC) to provide external assistance in judging whether evidence existed to establish the safety, efficacy, and clinical benefit of a medical service or product for national coverage decisions.⁷ Importantly, MCAC only proffers advice; the CMS retains control over final decisions. Moreover, only those national coverage decisions deemed in need of additional expertise go to MCAC. The CMS relies on its own medical experts and occasionally requests a formal health technology assessment (HTA) from the Agency for Healthcare Research and Quality (AHRQ).

In 1999 the CMS also sought to streamline its NCD process.⁸ The CMS stipulated it would respond in writing to a requester within ninety days of accepting a complete request (and if the requester submitted additional information, the agency would respond within ninety days of receiving it). If a referral was made to MCAC, the CMS would make a decision within sixty days of the committee’s recommendation. In recent years the CMS has attempted to further clarify its process, although it has never published explicit criteria for judging evidence.⁹

Study Data And Methods

Top The Evolution Of... Study Data And Methods Study Results Discussion And Policy... Editor's Notes NOTES

 
We reviewed all complete Medicare national coverage decisions from 1998 through August 2003, based on publicly available decision memoranda posted on the CMS Web site.¹⁰ Of eighty-two decisions, several were excluded because they pertained to minor coding or language changes (n = 7), dealt with exceptional circumstances (for example, coverage of liver transplants at nonauthorized hospitals during a local flood) (n = 3), or had incomplete decision memoranda (n = 3), leaving a final sample of sixty-nine.

Each memorandum was scrutinized independently by two trained reviewers who extracted data on numerous variables, including CMS review times (from the date the CMS accepted a request to the date it posted the final decision memorandum), the quality of evidence, and the final decision. Data were extracted with the aid of a data collection form, which was pilot-tested several times by our reviewers and amended for clarity and completeness before implementation.

For each technology, we coded the strength of supporting evidence in its favor, as reported by the CMS in decision memoranda, based on criteria developed by the U.S. Preventive Services Task Force (USPSTF).¹¹ Evidence was deemed "good" if it included consistent results from well-designed, well-conducted studies in representative populations; "fair" if evidence was sufficient to determine the effect on health outcomes but its strength was limited by the number, quality, or consistency of individual studies; or "poor" if evidence was insufficient to assess the effects on health outcomes because of the limited number or power of studies, flaws in their design or conduct, or lack of information on important health outcomes.

Following the USPSTF, we also considered a broader conception of evidence, taking into account "net benefit" (substantial, moderate, small, or zero/negative), based on trade-offs among benefits and harms.¹² We classified technologies according to the approach employed by the USPSTF accounting for both quality of evidence and net benefit (Exhibit 1).¹³

All data were entered into a Microsoft Access database, and analyses were conducted in Excel and SAS version 8. Interrater reliability was determined, and a kappa score was calculated for each item (0.84 for the item on quality of evidence). A consensus meeting was held if the two readers disagreed. We hypothesized that CMS decisions would be consistent with the quality of the evidence: that is, technologies with stronger evidence would be more likely to receive favorable reviews. We also hypothesized that decisions involving MCAC participation or HTAs would take much longer than those without such involvement, even when the decision was adjusted for the quality of evidence.

Study Results

Top The Evolution Of... Study Data And Methods Study Results Discussion And Policy... Editor's Notes NOTES

 
The sixty-nine technologies considered for national coverage decisions between January 1999 and August 2003 applied to a wide range of diseases and conditions, and they pertained mostly to medical devices (41 percent) and surgical procedures (23 percent) (Exhibit 2). The technologies constitute a diverse group, from air-fluidized beds for pressure ulcers to implantable cardioverter defibrillators and positron emission tomography (PET) scanning for various indications (each of which requires a separate decision).¹⁴

In 61 percent of cases, the CMS’s final verdict was to "cover with conditions." In 46 percent of all cases, coverage was limited to patients with more severe forms of the disease or condition in question. Other common restrictions included the imposition of diagnostic test thresholds (30 percent) and limiting coverage to patients who failed first-line therapy of other treatments (23 percent) or to specific settings of care (for example, coverage of transplants only in facilities with demonstrated experience) (13 percent).

Technologies with good evidence were more likely than those with fair or poor evidence to be covered (relative risk = 1.46, = .004) (Exhibit 3). In terms of net benefit, technologies most commonly had moderate net benefit but only fair evidence (twenty-five cases, Exhibit 4). In twenty cases there was insufficient information on net benefit. Where the evidence recommendation was A or B, the CMS covered the technology in 100 percent of cases.

View larger version (27K): [in this window] [in a new window]   EXHIBIT 3 Direction Of Medicare Coverage Decision, By Quality Of Evidence

View larger version (26K): [in this window] [in a new window]   EXHIBIT 5 Time To Medicare Coverage Decision, By Involvement Of MCAC And HTA

View larger version (34K): [in this window] [in a new window]   EXHIBIT 6 Time To Medicare Coverage Decision, By MCAC Participation And Quality Of Evidence

Discussion And Policy Implications

Top The Evolution Of... Study Data And Methods Study Results Discussion And Policy... Editor's Notes NOTES

 
Medicare and evidence-based medicine. The Medicare national coverage process has become more transparent and evidence based. The availability of decision memoranda and tracking sheets on the CMS Web site has made it much easier to follow these decisions and the rationales underlying them.

CMS decisions have generally been consistent with the strength of supporting evidence. Technologies with stronger evidence are much more likely to receive a favorable review. However, our research underscores the fact that there are gaps in the evidence base and that the quality of evidence available to the CMS is usually less than optimal. Despite the call for evidence-based decision making, most of the technologies considered by the CMS had only fair or poor supporting evidence, with few or no data available from randomized controlled trials.

Nevertheless, the CMS has generally found that there is at least fair evidence that the technology’s benefits outweigh its harms in some circumstances. Most frequently, the national process yields a favorable review, although Medicare places conditions on coverage. As others have noted in describing private health plans’ decision making, coverage policy involves much more than reviewing the rigor of the study design underlying the evidence and simply granting a yes/no verdict.¹⁵ Rather, it means sifting through the multifaceted evidence, balancing benefits and risks, and determining more finely when coverage is appropriate for which patients, under what conditions, and in what settings.

The creation of MCAC has provided the CMS with external expertise and advice, as well as a public forum for deliberation. However, MCAC participation is associated with longer review times—about eight months longer on average than reviews without MCAC involvement. It is not clear to what extent MCAC causes delays: It is possible that MCAC receives the more difficult and time-consuming cases in the first place. However, even when the quality of evidence is adjusted for, MCAC decisions are associated with much longer time frames. In addition, the presence of a formal HTA is also associated with extended review times. The longer review times for MCAC and HTA-assisted reviews are not surprising, because the committee takes time to convene, and assessments require time to prepare. However, the data underscore trade-offs between the intensity of the review and timeliness of the decision.

Moreover, the CMS has frequently failed to meet its own standards for prompt assessment, a finding consistent with other smaller studies on the subject.¹⁶ Several factors may cause elongated review times for national coverage decisions, including the complexity and controversy surrounding cases and the fact that on occasion the CMS considers additional materials provided by NCD requesters during the review process.¹⁷ However, even under the conservative assumption that all non-MCAC and non-HTA decisions involve additional information submitted to the CMS (in which case review deadlines increase from three months to six months), fourteen of the forty-four decisions (32 percent) exceeded the six-month window. Furthermore, review times have increased over the years, which could reflect the increased complexity of cases or more rigorous evaluations.¹⁸

The influence of costs and politics. Although our study did not directly examine the role of economic and political considerations, it seems probable that such factors play a role. Regarding costs, CMS officials have often denied that cost considerations play a role in coverage decisions, although they have also acknowledged that technologies with high budget impacts may receive a much more careful evidence review.¹⁹ CMS officials have also stated that because of the special challenges raised by using economic evaluation to inform decisions, they may consider cost outside the realm of traditional medical necessity determinations—for example, by limiting coverage to centers with evidence of unique expertise, as in the case of lung volume reduction surgery (LVRS) or left ventricular assist devices.²⁰

It is also important to acknowledge that medical technology attracts bipartisan and public support.²¹ The national coverage process is emblematic of broader tensions in the Medicare program over how to expand access to beneficial services while ensuring judicious use of public funds. Stakeholders in the process tend to focus on potential benefits of treatments rather than potential health risks or costs to Medicare. Not surprisingly, coverage decisions for tests and treatments for diseases that are well known to the public, such as PET scanning for Alzheimer’s disease, tend to attract greater political attention. Such decisions may take longer when evidence is complex, when consequences for beneficiaries’ health are less certain, or when there are differing views on how to interpret or apply available evidence. Our group is assessing these links in ongoing work.²²

Policy implications. The Medicare Prescription Drug, Improvement, and Modernization Act (MMA) of 2003 attempts to open and improve the coverage process further, by imposing stricter standards for CMS reviews, by requiring the CMS to make its rationale for decisions public, and by adding a public comment period before implementing NCDs (MMA Sec. 1013). It creates a council for technology and innovation within the CMS to coordinate coverage, coding, and payment for new technologies, including drugs.

Congress and other federal officials could help matters further by addressing gaps in the evidence base—for example, by funding more "pragmatic" clinical trials to aid public- and private-sector decisionmakers.²³ Section 1013 of MMA calls for research on outcomes, comparative clinical effectiveness, and appropriateness of health care and thus provides a potentially important vehicle for considering and prioritizing such studies. Medicare could also move to establish clear criteria for coverage and to engage MCAC at the earliest time possible.²⁴ Medical innovators who desire reimbursement can help by bringing good evidence to the review process in the first place.

Even with improvements, challenges will continue to confront policymakers, including the methods used for weighing evidence. The fact that the CMS is confronted with fair or poor evidence is not surprising and not necessarily alarming. As underscored by other papers in this issue of Health Affairs, in some cases the costs of obtaining clinical evidence from well-controlled trials can be prohibitively high, particularly for "small-ticket" medical devices.²⁵ Also, treatments that get into the national process are, by definition, those for which there is complex or uncertain evidence about their effectiveness (if the evidence was a "slam dunk" in favor of or against use, a national coverage decision would not be needed).

Although many practical issues remain, the CMS could shore up the methodological basis for making evidence determinations and judging net benefit, which today remain vague. Specifically, the CMS could make more use of formal methods to structure decision problems, to synthesize evidence and characterize the uncertainty surrounding a technology, and to determine when a decision should be postponed until additional information is collected.²⁶

Another issue pertains to the role of cost-effectiveness analysis (CEA) in coverage decisions. History suggests that it will be extremely difficult for the CMS to use CEA explicitly, because of concerns about the validity and transparency of CEA methods, difficulties in applying results to individual patients, and political opposition to explicit limit-setting exercises.²⁷ MMA is notably silent about cost-effectiveness, although CEA could be incorporated at a local level by competing private plans or pharmacy benefit managers (PBMs) administering the drug benefit.²⁸ Despite the challenges, CEA offers a powerful technique to help improve the value of health spending, and Congress and administration officials should explore ways in which it might be used to help target resources.²⁹

Study limitations. Our study has several limitations. First, as noted, there could be other factors influencing decisions that are not made public by the CMS or not easily codified, such as the budget impact or degree of political pressure associated with a technology. Moreover, our classifications of evidence into "good," "fair," and "poor" categories, and according to net benefit, are relatively crude and may not adequately capture the complexity involved. Similarly, our classification of decisions into the broad category "covered with conditions" does not capture the restrictiveness of the conditions. Other researchers might categorize variables differently, although our overall interrater reliability scores were high. It will be important in future work to consider more closely the quality of evidence and the nature of conditions placed on coverage.

Second, judgments about the quality of evidence for each technology were based on assessments by CMS staff, as coded by our research team, and were not independently validated. On the one hand, it might not be surprising that CMS memos on evidence quality would correlate with CMS coverage decisions, because CMS staff would strive to achieve consistency between the two. On the other hand, CMS decision memoranda are generally detailed about the type of evidence available to them—for example, about the lack of well-conducted randomized clinical trials or controlled studies. Moreover, documenting the CMS’s own reflections of the poor quality of evidence is important in itself.

Finally, we lack good information on why technologies were considered for national coverage decisions in the first place.

Medicare’s national cover age p olicy continues to evolve toward becoming a more open and evidence-based process. CMS decisions from 1998 to 2003 were generally consistent with the strength of evidence. Good clinical evidence from rigorous studies was usually lacking, although in most cases the CMS covered with conditions on grounds that there was at least fair evidence in certain circumstances that benefits outweighed harms. Our findings also underscore the trade-offs between rigorous review and timely decision making.

In the future it will be important to continue tracking the quality of evidence, as well as the timeliness and consistency of CMS decision making. The steady development of medical innovation, the aging of the baby boomers, and ongoing fiscal pressures will continue to challenge the CMS. Coverage policy can serve as a critical lever with which to help ensure that Medicare obtains good value for its spending.

Editor's Notes

Top The Evolution Of... Study Data And Methods Study Results Discussion And Policy... Editor's Notes NOTES

 
Peter Neumann (pneumann{at}hsph.harvard.edu) is an associate professor of policy and decision sciences at the Harvard School of Public Health in Boston, Massachusetts. Nomita Divi is project coordinator, Cost-Effectiveness of Preventing AIDS Complications Project, Massachusetts General Hospital (MGH), in Boston. Molly Beinfeld is research associate, Foundation for Informed Medical Decision Making, in Boston. Bat-Sheva Levine is an assistant in medicine, Department of Endocrinology, Children’s Hospital, Boston, and an instructor in pediatrics at the Harvard Medical School. Patricia Keenan is a postdoctoral fellow in aging and health economics at the National Bureau of Economic Research in Cambridge, Massachusetts. Elkan Halpern is director of statistics for the Institute for Technology Assessment (ITA) at MGH and chief statistician for the MGH Department of Radiology. Scott Gazelle is founder and director of the ITA at MGH.

This project was supported by Grant no. 046071 from the Robert Wood Johnson Foundation under its Changes in Health Care Financing and Organization (HCFO) initiative. Patricia Keenan gratefully acknowledges funding from the National Institute on Aging, Grant no. T32-AG00186.

NOTES

Top The Evolution Of... Study Data And Methods Study Results Discussion And Policy... Editor's Notes NOTES

 

1. U.S. Government Accountability Office, Medicare: Divided Authority for Policies on Coverage of Procedures and Devices Result in Inequities, Pub. no. 03-175 (Washington: GAO, April 2003).
2. Social Security Amendments, Sec. 1862[a][1], 1965.
3. This policy reflected a political compromise to those who wanted to maintain the program’s decentralized structure and a bow to practical considerations (that is, it would be difficult for the CMS centrally to assess the myriad medical technology advances). See S.B. Foote, "Why Medicare Cannot Promulgate a National Coverage Rule: A Case of Regula Mortis," Journal of Health Politics, Policy and Law 27, no. 5 (2002): 707–729[Abstract]; and S.B. Foote, "Focus on Locus: Evolution of Medicare’s Local Coverage Policy," Health Affairs 22, no. 4 (2003): 137–146.[Abstract/Free Full Text]
4. "Notice: Medicare Program; Revised Process for Making Medicare National Coverage Determinations," Federal Register 68, no. 187 (26 September 2003); and GAO, Medicare: Divided Authority.
5. "Medicare Program: Criteria and Procedures for Making Medical Services Coverage Decisions That Relate to Health Care Technology," Federal Register 54, no. 30 (30 January 1989): 4302–4318; and "Medicare Program: Procedures for Medical Services Coverage Decisions; Request for Comments," Federal Register 52, no. 82 (29 April 1987): 15560–15563.
6. R. Pear, "Medicare to Weigh Cost as a Factor in Reimbursement," New York Times, 21 April 1991; and "Medicare Program: Procedures for Making National Coverage Decisions," Federal Register 64, no. 80 (27 April 1999): 22619–22625.[Medline]
7. "Medicare Program: Establishment of the Medicare Coverage Advisory Committee and Request for Nominations for Members," Federal Register 63, no. 239 (14 December 1998): 68780.[Medline]
8. "Medicare Program: Procedures" (1999).
9. "Notice: Medicare Program" (2003); "Medicare Coverage Policy—Coverage Process: Criteria for Making Coverage Decisions," Federal Register 65, no. 95 (16 May 2000): 31124–31129; GAO, Medicare: Divided Authority; and Foote, "Focus on Locus."
10. Centers for Medicare and Medicaid Services, Medicare Coverage Database, cms.hhs.gov/mcd/index_list.asp?list_type=ncd (5 October 2004).
11. Agency for Healthcare Research and Quality, "Task Force Ratings," www.ahrq.gov/clinic/3rduspstf/ratings.htm (5 October 2004).
12. R.P. Harris et al., "Current Methods of the U.S. Preventive Services Task Force: A Review of the Process," American Journal of Preventive Medicine 20, no. 3 Supp. (2001): 21–35.
13. Ibid.
14. The full list of technologies is available from the CMS Web site, www.cms.hhs.gov/mcd/index_list.asp?list_type=nca, and from the authors; send e-mail to Peter Neumann, pneumann{at}hsph.harvard.edu.
15. A.M. Garber, "Evidence-based Coverage Policy," Health Affairs 20, no. 5 (2001): 62–82.[Abstract/Free Full Text]
16. GAO, Medicare: Divided Authority; and U.S. Department of Health and Human Services, National Coverage Determinations, Report submitted to Congress in response to requirements of Section 522(a) of the Medicare, Medicaid, and SCHIP Benefits Improvement and Protection Act (BIPA) of 2000, P.L. 106-554 (Washington: DHHS, 4 June 2002).
17. CMS, Report to Congress on National Coverage Determinations for Fiscal Year 2002 (Baltimore: CMS, 2003).
18. Originally MCAC required panels to meet first, and then required a second meeting of an executive committee. The executive committee was eliminated in 2002, and review times may shorten in the future accordingly. See CMS, "Medicare Coverage—MCAC," 24 November 2000, www.cms.hhs.gov/mcac/8b1-b1.asp (23 November 2004); and CMS, "Revised Medicare Charter," 22 November 2002, cms.hhs.gov/providerupdate/regs/cms3104n.pdf (22 November 2004).
19. M. O’Riordan, "Reaction to Medicare ICD Decision: Experts Dismayed Full MADIT II Criteria Not Included in Coverage," Heartwire (13 June 2003).
20. S.R. Tunis, "Economic Analysis in Healthcare Decisions," American Journal of Managed Care 10, no. 5 (2004): 301–304.[ISI][Medline]
21. R. Rettig, "Medical Innovation Duels Cost Containment," Health Affairs 13, no. 2 (1994): 8–27; M. Kim, R. Blendon, and J. Benson, "How Interested Are Americans in New Medical Technologies? A Multicountry Comparison," Health Affairs 20, no. 5 (2001): 194–201[Free Full Text]; and R.D. Arnold, The Logic of Congressional Action (New Haven, Conn.: Yale University Press, 1990).
22. P.S. Keenan, "Role of Clinical Evidence and Other Factors in Medicare National Decisions to Cover New Treatments" (Unpublished doctoral thesis, Harvard University, 2004).
23. S.R. Tunis, D.B. Stryer, and C.M. Clancy, "Practical Clinical Trials: Increasing the Value of Clinical Research for Decision Making in Clinical and Health Policy," Journal of the American Medical Association 290, no. 12 (2003): 1624–1632[Abstract/Free Full Text]; L. Kessler et al., "Clinical Use of Medical Devices in the ‘Bermuda Triangle,’ " Health Affairs 23, no. 1 (2004): 200–207[Abstract/Free Full Text]; and Garber, "Evidence-based Coverage Policy."
24. GAO, Medicare: Divided Authority; Foote, "Why Medicare Cannot Promulgate a National Coverage Rule"; and M.R. Gillick. "Medicare Coverage for Technology Innovations—Time for New Criteria?" New England Journal of Medicine 350, no. 21 (2004): 2199–2203.[Free Full Text]
25. See, for example, E.P. Steinberg and B.R. Luce. "Evidence based? Caveat Emptor!" Health Affairs 24, no. 1 (2005): 80–92[Abstract/Free Full Text]; S.M. Teutsch, M.L. Berger, and M.C. Weinstein, "Comparative Effectiveness: Asking the Right Questions, Choosing the Right Method," Health Affairs 24, no. 1 (2005): 128–132[Abstract/Free Full Text]; and K. Claxton, J.T. Cohen, and P.J. Neumann, "When Is Evidence Sufficient?" Health Affairs 24, no. 1 (2005): 93–101.[Abstract/Free Full Text]
26. Claxton et al., "When Is Evidence Sufficient?"
27. P.J. Neumann, "Why Don’t Americans Use Cost-Effectiveness Analysis?" American Journal of Managed Care 10, no. 5 (2004): 308–312[ISI][Medline]; and Tunis, "Economic Analysis in Healthcare Decisions."
28. P.J. Neumann. "Evidence-based and Value-based Formulary Guidelines," Health Affairs 23, no. 1 (2004): 124–134.[Abstract/Free Full Text]
29. Neumann, "Why Don’t Americans Use Cost-Effectiveness Analysis?"

                      What's this?