|
Evidence, Economics, And Emphysema: Medicare’s Long Journey With Lung Volume Reduction Surgery
Use of lung volume reduction surgery (LVRS) increased in the mid-1990s as a palliative therapy for severe emphysema. Rapid growth in procedure volume despite little evidence supporting its safety and effectiveness prompted the Centers for Medicare and Medicaid Services (CMS) to suspend payments and cosponsor a nationwide randomized controlled trial to evaluate the procedure. In this paper we describe the trial and its influence on the CMS’s recent coverage decision for LVRS. We describe the implications of this study for evidence-based evaluation of surgical procedures and Medicare’s potential role in evaluating experimental treatments that affect its beneficiaries.
The national emphysema treatment trial (NETT) is a story of the art, science, and politics of designing and conducting a large, multicenter clinical trial to evaluate a controversial and costly surgical procedure for the purpose of making an evidence-based national coverage decision. The NETT involved an unusual collaboration of three federal agencies: the Centers for Medicare and Medicaid Services (CMS, then HCFA); the National Heart, Lung, and Blood Institute (NHLBI); and the Agency for Healthcare Research and Quality (AHRQ). It was designed to assess the clinical and economic impact of lung volume reduction surgery (LVRS) compared with standard medical therapy for patients with severe emphysema. In addition, the NETT also had a quasi-regulatory function. Just before the start of the NETT, the CMS announced that it would cover the procedure only for patients who agreed to participate in the trial. The rationale was that the results of the trial would be used to inform the CMS’s national coverage decision for the procedure. In addition, the CMS policy is likely to influence private health plans’ coverage and reimbursement decisions. The NETT was conducted and analyzed in accordance with standard practice for clinical trials. Despite, or perhaps because of, the rigorous design, the main outcome of the study was mixed, and many felt that the results did not show convincing proof of meaningful benefit. The economic analysis also suggested that the procedure had relatively poor cost-effectiveness in the short run. Nevertheless, a few months after the release of the trial results, Medicare agreed to cover LVRS, and designated surgery centers have begun marketing the procedure to the public.1 In this paper we describe the history of LVRS and the NETT, and how this history and the results of the trial influenced the CMS’s coverage decision for LVRS. We place this decision in the context of the new CMS initiative in evidence-based medicine and other recent coverage decisions for costly procedures. We conclude by describing the implications of the NETT study for future studies of expensive surgical procedures that will primarily affect Medicare beneficiaries.
The history of LVRS prior to the NETT is an excellent example of a convergence of clinical, economic, and legal factors that foster rapid diffusion of a technology in advance of evidence of its efficacy and cost-effectiveness. For tens of thousands of patients with severe emphysema, quality of life is often poorer than for patients with cancer, coronary artery disease, and AIDS.2 Family members often must care for emphysema patients’ basic needs, including cooking, cleaning, bathing, and dressing. Standard medical treatment offers only modest symptom relief, and no intervention except smoking cessation early in the course of the condition has been shown to substantively alter the course of disease. As with other serious medical conditions, this combination of poor quality of life coupled with lack of effective therapies creates an intense demand among patients and their families for new technologies that promise symptom relief, improved survival, or both, regardless of the cost. Originally developed in the 1950s, LVRS is a surgical approach to reducing the volume of the hyperinflated lung parenchyma that is characteristic of severe emphysema. Although 75 percent of patients reported clinical improvement in an early case series, the lack of objective documentation of benefit and an operative mortality of 18 percent prevented widespread acceptance of the procedure.3 In 1995 Joel Cooper and colleagues reported a modification of the original volume reduction operation in which diseased lung tissue was removed from both lungs via a surgical procedure called a median sternotomy. After favorable results from a small, uncontrolled study of patients with severe emphysema who underwent LVRS were reported in 1994, it was not surprising that the pressures to adopt this technology were strong.4 Patient advocates posted testimonials on a thoracic surgery Web site, and favorable reports appeared in the popular press.5 When LVRS was gaining popularity, it had no specific billing code. Physicians and hospitals thus obtained reimbursement by submitting preexisting but non-specific billing codes for multiple resections of lung tissue. LVRS is neither technically difficult nor costly, nor does it require the purchase of costly capital equipment. Because Medicare’s reimbursement for the nonspecific lung procedures was fairly generous, there were few financial disincentives to performing the operation. Given a prevalent, severely disabling disease; a technically simple, low-cost, and potentially effective procedure; and a relatively generous reimbursement climate, it is perhaps not surprising that it took little time for several institutions throughout the country to develop LVRS programs. In less than eighteen months from the initial published reports, more than 1,200 LVRS procedures had been performed on Medicare beneficiaries nationwide.6 These procedures were also likely performed on patients in private health plans during the same time period. Regulatory response: the National Emphysema Treatment Trial. After identifying sharp increases in heretofore little used claims for lung resection in 1994–95, the CMS asked AHRQ to conduct an evidence review of LVRS. In addition, the CMS learned from an internal claims review that mortality at six months for patients undergoing LVRS approached 17 percent.7 Based on this finding and the AHRQ review, which found no compelling evidence of benefit, the CMS announced in December 1995 that Medicare would not pay for LVRS, on the grounds that insufficient safety and efficacy evidence was available to support reimbursement.8 To what degree did economic considerations figure into the CMS’s decision to suspend reimbursement for LVRS? Certainly, agency officials were concerned that hundreds of millions of dollars would be spent on a procedure whose clinical value was unknown and that might be ineffective or inferior to medical therapy.9 On the other hand, the political cost of denying patients access to a potentially beneficial procedure was likely to be great. With many uncertainties in play, regulatory officials made an unprecedented decision: When the CMS announced that it would suspend payment for LVRS, it also announced that it would collaborate with the National Institutes of Health (NIH), which would sponsor a multicenter randomized controlled trial (RCT) to evaluate the efficacy of LVRS in conjunction with the NHLBI. Furthermore, the CMS agreed to pay for the costs of the procedure, an exception to its usual policy of paying only for routine medical care that is part of clinical trials.10 Apparently, the CMS did not require additional regulatory or statutory authority to provide coverage for the procedure during the trial. Science meets politics in a clinical trial. After a competitive, peer-reviewed bidding process sponsored by the NHLBI, eighteen centers around the United States were ultimately invited to participate, including Washington University in St. Louis, home of the rejuvenator of the procedure, Joel Cooper. Given the obvious economic implications of LVRS, several centers proposed substudies to conduct cost-effectiveness analyses alongside the trial. AHRQ responded by providing funding for such an analysis. The CMS supported the cost-effectiveness study by providing claims and reimbursement records for trial participants. The political ramifications of the CMS decision became apparent even before the trial had enrolled its first patient. In the fall of 1996 Congress passed a law requiring the CMS to provide a report that included an analysis of all available data for LVRS and recommendations as to the appropriateness and conditions of Medicare coverage of the procedure. On 17 April 1997, at hearings conducted in part to provide a forum for those concerned with Medicare’s decision to cover LVRS only under the NETT, Rep. Jim Ramstad (R-MN) expressed "concern that the decision not to extend Medicare coverage for the procedure [LVRS] could negatively affect the lives of thousands of older Americans who suffer from this disease."11 At the same hearing, Cooper also criticized the CMS’s decision to deny payment for LVRS outside of the trial. Just prior to the start of the trial, Cooper announced that Washington University would withdraw from the multicenter study, on the grounds that its institutional review board (IRB) felt that a controlled study was unethical, given what they saw as obvious benefits of the procedure.12 Cooper continued to lobby vigorously for the procedure (and against the trial) during the course of the study, likening the controlled trial to "hiding behind a scientific study to ration the procedure."13 Nevertheless, despite the political difficulties, in October 1997 seventeen centers began enrolling patients.
The Statistical Coordinating Center at the Johns Hopkins University and participating centers were responsible for ensuring the quality of the trial design and conduct of the study.14 The problem of crossover—patients randomized to the medical care arm who later obtain surgery outside of the trial—was a major concern at the start of the trial because of the level of publicity surrounding the procedure. Crossover ultimately proved inconsequential. Loss of patients at follow-up, another problem that can plague large clinical trials, was minimal: At the time of publication of the main outcome paper, 99 percent of all surviving patients continued to complete the telephone or clinic visits. For all of its success, the NETT was not without challenges. Patient recruitment proved difficult. The original enrollment goal of 4,500 patients was later reduced to 2,500 when it became clear that recruitment was going to be far below targets.15 When interim analysis identified a subset of patients with excess mortality and no apparent benefit from surgery, many cited this outcome as evidence that the procedure was unsafe, further adding controversy to the trial and reducing the number who presented for screening.16 Ultimately, 1,218 patients were randomized. Interestingly, slow enrollment halted a planned clinical trial in Canada to be conducted roughly within the same time frame as the NETT. The Canadian researchers also planned a cost-effectiveness analysis that would ostensibly inform coverage policy. In May 2003 the NETT research group published its main findings, which showed no improvement in survival for surgical patients but differential improvements in exercise capacity and respiratory-specific quality of life for a percentage of the surgical patients.17 A secondary analysis suggested differential relative benefits for patient subgroups defined by combinations of regional predominance in emphysema distribution (by computed tomography [CT] scan); and (2) exercise capacity following pulmonary rehabilitation. One subgroup showed statistically significant improvements in survival for the surgical cohort. Surgical patients in three of the four subgroups had improvements in exercise capacity and quality of life compared with medical therapy patients. The cost-effectiveness of LVRS compared with medical therapy during the three years following initiation of treatment was judged to be unfavorable, at $189,000 per quality-adjusted life year (QALY) gained, compared with the often-cited conventional level of $50,000 per QALY.18 In the subgroups, the cost-effectiveness analysis was more favorable for patients with upper-lobe emphysema and low exercise capacity but less favorable in the two remaining subgroups. Both the clinical and cost-effectiveness results were published simultaneously in the New England Journal of Medicine, an unprecedented occurrence that underscored the interest in both the clinical and economic outcomes for the NETT. Given the relatively modest clinical benefits for the main analysis, uncertain significance of the subgroup analysis, and poor short-term cost-effectiveness, the accompanying editorial was relatively circumspect in its assessment of the role of the procedure in clinical practice.19 The CMS faced a difficult decision about coverage, heightened by uncertainty about the rate at which LVRS would be adopted after the trial.
At the same time that the trial results were published, the CMS released a national coverage memorandum for LVRS for comment. LVRS would be covered for patients with upper- or lower-lobe predominant emphysema with low exercise capacity, or upper-lobe emphysema with high exercise capacity. These groups constituted three of the four subgroups from the trial for whom benefits were identified. The CMS originally specified that LVRS is reasonable and necessary only when performed at facilities that were identified by the NHLBI as meeting the thresholds for participation in the NETT, although it noted that it was particularly interested in comments on the "potential for restricting LVRS to certain facilities that meet predefined criteria." The final rule was amended to include sites that have been approved by Medicare as lung transplant facilities.20 The CMS implemented its final coverage policy 5 January 2004. Concerns about the CMS’s policy. Several analysts have noted that the CMS’s relatively generous coverage decision for LVRS seems disproportionate to both the weight of the evidence and incremental benefit for LVRS patients shown in the trial.21 Findings from subgroup analyses of a single clinical trial are considered relatively weak evidence, particularly in situations where the trial fails to show a hypothesized benefit for the experimental group. Certainly, the U.S. Food and Drug Administration (FDA) would not approve drugs in such instances; rather, the agency would request that the manufacturer conduct a separate trial that considered only the subgroup(s) in question. Because there was no prior hypothesis of or biological rationale for differential benefit based on distribution of emphysema and presurgical exercise capacity, the post hoc subgroup analysis is open to criticisms that the investigators undertook a "data dredging" exercise to identify groups with the findings they sought: improvement in survival, exercise capacity, and quality of life. As a practical concern, the CMS’s coverage policy excludes only patients with high exercise capacity prior to surgery. Given that exercise ergometry is technically difficult to standardize and relatively easy to manipulate (patients wanting LVRS could simply not exercise as hard), it may be difficult to limit LVRS based on exercise capacity. In effect, the CMS may have designed a coverage policy that includes all patients meeting the original NETT eligibility criteria. Limiting LVRS reimbursement to NETT centers and Medicare-designated lung transplant facilities could also be criticized as being inconsistent with evidence and the CMS’s own rationale for limiting high-risk procedures to designated centers of experience. Studies have shown that patients undergoing high-risk surgical procedures at institutions and by surgeons with the most experience have better outcomes than patients at lower-volume facilities.22 The NETT surgical centers were probably, with one exception, the only centers in the United States that had substantive experience with the procedure.23 When the CMS suspended coverage for LVRS, the number of procedures performed outside of the NETT fell to a trickle, as most private insurers suspended coverage while awaiting results from the trial.24 Expanding coverage to include Medicare-designated lung transplant centers (that were not also NETT facilities) thus is potentially problematic, since few surgeons at those centers had recent experience with the procedure. What shaped the CMS’s coverage policy? The issues noted above might lead one to conclude that in suspending its evidence policy for LVRS, the CMS was bowing to political, advocacy, professional, and other pressures. Although there is substance to this line of reasoning, it is also important to consider pragmatic issues and the state of the evidence. First, patients in the surgical arm of the NETT did demonstrate statistically significant improvements in pulmonary functioning, exercise capacity, and respiratory-specific quality of life compared with those in the medical therapy arm. Although one might argue that the clinical significance was modest, such distinctions would not necessarily be valid points of contention unless the sponsors had decided a minimal clinical difference beforehand. Interestingly, neither the CMS nor the trial sponsors defined clinical significance thresholds for functional or quality-of-life endpoints at the onset of the trial. Second, one of the early stated goals of the NETT was to refine the eligibility criteria for the procedure—that is, to determine whether it was possible to identify subpopulations with differential benefit. Although those subpopulations did not turn out to be the ones that were initially postulated, it was clear at the onset that there was much to learn about this procedure. Third, the NETT is certainly going to be the only large RCT of LVRS with long-term endpoints. The trial cost more than $125 million, including procedural, patient care, and research costs.25 Given such expense, and the well-recognized recruitment difficulties, there is little interest in mounting a second controlled study to validate the findings. It is difficult to expect patients, clinicians, or health insurers to wait for more evidence on subgroups, since no trial of the magnitude and rigor of the NETT will be conducted hence. There are also arguments in favor of expanding coverage to more surgical centers. During the trial, it became clear that many patients and their families had great difficulty traveling many miles to receive care. The evaluation process to determine eligibility often took days, and for those enrolled, six to ten weeks of pulmonary rehabilitation were required prior to surgery. The seventeen NETT centers leave many geographic "holes" without LVRS centers for hundreds of miles. Expanding coverage to the forty-four lung transplant facilities in the United States will greatly improve access both to the evaluation and rehabilitation process and to the procedure itself. There are short-term concerns about experience, but these are mitigated by the fact that lung transplant centers are already skilled at caring for patients with severe respiratory disease. Did political considerations ultimately influence the CMS’s coverage policy? If the CMS deemed it appropriate to base coverage on subgroup analyses, it could have restricted coverage to those with upper-lobe disease and low exercise capacity, since this subgroup showed both a mortality gain and a much greater benefit of exercise and quality of life compared with the other subgroups. Because less than 25 percent of all patients in the trial met these criteria, such a restriction would have greatly reduced potential procedure volume nationwide. Second, expanding coverage to include lung transplant centers allowed the CMS to permit LVRS at the home of its most vocal proponent, Joel Cooper. Cooper originally proposed expanding coverage to CMS-designated lung transplant centers (Washington University in St. Louis is a designated center). Did cost-effectiveness play a role in the decision? Many experts agreed beforehand that the cost-effectiveness of LVRS should be considered relative to outcomes and costs for other medical and surgical procedures.26 Although the CMS did support the cost-effectiveness study that was conducted alongside the trial, there is little evidence that the results of the cost-effectiveness analysis played an important role in its coverage decision. Calculated over the observation period for the trial, the cost-effectiveness of LVRS was unfavorable compared with that of other common and reimbursed thoracic surgical procedures. The projected long-term cost-effectiveness (derived through a statistical modeling approach) is somewhat more favorable but also quite uncertain, largely because of clinical uncertainty about the duration of any survival or quality-of-life benefit. If these considerations were taken into account, use of LVRS would likely be greatly restricted. CMS officials have stated that costs do not play a role in their coverage decisions.27 Although there is no specific regulatory language prohibiting the CMS from considering economic evidence in its decisions, past attempts to incorporate economics in the regulatory process have failed, largely because of political concerns.28 Indeed, as has been noted, adopting economic criteria implies that the CMS will deny a clinical benefit because of cost, a task that is much more difficult politically than even the current sensitive task of weighing evidence of clinical risks and benefits.29 An important precedent of the NETT was the CMS’s willingness to cosponsor the cost-effectiveness component of study. This may reflect the agency’s interest in developing a process of conducting economic analyses alongside clinical trials that have great importance to their covered population.
Surgical procedures are essentially unregulated in the United States; as a result, many new procedures enter clinical practice with little evidence that allows meaningful assessment of their clinical benefits and risks. LVRS demonstrated that when a new surgical procedure is designed to treat a highly prevalent, severely disabling condition with few therapeutic alternatives, patient and physician demand can "drive" utilization well ahead of evidence of benefit, and serious risks become known only after many patients have been treated. Because of these issues, the CMS’s decision to suspend payment and sponsor a clinical trial of LVRS is a critical policy development. Will the NETT experience become a paradigm for future surgical trials?30 We think that this will depend on a number of lessons that can be drawn from the trial. Agency collaboration. First, collaboration between the CMS and other federal agencies that evaluate new medical technologies is important. A regulatory "Bermuda Triangle" often exists between the CMS, NIH, and FDA, where important devices and procedures are implicitly or explicitly covered, then enter widespread use prior to the availability of high-quality evidence supporting their clinical effectiveness and safety.31 The FDA and NIH are in the best position to alert the CMS when important technological innovations are moving from research settings to clinical use, particularly technologies that are licensed and regulated by the FDA or invented with NIH funding. Paradoxically, it was the CMS’s own internal audits that found sizable mortality concerns for patients undergoing LVRS. The three agencies would benefit from periodic, coordinated reviews of the medical technology landscape so that proactive evaluations can take place for important emerging procedures. Part of the review procedure may be developing criteria that would trigger a recommendation for conducting jointly sponsored, large-scale RCTs. Consensus about acceptable quality of evidence. Second, in cases where it appears that a clinical trial is desirable, the CMS should work early on with external investigators and the sponsoring federal research agency (such as the NHLBI) to develop consensus about the need for high-quality evidence. This is important both for gaining acceptance from the clinical community and for elucidating all relevant issues pertaining to the procedure. Prior to the CMS’s decision to suspend payment for LVRS, it cosponsored a workshop with the NHLBI where experts from many disciplines agreed that the state of knowledge regarding LVRS was such that a multicenter RCT was needed. External peer review. Third, high-quality external peer review is necessary to ensure the integrity of the process of selecting centers to participate in a trial. As the NETT also demonstrated, this does not ensure that those who are successful applicants will be neutral in their appraisal of the state of the technology. Indeed, those with the most experience in experimental procedures (and thus most qualified to participate in a trial) are often the most vocal proponents. As occurred with Cooper and the NETT, such proponents often state that the evidence is such that controlled trials are unethical. It is important to counter such arguments forcefully during the design phase of the study, as they are part of a longstanding double standard that exists for evaluating surgical (versus medical and pharmaceutical) treatments. Publication of a consensus statement from the aforementioned NHLBI/CMS workshop on the need for a randomized trial would have been helpful to counter criticism from those who felt that a trial was unwarranted, and would have served as a reference for IRBs that might otherwise be influenced by advocates for adopting new technologies prior to thorough evaluation. The arguments against controlled surgical trials have been refuted on both ethical and scientific grounds.32 A few of the many threats to study design that occur in discussions of surgical trials include differential eligibility criteria for cases and controls, use of inappropriate or sham treatments as controls, and allowing patients who are randomized to the nonexperimental arm to cross over to the treatment arm. Indeed, such design problems were apparent in a privately funded study that was initiated to evaluate LVRS (the study failed to achieve recruitment goals and was cancelled).33 In the case of LVRS, an argument could be made that blinding (that is, a sham surgery arm) would have improved the rigor of the study by removing the placebo effect of surgery. Such sham-controlled trials have often yielded surprising results and called into question the therapeutic value of the procedure.34 Because the NETT was not a blinded study, it is open to criticism that quality-of-life differences between treatment and control can be accounted for in part by the placebo effect. Predefined "significant clinical benefit." We also recommend that clinical and regulatory officials work together to prespecify levels of clinical benefit that would be considered clinically as well as statistically significant. Trials would then be powered to test hypotheses against these levels of difference. Applying a definition of clinical significance to the treatment and patient population in question would be a valid interpretation of Medicare’s regulatory language specifying that "reasonable and necessary" services should be covered.35 When multiple clinical endpoints are being considered, the clinical study should be designed to have enough patients to test the primary and secondary hypotheses as well as any planned subgroup analyses. In the case of the NETT, gathering opinions about a clinically meaningful difference in exercise ergometry beforehand would have reduced problems with interpreting the findings later. Of note, economic evaluation could also be used to inform the level of benefit that is considered reasonable value for money. For example, economic modeling could have been used to translate exercise thresholds and projected changes in life-expectancy gains into QALYs, the most widely used metric of benefit in cost-effectiveness analysis. Expected intervention costs could be assessed relative to benefits compared to standard therapy. Such a process would avoid cost-effectiveness analysis thresholds that are anathema to many. Costs of experimental treatment. Another issue is whether the CMS should pay the costs of the experimental treatment. There are several reasons why the CMS should consider making such payment a rule rather than an exception for studies of surgical procedures. Unlike drugs and devices, surgical procedures typically do not have sponsors that can cover the costs of treatment, as drug and device manufacturers do. By paying for procedures in trial settings, the CMS lowers an important financial barrier to participation. Patient recruitment can be increased, and surgeons do not have to bear the expense (both real and in terms of opportunity costs) of donating their time during the evaluation phase. Larger trials with power to test multiple endpoints would become more feasible. In addition, surgeons who have less conflict of interest could be recruited to participate. Of note, Cooper is a major stakeholder in the company that manufactured the strips used to seal the lungs of LVRS patients. Such obvious financial conflicts can run afoul of rules regarding federally sponsored clinical trials. Long-term follow-up. Clinical trials are by design of limited duration, but trials conducted in collaboration with the CMS have the advantage of permitting long-term follow-up through claims and Medicare vital status files. The opportunity for post-trial evaluation using observational epidemiology methods and claims data analysis should be strongly considered in cases where the treatment is expected to influence patient outcomes long after the study is completed or there is a need to monitor use and outcomes for patients after the coverage decision has been made. Such evaluations would require that the CMS adopt an intra- or extramural research program or work with AHRQ or the NIH to undertake evaluations of coverage decisions.
Scott Ramsey (sramsey{at}fhcrc.org) is an associate member of the Fred Hutchinson Cancer Research Center in Seattle, Washington. He is also an associate professor in the Division of General Internal Medicine and (as an adjunct) in the Pharmaceutical Outcomes, Research, and Policy Program, Department of Pharmacy, at the University of Washington, also in Seattle. Sean Sullivan is a professor of pharmacy and health services and director of that program. The opinions expressed in this paper are those of the authors and do not necessarily reflect those of the NETT investigators or sponsors.
This article has been cited by other articles:
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Top
History Of Lung Volume...
Trials And Tribulations
