|
Evidence, Politics, And Technological Change
In few fields of public policy are the use and cost of services so powerfully driven by technological change as they are in medicine. To manage technology, policy-makers have expanded their investment in evaluative research. This paper addresses three underexamined challenges in using evidence: those inherent in the dynamics of technological change itself; those inherent in the analytical enterprise; and the ways in which political factors shape the translation of evidence into policy decisions. The design of institutional arrangements and processes that seek to blend evidence with politics merit closer attention, and existing cross-national arrangements deserve careful study.
Policymakers in industrialized countries feel an intensifying imperative to manage the health and economic impact of new and emerging technologies, and many understandably find this duty conceptually frustrating and elusive. Decisions on how to handle the explosive field of medical innovation, moreover, translate directly into regulatory and budgetary decisions—who gets what medical care, on what terms—that carry arcane analytical issues straight onto the radar screens of public opinion and myriad organized groups. Managing innovation in medical technology is a formidable task. In few fields of public policy are the use and cost of services so powerfully driven by rapid innovation as they are in medicine, and new drugs, devices, and procedures continually emerge. But invention is one thing, adoption another: The fruits of medical progress do not appear in definitive form on the physician’s or policymaker’s doorstep. Adoption decisions face uncertainty about the extent of risks and benefits associated with an emerging technology, the size of the population that may benefit, and new uses that may be discovered later. These innovations also raise economic questions. They make it difficult to balance health (and therefore social) budgets and trigger debate about opportunity costs of the marginal health care dollar.
These challenges have spurred the design of analytic techniques that use rigorous evidence to tell practitioners, payers, and policymakers which technologies work well and which should be modified or abandoned. Examples of their use include the U.K. National Institute for Clinical Excellence (NICE), France’s Agence National d’Accréditation et d’Evaluation en Santé (ANAES), and new in-house analytic capacities in the U.S. Centers for Medicare and Medicaid Services (CMS). This portfolio of evidence-based techniques—randomized controlled trials (RCTs), cost-effectiveness studies, and more—gains legitimacy from the same claims to "hard" scientific validity that biomedical research itself asserts. However, as does medical science itself, the interpretation and application of analytic findings vary greatly among and within countries. Bypass surgery and percutaneous transluminal coronary angioplasty (PTCA), for example, are common cardiac revascularization procedures that have been extensively evaluated. Yet international rates of its use are stunningly different (Exhibits 1
Clearly, empirical analyses do not provide off-the-shelf policy decisions. Value judgments, different institutional contexts, and stakeholders—what we call politics, as shorthand—shape how policymakers respond to analytical recommendations. The stakeholders—scientists, physicians, patients, policymakers, purchasers, and insurance institutions—all seek evidence to guide adoption decisions. But all bring distinct readings of the evidence to decisions that may have heart-rending implications for quality, cost, and fairness. The ensuing dilemmas must be managed politically—which is too often taken to mean nonrationally. Rational technology policy must cope with conflicts over values and interests, and the coping can hardly be other than political. Politics and analysis need not exclude each other, however; the inescapably political process of managing technology may gain legitimacy and defuse frustration if it is informed by analysis of the benefits that health spending buys. This paper addresses three underexamined challenges in this respect: those inherent in the dynamics of technological change itself; those inherent in the analytical enterprise; and those inherently political factors that shape the translation of evidence into policy decisions.
Managing technological change challenges policymakers in many ways. First, large investments by industrialized countries in medical research and development (R&D), combined with the growing demand for effective technologies to manage aging, and often chronically ill, populations, have accelerated innovation. In 2002 alone, for example, the U.S. Food and Drug Administration (FDA) approved some 89 new drugs and biological agents, 172 new indications of use, and more than 4,000 new or improved devices.2 Moreover, many advances in clinical procedures, such as nerve-sparing prostatectomy or islet cell transplantation, are not related to new products and not reflected in the activities of the FDA. Second, the medical profession modifies and expands the application of technologies in clinical practice. Incremental technical improvements in performance tend to expand the universe of patients within a given disease category who benefit. Only 4 percent of coronary disease patients treated with coronary artery bypass graft (CABG) surgery today would have met the eligibility criteria of the trials that established its initial value.3 Improvements in surgical technique expanded the use of CABG surgery to patients with acute myocardial infarction, acute cardiogenic shock, women, the elderly, and patients with multiple comorbidities—a pattern that holds for many other technologies. Minimally invasive laparoscopic surgery approaches reduced postoperative pain and recovery time, as well as treatment cost per patient. The target population for these procedures, therefore, has expanded, often to less sick patients (for whom the risks of the procedure are now acceptable) and to sicker patients, who initially were too risky to be candidates. The ability to expand target populations suggests that the elasticity of demand for medical services is greater than commonly appreciated.4 Because technological change often reduces cost per patient and improves quality, thereby expanding demand, improvements in efficiency do not necessarily yield global cost savings—a dilemma that understandably troubles policymakers. Unexpected innovations may also emerge from seemingly routine practices. For example, alpha-adrenergic receptor blocking agents first introduced for hypertension, were found some twenty years later to help reduce symptoms of benign prostatic hyperplasia.5 Widespread use is often a precondition for identification of new therapeutic indications of use. A study of the top twenty blockbuster drugs from 1993 found that secondary indications exceeded 40 percent of revenues; a similar pattern held for medical devices.6 Such serendipity is at once an indispensable path to better quality and a source of disturbance to analytical calculi and policymakers’ expectations. In short, the policy implications of analytical studies that address the state of a technology at one point in time may be challenged by highly dynamic patterns of evolution and adaptation. Policymakers may, therefore, watch evidence-based answers morph before their eyes into fresh questions.
Managing technology is not one task but many, not static but dynamic. Better information on the effectiveness and cost-effectiveness of medical technologies can guide policy making, and industrialized countries have expanded their investment in clinical evaluative research. Yet challenges inherent in the analytical enterprise remain. Rigorous evidence, from RCTs and other well-controlled clinical studies, can inform policy decisions about technologies’ efficacy and safety. The chief advantage of the RCT over other methods is that it reduces bias; because patients and physicians do not select the treatment, estimates of effectiveness cannot be corrupted by subtle differences between patients in the treatment arms with respect to the outcomes of interest. Sponsors, patients, and physicians want to establish the benefits of new technologies and introduce them rapidly into general use following regulatory approval. RCTs, therefore, are typically conducted in specialized centers with well-defined populations to facilitate the efficient testing of hypotheses. They have limited time frames and seldom measure long-term effectiveness or safety. They also limit patient heterogeneity and therefore may not be generalizable to all potential recipients. The specialized skill of participating centers raises further questions about generalizability. Regulatory decisions, therefore, rest on limited information about outcomes of therapy in general use; reducing uncertainty requires postmarketing studies. Payment policies may focus on both clinical and economic endpoints. In the past, cost-effectiveness analyses were often done after clinical evidence had been collected. The recent trend is to conduct these analyses prospectively within RCTs. Typically, economic endpoints are of secondary consideration in these trials—that is, trial sample size (based on the clinical outcome) may be inadequate to show a definitive statistical difference in costs or cost-effectiveness. Statisticians and health economists continually debate about preferred methodologies, about whether cost-effectiveness or net health benefit is the appropriate metric, and about how to construct coherent data summaries that will allow policy-makers to evaluate research data with due regard for statistical uncertainty.7 International standards for measuring resource use and accurately determining costs would ease the transfer of economic studies from one setting to another. But, even so, the usefulness of cost-effectiveness analysis, which requires defining marginal costs and benefits, will depend on the appropriate selection of the therapeutic alternatives to which a new technology is being compared.8 Because technologies evolve in clinical practice, evaluations should be revisited after technologies are introduced. Observational studies and "pragmatic" or "practical" RCTs in high-cost, high-prevalence conditions can contribute by selecting clinically relevant interventions to compare; by including diverse populations of patients from a variety of practice settings; and by collecting data on a broad range of health outcomes.9 The movement toward "large, simple" trials, which seek a broader representation of patients and practitioners, tries to make clinical research more efficient and economical while holding bias at bay. Yet these trials tend to be large and expensive, and scientists who seek to answer a clinical question must decide whether freedom from bias is worth the extra cost. Therefore, observational, decision-analytical, and cost-effectiveness studies that are quasi-experimental or collect no new data but draw estimates of prevalence and effect size from the literature—continue to be popular. Trials of new clinical procedures, pragmatic trials, and observational studies are mostly supported by the public sector, especially if patents no longer protect the technology in question. The public sector, however, invests less in this type of research than the private sector, which commits around 30 percent of its large and growing R&D budgets to such studies. If these inadequate funding levels continue, considerable uncertainty will remain about the value of many evolving technologies, which will constrain policy and clinical decisions.
If the growing body of analysis and evidence is to help policymakers hit the moving target of technological change, they must be inserted into existing "steering" tools. On the supply side, countries in the Organization for Economic Cooperation and Development (OECD) use public planning and regulation to provide, distribute, or limit the spread of technologies, facilities, and professionals. On the demand side, Beveridge-type systems (single payers, funded by general revenues) use global and regional budgets, whereas Bismarckian systems (social insurance, funded by employers and workers) use coverage and reimbursement decisions made by insurers (sickness funds) to control the use of technology. Clinical practice guidelines issued by specialty societies, government agencies, or insurers aim to promote appropriate use. Here, too, dynamics—in this case, value considerations—complicate analysts’ and policymakers’ task in translating evidence into policy. Drug or device regulatory decisions. Differences between Europe and the United States in the regulation of drugs and devices are diminishing because of harmonization efforts. Increasingly, premarketing controls tap quantitative data from RCTs, which are essential to determining whether a new technology is safe and efficacious for a particular use.
Such evidence has improved policy decisions by offering quantitative insight into how well technologies work and for whom, but such data do not eliminate vexing trade-offs between the benefits provided (considering the available alternatives) and the acceptability of risks incurred to achieve them. These judgments depend on the interests and values of stakeholders. Reputational effects weigh heavily with agencies such as the FDA that operate under congressional oversight. The FDA has been intermittently criticized both for approving drugs or devices that proved to be unsafe and for approving efficacious therapies too slowly.10 In 1992, political pressure from activist groups helped persuade the FDA to enact a rapid approval process, which allowed approval of AIDS therapeutics with a higher level of uncertainty in the risk-to-benefit ratio, paving the way for anti-retroviral therapies like dideoxyinosine.11 Heart failure is another life-threatening condition for which qualitative concerns shape trade-offs between quality of life and survival. For example, flosequinan, an oral inotropic heart failure drug, was approved in the early 1990s to improve quality and function among recipients, but after its introduction it was found to reduce survival and therefore was withdrawn from the U.S. market. When the risk-benefit trade-off was later posed to patients with heart failure, however, 40 percent of those questioned would accept the higher risk of death ( Planning decisions. Most European countries require hospitals to obtain a public license for expensive high-technology devices and procedures, such as open-heart surgery units and nuclear medicine imaging. Such government mechanisms, which typically have more teeth than the comparable U.S. certificate-of-need (CON) system, may powerfully constrain the speed of diffusion. The empirical evidence required for such decisions includes needs assessments of regional populations and data on whether patient volume is an important determinant of physician and institutional performance. The decision then requires trade-offs between centralization of services and access. Planning agencies must deal with political pressures from academic and other large health care institutions seeking to expand or develop services that increase their prestige or marketability. Such supply-side decision-making processes remain, despite their importance, an underresearched black box. Planning tools can determine the geographic distribution of medical services, particularly "big-ticket" items, but small-ticket items, the major drivers of the health care budget, largely elude this. OECD countries increasingly tend to view supply-side controls as potent but crude, necessary but insufficient for rational allocation of technologies, and complementary methods of managing technology, including payment policies, have gained favor. Payment policies. National health systems use budget caps at the national, regional, or hospital level to force explicit choices among technologies. Regional administrators in the U.K. National Health Service (NHS) may have to decide, for instance, whether to buy a magnetic resonance imaging (MRI) machine for one or two major centers or purchase mammography equipment for, say, seven district hospitals. Budget-driven constraints that dampen the general rate of diffusion of high-cost technologies do not necessarily trigger selection of the most effective or cost-effective ones, however. The rational allocation of resources within fiscal limits presupposes the existence of information on the relative effectiveness and costs of medical interventions, which is often not available.14 Recently, national health systems have strengthened the analytical enterprise; for example, the United Kingdom created NICE, which advises the NHS about the clinical and cost-effectiveness of new, and often costly, interventions. If technologies are found to be cost-effective, the Health Trusts within the NHS are obligated to make resources available to fund them. By contrast, countries with Bismarck-type systems make detailed coverage decisions and perhaps even promulgate basic benefit packages that define where society’s obligations end and individual responsibility begins. These coverage and reimbursement decisions have become better informed regarding effectiveness and cost-effectiveness. In Australia and certain provinces of Canada, for example, sponsors must provide cost-effectiveness data to obtain approval for public reimbursement of a new drug. Decisionmakers also want to gauge the (short-term) impact of covering an intervention on their overall health care budgets, which reflects price and anticipated volume. If payers decide to cover, they can limit diffusion by regulating volume, certification of providers, or reimbursement levels. The trend toward evidence-based decisions about coverage and reimbursement has made systems more rational and transparent, but it too faces challenges.
Cost-effectiveness analyses can encourage the purchase of "good value for money," but the criteria for this are rarely made explicit in any Western system of health care financing. Should these cost-effectiveness ratios be used, as some health economists argue, as strict thresholds? A ratio with $50,000 as the cut-off point would exclude neurosurgery, for example (Exhibit 3  ). Such rules may slight important considerations—for example, whether the technology in question is established or emerging, which determines whether the cost-effectiveness ratio will be static or evolving. For instance, patients with end-stage heart failure, who typically experience one-year mortality rates of 75 percent, were recently found to derive improvements in survival and quality of life from the long-term implantation of left ventricular assist devices (LVADs).15 The survival benefits derived were substantial—nearly a 50 percent reduction in mortality—and the quality of life resulting from their use surpassed that of other therapies. Despite these important gains for individuals, this therapy adds few additional years of life in the aggregate, and device side effects and lengthy hospitalizations made the therapy (when initially tested) quite costly. Blue Cross and Blue Shield projected a cost-effectiveness ratio, based on limited information about the cost of medical therapy, between $500,000 and $1.4 million per quality-adjusted life year (QALY), which exceeds all conventional cut-off criteria.16 Early evidence suggests, however, that the effectiveness of the procedure and associated treatment costs are greatly improving. Using a strict criterion of cost-effectiveness at this point would ignore the prospect for improvement in the technology, dash the hopes of a terminally ill population, and preclude the development of a promising therapy.
Moreover, cost-effectiveness ratios do not adequately address equity and distributive justice. The use of QALYs to measure effectiveness may encourage political divisiveness by juxtaposing the interests of the young and the old, who have inherently less capacity, measured by life expectancy, to benefit from interventions. In the early 1990s, therefore, the Dutch suggested using "social utility" considerations to ensure equity in allocation of funding to the elderly, who on cost-effectiveness ratios alone might be denied needed services.17 That cost-effectiveness ratios cannot constitute exclusive and absolute cut-off criteria is well illustrated by the work of NICE, which has generated heated debate as it confronts the complexity of integrating quantitative evidence with qualitative considerations.18 Such agencies, however extensive the representation of consumers and other stakeholders, are bound to be criticized for lack of transparency, lack of inclusiveness in their processes, and lack of timeliness in making decisions in controversial cases. Controversy over the application of cost-effectiveness ratios may be especially troubling for technologies that benefit life-threatening or seriously debilitating conditions with few therapeutic alternatives. The LVAD is one case in point. Another is the use of beta interferon and glatiramer acetate as treatments for multiple sclerosis. NICE first recommended against using these drugs, based on cost-effectiveness, but then decided to recommend them for a circumscribed population. In these cases, evidence about effectiveness and cost cannot entirely drive policy making: The health system needs flexibility to decide whether clinical conditions and patients’ political preferences justify the use of these technologies. Some "fat" in medical systems must be tolerated because it provides reserves and insulation that are essential to survival.
Cost-effectiveness data are best suited for use in well-focused clinical areas (for example, ischemic heart disease) with alternative treatment options (Exhibit 4
Throughout the twentieth century, technological change generated new forms of medical care that extend human life; reduce pain, risk, and disability; and often prove to be very costly. As health services research has grown, the hope that rigorous clinical and economic evidence could inform such policy tools as supply-side restrictions, global budgets, and coverage and payment policies and depoliticize difficult policy decisions has grown, too. More rigorous evidence has indeed made technology policy decisions better grounded, but the analytical enterprise faces limits that derive from one source: namely, whereas evidence is static (it offers findings for a state of affairs that are fixed in time and space), innovation is dynamic (showing up differently over time and across space). Here we have explored three manifestations of these dynamics. First, technologies "develop" new uses for the benefit of populations (new and old) in the course of their application. Analysis, therefore, often is a chapter behind the class, offering answers to yesterday’s questions. Second, even the most rigorous evidence-based studies get enmeshed in a ceaseless "question and answer" dynamic. Randomized trials may, by their design, exclude some populations to whom a new intervention will almost surely be applied. Reliable cost data are less readily available than measures of health outcomes are. Current methods of resource-based costing leave much to be desired. Recent work on the statistical validity of cost-effectiveness ratios identifies deficiencies in that metric, which awaits methodological refinement. Randomized trials of emerging technologies should be complemented by observational studies and so-called practical randomized trials that explore the relative cost-effectiveness of familiar alternative interventions for common conditions and that diminish uncertainty about the value of the interventions as they evolve. Within limited research budgets, criteria must be set and choices must be made—procedures that will inescapably bear traces, at least at the margin, of patterns of support for innovations among suppliers, organized interests, and the bureaucracy. Third, evidence is supposedly quantitative, value- and interest-neutral, and apolitical. Policy making, however, must wrestle with conflicts of value and therefore has an irreducible political component. Evidence cannot be constructively inserted into policy if one assumes that science speaks for itself and its will should be done because the "answer" is in hand or that findings can be tossed into rough political seas. Regulatory decisionmakers must struggle with value judgments as they weigh risks versus benefits; payers and sickness funds must contemplate costs and benefits in their full complexity. Translating analysis into policy is itself a highly dynamic process, and these dynamics manifest themselves as surprises (in the evolving application of technology), uncertainties (in interpretation and extrapolation of evidence), and value judgments (as policymakers consult evidence to decide what technologies to cover). Consumer groups, the media, industry, and public and professional opinion crowd the table. These considerations cross into terrain that is more commonly the preserve of political theory than of welfare economics—namely, that of democracy, equity, participation, and transparency. Our original question has thus evolved from one of institutional accommodation (how best to integrate analysis into supply/demand controls) to one of institutional innovation (how to integrate myriad stakeholders and claimants into priority setting). Legitimacy is not born (of science) but made—that is, "socially constructed." Nowadays each increment of analytical progress is matched by an increment of political organization among groups that do not want to see their interests relegated to the disfavored quadrants or percentiles of an analyst’s league table. Those who would strengthen the legitimacy of analysis must conjure explicitly with political choices, which are no less difficult than analytical choices about technology. Town meetings and public hearings promote procedural transparency, but people may insist on institutional representation, for instance, on advisory boards and decision-making bodies. How should such representation be organized? Should we include patients, consumers, or both? What are the moral claims of representation by age, socioeconomic status, race/ethnicity, and genetic categories? And if representation and transparency improve, does one get the intended legitimacy and buy-in for "efficient" decisions, or is efficiency jettisoned as an unintended consequence of democratic participation? What are the procedural requisites of an ethically acceptable approach to explicit priority setting? Analysis and evidence-based projects have done much to clarify the stakes of policy decisions about technology and (at times) to fortify political legitimacy of those decisions. A distinct but complementary project—the design of institutional arrangements and processes that seek to integrate the findings and counsels of evidence with the larger qualitative world of surprise, uncertainty, and value tension that springs from and heightens the dynamics of technology policy—has advanced much more slowly, in part because it is too seldom recognized as legitimate and integral to rational decision making. Many countries now experiment with a range of institutional innovations that try to interfuse qualitative and quantitative "data," to blend evidence with politics, and they deserve sophisticated scrutiny in careful cross-national case studies. Without a higher synthesis of evaluative science and political analysis, the admirable rigor of evidence-based advice may risk irrelevance.
The authors are affiliated with the International Center for Health Outcomes and Innovation Research (InCHOIR) at Columbia University in New York City and the Organization for Economic Cooperation and Development (OECD) in Paris. Annetine Gelijns (acp10{at}columbia.edu) is an associate professor of surgical science and of health policy and management; Larry Brown is a professor of health policy and management; Corey Magnell is a postdoctoral research fellow in surgery; and Alan Moskowitz is an associate professor of clinical medicine at Columbia. Elettra Ronchi is coordinator of health and biotechnology activities at the OECD, Directorate for Science, Technology, and Industry. The authors thank the Merck Foundation (except Elettra Ronchi) and the Organization for Economic Cooperation and Development for support, and Par Atwal and anonymous reviewers for their helpful comments. Views are those of the authors and do not necessarily reflect the view of the Organizations for Economic Cooperation and Development or its member countries.
This article has been cited by other articles:
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Top
Challenges Inherent In The...
Challenges Inherent In The...
5 percent) to achieve a better quality of life (a five-point change on the Minnesota Living with Heart Failure Scale).
