|
PERSPECTIVEEvidence-Based Guidelines As a Foundation For Performance Incentives
Clinical guidelines, which increasingly build upon impartial analysis of evidence from well-designed studies, have become highly credible sources of information about what forms of care are effective. Consequently, they are attractive as foundations for performance incentives. Unfortunately, they are often complex, and frequently it is infeasible to gather the information required to assess compliance with guidelines at reasonable cost. I discuss the problems in implementing evidence-based guidelines and steps that could be taken to make them more useful as a basis for performance measurement.
It is no longer controversial to assert that rewards for meeting quality or performance criteria can change medical practice. Employers and health plans are committed to "pay-for-performance" initiatives, and the Medicare Prescription Drug, Improvement, and Modernization Act (MMA) of 2003 contains major provisions to develop financial incentives to improve hospital quality.1 Several years of experience confirm that the challenges of implementing incentives to promote better-quality care are not easily surmounted.2 Arguably, the biggest obstacle has been the mismatch between the size of the incentives and the investment needed to measure, then meet, quality targets. But a larger payout is not all that is needed. Changes in program design and implementation will also be necessary. Improvements in the ways that evidence-based clinical guidelines are adapted for use in performance measures would be an important step in that direction. Clinical guidelines would not have such an important role in the design of performance incentives if it were easy to reward providers for improvements in the outcomes of the care they give. Outcome-based performance measures have obvious appeal: If a doctor or hospital achieves lower rates of complications, faster recoveries, and lower mortality, does it matter how? However, the practical barriers to their broader use are daunting. David Eddy writes: When the main health outcomes for an important condition are infrequent, delayed, weakly controllable, or heavily confounded, blind adherence to outcomes will produce inaccurate results...A poorly designed outcome measure can easily do harm than good. The solution is to use more process measures.3 Since outcome measures are unlikely to replace process measures anytime soon, it is imperative to ensure that the process measures used are valid indicators of health care quality. Evidence-based clinical guidelines, which have the credibility that comes from a combination of impartiality and deep clinical and scientific expertise, are attractive in this regard. Many are based on rigorous analyses of available evidence, carrying the imprimatur of specialty societies and other well-regarded expert groups. Furthermore, the best guidelines produced by the American College of Physicians (ACP), the American College of Cardiologists (ACC), the American Heart Association, the U.S. Preventive Services Task Force, and other prestigious groups, are clear and detailed, lending themselves to adaptation as criteria for the quality of care.
Despite their virtues—and the likelihood that no readily available alternative has a stronger scientific foundation—evidence-based guidelines are not an ideal platform for performance incentives. To be used effectively, of course, a high-quality guideline must exist. The number of conditions for which guidelines are available is steadily increasing, but many patients receive care for conditions or combinations of conditions for which suitable guidelines are not available.4 Furthermore, evidence-based clinical guidelines are rarely written with performance incentive programs in mind. They are intended to improve clinical care by describing a set of actions that physicians should consider when managing patients with specific health conditions, and sometimes to influence financing and other aspects of policy that could affect guideline implementation. Flexibility. Most well-accepted guidelines, especially those produced by specialty societies, leave considerable discretion to the treating physician. The authors anticipate the objections that practicing physicians might raise, and most guidelines acknowledge numerous circumstances in which exceptions to their general recommendations are justified. To gain the approval of physicians, many of the recommendations incorporated in clinical guidelines are vague enough to permit flexibility ("choose the medication based on the patient’s medical history, preferences, and ability to comply with follow-up care"). Others trade vagueness for detail and complexity by explicitly describing clinical nuances and many modifying considerations. Flexibility, whether obtained by vagueness or complexity, makes it difficult to determine whether the care a person received was consistent with the recommendations. Guidelines that address controversial practices, such as prostate cancer screening, commonly state that individual preferences are critical to determining which patients should receive the intervention. Without interviewing each patient directly, it can be impossible to determine whether the care complied with the guideline.5 Even when the guideline itself is unambiguous, costly chart review is usually necessary to determine whether it was followed. Few institutions or physician practices have information systems that enable them to extract detailed clinical data, comparable to the contents of a paper chart. And even a paper chart can lack important detail. Electronic health information has been a cornerstone of nearly all plans for quality improvement in hospitals and doctors’ offices for many years, but the effective deployment of electronic medical records and related advances has proceeded remarkably slowly. Until providers are able to use powerful information systems to improve the accuracy and drive down the cost, the translation of detailed evidence-based guidelines into performance measures will proceed slowly and imperfectly.6 Accountability. Accountability is one of the biggest challenges to quality improvement and measurement. From a payer’s or purchaser’s point of view, when all of a patient’s care is provided by an integrated system or group, such as Kaiser Permanente or a Department of Veterans Affairs (VA) facility, assigning responsibility for care is straightforward.7 That is, it is possible to identify who should receive a performance incentive, even if the group itself must develop internal mechanisms to ensure that the performance standards are met. The challenges are greater in a typical fee-for-service or preferred provider organization (PPO) environment. Even the conceptual basis for assigning responsibility is unclear when a patient is treated by multiple physicians, some of whom the patient selects without the concurrence or even knowledge of the others. An adult with diabetes mellitus could receive care regularly from an internist, cardiologist, ophthalmologist, and podiatrist, each of whom could adjust medications and share in the monitoring of disease complications and the side effects of treatment. Who is responsible if the patient fails to receive a recommended treatment such as an angiotensin-converting enzyme (ACE) inhibitor? If the patient requires a toe amputation that should have been preventable, which of several physicians and nurses caring for the patient should be considered responsible? To what degree does the patient bear responsibility? Accountability for health outcomes remains a challenge in nearly every setting, but it is most severe for the majority of Americans who do not receive care from an integrated group. Guidelines have offered little help. Few discuss who should be responsible for each aspect of care.8 Limits in the evidence base. Limits in the evidence base available to guideline authors have repercussions for those guidelines’ application to performance measurement. Randomized controlled trials, the most respected source of information for clinical guidelines, are conducted in relatively small and often unrepresentative populations. To make recommendations that apply to most of the patients that doctors see in their offices, guideline authors nearly always extrapolate to groups that were not adequately represented in the trials. Do estimates of the efficacy of mammography at diagnosing early-stage breast cancer in middle-aged women apply to younger populations that were not included in the large studies? Do results of tests for colon cancer in high-risk patients apply to low-risk groups? In the act of translating rigorous evidence into specific recommendations for patient care, judgment must be applied, and different experts can reach different conclusions from the same evidence. Strong and weak evidence. The best guidelines clearly distinguish between recommendations that are strongly supported by direct evidence and those that depend on extrapolation or judgment. But it is not always clear how to use such information. Performance measures that are based only on the most rigorous and direct evidence often apply to a small, narrowly defined patient population. Sometimes performance incentives will be superfluous for that population, because so many of them already receive appropriate care. Performance measures based on more speculative recommendations will greatly expand the scope of care that can be assessed, but they risk sacrificing validity. Costs and quality standards. Finally, nearly all U.S. evidence-based guidelines and performance incentive systems share an expressed goal of improving quality of care. Yet "quality of care" can be interpreted in different ways. Most modern guidelines are designed to promote care that has proved effective while discouraging practices that are of unknown effectiveness or ineffective. But beyond basic agreement that high-quality care excludes ineffective or harmful practices, there are fundamental, frequently unstated differences in interpretation. Most importantly, some guidelines consider the value, or cost-effectiveness, of care, rather than effectiveness alone. The distinction is crucial. It is all but a cliché to describe the rate at which beta-blockers and ACE inhibitors are prescribed for heart failure patients as a key measure of the quality of care. These inexpensive drugs are highly effective. The left ventricular assist device (LVAD) is also an effective treatment for patients with severe congestive heart failure, but few if any guidelines recommend LVAD placement, and the rate of LVAD placement does not appear in performance or quality-of-care measures.9 The LVAD is a relatively new technology, but the tremendous cost of keeping a patient alive with the device has undoubtedly dampened any enthusiasm for building performance measures around its use. Few if any guidelines explicitly state that they are intended to maximize quality of care without regard to cost. Indeed, advocates for quality improvement efforts, including performance incentive programs, often claim that quality can be improved while lowering cost. But many, if not most, medical interventions that improve health outcomes increase spending. Explicit statements that cost-effectiveness (producing the best outcomes for a given cost) is the basis for including a specific recommendation in a guideline remain uncommon, at least in the United States. Absent such an explicit statement of goals, it is difficult to know whether and how performance improvement programs, which themselves are often silent about cost-effectiveness, should reward providers who offer care that is effective but not cost-effective. For those programs that are implicitly designed to reward value, evidence-based guidelines that ignore cost help with only part of the task. Deficits in state-of-the-art guidelines. Many of these issues are manifest in state-of-the-art guidelines that may advance patient care but whose translation into performance measures poses enormous challenges. For example, guidelines recently issued by the ACC and the American Heart Association concerning the management of ST-segment elevation myocardial infarction, a common form of heart attack, are admirably comprehensive and detailed. Furthermore, like other sophisticated guidelines, they clearly indicate the quality of evidence supporting each recommendation. But the guideline document is more than 200 published pages long, has 1,400 references, and requires clinical information that would be difficult to acquire without in-depth chart review.10 Many other guidelines are much simpler yet also require information that is not readily available from electronic sources—for example, the recent ACP guidelines for the evaluation of patients with chronic stable angina are a model of precision and balance between simplicity and accuracy.11
Guidelines such as these are designed to guide patient care, not to provide a framework for rewarding better clinical performance. Some of the problems in translating them into performance measures will diminish over time, especially when better information systems are ubiquitous. Until then, how can authors and expert groups produce guidelines that better support performance incentives? Tie guidelines to strong incentives. Authors of guidelines should first recognize that employers and health plans trying to implement performance incentives are an important audience for their efforts. As these authors know, when guidelines are tied to strong incentives, they are much more likely to be used. Thus, their efforts will have greater impact if they produce guidelines that can be readily adapted for performance measurement. The communities that develop evidence-based clinical guidelines and those that develop performance incentive systems are largely distinct, and better communication between them will facilitate the translation of guidelines into performance measures. Use data from diverse settings. To help make their guidelines translatable, authors should learn more about the kinds of data that are available—and unavailable—in diverse health care settings. Everything from specialized questionnaires to chart review to administrative or population-level data has been used to assess the quality of health care. Guideline authors, especially those expert in evaluating clinical evidence, are well aware that critical information needed to guide therapy is often missing from the literature. They could readily appreciate that the data used in published studies are more detailed than what would typically be available in administrative databases and the limited clinical data used for performance measurement. By applying their knowledge of the evidence base underlying the guidelines and their skills at interpreting the data, they could offer useful advice to the performance-incentive community. They could help determine, for example, whether the data available in many performance measurement efforts are simply too limited to enable conclusions about the quality of care. They could also identify areas in which quality is more easily judged or in which targeted data collection could, at reasonable cost, provide much information about quality. Match magnitude of incentives with importance of performance criteria. Even in the best-designed programs, not all performance criteria are equally important. Ideally, pay-for-performance programs would reserve large rewards for meeting quality standards that lead to large benefits, while providing smaller incentives for meeting performance criteria that correspond to small health benefits.12 For example, delays in administering appropriate therapy for a heart attack or stroke greatly increase mortality and disability; the incentives to treat these conditions quickly and appropriately should thus be larger than the incentives for other practices that should be encouraged but produce smaller benefits, such as avoiding the use of ineffective antibiotics for treatment of the common cold.13 Some programs respond to the challenge by assigning greater weight to the more important criteria when calculating an overall performance score. Typically, however, there is only a loose relationship between the magnitude of the incentive and the importance of the specific performance measure. Improving the match between the incentive and the benefit, of course, may require much analytic effort and detailed information. But some of the work is already done during guideline development. Particularly when they address complex problems or detailed management strategies, evidence-based guidelines could help by offering information about the benefit corresponding to each specific recommendation they make, or at least by assigning a priority to each recommendation. Many guidelines already contain qualitative statements about the extent of benefits. Furthermore, the evidence review that leads to state-of-the-art guidelines is a source of quantitative estimates of benefit. Consider guideline authors’ opinions. Sometimes a treatment may seem to be beneficial, but convincing studies have not been completed. What should be done about the large gray areas, in which there is substantial but not compelling evidence of benefit? Frequently, for example, multiple observational studies, all of them subject to potential bias, have shown that a particular clinical strategy is associated with better outcomes. Some experts will conclude that such information is always insufficient to support a recommendation for the strategy. But some observational studies are more convincing than others. The opinions of the guideline authors, who know the evidence and clinical context well, would be very informative. Some of the recommendations might be strong candidates as performance measures if, for example, there were a consensus that the recommendation was almost certain to lead to better outcomes despite the lack of compelling studies. Be explicit about guideline goals. Finally, both the authors of guidelines and those who design incentive payments need to be explicit about what they are trying to achieve. Are they trying to promote effective care, not considering costs, or high-value, cost-effective care? Do they want to improve patient satisfaction, or are they primarily concerned with mortality, functional status, and physical discomfort? The goals of guideline authors will not always be identical to those of performance incentive systems, but guidelines can still be useful if they are explicit about the rationale for and consequences of following each recommendation. None of these steps is a substitute for better data and thoughtfully applied incentives. However, efforts to promote better-quality care are gaining momentum as large employers and policymakers have come to view poor quality as a crisis for U.S. health care. They will not wait for health care providers to put better information systems in place. As the validity and reliability of performance measures improve, it will become appropriate to increase the magnitude of the incentives. With better underlying performance measures and stronger incentives, we are likely to accelerate our progress toward a health care system that rewards, and produces, better health.
Alan Garber (garber{at}stanford.edu) is a staff physician at the Department of Veterans Affairs (VA) Palo Alto Health Care System and associate director of the VA Center for Health Care Evaluation. Garber is the Henry J. Kaiser Jr. Professor at Stanford University, where he is also a professor of medicine, economics, and health research and policy. He is the founding director of the Center for Health Policy and the Center for Primary Care and Outcomes Research at Stanford’s School of Medicine and research associate and director of the Health Care Program, National Bureau of Economic Research. This work is supported in part by the Homer Laughlin Endowment and by an Investigator Award in Health Policy Research from the Robert Wood Johnson Foundation. The author has benefited from comments by and conversations with Jay Bhattacharya, Douglas Owens, Sara Singer, Kathy McDonald, Arnold Milstein, and Robert Wachter, none of whom should be assumed to endorse the views expressed here.
This article has been cited by other articles:
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Top
Uneasy Application Of Guidelines...
Bridging The Gap Between...
