Health Affairs, 23, no. 1 (2004): 77-87 doi: 10.1377/hlthaff.23.1.77 © 2004 by Project HOPE	New Online   House Health Reform Bill   Paying for Reform   Vetting AHIP's Report   HIV/AIDS Costs   Brief: Insurance Reform   HA Blog Top 10

This Article Abstract Reprint (PDF) Submit a response to this article Alert me when this article is cited Alert me when Comments are posted Alert me if a correction is posted Services E-mail this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to My Personal Archive Download to Citation Manager Reprints & Permissions Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Califf, R. M. Search for Related Content PubMed PubMed Citation Articles by Califf, R. M. Related Collections Pharmaceuticals Research And Technology

Environment

Defining The Balance Of Risk And Benefit In The Era Of Genomics And Proteomics

Abstract

 
The ability to measure the function of genes and proteins has spawned the construct of personalized medicine, in which patients’ own risks and preferences are used to choose diagnostic and therapeutic strategies. The complexity of clinical data required to guide personalized medicine calls for improvements in our system of clinical research, including (1) overhauling it to produce networks that can do adequate-size pragmatic trials; (2) synchronization of regulatory and payment systems to encourage adequate studies; and (3) an investment in education of providers and patients to improve the understanding of the probabilistic predictions forming the basis of personalized medicine.

The concept of personalized, prospective medicine improves the potential for successful, sophisticated evaluation of the balance of risk and benefit because it embodies the belief that technology is not simply effective or ineffective, but that it is likely to be more effective in some people and less effective or even harmful in others.² In theory, the attributes of technology can be tailored to the needs of the individual.³ The anticipated effect of personalized medicine also fits with the increasing belief that people should be responsible for their own decision making to the greatest extent possible; if they are given information about the benefits and risks of medical products tailored to their biology and their preferences, one could reasonably hope that they would make more rational health care choices.

As a consequence of recent technological advances, chronic disease increasingly drives health care costs and decisions. The construct of personalized health care is built around prevention of the consequences of chronic disease. The belief has been promulgated that if one can understand the composition of one’s genes, proteins, and metabolites, in the era of sophisticated molecular imaging, knowledge of the presence of disease or propensity for its development, its prevention, and its treatment will be individualized and much more potent.⁴ Although this concept is alluring, predicting the impact of an intervention over many years is a different endeavor than measuring its short-term effect. The large increase in the combinations of personal characteristics and associated treatments will increase rather than decrease the need for clinical outcome data to determine whether potential variations in preventive efforts or treatment produce good or bad health outcomes. Indeed, our current knowledge about generalizable principles for assessing technologies and therapies suggests that we need to improve the system of evaluation of medical technologies (drugs, devices, and procedures) in three critical ways: (1) increase the amount of outcome-based clinical data from comparative randomized trials; (2) develop more harmonious approaches that encourage and incentivize several steps in approval and labeling of technologies for chronic disease; and (3) invest in educating providers, payers, and consumers to better understand the probabilistic nature of therapeutic decisions. In essence, as outlined more than a decade ago, personalized medicine means more need for technology assessment, not less.⁵

The Current System

Top Editor's Notes The Current System Lessons From Clinical Trials Approaches To Improving The... Notes

 
To a large extent, the current system of medical product approval by the Food and Drug Administration (FDA) and much of medical marketing is built upon an antiquated view of clinical therapeutics. This perspective holds that since the rational development of medical products is based on identification and characterization of biological targets derived from pathophysiologic reasoning, the health consequences of using those products should be predictable based on similar reasoning. In theory, given targets that are within a disease pathway and that can be modified, therapies that modify those targets can be developed and tested in specific subgroups, concentrating the power of the treatment effect on those with the most to gain. The therapy’s effect on biomarkers can be measured quickly with relatively small sample sizes; one can prove, for example, whether a drug lowers blood glucose, LDL cholesterol, or blood pressure or improves a psychological scale of depression in hundreds rather than thousands of patients. Based on pathophysiological reasoning and a few simplifying assumptions, one can create a rational process for estimating which treatments are most effective for specific "types" of patients identified through clinical or genetic characteristics and then for developing progressively more efficient ways to deliver therapies to them.

As our knowledge of the evidence needed to understand the balance of risks and benefits with technology has advanced, the need to reexamine this construct has become apparent. Of course, many technical business issues come into play when a decision is made to develop a drug or device for marketing, including intellectual property rights, market size, competing products, pricing controls, and distribution channels. If these issues are resolved, as payers and consumers become increasingly sophisticated and our knowledge of the principles of therapeutics improves, the industry has three hurdles to consider in planning research and development: the FDA’s decision to allow a product on the market, payers’ decisions to provide coverage, and the beliefs of consumers/patients and providers about whether the product should be used, given that it is available and covered.⁶

Lessons From Clinical Trials

Top Editor's Notes The Current System Lessons From Clinical Trials Approaches To Improving The... Notes

 
Eleven principles. In a recent review, a series of lessons from cardiovascular clinical trials pointed out the complexity of providing convincing evidence about personalized medicine.⁷ These lessons are summarized as follows: (1) Treatment effects are modest; (2) qualitative interactions are uncommon, but quantitative interactions are usual; (3) unintended targets are common; (4) interactions are unpredictable; (5) long-term effects deserve evaluation; (6) class effects can be uncertain; (7) most therapies produce a combination of helpful and harmful effects; (8) most beneficial therapies do not save money, but they are incrementally cost-effective; (9) applying the results of clinical trials is beneficial; (10) some areas of cardiovascular medicine are underserved; and (11) participation is imperative.⁸

The case of hormone replacement therapy (HRT) has been instructive in this regard. After years of increasingly strong recommendations for women to use HRT, the world was startled by the revelation that the most commonly prescribed combination of estrogen and progestin not only failed to protect women from vascular disease but actually increased the risk of vascular events.⁹ Perhaps more surprising to veteran clinical trialists was the demonstration that HRT did not improve quality of life except in the relatively small cadre of women with active hot flashes. Despite the widespread belief to the contrary and many (likely selectively published) clinical trials with small sample sizes, two adequate-size trials showed HRT to have no greater effect than placebo on broad and specific measures of quality of life, including cognitive functioning; most recently, a modest detrimental effect on cognitive functioning was documented.¹⁰ Despite evaluation of multiple subgroups, based on both biological and clinical findings, the investigators failed to find a subgroup with benefit for cardiovascular risk, but they did identify women with active hot flashes as benefiting with regard to quality of life.

Because most technologies, particularly drugs, affect both intended and unintended biological targets, it is hazardous to depend on biological measurements to reliably predict clinical outcomes.¹¹ A litany of therapies showing a beneficial effect on one aspect of disease-related biology have turned out to be neutral or detrimental (such as HRT, troglitazone, cisapride) when overall effects on risk and benefit were measured; in other cases, the long-term benefits of treatment far outstripped the predicted benefit, which was based on biological insight (such as statins, angiotensin-converting enzyme, or ACE, inhibitors, aspirin).¹² Importantly, these findings do not detract from the importance of biomarkers to enhance our understanding of biology; treatments that lower blood pressure will almost always reduce stroke, and those that reduce LDL cholesterol will almost always reduce atherosclerosis. It is not usually the intended pathways that create the problem with surrogates in predicting the integrated balance of risk and benefit. Instead, the culprits are the unintended or previously unknown biological pathways.

Modest effect on outcomes. While rare cases of major curative technologies can be cited, for the most part new therapies add incrementally if at all to the clinical outcome of patients with, or at risk of developing, disease. Since treatment effects on clinical outcomes are generally quite modest (less than 25 percent event-rate reductions on a relative scale), the sample sizes required to measure them are much larger than traditionally believed, and these relative benefits translate into small effects on absolute scale.¹³ This issue is exacerbated because common chronic diseases increasingly have many available therapies, making small, marginal differences highly likely. Because of the treachery of the play of chance, there is simply no substitute for enrolling adequate numbers of patients into randomized controlled trials to give reliable, reproducible answers about a technology’s health outcomes.

A prevalent approach to these issues has been to assume that breakthroughs in our understanding of biology will allow us to segregate patients with (or at risk of) a disease into two groups: those who will and those who will not benefit from a new therapy. The promise of genomics and proteomics is largely built on the theory that these technologies will enable us to judge the risks and benefits of therapies in much smaller, focused groups of patients. While a few notable successes have occurred, progress has been slow, and many heralded advances have not been replicated in validation studies.¹⁴ Indeed, a high proportion of genomic findings about disease susceptibility have been false positive, and given that most subgroup findings in clinical trials have been false positive, the use of multiple subgroups based on genomic or proteomic findings is cause for great concern.¹⁵

In two recent examples that could portend the future of personalized medicine, subgroups based on clinical characteristics have been used to make major decisions regarding coverage by the Centers for Medicare and Medicaid Services (CMS).¹⁶ In one case, implantable cardioverter defibrillators were funded by the CMS only if the duration of the QRS complex on the electrocardiogram (a measure of the duration of depolarization of the left ventricle) exceeded 12 milliseconds; in the other case, lung volume reduction surgery (LVRS) was funded only for patients with particular clinical characteristics. The latter case is particularly interesting because the CMS actually funded the multicenter clinical trial in a laudable effort to evaluate the technology in the interest of patients covered by the CMS. In both of these cases, there is little doubt that adequate randomized trials gave us much more confidence about who should be treated (and, accordingly, who should be reimubursed for treating them). However, an important question is raised by these examples: Would the subgroup findings be replicated if the trials were repeated? Without large, confirmatory studies, we run a significant risk of making policy decisions based on findings that occurred simply through random variation. Given the residual uncertainty with these relatively simple examples, if multiple genomic or proteomic variations are found that could explain clinical response variability, the amount of clinical data required to validate and assure the findings will likely increase by more than an order of magnitude.

Similar effects. Another critical theory about therapeutics is that drugs within the same "class" essentially have the same effect. This approach allows the shortcut of developing follow-on drugs in the same class as the first that can be presumed to have the same effect on the clinical outcomes of interest. Unfortunately, multiple examples dispel the notion that we can simply accept the class effect without more detailed evaluation.¹⁷ There are two basic reasons for this finding: First, different molecules aimed at the same target can have different effects on that target and on other targets that could alter the efficacy of the drug; second, the differences in structure and function of different molecules could alter the balance of risk and benefit through either greater or lesser toxic effects.

In addition to antibiotics and thiazide diuretics, beta-blocking drugs and statins are among the oldest classes of dramatically beneficial drugs. Except for their obvious effect in blocking beta receptors and inhibiting the pathway involving HmG CoA reductase in multiple tissues, the biology of the human response to beta blockade and statins is complex and poorly understood. However, not all of the trials comparing beta-blockers with placebo have shown a reduction in mortality; a large trial funded by the National Institutes of Health (NIH) failed to demonstrate a reduction in mortality with bucindolol.¹⁸ Most recently, a head-to-head trial of two beta-blockers, both of which had been shown independently to reduce the risk of death, showed carvedilol to be superior to short-acting metoprolol in reducing mortality.¹⁹ Despite the fact that one statin was removed from the market because of an unacceptably high rate of rhabdomyolysis (kidney damage) in the absence of evidence of a superior effect on clinical outcomes, no head-to-head comparison of the statins has been completed.²⁰

Assessment of therapies in combination. The basis of the development of each medical product has been an assessment of the effect of each therapy in isolation, and this "purified" assessment is obviously critical. However, we know that when two therapies are combined, the total effect is not easily predictable, yet the number of drugs taken concurrently by patients continues to expand. Sorting out whether combinations are additive, synergistic, neutral, or detrimental requires sample sizes more than double the number required to determine the effect of an individual treatment. This issue will become especially important as drug/device combinations become more common and as the industry begins to package multiple drugs in the same pill to deal with people’s difficulty with taking multiple pills.²¹

Long-term effects. In the current state of therapeutics, many therapies are administered for decades. In the future, as individual disease susceptibility is defined by genomics and proteomics, therapies might be given for a lifetime. We now have many examples of surprising beneficial or detrimental effects in the long term that were not predicted based on short-term results. The recent reports of a reduction in prostate cancer by finasteride, balanced by an increase in the severity of the cancer in those who develop it, points out both the importance of and uncertainty about the long-term effects of drugs taken for chronic disease.²² In the case of nonsteroidal anti-inflammatory drugs (NSAIDs), it remains unclear whether the long-term effects are beneficial or detrimental and whether different NSAIDs or COX-2 inhibitors might yield different long-term outcomes.²³

Development of imaging. One of the most dramatic effects of our national investment in technology has been the proliferation of imaging technology and diagnostic tests aimed at stratifying risk, which is a critical element of the strategy of personalized medicine. Intuitively, better and more accurate tests should be beneficial. At least in the short run, however, the story is not so clear. More accurate diagnostic tests that lead to ill-advised intervention can be both expensive and detrimental to health.²⁴ The use of anti-arrhythmic drugs based on Holter monitoring provides a classical example; the Holter monitor provided an accurate report of ventricular dysrhythmias, but the drugs used to the treat the dysrhythmias increased the risk of death rather than decreasing it.²⁵ Similar issues have been raised with the Swan-Ganz catheter, which measures intracardiac pressures and cardiac output but could lead to increased mortality because of the excessive use of inotropic drugs in monitored patients.²⁶ As we enter an era of risk stratification using genomic and proteomic arrays as well as high-resolution imaging using multiarray computed tomography (CT) and magnetic resonance imaging (MRI), often combined with molecular imaging, appropriate methods for understanding the downstream effects of such powerful technology remain elusive, and these methods have not yet been developed to a refined state.

Complexity of decision making. The complexity of decision making in an environment of multiple effective therapies with staggering costs and potential downstream effectiveness or harm should not be underestimated. Studies of human decision making have underscored a point known to marketers for years: Decision making is influenced, and in many cases dominated, by factors other than a rational weighing of the probability that decisions will be aligned with stated values.²⁷ Yet little effort has been expended to ensure that decisionmakers are able to grasp scientific results. Labels on medical products contain far more information than providers can process, and common communication strategies do a poor job of portraying the balance of risk and benefit in an unbiased manner.²⁸ A large segment of the provider community has poor numeracy skills, as do even highly educated patients.²⁹

Approaches To Improving The System

Top Editor's Notes The Current System Lessons From Clinical Trials Approaches To Improving The... Notes

 
The essential message when the promise of personalized medicine is combined with our improved understanding of principles of therapeutics is as follows: If our goal is to offer patients/consumers choices for preventing and treating chronic disease that are tailored to their individual needs based on the best scientific assessment of the value of new therapies, our current system of technology evaluation is outdated and inadequate. In fact, instead of decreasing the need for large clinical outcome studies, the demands of personalized medicine will further exacerbate the current state of affairs in which science outdistances policy. To provide the data needed for rational choices based on genetic, proteomic, and clinical data, the FDA, payers, and consumers can no longer be regarded as relatively independent links in the chain of therapeutics; efforts must be made to join them without sacrificing the ingenuity and creativity that have been the hallmark of American medicine. The balance of risk and benefit of therapies that have been marketed based on biomarker findings will need to be confirmed or refuted by adequately powered clinical outcome trials. Subgroup findings will require replication in independent samples. Head-to-head comparisons will be needed to discern which therapies truly have a more favorable balance of risk and benefit, and these comparisons will require large sample sizes evaluated over prolonged time periods.

Overhauling U.S. clinical research. To meet this critical need, as discussed in detail by others, we need to overhaul our national system of clinical research. Recent deliberations of the Clinical Research Roundtable have pointed out the shortage of personnel, funding, and efficient systems to deliver research results that can inform the choices that individuals and delivery systems must make among technologies.³⁰ The NIH has put considerable energy into a "Roadmap" that promises to reengineer the clinical research infrastructure to develop large networks of providers, patients, and families who have the education and the capacity to increase—by an order of magnitude—the amount of research being done.³¹ Such an improvement in efficiency will only be achieved if these networks use common data standards and nomenclature via electronic data exchange networks that allow interoperability of clinical, billing, and research data, to reduce cost per unit of research data.³²

Need for pragmatic trials. To obtain the kind of evidence that modern clinical research demands to reliably support decisions, many more trials aimed at providing generalizabile comparative clinical outcome results will be needed. The funding for these "pragmatic" trials now is woefully inadequate, as opposed to "explanatory" trials intended to elucidate pathophysiogical principles.³³ Regulatory and payment systems must be better synchronized so that the discovery and implementation of strategies that truly improve health are adequately rewarded, while technologies and strategies that produce harm or inadequate benefit compared with alternatives are weeded out. Given the increasing dominance of chronic disease, these systems need to consider mechanisms that will encourage early marketing of technology (to produce incentives for continued development) followed by ongoing assessment (to ensure accurate depiction of true effects).

One example of such an approach should be the increased use of provisional approvals for chronic disease therapies. The first phase of approval might be granted based on biomarkers and preliminary evidence of safety and efficacy, but it might be accompanied by the requirement to perform longer-term studies with clinical outcome measures as endpoints. Such a system is in place for priority drugs that provide hope for a disease that has no effective treatment, but it might stimulate more appropriate technology development as a more routine system.

Some have argued that postmarketing surveillance will be adequate to understand the longer-term implications of therapeutic choices, but the experience with previously mentioned examples such as HRT, anti-arrhythmic drugs, beta-blockers, and COX-2 inhibitors points out that this approach will not succeed. While postmarketing surveillance, particularly if improved from its current version, can detect adverse effects of drugs that are not part of the natural history of the disease under treatment or device failures, it cannot answer the complex questions needed to provide decisionmakers with the information they need for comparative decisions.³⁴ Especially when the effects are related to a chronic disease in and of themselves, such as cardiovascular effects of anti-inflammatory drugs, only long-term randomized comparisons will be sufficient.

Two-track system. Conceptually, it is reasonable to expand this construct of provisional approvals followed by longer-term studies to all therapies given chronically or used to treat a chronic disease. The evaluation of a chronic disease therapy might then be thought of as being a two-track system. The first track provides short-term evidence about whether the therapy is on balance beneficial in a relevant population. Since the true effects of the therapy cannot have been measured over a long period of time, the assessment must rely on biomarkers, short-term outcomes, and proof of absence of unusual adverse reactions. Labeling and promotion would reflect the uncertainty inherent at this level of measurement, and payment would be at a lower level or limited to providers and systems with a high level of expertise.

The second track, which could be started simultaneously with the first track, would measure the broad swath of outcomes, including survival, freedom from adverse events, quality of life, and cost. The populations studied would be broad, including all relevant age ranges with the mixture of comorbidities likely to be seen in routine practice. Success in this arena would lead to broad labeling, higher rates of payment, and incorporation into performance measures as part of the delivery of high-quality health care. This latter step would set up a reward system that could reduce marketing costs for successful chronic disease therapies and technologies, since failure to meet a professionally defined performance measure could lead to reduced payment and labeling of the omission as a medical error.

Educating the public. A massive effort is needed to educate the public about how to understand when a therapy or diagnostic strategy is likely to provide a positive balance of risk and benefit. This effort must begin by reshaping providers’ ability to understand and transmit quantitative information about the risks and benefits of technologies and approaches. Although our understanding of decision making and the human processing of quantitative information is evolving rapidly, little of this subject is taught in health professions schools, nor has the system evolved to enhance the degree to which information truly informs consumers and decisionmakers of what they need to know to make complex, prospective decisions.³⁵

Professional standards. The growing effort to create professional standards through clinical practice guidelines and performance measures has provided a route to actualize a more rational use of diagnostic and therapeutic technology.³⁶ Early efforts to pay differentially for adherence to performance measures provide an approach to focusing providers and industry on proving value, since quantitative demonstration of outcome benefit in adequate randomized trials provides a route to improve outcomes and an impetus to fund and use beneficial products.

The system is not broken. People are living longer and experiencing much greater freedom from disability than ever before. Yet if our regulatory and delivery systems can catch up with the rapid pace of technology evaluation, even greater progress can be made. Eschewing unhelpful or harmful technologies will leave room for rewarding those who produce and deliver technologies that are beneficial. In this new era of efforts to personalize medical decision making, it is critical to understand that knowledge of genes and proteins only serves as a substrate for much greater production of clinical outcomes data and a renewed national priority to improve the ability of health care providers and the public to deal with the derivative knowledge.

Editor's Notes

Top Editor's Notes The Current System Lessons From Clinical Trials Approaches To Improving The... Notes

 
Robert Califf is director of the Duke Clinical Research Institute and the Donald F. Fortin Professor of Cardiology, Division of Cardiology, Department of Medicine, at the Duke University Medical Center in Durham, North Carolina.

This work was supported in part by Agency for Healthcare Research and Quality (AHRQ) Centers for Education and Research on Therapeutics (CERTs) Cooperative Grant no. U18HS10548. The author thanks Penny Hodgson and John Daniel for their editorial assistance in preparing this paper.

Notes

Top Editor's Notes The Current System Lessons From Clinical Trials Approaches To Improving The... Notes

 

1. C.J. Murray, A.D. Lopez, and D.T. Jamison, "The Global Burden of Disease in 1990: Summary Results, Sensitivity Analysis and Future Directions," Bulletin of the World Health Organization (June 1994): 495–509; K.G. Manton and X. Gu, "Changes in the Prevalence of Chronic Disability in the United States Black and Nonblack Population above Age 65 from 1982 to 1999," Proceedings of the National Academy of Sciences (U.S.A.) (22 May 2001): 6354–6359; D.M. Cutler and M. McClellan, "Is Technological Change in Medicine Worth It?" Health Affairs (Sep/Oct 2001): 11–29; D.B. Mark and M.A. Hlatky, "Medical Economics and the Assessment of Value in Cardiovascular Medicine," Part 1, Circulation (23 July 2002): 516–520; and D.B. Mark and M.A. Hlatky, "Medical Economics and the Assessment of Value in Cardiovascular Medicine," Part 2, Circulation (30 July 2002): 626–630.
2. R. Snyderman and R.S. Williams, "The New Prevention: Research on Causes of Disease Must Be Applied to Remaking Our Care System," Modern Healthcare (26 May 2003): 19.
3. R.S. Williams, H.F. Willard, and R. Snyderman, "Personalized Health Planning," Science (25 April 2003): 549; G.S. Ginsburg and J.J. McCarthy, "Personalized Medicine: Revolutionizing Drug Discovery and Patient Care," Trends in Biotechnology (1 December 2001): 491–496; and F.S. Collins et al., "A Vision for the Future of Genomics Research: A Blueprint for the Genomic Era," Nature (April 2003): 835–847.
4. J.S. Ross and G.S. Ginsburg, "The Integration of Molecular Diagnostics with Therapeutics: Implications for Drug Development and Pathology Practice," American Journal of Clinical Pathology (January 2003): 26–36.
5. P.M. Ellwood, "Shattuck Lecture—Outcomes Management: A Technology of Patient Experience," New England Journal of Medicine 318, no. 23 (1988): 1549–1556[Web of Science][Medline]; and V.R. Fuchs and A.M. Garber, "The New Technology Assessment," New England Journal of Medicine 323, no. 10 (1990): 673–677.[Web of Science][Medline]
6. R.M. Califf and D.L. DeMets, "Principles from Clinical Trials Relevant to Clinical Practice," Part 1, Circulation (20 August 2002): 1015–1021; and R.M. Califf and D.L. DeMets, "Principles from Clinical Trials Relevant to Clinical Practice," Part 2, Circulation (27 August 2002): 1172–1175.
7. D.L. DeMets and R.M. Califf, "Lessons Learned from Recent Cardiovascular Clinical Trials," Part 1, Circulation (6 August 2002): 746–751; and D.L. DeMets and R.M. Califf, "Lessons Learned from Recent Cardiovascular Clinical Trials," Part 2, Circulation (13 August 2002): 880–886.
8. Califf and DeMets, "Principles from Clinical Trials."
9. S. Hulley et al., "Randomized Trial of Estrogen plus Progestin for Secondary Prevention of Coronary Heart Disease in Postmenopausal Women," Journal of the American Medical Association (19 August 1998): 605–613; and J.E. Rossouw et al., "Risks and Benefits of Estrogen plus Progestin in Healthy Postmenopausal Women: Principal Results from the Women’s Health Initiative Randomized Controlled Trial," Journal of the American Medical Association (17 July 2002): 321–333.
10. S.R. Rapp et al., "Effect of Estrogen plus Progestin on Global Cognitive Function in Postmenopausal Women: The Women’s Health Initiative Memory Study: A Randomized Controlled Trial," Journal of the American Medical Association (28 May 2003): 2663–2672; and J. Hays et al., "Effects of Estrogen plus Progestin on Health-Related Quality of Life," New England Journal of Medicine (8 May 2003): 1839–1854.
11. T.R. Fleming and D.L. DeMets, "Surrogate Endpoints in Clinical Trials: Are We Being Misled?" Annals of Internal Medicine (October 1996): 605–613.
12. Troglitazone was withdrawn from the market 21 March 2000. See "Troglitazone (Rezulin)," DiabetesMonitor.com, 28 March 2000, www.diabetesmonitor.com/rezulin.htm (3 November 2003); D. Layton, C. Key, and S.A. Shakir, "Prolongation of the QT Interval and Cardiac Arrhythmias Associated with Cisapride: Limitations of the Pharmacoepidemiological Studies Conducted and Proposals for the Future," Pharmacoepidemiology and Drug Safety (January–February 2003): 31–40; P.M. Ridker et al., "Inflammation, Pravastatin, and the Risk of Coronary Events after Myocardial Infarction in Patients with Average Cholesterol Levels," Circulation (1 September 1998): 839–844; ACE Inhibitor Myocardial Infarction Collaborative Group, "Indications for ACE Inhibitors in the Early Treatment of Acute Myocardial Infarction: Systematic Overview of Individual Data from 100,000 Patients in Randomized Trials," Circulation (9 June 1998): 2202–2212; and Antiplatelet Trialists’ Collaboration, "Collaborative Overview of Randomised Trials of Antiplatelet Therapy—I: Prevention of Death, Myocardial Infarction, and Stroke by Prolonged Antiplatelet Therapy in Various Categories of Patients," British Medical Journal (8 January 1994): 81–106.
13. R. Peto and C. Baigent, "Trials: The Next Fifty Years; Large Scale Randomised Evidence of Moderate Benefits," British Medical Journal (31 October 1998): 1170–1171.
14. B.R. Winkelmann et al., "Genetics of Coronary Heart Disease: Current Knowledge and Research Principles," American Heart Journal (October 2000): S11–S26; and S. Yusuf et al., "Analysis and Interpretation of Treatment Effects in Subgroups of Patients in Randomized Clinical Trials," Journal of the American Medical Association (3 July 1991): 93–98.
15. M. Donahue, W. Kraus, and C.B. Granger, "Genetics in Heart Failure: Practical Incorporation of This New Biologic Dimension," American Heart Journal (December 2002): 938–940.
16. D. Geddes et al., "Effect of Lung-Volume-Reduction Surgery in Patients with Severe Emphysema," New England Journal of Medicine (27 July 2000): 239–245; and A.J. Moss et al. for the MADIT-II Investigators, "Prophylactic Implantation of a Defibrillator in Patients with Myocardial Infarction and Reduced Ejection Fraction," New England Journal of Medicine (21 March 2002): 877–883.
17. C. Furberg, D.M. Herrington, and B.M. Psaty, "Are Drugs within a Class Interchangeable?" Lancet (2 October 1999): 1202–1204.
18. Beta-Blocker Evaluation of Survival Trial Investigators, "A Trial of the Beta-Blocker Bucindolol in Patients with Advanced Chronic Heart Failure," New England Journal of Medicine (31 May 2001): 1659–1667.
19. P.A. Poole-Wilson et al., "Comparison of Carvedilol and Metoprolol on Clinical Outcomes in Patients with Chronic Heart Failure in the Carvedilol or Metoprolol European Trial (COMET): Randomised Controlled Trial," Lancet (5 July 2003): 7–13.
20. R. SoRelle, "Baycol Withdrawn from Market," Circulation (August 2001): E9015–9016.
21. N.J. Wald and M.R. Law, "A Strategy to Reduce Cardiovascular Disease by More Than 80 Percent," British Medical Journal (28 June 2003): 1419.
22. I.M. Thompson et al., "The Influence of Finasteride on the Development of Prostate Cancer," New England Journal of Medicine (17 July 2003): 215–224.
23. M.A. Konstam, "Matters of the Heart: Assessing the Cardiovascular Safety of New Drugs," American Heart Journal (October 2003): 561–562; and D. Mukherjee and E.J. Topol, "Pharmaceutical Advertising versus Research Spending: Are Profits More Important Than Patients?" American Heart Journal (October 2003): 563–564.
24. R.A. Deyo, "Cascade Effects of Medical Technology," Annual Review of Public Health 23, no. 1 (2002): 23–44.[CrossRef][Medline]
25. Cardiac Arrhythmia Suppression Trial (CAST) Investigators, "Preliminary Report: Effect of Encainide and Flecainide on Mortality in a Randomized Trial of Arrhythmia Suppression after Myocardial Infarction," New England Journal of Medicine (10 August 1989): 406–412.
26. A.F. ConnorsJr. et al., "The Effectiveness of Right Heart Catheterization in the Initial Care of Critically Ill Patients," Journal of the American Medical Association (18 September 1996): 889–897.
27. E. Shafir, I. Simonson, and A. Tversky, "Reason-Based Choice," Cognition (Oct–Nov 1993): 11–36.
28. M.G. Ziegler, P. Lew, and B.C. Singer, "The Accuracy of Drug Information from Pharmaceutical Sales Representatives," Journal of the American Medical Association (26 April 1995): 1296–1298.
29. I.M. Lipkus, G. Samsa, and B.K. Rimer, "General Performance on a Numeracy Scale among Highly Educated Samples," Medical Decision Making (Jan–Feb 2001): 37–44.
30. N.S. Sung et al., "Central Challenges Facing the National Clinical Research Enterprise," Journal of the American Medical Association (12 March 2003): 1278–1287.
31. E.A. Zerhouni, "NIH Director Elias A. Zerhouni, M.D., Reflects on Agency’s Challenges, Priorities—Interview by Catherine D. DeAngelis," Journal of the American Medical Association (26 March 2003): 1492–1493.
32. Institute of Medicine Board on Health Sciences Policy, Exploring Challenges, Progress, and New Models for Engaging the Public in the Clinical Research Enterprise: Clinical Research Roundtable Workshop Summary, 2003, www.nap.edu/books/0309089468/html (3 November 2003).
33. S.R. Tunis, D.B. Stryer, and C.M. Clancy, "Increasing the Value of Clinical Research for Decision Making in Clinical and Health Policy," Journal of the American Medical Association (24 September 2003): 1624.
34. R.M. Califf et al., "Integrating Quality into the Cycle of Therapeutic Development," Journal of the American College of Cardiology (4 December 2002): 1895–1901; and R.J. Gibbons, S. Smith, and E.M. Antman, "The ACC/AHA Clinical Practice Guidelines, Part I: Where Do They Come From?" Circulation (17 June 2003): 2979–2986.
35. B.L. Strom et al., for the CERTs Risk Assessment Workshop Participants, "Risk Assessment of Drugs, Biologics, and Therapeutic Devices: Present and Future Issues," Pharmacoepidemiology and Drug Safety, www3.interscience.wiley.com/cgi-bin/fulltext/104534045/PDFSTART (17 November 2003).
36. K.P. Weinfurt, "Outcomes Research Related to Patient Decision Making in Oncology," Clinical Therapeutics (February 2003): 671–683; and E. Shafir, I. Simonson, and A. Tversky, "Reason-Based Choice," Cognition (Oct–Nov 1993): 11–36.

                      What's this?

This article has been cited by other articles:

				M. V. Pauly Is It Time To Reexamine The Patent System's Role In Spending Growth? Health Aff., September 1, 2009; 28(5): 1466 - 1474. [Abstract] [Full Text] [PDF]

				R. M. Califf, R. A. Harrington, L. K. Madre, E. D. Peterson, D. Roth, and K. A. Schulman Curbing The Cardiovascular Disease Epidemic: Aligning Industry, Government, Payers, And Academics Health Aff., January 1, 2007; 26(1): 62 - 74. [Abstract] [Full Text] [PDF]

				L. P. Garrison Jr. and M.J. F. Austin Linking Pharmacogenetics-Based Diagnostics And Drugs For Personalized Medicine Health Aff., September 1, 2006; 25(5): 1281 - 1290. [Abstract] [Full Text] [PDF]

				R. M. Califf Simple Principles of Clinical Trials Remain Powerful JAMA, January 26, 2005; 293(4): 489 - 491. [Full Text] [PDF]

				R. M. Califf, T. Ryan, P. Douglas, and P. J. Goldschmidt-Clermont A time of accelerated change in academic cardiovascular medicine: Implications for academic divisions of cardiology and their training programs J. Am. Coll. Cardiol., November 16, 2004; 44(10): 1957 - 1965. [Abstract] [Full Text] [PDF]

				R. Snyderman The Clinical Researcher--An "Emerging" Species JAMA, February 18, 2004; 291(7): 882 - 883. [Full Text] [PDF]