|
PERSPECTIVESeparating Fact From Fiction: A New Role For Health Affairs
Although policymakers may become sensitized on an issue by compelling anecdotes and may be motivated by ideology, increasingly, actual decision making requires the presentation and acceptance of hard evidence, whether or not it is valid or complete. Health Affairs has demonstrated a remarkable record of adapting to the times by moving successfully to Web-based publishing and an interactive blog. As a highly respected and nonpartisan health policy journal, Health Affairs is well positioned to take up the challenge of reducing out-of-control evidence.
ONE HUNDRED THIRTY thousand pages of Medicare regulations stifle provider innovation. We know that because conservative politicians such as Newt Gingrich tell us this every chance they get. The evidence? A decade ago, the estimable Mayo Clinic added up the pages; who, after all, doesn’t believe the Mayo Clinic? This nugget, demonstrating regulation run amok, even made it into the talking points that candidate George Bush used against Al Gore in one of their 2000 debates, although Bush managed to mangle the details. The only problem is that the number 130,000 is wrong—not just a little wrong, but about 127,500 pages wrong. I know this because as a senior political appointee at the Centers for Medicare and Medicaid Services (CMS), I was selected to defend the number in a congressional hearing. In fact, most of what Mayo counted as pages of regulations were newsletters, nonprecedential payment appeal decisions, and other assorted tidbits, many going back fifteen years. Medicare-related? Yes. Regulations? Not even close. These 130,000 pages became an urban myth partly because then CMS administrator Nancy-Ann DeParle wisely decided not to emulate the experience of Virginia Senator William Scott, who in 1974 responded to a published survey finding him the dumbest senator by calling a press conference to deny the libel; 2,500 pages, after all, is a lot of pages. Evidence versus anecdote. In a world of fantasy weapons of mass destruction, it is not surprising that health care myths arise. But health care is supposed to be particularly oriented to reliance on evidence. It helps that many claims have to make it through peer review in the best journals, such as Health Affairs. But even taking a broad view of what constitutes acceptable evidence—from randomized trials, through a variety of disciplined quantitative and qualitative research, literature synthesis, and even insightful journalism—many claims are just plain wrong, yet they find broad acceptance. Some have argued that researchers and policymakers fundamentally respond to different kinds of information, with the former relying on data and the latter responding largely to anecdote. (In response to Michael Moore’s movie SiCKO, anecdotes demonstrating the supposed inferiority of foreign health systems are popping out all over, no matter what the data show.) However, in health care and other policy domains, whatever the sensitizing role of anecdote on policymakers, actual policy making eventually requires an appeal to the hard evidence—whether valid or not. Misuses of evidence. Some "cook" the data, using questionable but obscure assumptions that produce the desired results. Others assert the evidence and hope that no one actually assesses the underlying support, as when self-interested parties incorrectly claim that the current Medicare Premier Hospital pay-for-performance demonstration saves money.1 Sometimes the distortion is so brazen that unwitting readers assume it must be true, especially when presented by a seemingly credible source. Recently, the just departed acting CMS administrator defended Medicare Advantage overpayments with the assertion that, "A decade ago Congress and President Bill Clinton created a new program within Medicare that allowed patients in the system to receive care through privately adminstered health insurance."2 This attempt to align President Clinton with overpayments to private plans distorts the fact that 1982 legislation set up Medicare risk-contracting private plans and that Congress and the Clinton administration actually cut payments to plans a decade ago. Another approach is to apply a new brand—for example, "pay-for-performance"—to an approach that actually has had a long and undistinguished track record. In this case, using other terminology, managed care plans have rewarded or penalized contracting physicians for more than two decades, with mixed and surely nontransformational results. Some conflate argument with evidence, albeit using logic and inference, sometimes from other sectors of the economy. Thus, proponents of "focused factories" claim that these entities provide higher-quality care more efficiently than general hospital dinosaurs can.3 Those of us who resist this call for specialized centers for every body part argue that patients increasingly have multiple, interacting chronic conditions best handled by generalists and multidisciplinary organizational structures. Evidence would surely help resolve what now is largely an ideological dispute. Limitations of evidence. Not that evidence would necessarily settle the matter. Often, the most important policy disputes reflect important differences in political values; here, the debate properly should occur on the grounds of ideology. Opinions are important, but they need to be labeled as such. Unfortunately, policy making does demand data. So the temptation to ground all argument in an appeal to evidence, however irrelevant or wrong, is powerful. Ignoring the evidence. Advocates often simply ignore existing evidence. Another round of teeth-gnashing that the cost of employer-sponsored health insurance makes U.S. firms noncompetitive in global markets has begun. We’ve been here before—in the pages of Health Affairs, Uwe Reinhardt, Mark Pauly, and Joe Newhouse each have offered strong, complementary arguments, and evidence, that the noncompetitive proposition is, at the very least, greatly exaggerated. Nor does evidence matter to many proponents of market competition. Victor Fuchs in Health Affairs in 1988 explained that the conditions needed for health care competition generally do not exist in most markets.4 Researchers from the Center for Studying Health System Change later documented in these pages that the situation has worsened over time.5 Proponents simply ignore this evidence. Projections versus evidence. For their part, some governors and presidential candidates like to fund their insurance expansion proposals from major savings resulting from adoption of electronic health records (EHRs) and disease management. Indeed, an optimistic RAND analysis—not evidence—projected huge potential savings from EHRs, resulting from major reductions in redundant medical interventions, while ignoring the reality that physicians have no incentive to actually reduce redundancy when they are paid more for maintaining it. Yet EHR savings projections routinely cite the RAND piece, summarized in Health Affairs, but naturally ignore the accompanying commentaries that cast substantial doubt on the robust savings estimates.6 The evidence about disease management based in third-party vendors displays the differences between "might work," "should work," and "does work." Even for the limited number of conditions for which disease management is being applied, there is scant evidence of cost-effectiveness.7 I know, "the absence of evidence is not proof of absence." It’s high time to produce the evidence that vendor-based disease management saves money. The needed corrective. By now, I am sure that many readers are silently protesting that I have mischaracterized the evidence on the issues I have chosen as examples of ignoring or misusing evidence. That’s the point. Even sorting out the validity of evidence is subject to interpretation. But that makes it even more important to try. On most important issues where evidence is being misused, one can find Health Affairs papers that provide the needed corrective. Yet advocates understandably cherry-pick the analyses that support their argument. The corrective gets lost in the wash of policy advocacy. The new Health Affairs blog provides a very useful forum for issues to be joined. But what are often ideological arguments may not expose the inaccuracy of the evidence being misused or missed. As noted, there are various reasons why inaccurate information takes root and grows without needed pruning. The assumption that the clash of opposing arguments relying on selective use of evidence will illuminate the truth might not work well here, especially when actual evidence on an issue might not produce advocates. A new challenge for the journal Health Affairs seamlessly adopted Web publication and a blog. Now it could take on the challenge of out-of-control evidence, not only in the papers it publishes, which the peer-review process handles reasonably well, but especially for claims made elsewhere on important policy issues. Assign an editor to solicit short pieces from experts akin to "fact checking," but at a more sophisticated level of analysis. Borrowing from ESPN SportsCenter’s "plays of the month," twice a year count down to the five most egregious evidence distortions of the past six months. Or something else. Happy Anniversary, Health Affairs, and keep up the good work.
Bob Berenson (rberenso{at}ui.urban.org) is a senior fellow at the Urban Institute in Washington, D.C.
| ||||||||||||||||||||||||
Top
NOTES
