This Article Abstract Submit a response to this article Services E-mail this article to a friend Alert me to new issues of the journal

P E R S P E C T I V E S
V A R I A T I O N S
W E B  E X C L U S I V E   

7 October 2004  

Perspective:
The Dartmouth Data:
Moving From Analysis To Action

The promise of real-time,provider-specific data adds
new life to the nation’s efforts to reduce
variations and improve its health system.


By James R. Knickman

ABSTRACT:


The work of Jack Wennberg and colleagues on variations in medical practice and outcomes has clarified our understanding of the relationships between supply, costs, and quality of medical care. Variations persist, however, in spite of our knowledge of them. Progress in reducing these variations will require technical, political, and regulatory solutions to problems surrounding financial incentives and health care delivery. Wennberg’s paper uses Medicare claims data to focus on specific providers rather than geographic regions. However, “real-time” availability of such data is necessary to maximize their usefulness, and collective action will be required to overcome obstacles to that goal.

In some ways the paper by John Wennberg and colleagues sounds similar to much of their previous work on variations in medical care use: They report striking variations in the delivery of care to seemingly similar patients at what are generally considered high-quality medical centers.¹ As is often the case with work from the Dartmouth research team, the reader is left wondering why we tolerate such large amounts of variation in costly service delivery when there is so little, if any, evidence that high-spending providers produce better or more valued outcomes. In most other industries, market forces would drive such providers from the market or force clear evidence that the higher costs produced something of added value to the patients treated in these hospitals.

Focus on individual providers. In another way, though, this paper is not at all like previous work by Wennberg and colleagues because it focuses on specific providers, not specific communities. In the past, the Dartmouth data could support analysis focused only on the community, rather than a specific provider. In communities with multiple hospitals, the care patterns for these hospitals end up being blended at the community level. The focus on communities is crucial to anyone interested in community health or the experience of a population of people living in a specific place. However, the absence of data for specific hospitals allowed hospital managers to sometimes ignore the findings from the Dartmouth work, claiming that unusually high spending rates in a community could be attributable to the patterns in other hospitals in the community, not the practice patterns in the hospital they manage.

Although the actual findings reported in Wennberg and colleagues’ paper are important and advance our understanding of variations in how hospitals deliver care to people at the end of life, the most important news here is the new potential for the Dartmouth Atlas data. Now, hospital managers have no reason not to listen closely to the findings about variation in utilization patterns. Now, health plans and those who pay for care have access to an important data stream that can be used to monitor patterns in spending, use, and outcomes. Managers can use the information to ask tough questions of their physicians and nurses. Purchasers of care can use the information to explore new approaches to selective contracting or incentive reimbursement. And some patients may be able to use the information to make decisions about the type of hospital they wish to go to when facing a serious medical problem. Importantly, hospital quality can be measured with concrete data on experience rather than on perceived reputation.

The hospital-specific data can begin to be used creatively to advance quality improvement agendas. Hospitals can easily compare their outcomes or utilization patterns for many different procedures, tests, or types of patients with those of other hospitals. The Medicare claims data system represents a rich source of comparable information on how care is delivered for almost every hospital in the country. And since the Dartmouth researchers have shown that variations across Medicare patients closely mirror variations across other types of patients, hospital leaders can be confident that the patterns identified for Medicare patients probably apply to most of the hospital’s patient population.²

Toward real-time Medicare data. The paper by Wennberg and colleagues hints at one other new advance that could make the data even more useful for quality improvement: having the Medicare data available in near “real time” (about six months following a hospitalization) rather than the current lag of eighteen months. Lags in data availability do not affect the usefulness of the data for basic research—as has been done by Dartmouth researchers and others over the years. However, actual providers of care may legitimately argue that bad outcomes recorded in the data reflect practice approaches that have changed over the eighteen-month lag period during which data are cleaned and assembled. Having data available quickly provides much more convincing tracking information about outcomes and utilization patterns across hospitals.

Having real-time data also opens up the potential to use Medicare information for evaluation purposes. Hospitals can experiment with quality improvement initiatives and get evaluative information about improvement or lack of improvement relatively quickly from the Medicare files. While hospitals can use their own internal data to track changes in outcomes associated with quality improvement initiatives, they have not had access to comparative data from other hospitals to help understand the impacts of changes in delivery approaches.

The process for creating real-time Medicare data sets, unfortunately, is not simple. There are sizable technical hurdles and confidentiality issues. Making real-time claims data happen would require making the creation of these data sets a priority at the Centers for Medicare and Medicaid Services (CMS). Although the Medicare Prescription Drug, Improvement, and Modernization Act (MMA) of 2003 includes authorization for the CMS to make claims data available more quickly, the agency has many tasks assigned to it as part of MMA. The research community, the provider community, the payer community, and the quality improvement community—as well as private organizations such as philanthropies—should explore how to assist the CMS in making the development of real-time data a high priority and a feasible task.

Foundation interest in the Dartmouth data. A frequent set of questions raised about the work of Wennberg and his colleagues at Dartmouth is as follows: Have these data sets had any impacts to date? How likely is the analysis to make changes in the future? How do we move from analysis to action? These questions are often raised internally at the Robert Wood Johnson Foundation (RWJF)—a consistent funder of this research over the past ten years.

Our interest at RWJF in this work initially flowed from our concerns with the high costs of medical care. The Wennberg analysis seemed like an important approach to better understanding the dynamics of costs and cost inflation in the U.S. health care system. Our sense is that the various studies published as part of the Dartmouth Atlas project have clarified our understanding of costs as well as the huge role of availability in driving costs: The more supply in a geographical area of hospital capacity or specialty capacity, the more the use of these capacities.³ In a recently published paper that reviews the RWJF’s work in addressing the problem of high costs of medical care, a number of commentators pointed to the Dartmouth work as perhaps the RWJF’s most important research investment in this area.⁴

Over time, however, the interest in this work at RWJF has shifted to our efforts in the area of quality. In thinking about quality, it is crucial to understand the “overuse” and “underuse” dichotomy. Much of the research on quality funded by the foundation has focused on the problem of underuse. For example, the work done by Beth McGlynn and her colleagues at RAND documents underuse of many types of services that are known to be effective even among well-insured people.⁵ But the Wennberg analysis makes the complementary case that other types of services are probably overused tremendously, at least in some geographical areas.

The quality improvement agenda in this country probably can only be advanced with the redirection of resources from areas of overuse to areas of underuse. The focus needs to be on pushing for value in how health care dollars are allocated. The chances of designing public programs to address the lack of insurance coverage among forty-five million Americans also will be improved if unneeded services are eliminated from our delivery system, making insurance coverage more affordable for all.

The Dartmouth data also hold promise for understanding what types of initiatives might reduce disparities in health care delivery and outcomes across racial and ethnic groups. The data already have improved our understanding of the role that geography plays in driving disparities.⁶ In the future, we hope that these data can help evaluate natural experiments in which some providers are involved to reduce disparities.

However, we have not necessarily found the answer to the most difficult question: How do we move from analysis to action? Why hasn’t the continuous stream of analysis flowing from Dartmouth reduced geographic variation to date? Some ideas for making this happen are suggested in other papers and commentaries in this collection of papers. But unwarranted regional variations can only be eliminated if payers—and in particular the CMS—take on the issue and create financial incentives, regulatory standards, or new organizational and financial approaches to managing health care delivery. This, of course, is somewhat a technical task, but much more so a political one.

The author acknowledges the research assistance of Kate Muessig.

NOTES

1. J.E. Wennberg et al., “Use of Medicare Claims Data to Monitor Provider-Specific Performance among Patients with Severe Chronic Illness,” Health Affairs, 7 October 2004, content.healthaffairs.org/cgi/content/abstract/hlthaff.var.5.
2. J.E. Wennberg and D.E. Wennberg, eds., Dartmouth Atlas of Health Care in Michigan (Hanover, N.H.: Center for the Evaluative Clinical Sciences, Dartmouth Medical School, 2000).
3. M. Gold, “Geographic Variation in Medicare Per Capita Spending: Should Policy-Makers Be Concerned?” (Princeton, N.J.: Robert Wood Johnson Foundation, 2004).
4. C. Newbergh, “The Robert Wood Johnson Foundation’s Efforts to Contain Health Care Costs,” in To Improve Health and Health Care, vol. 7, ed. S.L. Isaacs and J.R. Knickman (San Francisco: Jossey-Bass, 2004), 57–80.
5. E.A. McGlynn et al., “The Quality of Health Care Delivered to Adults in the United States,” New England Journal of Medicine 348, no. 26 (2003): 2635–2645.
6. K. Baicker et al., “Who You Are and Where You Live: How Race and Geography Affect the Treatment of Medicare Beneficiaries,” Health Affairs, 7 October 2004, content.healthaffairs.org/cgi/content/abstract/hlthaff.var.33; and J. Skinner et al., “Racial, Ethnic, and Geographic Disparities in Rates of Knee Arthroplasty among Medicare Patients,” New England Journal of Medicine 349, no. 14 (2003): 1350–1359.

Jim Knickman (JKNICKM{at}rwjf.org) is vice president, research and evaluation, at the Robert Wood Johnson Foundation in Princeton, New Jersey.

DOI: 10.1377/hlthaff.var.121
©2004 Project HOPE–The People-to-People Health Foundation, Inc.