|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
How Does The Quality Of Care Compare In Five Countries?
International data on quality of medical care allow countries to compare their performance to that of other countries. The Commonwealth Fund International Working Group on Quality Indicators collected data on twenty-one indicators that reflect medical care in Australia, Canada, New Zealand, England, and the United States. The indicators include five-year cancer relative survival rates, thirty-day case-fatality rates after acute myocardial infarction and stroke, breast cancer screening rates, and asthma mortality rates. No country scores consistently the best or worst overall. Each country has at least one area of care where it could learn from international experiences and one area where its experiences could teach others.
Most industrialized countries share an interest in measuring, reporting, and improving the quality of medical care. Despite this interest, there have been limited internationally comparable data available on quality indicators, especially in areas involving medical care interventions. Collaboration between countries to produce internationally comparable data permits benchmarking and allows policymakers and clinicians to identify specific areas where individual countries could improve. In the United States, sparked by Institute of Medicine (IOM) reports focusing attention on gaps in the quality of medical care, interest in improving quality has expanded rapidly among policymakers, corporations, clinicians, the media, and the public.1 Despite this concern about the quality of care, U.S. policymakers and clinicians often recite the mantra, "Americans have the best medical care in the world."2 The empirical basis for this statement is unclear. The limited empirical international data on quality that existlife expectancy and infant mortality statisticsplace the United States in the bottom quartile of industrialized countries, although most observers do not attribute this poor performance primarily to the performance of the medical care system.3 This paper presents data collected for twenty-one quality indicators in five countries. Our intent is to draw attention to potential opportunities to improve medical care in the five countries; raise questions about why some countries do well on some measures and others do poorly; provoke debate within countries about health care priorities and policies; and stimulate efforts to examine, refine, improve, and collect additional data.
In 1999 the Commonwealth Fund convened a working group of quality measurement experts from governments in Australia, Canada, New Zealand, England, and the United States, along with academic researchers and representatives of institutions involved in medical care quality measurement.4 This group examined a variety of working definitions of quality, ultimately choosing one developed by the IOM: "the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge."5 The next step was to choose among the various measurement domains that give structure to the quality reporting effort. The five countries have each created similar measurement frameworks. Canadas framework was adopted to guide the data collection exercise mainly because of its comprehensiveness. The working group focused on developing indicators of the appropriateness and effectiveness of carethe extent to which care is delivered in accordance with established standards and achieves its desired results. Other quality measurement domains (such as equity and responsiveness) were left for future work, because of measurement difficulties (for example, continuity and safety) or because they are the subject of other measurement activities (for example, efficiency and access). The working group then identified indicators for collection, starting with lists of potential indicators reflecting each domain of health system performance. Indicators were evaluated using the following criteria. (1) Feasibility: Only indicators that were already being collected by one or more countries were candidates. (2) Scientific soundness: Only indicators that were deemed valid and reliable were considered. Since all of the indicators considered were already in use, determination of scientific soundness relied on existing reviews of the scientific evidence and approval by a consensus process or similar method in one or more countries. (3) Interpretability: Only indicators that allowed a clear conclusion for policy-makers were included. This meant that the indication had to have a clear direction (higher is either good or bad). (4) Actionability: Only measures of processes or outcomes of care that could be directly affected by health care policy or health care delivery system intervention were eligible. (5) Importance: Only indicators that reflected important health conditions accounting for a major share of the burden of disease, the cost of care, or policymakers priorities such as vulnerable populations were pursued. These criteria were applied in a five-step process. First, all indicators currently available in at least one countryan initial set of more than 1,000 indicatorswere assembled. The fifty most promising indicators were then selected based on the five criteria above. We then further assessed these indicators by collecting information on definition, numerator and denominator specifications, the population represented, periodicity of collection, and data sources for each country. Indicators with irreconcilable differences in specifications or that were not nationally representative in several countries were discarded. For the remaining thirty-five indicators, we applied an iterative process of collecting data in the five countries, evaluating the comparability of the specifications, and making adjustments, such as revising coding classifications or age standardization. Finally, we compared the face validity of preliminary data and investigated any unusual differences to increase the reliability of the indicators. We also reviewed the final data with experts in each country. There are numerous reasons to explain why this specific list of twenty-one indicators was selected and many others were not. Many potential indicators would require a review of medical records, which would be very costly without routine access to electronic medical records, and medical record-keeping practices vary considerably across countries. Some indicators were deemed difficult to interpret.6 Others were eliminated because of a relative lack of importance.7 Among the available indicators meeting all other criteria, several were deemed not to be internationally comparable.8
Data for all twenty-one indicators are summarized in Exhibit 1
None of the five countries consistently scores the best or worst on all of the indicators. In addition, each country has either the best or the worst score on at least one indicator. In other words, no country scores consistently the best or worst overall, and each country has at least one area of care where it could learn from international experience. Each country also has an area where it could teach others.
The results in Exhibit 1 Outcome indicators: survival rates. The first five indicators are five-year relative survival rates for various types of cancer.9 The relative survival rate is the ratio of the number of cancer survivors to the number of people of that age and sex in the same country who would have been expected to be alive after five years if they did not have cancer. It measures the additional deaths attributable to cancer, controlling for differences in underlying mortality patterns between countries. On these indicators, the range in performance was usually small. On most survival rates, the countries are within 10 percent of each other. One pattern that does stand out is that England is consistently at the low end of the distribution for cancer survival. This is consistent with previous comparisons of cancer survival between the United Kingdom and other European countries.10 Higher cancer survival rates are unquestionably a desired health care goal. Primary care, including health promotion and screening, can make a difference in the stage of diagnosis for the cancers studied, particularly cervical, breast, and colorectal cancer.11 Secondary and tertiary cancer care can also make a difference. Other factors such as financial barriers to care, waiting lists, and reluctance to seek care could also influence rates. Two related indicators of the outcomes of health care are the survival rate following a kidney or liver transplant.12 The survival rates for both were relatively low in the United States. Differences in the characteristics of patients receiving transplants could influence survival rates. Assuming that transplant recipients in the five countries are similar, the remaining differences are more likely to be attributable to differences in medical care.
Data on acute myocardial infarction (AMI) and ischemic stroke are also presented, although comparable data are available in only three of the five countries.13 AMI case-fatality rates are highest in Canada and lowest in Australia (Exhibit 2
Outcome indicators: avoidable events. The second group of outcome indicators shows the rates of certain health outcomes that are considered avoidable had appropriate care been delivered. These indicators include suicide rates, the incidence of vaccine-preventable diseases, asthma mortality rates, and smoking rates. Suicide rates were almost equal in Australia, Canada, New Zealand, and the United States but lower in England.15 Suicide rates among two groups of younger people show bigger differences.16 New Zealand had much higher suicide rates among young people than the other four countries; suicide is an area that is already receiving attention in New Zealand. The incidence rates of three vaccine-preventable diseasespertussis, measles, and Hepatitis Bshow that some countries have these diseases under better control than others do.17 Pertussis incidence was particularly high in Australia and Canada; measles incidence was higher in England than elsewhere; and Hepatitis B incidence was highest in the United States and Canada.
Another appropriateness indicator reflecting an avoidable outcome is the asthma mortality rate for people ages 539, the ages at which asthma is most reliably diagnosed (Exhibit 4
Smoking rates (as percentage of the population) were lowest in the United States and Canada (Exhibit 1 Process indicators. The five countries were similar in performance on several process indicators of appropriate care delivery and widely different on others. The differences in mammography rates between countries were relatively small.23 However, sizable differences were seen between countries in the cervical cancer screening rate for the population for whom screening is indicated.24 Cervical cancer screening was much more common in the United States than elsewhere.25 Influenza vaccination rates show that all five countries could prevent more influenza-related deaths among older people through vaccination.26 New Zealand, in particular, might investigate how it could increase its rate to the level of other countries. Polio vaccination rates were above 90 percent in Australia, England, and the United States and above 80 percent in Canada and New Zealand.27 Although polio has recently been absent in these countries, these low vaccination rates could allow it to recur, particularly in Canada and New Zealand.28 If one considers cost-effectiveness, it is hard to identify the clear ideal level for these process indicators; higher rates are better, but at a certain point the marginal returns are likely to be small. In the absence of such an ideal level, it is useful for countries to benchmark their rate against those in other countries. These comparisons show that countries deliver these health care interventions at generally similar rates, although opportunities exist for countries to raise their level to that of the best-performing country.
The comparisons on this initial set of quality indicators show that each country performs well in some areas and poorly in others compared with other countries. Each country could improve the quality of care. Australia performed well on many of the indicators. In particular, cancer survival rates were generally high (excepting childhood leukemia); breast cancer screening rates were high; asthma mortality was relatively low; and influenza and polio vaccination rates were high. However, the incidence of pertussis was much higher than elsewhere, suggesting an opportunity for improvement. In Canada, cancer survival rates were generally average or above average and were highest for childhood leukemia. Stroke case-fatality rates were relatively low. Transplant survival was also relatively high in Canada. However, AMI case fatality was higher in Canada than Australia and New Zealand in older age groups. This confirms previous findings and deserves further investigation.29 Pertussis incidence was much higher than in the other countries (except for Australia). Suicide rates were notably lower in England than in the other four countries. The polio vaccination rate there was the highest. However, cancer survival rates were lowest in England, as were breast and cervical cancer screening rates. This confirms previous European comparisons and suggests an opportunity for improvement. Measles incidence was also higher in England than elsewhere. In New Zealand, the improvement in asthma mortality over the past twenty years is a true success story, although there may be room for further improvement. The colorectal cancer relative survival rate was highest. However, the suicide rate in New Zealand, particularly among younger people, was much higher than elsewhere. Stroke case-fatality rates were higher among older age groups. Breast cancer screening and influenza and polio vaccination rates were relatively low. In the United States, breast cancer survival rates were higher than in the other countries. Cervical cancer screening rates were very high. One area for concern is that asthma mortality rates were increasing in the United States but decreasing in the other countries. Transplant survival rates were also relatively low in the United States. While the United States often performs relatively well for this set of indicators, it is difficult to conclude that it is getting good value for its medical care dollar from these data. The huge difference in the amount the United States spends on health care compared with the other countries could very well be justified if the extra money provided extra benefits. Population surveys have shown that the extra spending is probably not buying better experiences with the health care system, with the exception of shorter waits for nonurgent surgery.30 Earlier studies have shown the United States to be in the bottom quartile of population health indicators such as life expectancy and infant mortality.31 Our results also fail to reveal what the extra spending has bought, although there are many important places to look. The limitations of this indicator list preclude the definite conclusion that any country has the best quality of care. It should be emphasized that this initial set of twenty-one quality indicators was distilled from a starting compendium of more than 1,000. It is an opportunistic list, rather than a comprehensive list. Some indicators relate to health conditions that account for a large share of the burden of disease in these countries, while others (such as transplant survival rates) have smaller implications for population health. Some conditions that represent a large share of the disease burden, such as diabetes, are not represented at all. More work is clearly needed to expand the scope and depth of the indicator set so that it can be used to judge overall health system performance, and further investment in data collection and international harmonization of indicators to allow valid international comparisons are necessary. During the time frame of this project, which began in 1999, major improvements in quality measurement capabilities have been made in many countries, which indicates the potential for improvement. Most importantly, building on the Commonwealth Funds International Working Group on Quality Indicators and a similar effort undertaken by a group of five Scandinavian countries, the Organization for Economic Cooperation and Development (OECD) has undertaken an initiative to move this work forward, expand the number of countries involved, develop additional quality indicators, and institutionalize the collection of these indicators. We hope that these twenty-one indicators will be a first installment in ongoing efforts to conduct international quality comparisons.
Peter Hussey is a doctoral candidate at the Johns Hopkins Bloomberg School of Public Health in Baltimore, Maryland. Gerard Anderson (ganderso{at}jhsph.edu) is a professor there. Robin Osborn is assistant vice president of the Commonwealth Fund, in New York City, and director of its International Program in Health Policy. Colin Feek is deputy director-general, Clinical Services Directorate, in the New Zealand Ministry of Health. Vivienne McLaughlin is assistant secretary of the Health Priorities Branch, Australia Department of Health and Ageing. John Millar is executive director of Population Surveillance and Disease Control Planning, Provincial Health Services Authority, British Columbia. Arnold Epstein is the John H. Foster Professor and chair of the Department of Health Policy and Management, Harvard School of Public Health, in Boston, Massachusetts. The working group thanks Jonette McDonnell and the Australian Institute for Health and Welfare, who made the reporting of Australian data possible. This research was supported by the Commonwealth Fund and the Nuffield Trust.
This article has been cited by other articles:
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||