|
Variations In The Quality Of Care For Very-Low-Birthweight Infants: Implications For Policy
Much of the decline in childhood mortality over the past two decades is attributable to improvements in neonatal intensive care for very-low-birthweight infants. Yet large and persistent disparities persist in the quality of neonatal intensive care across hospitals. Improving care for infants now served by hospitals with poor outcomes can greatly reduce infant mortality, particularly among minority infants who are more likely to be very low birthweight and cared for by hospitals with poor outcomes. Referral of high-risk births to hospitals with the best outcomes is another promising strategy.
The United States has one of the highest infant mortality rates among industrialized countries. This is particularly true among black American infants, whose mortality rates are 2.5 times those of white American infants. More than two-thirds of all deaths among Americans under age fifteen occur in the first year of life, and nearly half occur within twenty-eight days of birth.1 Deaths are concentrated among infants with low birthweight (less than 2,500 grams at birth) and especially very low birthweight (less than 1,500 grams at birth). Low-birthweight infants constitute 7.6 percent of all U.S. births but account for nearly two-thirds of all infant deaths, while very-low-birthweight infants constitute 1.4 percent of all U.S. births but account for more than half of infant deaths.2 Thus, more than one-third of all deaths among children under age fifteen occur in the first year of life among infants with very low birthweight. Because of this, there has been great policy interest in reducing U.S. infant mortality rates. Efforts to reduce the proportion of low-birthweight and very-low-birthweight infants, however, have been largely ineffective. Medical interventions to identify women at risk and to reduce the rates of preterm birth have not been successful.3 Policy initiatives such as recent Medicaid expansions have had small effects on the rates of low birthweight.4 Indeed, the U.S. incidence of low birth-weight has actually risen during the past two decades.5 Nevertheless, the U.S. infant mortality rate has declined substantially over the same time period, from 12.6 per 1,000 births in 1980 to 6.9 per 1,000 in 2000.6 Much of this improvement has been attributed to improved perinatal care provided by neonatal intensive care units (NICUs).7 Two-thirds of the decline in mortality among very-low-birthweight infants has been attributed to increases in the effectiveness of newborn intensive care, associated with greater aggressiveness of respiratory and cardiovascular treatments.8 Despite this progress, however, few additional improvements in mortality or morbidity have been observed since 1995, ending a long trend of improving outcomes.9 Because neonatal intensive care has played a vital role in improving infant mortality, in this paper we assess the potential role of improving the quality of this care in reducing infant mortality. Specifically, we assess the potential of two alternative approaches to improve quality of care: collaborative quality improvement and selective referral. The collaborative approach systematically identifies "best practices" being used in hospitals with the best outcomes and then encourages the adoption of these practices at all hospitals.10 The evidence-based selective-referral approach relies on consumer choice and competition among health care providers.11 The basic idea is to improve infant outcomes through careful selection of high-quality providers rather than improving a given provider’s quality of care. Selective referral is being promoted by the Leapfrog Group, a business roundtable of large employers and insurers, which initially focused on referring patients to high-volume providers.12
Data for this study come from U.S. hospitals participating in the Vermont Oxford Network (VON) during 1994–2000. The VON is a voluntary, collaborative network of NICUs organized to improve the effectiveness, efficiency, and safety of care for newborns and their families through a coordinated program of randomized trials, outcomes research, education, and quality improvement projects. Because individual hospitals’ data are used only for research and quality improvement and never publicly reported, hospitals in the VON have little incentive to misreport their data. The VON has NICUs in forty-nine states and twenty-two countries. Advantages of the VON data. The data collected by the VON have many advantages for comparing quality of care across hospitals. First, the network database includes detailed, uniform clinical and treatment information on all infants weighing 501–1,500 grams at birth born at member institutions. We excluded infants born elsewhere and later transferred to VON hospitals because the hospital of birth has been shown to be the most important factor in infant survival.13 The second advantage is that the VON database monitors morbidity outcomes such as nosocomial infection, intraventricular hemorrhage, and chronic lung disease and follows each infant through all subsequent transfers to determine survival. Although a complete analysis of quality of care would necessarily incorporate morbidity of surviving infants, we focus on survival as the primary outcome of interest for simplicity and because survival has been found to be correlated with other morbidity measures at the hospital level.14 A final advantage is that the VON database includes detailed infant characteristics collected at the time of birth for the purposes of risk adjustment that are not generally available in discharge data, including exact birthweight; gestational age; one-minute APGAR (activity, pulse, grimace, appearance, respiration) score; and indicators for race, sex, multiple birth, major birth defects, vaginal delivery, and prenatal care. The VON risk-adjustment model using these variables compares well with physiologically based measures such as the Score for Neonatal Acute Physiology (SNAP) score, with an area under the Receiver Operating Characteristic (ROC) curve of 0.89 in these data.15 Sample size. By 2000 the VON database included approximately half of all very-low-birthweight infants born in the United States and 40 percent of NICUs. Although our sample is fairly representative of the general population, some important differences must be kept in mind. Our data are limited to hospitals with a NICU and do not capture any of the variation in quality of care between hospitals with and without a NICU. In addition, VON members tend to have larger NICUs, which is why the VON represents a larger fraction of infants than NICUs. In particular, small NICUs admitting fewer than ten very-low-birthweight infants annually are underrepresented among network hospitals, and mortality may be particularly high in such hospitals. Finally, VON hospitals participate in ongoing quality improvement activities that may reduce the variation in patient outcomes across hospitals. Thus, if anything, the variation in patient outcomes observed within the network may understate the variation that would be observed across all hospitals.
Large differences in quality of neonatal intensive care across hospitals exist even after differences in case-mix are adjusted for. The data in Exhibit 1  represent ninety hospitals in the VON database that were continuously enrolled during 1994–1999. Conventional estimates of hospital mortality rates from a single year of data tend to overstate the amount of variation because of the relatively small numbers of very-low-birthweight infants born at each hospital (for example, the median hospital shown in the exhibit admitted seventy-eight such infants annually). Therefore, we used estimates and confidence intervals for twenty-eight-day mortality rates that pool information from all years 1994–1999 and are adjusted for reliability using a hierarchical method.16
Mortality rates. The risk-adjusted mortality rate for the average hospital is 11.4 percent. However, there is nearly a threefold difference in mortality rates. Ten percent of hospitals have rates in excess of 14.5 percent, half again as high as the mean mortality rate. These estimates are fairly precise, with standard errors in the range of one to two percentage points of the mean. Fifteen of the hospitals with above-average mortality had significantly higher mortality than thirteen of the hospitals with below-average mortality rates; we denote these, respectively, as high- and low-mortality hospitals. Low-mortality hospitals were those ranked 1–7, 9–12, 14, and 17; high-mortality hospitals were those ranked 75–78 and 80–90. Analyses of these differences over time demonstrate that they are persistent into future years: Risk-adjusted twenty-eight-day mortality rates in 2000 were 15.1 percent among high-mortality hospitals, 6.7 percent among low-mortality hospitals, and 11.4 percent among the remaining hospitals. Thus, some hospitals appear to have consistently lower quality than others.
Implications of large variation.
This large variation in outcomes suggests that infant mortality can be reduced by improving the care of infants treated in hospitals with poor outcomes. Based on the hospitals plotted in Exhibit 1 Reducing racial disparities. Improving the care of infants treated in hospitals with poor outcomes may be particularly effective for reducing mortality among black infants, potentially reducing racial disparities in childhood mortality. Black infants are more likely than white infants to be of very low birthweight (3.1 percent versus 1.2 percent), so that any improvement in care for very-low-birthweight infants disproportionately reduces mortality among blacks. In the VON data, risk-adjusted mortality rates of black and white infants display similar variation across hospitals and are highly correlated (correlation >0.9 after adjusting for small-sample noise using hierarchical methods), which suggests that black infants will benefit at least as much as white infants from a policy that improves care for infants treated at high-mortality hospitals. In fact, black infants may benefit more because they are more likely to be born in such hospitals. In the most recent year of VON data (2000), half of black infants were born in a hospital with above-average risk-adjusted mortality, compared with 42 percent of white infants.
The evidence. There is growing evidence for the effectiveness of collaborations for quality improvement.17 For instance, a longitudinal study by Gerald O’Connor and colleagues found that a multidisciplinary team approach including audit and feedback, quality improvement training, and site visits reduced mortality rates from coronary artery bypass surgery.18 While collaborations that focus on very-low-birthweight infants have not been evaluated for their influence on mortality, they have been shown to be effective in achieving clinical goals for improvement and reducing treatment costs. A longitudinal study of the units in a VON collaborative demonstrated reductions in nosocomial infection and chronic lung disease reduced overall costs of care.19 A cluster-randomized trial of another VON collaborative designed to promote evidence-based surfactant therapy for preterm infants demonstrated that multidisciplinary teams exposed to the intervention treated infants with surfactant much sooner after birth than teams in a control group.20 The impact. Despite these important successes, most observers would probably agree with the conclusion of a recent survey "that the quality improvement movement in health care has not had the impact that many advocates and observers hoped for."21 Aside from the evidence from a few select collaboratives discussed above, there is little evidence in the peer-reviewed literature that quality improvement has had an impact on clinical outcomes.22 Even in the VON, wide variations in patient outcomes continue to persist across hospitals. However, most member hospitals have not yet participated in collaboratives. One reason for this limited success may be hospitals’ lack of strong incentives to improve quality. Policies that link patient referral and reimbursement to measures of provider quality, such as those being promoted by the Leapfrog Group and the Robert Wood Johnson Foundation’s Rewarding Results project, may have the additional benefit of increasing the participation in and effectiveness of quality improvement efforts.23
An alternative approach to lowering mortality rates for very-low-birthweight infants is for them to be cared for in higher-quality hospitals. This can be accomplished by moving infants to better hospitals or by public reporting of quality information that allows consumers to choose better hospitals.
Moving infants to better hospitals.
Basing referral on hospital characteristics.
The success of selective referral depends critically on our ability to reliably identify the "best" hospitals. In theory, the best hospitals could be identified based on their rates of adverse events among large samples of recently delivered infants, after perfectly adjusting for differences across hospitals in patient risk. In practice, because of the current lack of data and small samples of infants at each hospital, evidence-based referral decisions have been based on hospital characteristics (volume and level of the NICU) associated with better outcomes. For example, the Leapfrog Group’s evidence-based hospital referral standard in 2002–2003 required that infants who have an expected birthweight of less than 1,500 grams or a gestational age of less than thirty-two weeks or who have correctable major birth defects should be delivered at a regional NICU with an average daily census of fifteen or more.24 Unfortunately, while patient volume and other hospital characteristics are easily obtainable and are significantly related to patient outcomes in a statistical sense, they explain little of the variation in mortality across hospitals. This fact is apparent in Exhibit 2
Basing referral on past mortality rates. Not surprisingly, rankings based on past mortality rates at each hospital outperformed rankings based on volume and level of the NICU in their ability to prospectively identify mortality differences across hospitals. In comparison to volume, rankings based on past mortality forecasted far more of the hospital-level variation in mortality (34 percent versus 1 percent) and identified hospitals with larger and more statistically significant differences in mortality. In particular, the difference in mortality between hospitals ranked in the best and worst quintiles was more than five times larger when hospitals were ranked on past mortality rates (9 percent versus 19 percent) than when they were ranked on past volume (13 percent versus 15 percent).26 Relative impact. This type of evidence suggests that selective-referral strategies that rely on indirect quality indicators such as patient volume are likely to have little impact on mortality among very-low-birthweight infants. In contrast, more lives could be saved if referrals were based on infant outcome data, as routinely collected by the VON. Compared with using volume or NICU level, referrals based on past mortality identify hospitals (both good and bad) with larger differences in mortality and potentially affect more patients (since low-volume hospitals by definition serve few infants). In our earlier work with the VON data, we estimated that a referral strategy that moved all infants out of the lowest-ranked 20 percent and into the middle 60 percent of VON hospitals in 1999–2000 would have reduced the number of deaths among very-low-birthweight infants by 0.5 percent based on a historical volume standard, compared with 4.6 percent based on historical mortality experience. A more aggressive referral strategy that moved all infants into the best 20 percent of VON hospitals would result in larger total reductions in mortality (4.2 percent using volume, 34.2 percent using historical mortality) but similar conclusions regarding the superiority of using outcome data as the basis for referrals.
Availability of better hospitals.
Better hospitals are likely to be available within a reasonable distance of infants treated in high-mortality hospitals. However, it is impossible to answer this question directly because the VON data only cover a subset of infants and hospitals. In the VON data, there are no significant differences in mortality across census regions or states, which suggests that all of the variation in hospital quality occurs within areas rather than between them. Thus, the variation in mortality seen in Exhibit 1 Evaluating selective referral. Evaluating any actual referral strategy, of course, would be far more complicated. Such a strategy would be limited to pregnant women known to be at risk. Thus, one could not expect to move all infants out of the highest-mortality hospitals. On the other hand, such a standard would presumably be applied to hospitals outside of the VON, including many small hospitals without NICUs, and this would presumably lead to more lives saved. Moreover, moving patients among providers may generate trade-offs in family disruptions and potentially raise treatment costs. Little is known about how large such costs are likely to be, although a simulation of the impact of referring children undergoing cardiac surgery to high-volume hospitals found that this strategy could reduce mortality with only minimal increases in travel distances.27 Public reporting. Even without a formal selective-referral strategy in place, public reporting of hospital rankings could affect patient outcomes. Public reporting of patient mortality rates could place competitive pressure on all providers to improve, through hospitals’ concerns (whether real or imagined) that poor rankings will influence their ability to attract patients and staff. This competitive pressure could work to improve outcomes at all hospitals, even if the proportion of patients going to the best hospitals does not change. However, such high-stakes accountability also has a downside, as providers have increased incentives to distort their mortality rates through patient selection or misrepresenting their patient mix. There is some evidence that surgeons stopped doing coronary artery bypass graft surgery on certain high-risk patients because of pressures to improve their mortality rates, and, as a result, overall mortality rose for these patients.28 Thus, while unlikely to be as large as the direct effects of selective referral, the indirect effect of competitive pressure on patient outcomes is an open question. Practical hurdles. Selective-referral strategies face a number of practical hurdles that have so far limited their effectiveness. Outside of collaborative voluntary networks such as the VON, there is a paucity of good data from which to create measures of risk-adjusted outcomes. Thus, while organizations such as Leapfrog aim to use risk-adjusted patient outcome measures as the basis for selective referral, they must rely on cruder measures until better data become available. Another practical hurdle is that such a system inevitably involves public reporting of hospital performance measures, and little is known about how such information should be presented. There is some evidence that patients have misunderstood or ignored such reports in the past, and much work remains to be done.29 It is not yet known how to best improve care for infants now being served by hospitals with poor outcomes. Both the collaborative quality improvement and selective-referral approaches face important practical challenges. Perhaps the most important one is the development of reliable performance measures. Many current performance measures are either weakly related to patient outcomes (as in the case of volume) or unreliable because they are based on small samples of patients (as in the case of mortality rates at small hospitals). Further, reliable performance measures must be based on detailed and accurate data such as those collected by the VON.30 Not only are such data costly to collect, but their accuracy depends on providers’ incentives to misrepresent their data or to select patients to improve their measured performance.31 Nonetheless, the collaborative and selective-referral approaches hold promise for improving U.S. infant mortality rates. Unless there are new developments in neonatal technology or other improvements in perinatal care, there are unlikely to be further improvements in mortality for very-low-birthweight infants such as those observed in earlier decades. It will remain a major challenge for U.S. policy-makers to determine how to further reduce infant mortality.
Jeannette Rogowski (rogowsje{at}umdnj.edu) is University Professor in Health Economics at the University of Medicine and Dentistry of New Jersey (UMDNJ) School of Public Health in New Brunswick, New Jersey. Douglas Staiger is a professor in the Department of Economics at Dartmouth College in Hanover, New Hampshire, and is a research associate with the National Bureau of Economic Research. Jeffrey Horbar is a professor in the Department of Pediatrics at the University of Vermont College of Medicine and Vermont Oxford Network in Burlington. This research was funded by Grant nos. R01 HS13371 and P01 HS10858P from the Agency for Healthcare Research and Quality. The authors thank Jason Wang for outstanding research assistance. They also acknowledge the institutions that participate in the Vermont Oxford Network Database, without whose work this research would not have been possible.
This article has been cited by other articles:
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Top
Study Data And Methods
The Potential For Reducing...
