|
U.S. Adoption Of Computerized Physician Order Entry Systems
Computerized physician order entry (CPOE) has been shown to reduce preventable, potential adverse events. Despite this evidence, fewer than 5 percent of U.S. hospitals have fully implemented these systems. We assess empirically alternative reasons for low CPOE implementation using data from various sources. We find that CPOE is related to hospital ownership and teaching status; government and teaching hospitals are much more likely than other hospital types are to invest in CPOE. Hospital profitability is not associated with CPOE investment. Although greater diffusion of CPOE is needed, it might have to await continuing publicity efforts and substantial reimbursement system changes.
Recent advances in information technology provide hospitals with new opportunities to reduce their rates of medication errors. Computerized physician order entry (CPOE) systems—information systems that allow physicians to enter orders electronically—have been shown to reduce potential errors. Depending on the system, they can integrate orders with other patient information, allow physicians easy access to medical guidelines and cost information, process in-hospital orders to pharmacies, and create legible prescriptions. Studies show that CPOE has improved physicians’ prescribing practices.1 In fact, estimates of CPOE’s effect on error prevention range as high as 80 percent, although some studies have identified overlooked, technology-related errors that would lower estimated effects.2 Based on the potential to improve quality, national organizations such as the Institute of Medicine, Joint Commission on Accreditation of Healthcare Organizations, President’s Information Technology Advisory Committee, and Leapfrog Group have identified CPOE as an important technology for preventing medical error.3 Despite potential benefits, only an estimated 4–10 percent of U.S. hospitals have fully implemented CPOE.4 Here we examine which types of hospitals have acquired CPOE systems and why the rate is so low. We use data on CPOE ownership from the Leapfrog Group, a consortium of more than 130 Fortune 500 companies and other large private and public health care purchasers. The survey results represent Leapfrog’s effort to reward hospitals that demonstrate quality advances with positive publicity. Drawing on hypotheses in the literature, we consider two broad classes of theories for low CPOE ownership: financial theories and ownership status theories.5 Under financial theories, expense typically explains low CPOE acquisition rates. Cost estimates vary widely, from $3 million to $10 million for setup alone.6 An estimate of $8 million for setup and $1 million for annual operations translates to a per bed expense of about $32,000 initially and $4,000 ongoing for urban hospitals.7 Hospitals might not have the funds to invest in or maintain these systems. Even if they can afford these costs, it is unclear whether hospitals will profit from CPOE. There has been speculation about hospital savings from lower transcription costs, other resource uses, patient charges and hospital costs, personnel time, and malpractice costs.8 Aggregate savings, however, are both hard to measure and unproven. For example, because traditional hospital technology valuation methods do not capture all of CPOE’s benefits, scholars have argued that any valuation must include quality improvement.9 Yet quality improvements do not necessarily translate into a healthier bottom line for hospitals or physicians. Further, anecdote suggests temporary efficiency losses related to disrupted physician workflow from learning a new system.10 In an era of falling hospital revenues, with many hospitals operating in the red, investments of this scale could simply be unaffordable. The second set of theories broadly relates to hospital organization. Some hospital types might place more weight on quality improvement than others. Physicians in teaching hospitals might, for example, be more committed to innovation than other physicians. Literature examining the link between hospital ownership and quality has produced inconsistent results, with some studies finding nonprofit hospitals relatively innovative and others not, maybe because different technologies were studied, not all of which should optimally be acquired by each hospital.11 CPOE systems are an excellent test of these theories because of current recommendations that all hospitals acquire them.
CPOE investment. Data on CPOE ownership are from the Leapfrog Group’s Hospital Patient Safety Survey, Version 1 (2002 through April 2003), a compilation of hospital-reported progress on CPOE implementation. The survey included 1,189 observations, of which 937 were invited to respond and the remainder represented unsolicited respondents. Leapfrog surveyed hospitals in twenty-two geographic regions, which included metropolitan areas, regions within states, and states. These areas were chosen because Leapfrog members—employers and employer groups—approached Leapfrog to include their areas in the survey. Leapfrog created a list of all hospitals within the selected regions, and a representative contacted each hospital’s chief executive officer to urge participation, repeating the request if the hospital failed to participate after the first contact. Hospitals were aware of the public recognition they would receive for following Leapfrog’s suggested safety practices and were informed that they would be labeled as "nonresponders" if they failed to answer the survey. The majority of Leapfrog’s survey participants came from the targeted regions; however, Leapfrog welcomed participation by all hospitals. We excluded unsolicited respondents to mitigate self-selection bias. After we merged the Leapfrog data with other data sources, as described below, the final sample included 751 hospital observations from 19 states.12 Response rates were high; nearly half of the states had a response rate of more than 70 percent. More than 60 percent of the hospitals that were targeted and for which we have data come from California (260 hospitals), New York (114), and New Jersey (72). Although there is no a priori reason to believe that these hospitals are different from those in other states, one should be cautious in extending our results to all U.S. hospitals.
We generated dependent variables using the CPOE implementation levels as defined by Leapfrog. A hospital responding to the survey received one of four possible scores measuring its progress toward fulfilling the CPOE safety standard. We paraphrase the categories as willing to report publicly but not yet meeting criteria for a good early-stage effort, good early-stage effort, good progress, or full implementation (Exhibit 1
Explanatory variables. To measure financial status, we calculated each hospital’s net income per admission, using data from the 2000 Medicare Cost Reports. Using data from the 2000 American Hospital Association (AHA) annual survey, we also included a dummy variable to represent whether hospitals were system members. We believed that system membership might be important since systems have the ability to pool resources across institutions, take advantage of economies of scale, and possibly access debt markets relatively quickly.13 When hospital system membership was missing, we generated a dummy variable equal to 1 if the system membership variable was missing and to 0 otherwise. We also measured organizational structure by ownership type: nonprofit (religious, secular), for-profit, or government (city, state, county) owned. The distribution of hospital types in our sample (18 percent for-profit, 11 percent government, and 72 percent nonprofit) is roughly representative of distribution of types in urban areas, although our final sample excluded federal hospitals. Further, we included a dummy variable for teaching hospital status, defined as membership in the Council of Teaching Hospitals (COTH). To address other possible biases from omitted factors, we controlled for the number of hospital admissions and, to control for any nonlinear effect in admissions, its square. Finally, we adjusted for heteroskedasticity and accounted for correlations in local markets by clustering standard errors by Hospital Referral Regions (HRRs).
Exhibit 1  reports unadjusted CPOE status by hospital ownership and profit type. Contrary to our expectations, government hospitals are more likely than private nonprofit or for-profit hospitals to report full implementation or good progress on CPOE. These raw tabulations also hint that money-losing hospitals are also more likely to have invested in CPOE systems. In the regression analysis that follows, we investigate these correlations more thoroughly with controls.
Exhibit 2
Exhibit 3  reports odds ratios from logistic regressions relating CPOE investment to hospital financial status, ownership, and other factors. The first column examines which hospitals report data on CPOE. For-profit hospitals were less likely than nonprofit hospitals to respond to the Leapfrog survey. The 0.922 odds ratio for government ownership, although indicating a slightly lower reporting rate than nonprofits, is not statistically significantly different from 1. Reporting status is also correlated with teaching status and hospital size, although it is not correlated with either hospital profitability or membership in a system.
The next three columns in Exhibit 3 examine the link between hospital characteristics and progress in implementing CPOE. The progress levels follow Leapfrog criteria: level of CPOE implementation, at least a good early-stage effort, and at least good progress. We estimated the model as an ordered logit and report the results as odds ratios. Column 2 shows that given any level of investment, government status is correlated with greater investment than nonprofit status, and for-profit ownership is correlated with less investment than nonprofit status. In columns 2–4, the sample is limited to hospitals that report data on CPOE. Controlling for other factors, teaching hospitals are roughly three times as likely as nonteaching hospitals to satisfy the requirements for a "good early-stage effort" (column 3) or "good progress" (column 4). Among the control variables, larger hospitals are more likely than others to invest in CPOE systems. These results are both unsurprising and consistent with previous findings.14 In the case of system membership, the odds ratios for both membership variables (system membership and the dummy variable marking missing membership) are insignificant, and many are quite close to 1. In none of these models is net income per admission related to CPOE implementation, as can be seen by the fact that the odds ratios are all equal to 1.
Hospital ownership, however, is strongly correlated with CPOE (Exhibit 3 We ran sensitivity tests as a check on our claims that ownership, rather than unobserved characteristics related to the geographic area in which hospitals operate, explain these results. As discussed above, to account for this effect, we clustered the observations by HRR. The results generally retained statistical significance (data not shown). The odds ratios were statistically significant, at least at the 10 percent level for government ownership (CPOE, ordered logit p = .024; some CPOE, p = .009; good CPOE, p = .076). The results for for-profit hospitals were somewhat weaker (CPOE, ordered logit p = .070; some CPOE, p = .113; good CPOE, p = .032). We further tested the alternative hypothesis that location, rather than ownership, explains why hospitals implement CPOE, by using a fixed-effects approach with an indicator variable for the HRR in which each hospital operates. The differences among hospital types were quite similar to the previous model, confirming our hypotheses. Because so many HRRs included in the test perfectly predicted CPOE implementation, however, the significance tests were not reliable. We conducted a final set of sensitivity tests as a check on respondent bias. We repeated our analyses on the full sample of targeted hospitals, assuming that nonreporting hospitals were unlikely to have implemented CPOE. This is a reasonable assumption, given that several attempts were made to contact nonresponding hospitals, and hospitals had an interest in publicizing their efforts to implement CPOE. The magnitude of the coefficients was very similar with this change as in the results we present, although the standard errors were naturally larger, given the noise introduced by imputation (data not shown).
There is much speculation about the factors leading hospitals to invest in particular technologies. One hypothesis is the "resources" explanation: Hospital investment is determined primarily by uncommitted money to invest in projects. In this setting, one would expect positive net income or system ownership to matter for investment. Our results refute this explanation. Neither net income nor system membership is positively associated with CPOE investment. In contrast, we find a role for hospital ownership in explaining CPOE investment. Government hospitals are the most likely and for-profits the least likely, to invest in CPOE systems; nonprofits are in the middle. The differences among ownership forms are consistent with other evidence concerning technologies that are not perceived to generate profits.15 Indeed, in a case study of five sites that implemented CPOE, the authors found that administrators did not expect financial benefits from CPOE; in fact, several did not expect CPOE to pay for itself.16 Hospitals that invest in CPOE might place more value on quality than other hospital types. The relatively high level of CPOE investment by government hospitals is perhaps the most surprising result, particularly since the government sample was composed entirely of community hospitals, not federal hospitals such as Veterans Affairs hospitals, which have their own computerized patient record system.17 There are several possible explanations for the relatively high rate of government investment. First, government hospitals might have the most to gain by implementing CPOE systems. They might have the sickest patients and could benefit disproportionately from the outcome improvements. Alternatively, political interest in clinical safety might explain public hospital leaders’ willingness to invest in these systems. For example, several states have proposed or passed mandates requiring hospitals to address medication-related errors, adopt CPOE, or publicize such efforts.18 Although these requirements typically apply to all hospitals, public interest may differentially motivate public hospitals to acquire technology. Another explanation involves the technology itself. Perhaps the existing computer systems at government hospitals are more amenable to currently available CPOE products. Large hospital chains often contract with vendors to develop their own CPOE systems, a long process that might explain why government hospitals have moved toward early implementation. A final explanation is that there are different sources of decision-making authority among hospital types. Many physicians are opposed to CPOE systems, believing they are complicated to use or diminish the clinical experience.19 Perhaps physicians at public institutions are more involved in selecting vendors or developing systems than physicians at other hospital types. It may be that physicians are sufficiently powerful to prevent their adoption at private hospitals but not sufficiently powerful to delay their adoption at public institutions. Study limitations. Because the study relies on Leapfrog data, the results are necessarily limited by their quality. One such limitation is that Leapfrog’s membership suggested the survey sites. The sites, however, represent a wide distribution of cities and states across the country, making it unlikely that the site selection biased the results. Because the data set included only urban hospitals, the results cannot be generalized to rural areas, which have comparatively lower rates of computer-based patient record adoption than urban areas.20 That not all targeted hospitals responded creates another potential source of bias. However, response rates were high, and our quantitative tests to determine whether the results were undermined by selection bias were reassuring. Conversations with Leapfrog Group personnel, relying on unpublished work, confirmed our conclusion that there was little responder bias.21 In addition, California hospitals constitute a large share of the sample and operate in a unique regulatory environment. State law enacted in September 2000 requires hospitals to submit plans to reduce medication-related errors, including technological solutions.22 The plan submission deadline was 1 January 2002, with implementation no later than 1 January 2005. The influence of this requirement on California hospitals’ CPOE compliance before the Leapfrog survey (2002–03) is unclear, although the data show that California hospitals, which include a disproportionate share of government hospitals, were less likely than other hospitals to comply. It is likely, however, that the high share of California hospitals causes an understatement of our findings, if it has any effect at all. Finally, the hospital data were self-reported. We do not believe, however, that hospitals were likely to misrepresent efforts to implement CPOE because the survey results are both simple to understand and publicly available. Further, Leapfrog’s business membership has an interest in reviewing the survey results. Concluding comments. The most troubling feature of our results is the low rate of overall investment in CPOE. Despite many calls for implementation, only 4 percent of hospitals are in full compliance with CPOE standards, and only 17 percent have made good progress. Our results suggest that access to funds is not the only factor involved. Even if all hospitals were in the black, CPOE investment might not be much higher than it is now because hospital executives might worry that the technology would not benefit their hospitals financially. Two others issues are important. The first is diffusion of information about the benefits of CPOE. The only group for which CPOE ownership is fairly large is teaching hospitals, where one-third of institutions have made at least good progress toward implementation. These results suggest that information about CPOE’s benefits has not spread widely enough among key physicians or hospital decisionmakers outside of teaching institutions. Further, more study is needed to see if the benefits of CPOE adoption vary at different types of hospitals. The second is the importance of making patient safety central to hospitals’ missions. Although policymakers can change the information flow to hospitals, they cannot readily change hospitals’ missions. Thus, in the current environment, it could take a long while for CPOE systems to diffuse broadly throughout the health care system. Other research has shown, however, that hospitals do respond to financial incentives.23 Changing the reimbursement environment to favor adoption of CPOE systems with targeted funding, as Congress has considered, could offer a short-term solution to increasing investment in these systems and improving patient safety.24
David Cutler is a professor of economics at Harvard University in Cambridge, Massachusetts. Naomi Feldman is an assistant professor of economics at Ben-Gurion University in Be’erSheva, Israel. Jill Horwitz (jrhorwit{at}umich.edu) is an assistant professor of law at the University of Michigan Law School in Ann Arbor. The authors thank Mary Beth Landrum, George Williams, Catherine Eikel of the Leapfrog Group, and four anonymous reviewers for helpful comments on an earlier draft. They are grateful to Rob Huckman for data and advice; to Paula Payton for administrative assistance; and for research support from the National Institute on Aging, the Harvard Program for Health System Improvement, and the University of Michigan Law School John M. Olin Center for Law and Economics.
This article has been cited by other articles:
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Top
Study Data And Methods
Study Results
