Health Affairs, 24, no. 6 (2005): 1654-1663 doi: 10.1377/hlthaff.24.6.1654 © 2005 by Project HOPE	New Online   McAllen, TX & Beyond: An Expert Roundtable   Geography & Reform   Medicaid or Insurance Exchange?   Siren Song of New GME   Public Plan Option: Pro & Con

This Article Abstract Reprint (PDF) Submit a response to this article Alert me when this article is cited Alert me when Comments are posted Alert me if a correction is posted Services E-mail this article to a friend Similar articles in this journal Similar articles in ISI Web of Science Similar articles in PubMed Alert me to new issues of the journal Add to My Personal Archive Download to Citation Manager Reprints & Permissions Citing Articles Citing Articles via HighWire Citing Articles via ISI Web of Science (23) Citing Articles via Google Scholar Google Scholar Articles by Cutler, D. M. Articles by Horwitz, J. R. Search for Related Content PubMed PubMed Citation Articles by Cutler, D. M. Articles by Horwitz, J. R. Related Collections Hospitals Legal/Regulatory Issues Business Of Health Physicians Research And Technology Health Information Technology

DataWatch

U.S. Adoption Of Computerized Physician Order Entry Systems

Abstract

 
Computerized physician order entry (CPOE) has been shown to reduce preventable, potential adverse events. Despite this evidence, fewer than 5 percent of U.S. hospitals have fully implemented these systems. We assess empirically alternative reasons for low CPOE implementation using data from various sources. We find that CPOE is related to hospital ownership and teaching status; government and teaching hospitals are much more likely than other hospital types are to invest in CPOE. Hospital profitability is not associated with CPOE investment. Although greater diffusion of CPOE is needed, it might have to await continuing publicity efforts and substantial reimbursement system changes.

Despite potential benefits, only an estimated 4–10 percent of U.S. hospitals have fully implemented CPOE.⁴ Here we examine which types of hospitals have acquired CPOE systems and why the rate is so low. We use data on CPOE ownership from the Leapfrog Group, a consortium of more than 130 Fortune 500 companies and other large private and public health care purchasers. The survey results represent Leapfrog’s effort to reward hospitals that demonstrate quality advances with positive publicity.

Drawing on hypotheses in the literature, we consider two broad classes of theories for low CPOE ownership: financial theories and ownership status theories.⁵ Under financial theories, expense typically explains low CPOE acquisition rates. Cost estimates vary widely, from $3 million to $10 million for setup alone.⁶ An estimate of $8 million for setup and $1 million for annual operations translates to a per bed expense of about $32,000 initially and $4,000 ongoing for urban hospitals.⁷ Hospitals might not have the funds to invest in or maintain these systems. Even if they can afford these costs, it is unclear whether hospitals will profit from CPOE. There has been speculation about hospital savings from lower transcription costs, other resource uses, patient charges and hospital costs, personnel time, and malpractice costs.⁸

Aggregate savings, however, are both hard to measure and unproven. For example, because traditional hospital technology valuation methods do not capture all of CPOE’s benefits, scholars have argued that any valuation must include quality improvement.⁹ Yet quality improvements do not necessarily translate into a healthier bottom line for hospitals or physicians. Further, anecdote suggests temporary efficiency losses related to disrupted physician workflow from learning a new system.¹⁰ In an era of falling hospital revenues, with many hospitals operating in the red, investments of this scale could simply be unaffordable.

The second set of theories broadly relates to hospital organization. Some hospital types might place more weight on quality improvement than others. Physicians in teaching hospitals might, for example, be more committed to innovation than other physicians. Literature examining the link between hospital ownership and quality has produced inconsistent results, with some studies finding nonprofit hospitals relatively innovative and others not, maybe because different technologies were studied, not all of which should optimally be acquired by each hospital.¹¹ CPOE systems are an excellent test of these theories because of current recommendations that all hospitals acquire them.

Study Data And Methods

Top Study Data And Methods Study Results Discussion And Policy... NOTES

 
CPOE investment. Data on CPOE ownership are from the Leapfrog Group’s Hospital Patient Safety Survey, Version 1 (2002 through April 2003), a compilation of hospital-reported progress on CPOE implementation. The survey included 1,189 observations, of which 937 were invited to respond and the remainder represented unsolicited respondents.

Leapfrog surveyed hospitals in twenty-two geographic regions, which included metropolitan areas, regions within states, and states. These areas were chosen because Leapfrog members—employers and employer groups—approached Leapfrog to include their areas in the survey. Leapfrog created a list of all hospitals within the selected regions, and a representative contacted each hospital’s chief executive officer to urge participation, repeating the request if the hospital failed to participate after the first contact. Hospitals were aware of the public recognition they would receive for following Leapfrog’s suggested safety practices and were informed that they would be labeled as "nonresponders" if they failed to answer the survey.

The majority of Leapfrog’s survey participants came from the targeted regions; however, Leapfrog welcomed participation by all hospitals. We excluded unsolicited respondents to mitigate self-selection bias. After we merged the Leapfrog data with other data sources, as described below, the final sample included 751 hospital observations from 19 states.¹² Response rates were high; nearly half of the states had a response rate of more than 70 percent. More than 60 percent of the hospitals that were targeted and for which we have data come from California (260 hospitals), New York (114), and New Jersey (72). Although there is no a priori reason to believe that these hospitals are different from those in other states, one should be cautious in extending our results to all U.S. hospitals.

We generated dependent variables using the CPOE implementation levels as defined by Leapfrog. A hospital responding to the survey received one of four possible scores measuring its progress toward fulfilling the CPOE safety standard. We paraphrase the categories as willing to report publicly but not yet meeting criteria for a good early-stage effort, good early-stage effort, good progress, or full implementation (Exhibit 1). Forty-two percent responded to the survey yet did not meet the criteria for a good early-stage effort, 9.5 percent responded to the survey and have made good-early stage efforts, 11 percent responded to the survey and have made good progress in starting to implement CPOE, and the remaining 2.7 percent responded to the survey and have fully implemented CPOE. Of the 751 hospitals for which we have complete data, 264 (35 percent) did not respond to the Leapfrog survey. The fraction of hospitals that did not respond is fairly constant across hospital ownership types, which suggests that there is little bias from differential reporting by ownership type or financial status (Exhibit 1).

We also measured organizational structure by ownership type: nonprofit (religious, secular), for-profit, or government (city, state, county) owned. The distribution of hospital types in our sample (18 percent for-profit, 11 percent government, and 72 percent nonprofit) is roughly representative of distribution of types in urban areas, although our final sample excluded federal hospitals.

Further, we included a dummy variable for teaching hospital status, defined as membership in the Council of Teaching Hospitals (COTH). To address other possible biases from omitted factors, we controlled for the number of hospital admissions and, to control for any nonlinear effect in admissions, its square. Finally, we adjusted for heteroskedasticity and accounted for correlations in local markets by clustering standard errors by Hospital Referral Regions (HRRs).

Study Results

Top Study Data And Methods Study Results Discussion And Policy... NOTES

 
Exhibit 1 reports unadjusted CPOE status by hospital ownership and profit type. Contrary to our expectations, government hospitals are more likely than private nonprofit or for-profit hospitals to report full implementation or good progress on CPOE. These raw tabulations also hint that money-losing hospitals are also more likely to have invested in CPOE systems. In the regression analysis that follows, we investigate these correlations more thoroughly with controls.

Exhibit 2 reports summary statistics for the independent variables, separated by hospital ownership status and profitability. As in the general population, for-profit hospitals have fewer admissions than private nonprofit and government hospitals and are less likely to be teaching hospitals. Based on income per hospital admission, for-profit hospitals are the most likely and government hospitals, the least likely, to be profitable.

Controlling for other factors, teaching hospitals are roughly three times as likely as nonteaching hospitals to satisfy the requirements for a "good early-stage effort" (column 3) or "good progress" (column 4). Among the control variables, larger hospitals are more likely than others to invest in CPOE systems. These results are both unsurprising and consistent with previous findings.¹⁴ In the case of system membership, the odds ratios for both membership variables (system membership and the dummy variable marking missing membership) are insignificant, and many are quite close to 1. In none of these models is net income per admission related to CPOE implementation, as can be seen by the fact that the odds ratios are all equal to 1.

Hospital ownership, however, is strongly correlated with CPOE (Exhibit 3, columns 2–4). Government hospitals are consistently more likely than others to report higher levels of CPOE: Controlling for all other factors, they are almost three times as likely as nonprofit hospitals and seven times as likely as for-profit hospitals to satisfy the requirements for a "good early-stage effort" and, although the results are not statistically significant, more than twice as likely to have made good or full compliance.

We ran sensitivity tests as a check on our claims that ownership, rather than unobserved characteristics related to the geographic area in which hospitals operate, explain these results. As discussed above, to account for this effect, we clustered the observations by HRR. The results generally retained statistical significance (data not shown). The odds ratios were statistically significant, at least at the 10 percent level for government ownership (CPOE, ordered logit p = .024; some CPOE, p = .009; good CPOE, p = .076). The results for for-profit hospitals were somewhat weaker (CPOE, ordered logit p = .070; some CPOE, p = .113; good CPOE, p = .032).

We further tested the alternative hypothesis that location, rather than ownership, explains why hospitals implement CPOE, by using a fixed-effects approach with an indicator variable for the HRR in which each hospital operates. The differences among hospital types were quite similar to the previous model, confirming our hypotheses. Because so many HRRs included in the test perfectly predicted CPOE implementation, however, the significance tests were not reliable.

We conducted a final set of sensitivity tests as a check on respondent bias. We repeated our analyses on the full sample of targeted hospitals, assuming that nonreporting hospitals were unlikely to have implemented CPOE. This is a reasonable assumption, given that several attempts were made to contact nonresponding hospitals, and hospitals had an interest in publicizing their efforts to implement CPOE. The magnitude of the coefficients was very similar with this change as in the results we present, although the standard errors were naturally larger, given the noise introduced by imputation (data not shown).

Discussion And Policy Implications

Top Study Data And Methods Study Results Discussion And Policy... NOTES

 
There is much speculation about the factors leading hospitals to invest in particular technologies. One hypothesis is the "resources" explanation: Hospital investment is determined primarily by uncommitted money to invest in projects. In this setting, one would expect positive net income or system ownership to matter for investment. Our results refute this explanation. Neither net income nor system membership is positively associated with CPOE investment.

In contrast, we find a role for hospital ownership in explaining CPOE investment. Government hospitals are the most likely and for-profits the least likely, to invest in CPOE systems; nonprofits are in the middle. The differences among ownership forms are consistent with other evidence concerning technologies that are not perceived to generate profits.¹⁵ Indeed, in a case study of five sites that implemented CPOE, the authors found that administrators did not expect financial benefits from CPOE; in fact, several did not expect CPOE to pay for itself.¹⁶ Hospitals that invest in CPOE might place more value on quality than other hospital types.

The relatively high level of CPOE investment by government hospitals is perhaps the most surprising result, particularly since the government sample was composed entirely of community hospitals, not federal hospitals such as Veterans Affairs hospitals, which have their own computerized patient record system.¹⁷ There are several possible explanations for the relatively high rate of government investment. First, government hospitals might have the most to gain by implementing CPOE systems. They might have the sickest patients and could benefit disproportionately from the outcome improvements. Alternatively, political interest in clinical safety might explain public hospital leaders’ willingness to invest in these systems. For example, several states have proposed or passed mandates requiring hospitals to address medication-related errors, adopt CPOE, or publicize such efforts.¹⁸ Although these requirements typically apply to all hospitals, public interest may differentially motivate public hospitals to acquire technology.

Another explanation involves the technology itself. Perhaps the existing computer systems at government hospitals are more amenable to currently available CPOE products. Large hospital chains often contract with vendors to develop their own CPOE systems, a long process that might explain why government hospitals have moved toward early implementation.

A final explanation is that there are different sources of decision-making authority among hospital types. Many physicians are opposed to CPOE systems, believing they are complicated to use or diminish the clinical experience.¹⁹ Perhaps physicians at public institutions are more involved in selecting vendors or developing systems than physicians at other hospital types. It may be that physicians are sufficiently powerful to prevent their adoption at private hospitals but not sufficiently powerful to delay their adoption at public institutions.

Study limitations. Because the study relies on Leapfrog data, the results are necessarily limited by their quality. One such limitation is that Leapfrog’s membership suggested the survey sites. The sites, however, represent a wide distribution of cities and states across the country, making it unlikely that the site selection biased the results. Because the data set included only urban hospitals, the results cannot be generalized to rural areas, which have comparatively lower rates of computer-based patient record adoption than urban areas.²⁰

That not all targeted hospitals responded creates another potential source of bias. However, response rates were high, and our quantitative tests to determine whether the results were undermined by selection bias were reassuring. Conversations with Leapfrog Group personnel, relying on unpublished work, confirmed our conclusion that there was little responder bias.²¹

In addition, California hospitals constitute a large share of the sample and operate in a unique regulatory environment. State law enacted in September 2000 requires hospitals to submit plans to reduce medication-related errors, including technological solutions.²² The plan submission deadline was 1 January 2002, with implementation no later than 1 January 2005. The influence of this requirement on California hospitals’ CPOE compliance before the Leapfrog survey (2002–03) is unclear, although the data show that California hospitals, which include a disproportionate share of government hospitals, were less likely than other hospitals to comply. It is likely, however, that the high share of California hospitals causes an understatement of our findings, if it has any effect at all.

Finally, the hospital data were self-reported. We do not believe, however, that hospitals were likely to misrepresent efforts to implement CPOE because the survey results are both simple to understand and publicly available. Further, Leapfrog’s business membership has an interest in reviewing the survey results.

Concluding comments. The most troubling feature of our results is the low rate of overall investment in CPOE. Despite many calls for implementation, only 4 percent of hospitals are in full compliance with CPOE standards, and only 17 percent have made good progress. Our results suggest that access to funds is not the only factor involved. Even if all hospitals were in the black, CPOE investment might not be much higher than it is now because hospital executives might worry that the technology would not benefit their hospitals financially.

Two others issues are important. The first is diffusion of information about the benefits of CPOE. The only group for which CPOE ownership is fairly large is teaching hospitals, where one-third of institutions have made at least good progress toward implementation. These results suggest that information about CPOE’s benefits has not spread widely enough among key physicians or hospital decisionmakers outside of teaching institutions. Further, more study is needed to see if the benefits of CPOE adoption vary at different types of hospitals.

The second is the importance of making patient safety central to hospitals’ missions. Although policymakers can change the information flow to hospitals, they cannot readily change hospitals’ missions. Thus, in the current environment, it could take a long while for CPOE systems to diffuse broadly throughout the health care system. Other research has shown, however, that hospitals do respond to financial incentives.²³ Changing the reimbursement environment to favor adoption of CPOE systems with targeted funding, as Congress has considered, could offer a short-term solution to increasing investment in these systems and improving patient safety.²⁴

Editor's Notes

 
David Cutler is a professor of economics at Harvard University in Cambridge, Massachusetts. Naomi Feldman is an assistant professor of economics at Ben-Gurion University in Be’erSheva, Israel. Jill Horwitz (jrhorwit{at}umich.edu) is an assistant professor of law at the University of Michigan Law School in Ann Arbor.

The authors thank Mary Beth Landrum, George Williams, Catherine Eikel of the Leapfrog Group, and four anonymous reviewers for helpful comments on an earlier draft. They are grateful to Rob Huckman for data and advice; to Paula Payton for administrative assistance; and for research support from the National Institute on Aging, the Harvard Program for Health System Improvement, and the University of Michigan Law School John M. Olin Center for Law and Economics.

NOTES

Top Study Data And Methods Study Results Discussion And Policy... NOTES

 

1. G. Kuperman and R.F. Gibson, "Computer Physician Order Entry: Benefits, Costs, and Issues," Annals of Internal Medicine 139, no. 1 (2003): 31–39[Abstract/Free Full Text]; J.M. Teich et al., "Effects of Computerized Physician Order Entry on Prescribing Practices," Archives of Internal Medicine 160, no. 18 (2000): 2741–2747[Abstract/Free Full Text]; and R.S. Evans et al., "A Computer-Assisted Management Program for Antibiotics and Other Antiinfective Agents," New England Journal of Medicine 338, no. 4 (1998): 232–238.[Abstract/Free Full Text]
2. D.W. Bates et al., "The Impact of Computerized Physician Order Entry on Medication Error Prevention," Journal of the American Medical Informatics Association 6, no. 4 (1999): 313–321[Abstract/Free Full Text]; D.W. Bates et al., "Effect of Computerized Physician Order Entry and a Team Intervention on Prevention of Serious Medication Errors," Journal of the American Medical Association 280, no. 15 (1998): 1311–1316[Abstract/Free Full Text]; and (regarding overlooked errors) R. Koppel et al., "Role of Computerized Physician Order Entry Systems in Facilitating Medication Errors," Journal of the American Medical Association 293, no. 10 (2005): 1197–1203.[Abstract/Free Full Text]
3. K. Adams and J. Corrigan, eds., Priority Areas for National Action: Transforming Health Care Quality (Washington: National Academies Press, 2003); testimony of Dennis O’Leary, "JCAHO—Patient Safety: Instilling Hospitals with a Culture of Continuous Improvement," before the Senate Committee on Governmental Affairs, 2003; President’s Information Technology Advisory Committee, Revolutionizing Health Care through Information Technology (Washington: Executive Office of the President, June 2004); and the Leapfrog Group’s home page, www.leapfroggroup.org.
4. J.S. Ash et al., "Computerized Physician Order Entry in U.S. Hospitals: Results of a 2002 Survey," Journal of the American Medical Informatics Association 11, no. 2 (2004): 95–99.[Abstract/Free Full Text]
5. E.G. Poon et al., "Overcoming Barriers to Adopting and Implementing Computerized Physician Order Entry Systems in U.S. Hospitals," Health Affairs 23, no. 4 (2004): 184–190[Abstract/Free Full Text]; and T.A. Massaro, "Introducing Physician Order Entry at a Major Academic Medical Center: I. Impact on Organizational Culture and Behavior," Academic Medicine 68, no. 1 (1993): 20–25.[Web of Science][Medline]
6. Poon et al., "Overcoming Barriers," citing Advisory Board Company, Computerized Physician Order Entry: Lessons from Pioneering Institutions (Washington: Advisory Board Company, 2001), 11.
7. First Consulting Group, Computerized Physician Order Entry: Costs Benefits and Challenges (Long Beach, Calif.: First Consulting Group, 2003).
8. H.S. Mekhjian et al., "Immediate Benefits Realized following Implementation of Physician Order Entry at an Academic Medical Center," Journal of the American Medical Informatics Association 9, no. 5 (2002): 529–539[Abstract/Free Full Text]; Kuperman and Gibson, "Computer Physician Order Entry"; W.M. Tierney et al., "Physician Inpatient Order Writing on Microcomputer Workstations: Effects on Resource Utilization," Journal of the American Medical Association 269, no. 3 (1993): 379–383[Abstract/Free Full Text]; R. Taylor, J. Manzo, and M. Sinnett, "Quantifying Value for Physician Order-Entry Systems: A Balance of Cost and Quality," Healthcare Financial Management 56, no. 7 (2002): 44–48; and J. Warner, Building the Business Case for Safety: Financial Returns on Investments to Reduce Medical Errors and Malpractice Claims (Berkeley: Goldman School of Public Policy, University of California, 2004).
9. Taylor et al., "Quantifying Value."
10. Poon et al., "Overcoming Barriers"; and C. Broder, "Hospitals Share Lessons Learned from CPOE," California Healthline, 3 February 2003, www.californiahealthline.org/index.cfm?Action=dspItem&itemID=102454&classcd=CL126 (20 September 2005).
11. F. Sloan, "Not-for-Profit Ownership and Hospital Behavior," in Handbook of Health Economics, vol. 1, ed. A.J. Culyer and J.P. Newhouse (New York: Elsevier Science B.V., 2000), 1141–1174.
12. California, Colorado, Connecticut, Florida, Georgia, Kansas, Massachusetts, Minnesota, Missouri, Mississippi, New Jersey, New York, Tennessee, Texas, Washington, and Wisconsin. A small number of hospitals in Arkansas, Illinois, and North Dakota were targeted but did not respond to the survey and were excluded from most of our analysis. Of the 937 targeted hospitals, 186 observations were lost because of matching problems with the merged databases.
13. K. Madison, "Multihospital System Membership and Patient Treatments, Expenditures, and Outcomes," Health Services Research 39, no. 4, Part 1 (2004): 749–769.[CrossRef][Web of Science][Medline]
14. D.P. Lorence, A. Spink, and M.C. Richards, "EPR Adoption and Dual Record Maintenance in the U.S.: Assessing Variation in Medical Systems Infrastructure," Journal of Medical Systems 26, no. 5 (2002): 357–367.[CrossRef][Medline]
15. J.R. Horwitz, "Making Profits and Providing Care: Comparing Nonprofit, For-Profit, and Government Hospitals," Health Affairs 24, no. 3 (2005): 790–801.[Abstract/Free Full Text]
16. First Consulting Group, Computerized Physician Order Entry.
17. D.F. Doolan and D.W. Bates, "Computerized Physician Order Entry Systems In Hospitals: Mandates and Incentives," Health Affairs 21, no. 4 (2002): 180–188. 2003 Fla. Laws, ch. 2003-416, Sec. 36(2)(d).[Abstract/Free Full Text]
18. For example, see Cal. Health & Safety Code Sec. 1339.63 (West 2004); S.B. 4146, 226th Leg. (NY 2003); S.B. 562, 105th Leg. (FL 2003); H.B. 63B, Special Session B (FL 2003); S.B. 1464, 106th Regular Session (FL 2004); and S.B. 560, 105th Regular Session (FL 2003).
19. Poon et al., "Overcoming Barriers."
20. Lorence et al., "EPR Adoption."
21. C. Eikel et al., "Factors Influencing Hospital Patient Safety Achievements: An Analysis of Leapfrog Hospital Survey Data," Poster session presentation, AcademyHealth Annual Research Meeting, San Diego, California, June 2004.
22. California Health and Safety Code, sec. 1339.63.
23. Horwitz, "Making Profits and Providing Care."
24. Nine bills addressing CPOE, as well as other technology meant to improve patient safety, have been introduced since 2001. Two bills from the 108th Congress appear to be advancing: S. 720, 108th Cong., 2d sess. (2004); and H.R. 663, 108th Cong., 2d sess. (2004).

                      What's this?

This article has been cited by other articles:

				A. K. Jha, C. M. DesRoches, E. G. Campbell, K. Donelan, S. R. Rao, T. G. Ferris, A. Shields, S. Rosenbaum, and D. Blumenthal Use of Electronic Health Records in U.S. Hospitals N. Engl. J. Med., April 16, 2009; 360(16): 1628 - 1638. [Abstract] [Full Text] [PDF]

				J. Aarts and R. Koppel Implementation Of Computerized Physician Order Entry In Seven Countries Health Aff., March 1, 2009; 28(2): 404 - 414. [Abstract] [Full Text] [PDF]

				K. G. Shojania and W. Levinson Clinicians in Quality Improvement: A New Career Pathway in Academic Medicine JAMA, February 18, 2009; 301(7): 766 - 768. [Full Text] [PDF]

				N. Menachemi, R. G. Brooks, E. Schwalenstocker, and L. Simpson Use of Health Information Technology by Children's Hospitals in the United States Pediatrics, January 1, 2009; 123(Supplement_2): S80 - S84. [Abstract] [Full Text] [PDF]

				M. F. Furukawa, T.S. Raghu, T. J. Spaulding, and A. Vinze Adoption Of Health Information Technology For Medication Safety In U.S. Hospitals, 2006 Health Aff., May 1, 2008; 27(3): 865 - 875. [Abstract] [Full Text] [PDF]

				D. P. Scanlon, J. B. Christianson, and E. W. Ford Hospital Responses to the Leapfrog Group in Local Markets Med Care Res Rev, April 1, 2008; 65(2): 207 - 231. [Abstract] [PDF]

				Y. Xiao, F. Dexter, P. Hu, and R. P. Dutton The Use of Distributed Displays of Operating Room Video When Real-Time Occupancy Status Was Available Anesth. Analg., February 1, 2008; 106(2): 554 - 560. [Abstract] [Full Text] [PDF]

				J. S. Ash, D. F. Sittig, E. G. Poon, K. Guappone, E. Campbell, and R. H. Dykstra The Extent and Importance of Unintended Consequences Related to Computerized Provider Order Entry J. Am. Med. Inform. Assoc., July 1, 2007; 14(4): 415 - 423. [Abstract] [Full Text] [PDF]

				A. K. Jha, T. G. Ferris, K. Donelan, C. DesRoches, A. Shields, S. Rosenbaum, and D. Blumenthal How Common Are Electronic Health Records In The United States? A Summary Of The Evidence Health Aff., November 1, 2006; 25(6): w496 - w507. [Abstract] [Full Text] [PDF]