Health Affairs, 22, no. 2 (2003): 243-254 doi: 10.1377/hlthaff.22.2.243 © 2003 by Project HOPE	New Online   Obama's Success & Next Steps   3 Health Policy Options for President Obama   A Hybrid Proposal   Bipartisan Compromise

This Article Abstract Reprint (PDF) Submit a response to this article Alert me when this article is cited Alert me when Comments are posted Alert me if a correction is posted Services E-mail this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to My Personal Archive Download to Citation Manager Reprints & Permissions Citing Articles Citing Articles via ISI Web of Science (14) Citing Articles via Google Scholar Google Scholar Articles by Chen, J. Articles by Krumholz, H. M. Search for Related Content PubMed PubMed Citation Articles by Chen, J. Articles by Krumholz, H. M. Related Collections Hospitals Quality Of Care

DataWatch

JCAHO Accreditation And Quality Of Care For Acute Myocardial Infarction

Abstract

 
We examined the association between JCAHO accreditation of hospitals, those hospitals’ quality of care, and survival among Medicare patients hospitalized for acute myocardial infarction. Hospitals not surveyed by JCAHO had, on average, lower quality (less likely to use aspirin, beta-blockers, and reperfusion therapy) and higher thirty-day mortality rates than did surveyed hospitals. However, there was considerable variation within accreditation categories in quality of care and mortality among surveyed hospitals, which indicates that JCAHO accreditation levels have limited usefulness in distinguishing individual performance among accredited hospitals. These findings support current efforts to incorporate quality of care in accreditation decisions.

JCAHO accreditation is awarded on the basis of a hospital’s compliance with a set of standards, which surveyors use in assessing performance during hospital site visits.⁵ Standards are assessed in patient assessment and care, patients’ rights, clinical ethics, organizational leadership, human resources management, and information management.⁶ JCAHO’s philosophy is that "if hospitals complied with relevant standards then hospitals would be likely to achieve good outcomes."⁷ However, evidence demonstrating that JCAHO accreditation can distinguish differences in hospitals’ quality of care or patient outcomes is limited. Studies have found little correlation between accreditation and general hospital mortality, and no differences in rates of medication error between accredited and nonaccredited hospitals.⁸ However, a robust assessment of whether JCAHO accreditation correlates with disease-specific quality measures has yet to be conducted.

Acute myocardial infarction (AMI) is well suited for a study of accreditation because it is a common diagnosis and a major cause of mortality for which quality performance measures have been established from authoritative clinical guidelines. The availability of data from the Cooperative Cardiovascular Project (CCP), a national project to assess and improve the care of Medicare patients hospitalized with AMI, provides a unique opportunity to assess whether JCAHO hospital accreditation is associated with use of guideline-recommended therapies and clinical outcomes.

Study Methods

Top Study Methods Study Results Discussion NOTES

 
JCAHO accreditation. A hospital seeking to obtain JCAHO accreditation is visited every three years by a survey team that observes hospital operations, conducts interviews, and reviews medical documentation for compliance with a set of standards in forty-five performance areas.⁹ JCAHO surveyors assign a score in each performance area and determine an "accreditation level" based on a hospital’s overall score and whether JCAHO cited specific areas for improvement (for example, type I recommendations, which indicate the need to resolve unsatisfactory compliance). JCAHO accreditation levels during our study period were (in descending order of compliance) accreditation with commendation, accreditation without (type I) recommendations (hereafter referred to as "accreditation"), accreditation with (type I) recommendations, conditional accreditation, and not accredited. We obtained data from JCAHO on hospital accreditation level and summary scores for hospitals surveyed between 1994 and 1998. Hospitals that had neither a summary score nor an accreditation level reported were considered to be not surveyed.

Cooperative Cardiovascular Project. The CCP sample included 234,769 fee-for-service (FFS) Medicare hospitalizations from acute care, nongovernmental hospitals with a principal discharge diagnosis of AMI, excluding readmissions for AMI, between January 1994 and February 1996 in all fifty states and the District of Columbia.¹⁰ Medical records of sampled patients were abstracted for patients’ clinical characteristics, in-hospital treatments, and vital status.

Our study consisted of CCP patients age sixty-five and older who were hospitalized with clinically confirmed AMI. We excluded patients who did not have confirmed AMI (n = 31,186), were younger than age sixty-five (17,593), were readmitted for AMI (25,185), were admitted by interhospital transfer (42,277), had a terminal illness or metastatic cancer (4,616), were hospitalized outside the United States (1,760), had unverified mortality (357), and were admitted to hospitals for which American Hospital Association (AHA) data were unavailable (2,363). We also excluded patients admitted to hospitals surveyed by JCAHO for which data were not available (14,598). In total, the study cohort comprised 134,579 patients.

Quality of AMI care and outcomes. Quality of care for AMI was assessed using a set of clinical performance measures from the Centers for Medicare and Medicaid Services (CMS) that assess the use of therapies among patients (ideal candidates) who would benefit from and did not have contraindications for particular treatments. We evaluated the following quality performance measures: use of aspirin or beta-blockers within forty-eight hours of admission, aspirin or beta-blockers anytime during hospitalization, and acute reperfusion therapy (thrombolytic agents or primary angioplasty) within six hours of admission.¹¹ We examined patient outcomes using thirty-day mortality because the benefits of high-quality hospital care should be evident within this period. Dates of death were obtained from the Medicare Enrollment Database and the Social Security Administration’s Master Beneficiary Record.

Statistical analysis. Chi-square tests and analyses of variance were used to compare differences across hospitals, and the Cochrane-Armitage test was used to evaluate for linear trends in therapy or mortality rates associated with higher accreditation ranking. We compared hospitals’ risk-standardized thirty-day mortality rates using the Medicare Mortality Predictor System (MMPS), a disease-specific mortality prediction model for elderly patients.¹² Using logistic regression, we calculated a risk-standardized mortality rate that estimated thirty-day mortality for hospitals in each JCAHO accreditation group, assuming that they had the same patient characteristics as the overall sample. Because of the correlation between hospital characteristics, physician characteristics, and JCAHO accreditation, the primary analyses were risk-adjusted for patient characteristics only; secondary analyses that added adjustment for hospital and physician characteristics were also performed.

To assess the heterogeneity of hospitals’ performance within JCAHO accreditation categories, we calculated the observed use of AMI therapies in ideal patients and risk-standardized thirty-day mortality for individual hospitals. Standard deviations and twenty-fifth through seventy-fifth percentiles were evaluated to determine variations within categories in AMI therapy use and thirty-day mortality rates within a particular JCAHO accreditation group. To ensure stability in these estimates, we restricted our analyses to hospitals with at least twenty-five observations. Statistical calculations were performed using STATA 7.0.

Study Results

Top Study Methods Study Results Discussion NOTES

 
The final study cohort consisted of 134,579 patients treated at 4,221 hospitals. Approximately one-quarter of hospitals in the study sample were not surveyed by JCAHO (Exhibit 1). Most of the surveyed hospitals received accreditation with recommendations. Nonsurveyed hospitals were more likely than surveyed hospitals were to be smaller-volume centers, nonteaching hospitals, publicly owned, and located in a rural setting, and most lacked on-site facilities for cardiac procedures. Surveyed hospitals with higher accreditation levels tended to be larger-volume teaching centers and to be located in urban settings with on-site facilities for cardiac procedures (Exhibit 1).

Nonsurveyed hospitals had higher thirty-day mortality rates than surveyed hospitals had (HR 1.15, p < .001) when patient characteristics were adjusted for. The increased hazard associated with nonsurveyed hospitals was attenuated after adjustment for hospital and physician characteristics, but remained significant (HR 1.08, p < .001).

However, we found considerable variation in risk-standardized thirty-day mortality rates within each accreditation level (Exhibit 4). The interquartile ranges of risk-standardized thirty-day mortality were primarily located between 15 percent and 25 percent across all accreditation levels.

Discussion

Top Study Methods Study Results Discussion NOTES

 
In our study, nonsurveyed hospitals had lower use of AMI therapies and worse thirty-day outcomes than did hospitals surveyed by JCAHO. However, among surveyed hospitals there were only modest differences in the use of AMI therapies, with the greatest variation observed for the use of beta-blockers. Patients admitted to hospitals accredited with commendation had lower thirty-day mortality rates than those of patients admitted to hospitals in lower accreditation levels. However, we observed much variation in quality measures and outcomes within each JCAHO accreditation category across hospitals.

These findings suggest that the JCAHO standards-based accreditation system has only a modest ability to assess quality of AMI clinical care at any particular hospital. Accreditation does provide some information concerning hospitals’ quality of care and outcomes in the aggregate. Indeed, knowing that a hospital participated in the JCAHO survey process suggests superior quality and outcomes compared with nonsurveyed hospitals. It is unknown, however, whether the process of undergoing JCAHO accreditation improves quality of care or whether this association reflects self-selection against JCAHO evaluation by more poorly performing hospitals.

In contrast, accreditation levels were of limited value in differentiating quality among surveyed hospitals. Although beta-blocker use was higher across successive accreditation levels, the absolute differences in rates across accreditation groups were small. Furthermore, there was considerable hospital-level variation in the use of aspirin therapy, the use of beta-blocker therapy, and thirty-day mortality rates within all JCAHO accreditation groups. There were hospitals with high and low rates of AMI therapy use and thirty-day outcomes in all JCAHO accreditation categories, even among hospitals with JCAHO conditional accreditation and non-surveyed hospitals. Thus, a higher JCAHO accreditation level was not necessarily a guarantee of higher-quality care or better outcomes in the management of AMI.

To place the mortality difference across accreditation categories in perspective, the relative difference in risk for thirty-day mortality between surveyed and non-surveyed hospitals was approximately 15 percent. In contrast, an examination of hospital volume and AMI mortality in the CCP data set identified a 17 percent relative risk difference in thirty-day mortality between the smallest and largest hospital volume quartiles.¹³ Accreditation’s ability to predict short-term mortality after AMI appears comparable to that of hospital volume.

Reasons for lack of quality differentiation. There are several reasons why standards-based JCAHO accreditation levels may not be able to differentiate hospitals on the basis of quality. First, many of the JCAHO standards do not assess quality in day-to-day patient care activities. For example, a high degree of compliance with administrative or managerial standards is unlikely to have much bearing on whether patients receive aspirin on admission for AMI, yet these areas account for more than half of all points in a JCAHO survey. Identifying hospitals that are well managed, while informative, is likely to be different than identifying hospitals that provide high-quality clinical care.

Second, the wide range of hospital compliance with JCAHO standards within a single accreditation level may dilute any differences in quality. For example, the category of "accredited with type I recommendations" does not distinguish between hospitals with a single recommendation or many. Similarly, hospitals are assigned conditional accreditation whether they received one citation or several.

Third, JCAHO surveyors exhibit discretion when determining how deeply to probe for potential problems during a survey visit.¹⁴ The impact of variation by and between observers and the reliability of the JCAHO accreditation process are unknown. The Joint Commission itself has recognized that levels of accreditation can be subjective. Commenting on the recent removal of the accreditation with commendation rating, JCAHO president Dennis O’Leary stated that "the distinction between those who get commendation and those who fall just short is artificial in many respects."¹⁵

JCAHO as an accreditor of quality. Given that JCAHO accreditation cannot differentiate hospitals on the basis of clinical performance, the question is whether JCAHO would be an effective force for assessing and improving quality. There would be several potential advantages for having JCAHO evaluate quality in addition to its current standards-based accreditation.

First, JCAHO has the administrative machinery necessary to evaluate hospitals. Thus, there are financial and logistical benefits to having it evaluate both standards and quality. Second, JCAHO accreditation is sought nearly universally; as a result, the effect of codifying quality into accreditation decisions would be readily disseminated across the country. Third, as the nation’s most widely accepted accreditor, JCAHO would likely meet with less resistance from hospitals to the inclusion of quality measures as a natural extension of the accreditation process rather than an entirely new review process.

In spite of the advantages, several challenges remain. First, although JCAHO is an independent institution, it has close ties to the industry it oversees. In a recent critique, the Office of Inspector General of the Department of Health and Human Services reported that JCAHO’s stance is "moving towards collegiality rather than regulatory," suggesting a lack of impartiality in evaluating hospitals.¹⁶ Similar concerns have been raised by public-interest groups, which note that half of the members of JCAHO’s board of commissioners are from within the industry it is supposed to regulate (the American Hospital Association and the American Medical Association).¹⁷

Second, while JCAHO has indicated a willingness to incorporate quality into accreditation decisions, the specific details are lacking. It is less likely that hospitals will feel an incentive to improve quality if their accreditation is not placed at risk. However, critics contend that very few hospitals are denied accreditation.¹⁸ Whether placing accreditation in jeopardy on the basis of quality will lead to quality improvement remains to be seen.

Third, JCAHO’s current system for quality measurement is limited. To its credit, JCAHO has recognized that standards and performance measurement are complementary to assessment of hospital care and has embarked upon a program to integrate clinical performance measures into accreditation decisions.¹⁹ In the mid-1990s JCAHO began requiring accredited hospitals to submit performance data through its "Oryx initiative." Hospitals contracted with vendors to collect data and developed automated databases that feed performance measures back to hospitals and JCAHO each quarter.

However, the Oryx methodology raises questions regarding its validity for measuring quality. Hospitals could select any six of more than 2,000 performance measures. This is problematic because of the potential for hospitals to "game the system" by selecting measures at which they already do well. Also, there is wide variation in clinical importance in the performance measures, which range from length-of-stay, mortality or readmission rates, use of procedures such as cesarean section, patient fall rates, or use of restraints. There is no guarantee that a particular hospital-chosen performance measure represents meaningful differences in quality. Finally, hospitals are evaluated against peers, but the comparison groups are different for each vendor’s system. The benchmark group could range from hospitals that used the same vendor’s measure, or it may include nonvendor data from the CMS or state health departments, all of which limit Oryx’s ability to determine national benchmarks for quality.

To mitigate these limitations, JCAHO recently began requiring hospitals to report a set of "core performance measures" from among four medical conditions (AMI, heart failure, community-acquired pneumonia, and pregnancy) with specific definitions for numerators and denominators.²⁰ The advantages of using these core measures is that the clinical consensus underlying the quality indicators ensures that they can be compared across both hospitals and time. It is too early to tell whether the reporting of these core measures will affect hospital accreditation or lead to improvements in patient care and outcomes.

Additional elements of success. Besides selecting appropriate performance measures, establishing standardized benchmarks, and codifying the effects of quality on accreditation, we believe that several additional key elements are necessary if accreditation is to have a substantial and lasting public role for monitoring hospital quality.

Publication of quality measures. Public release of comparative hospital data will allow patients and purchasers to make purchasing decisions based on quality. Consumers would "vote with their feet" in selecting health plans incorporating hospitals that emphasize quality. Purchasers could contract for care based on quality and thereby receive greater value for their health care dollars. Moreover, providers could use explicit measures of quality when negotiating contracts, rather than relying on subjective measures or purchasers’ perceptions of quality.

Rewards for success. Purchasers’ decisions to contract on the basis of quality need not be punitive. The Pacific Business Group on Health (PBGH) has negotiated with several health plans in California to place $8 million at risk for meeting performance measures on patient satisfaction, preventive care measures, and cesarean section rates.²¹ In addition, the Leapfrog Group has embarked on a program to reward hospitals for meeting requirements for hospital safety, evidence-based hospital referral, and physician staffing in intensive care units.²² These examples demonstrate that purchasers are amenable to pursuing reimbursement that rewards superior quality.

Influence of government. The federal government is uniquely positioned to motivate changes in JCAHO accreditation because of accreditation’s role in securing hospitals’ Medicare reimbursement. This criterion could be leveraged to improve the quality of care for the elderly by having Medicare pay more (or less) depending on providers’ quality measures for diseases prevalent in the elderly; by providing a highly visible distinction for hospitals that achieve high standards of performance; or even by tying participation in Medicare to minimum quality-of-care standards.

Our study suggests that an exclusively standards-based accreditation is a limited tool for comparing hospital quality of care, because of the considerable heterogeneity of performance within accreditation levels across hospitals; this highlights the need to measure and report quality indicators directly. The integration of standardized quality measures into the next generation of JCAHO accreditation may address this deficiency. Nevertheless, there are major challenges for JCAHO, as it ponders how to integrate quality into its accreditation process.

Editor's Notes

 
At the time this research was conducted, Jersey Chen was a student at Yale University School of Medicine; he is now a resident in internal medicine at Beth Israel Deaconess Medical Center in Boston. Saif Rathore is a lecturer at the Yale University School of Medicine. Martha Radford is system director at Yale New Haven Health in New Haven, Connecticut. Harlan Krumholz is a professor at the Yale University School of Medicine.

The authors thank Maria Johnson for her editorial assistance; Yun Wang, Paul Hung, and Bryon Butts for their technical assistance; Jerod Loeb for his review of prior drafts; and the people and organizations involved in the Cooperative Cardiovascular Project. Harlan Krumholz was a chair of the Cardiovascular Conditions Clinical Advisory Panel for the development of JCAHO’s core indicators. The analyses upon which this manuscript is based were performed under Contract no. 500-99-CT01, titled "Utilization and Quality Control Peer Review Organization for the State of Connecticut," from the Centers for Medicare and Medicaid Services (CMS), U.S. Department of Health and Human Services (HHS). The content of this paper does not necessarily reflect the views or policies of HHS, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. government. The authors assume full responsibility for the accuracy and completeness of the ideas presented. This paper is a direct result of the Health Care Quality Improvement Project initiated by the CMS, which has encouraged identification of quality improvement projects derived from analysis of patterns of care, and therefore required no special funding on the part of the contractor.

NOTES

Top Study Methods Study Results Discussion NOTES

 

1. J.S. Roberts, J.G. Coale, and R.R. Redman, "A History of the Joint Commission on Accreditation of Hospitals," Journal of the American Medical Association 258, no. 7 (1987): 936–940, with a published erratum in the Journal of the American Medical Association 258, no. 19 (1987): 2698.[Medline]
2. Ibid; and Social Security Act, Sec. 1865, 42 U.S. Code 1395bb.
3. "Plastering Its Gold Seal Everywhere, Bridgeport Shows Off Its Commendation," Profiles in Healthcare Marketing 13, no. 1 (1997): 27–32.
4. U.S. Department of Health and Human Services, Office of Inspector General, The External Review of Hospital Quality: A Call for Greater Accountability, Pub. no. OEI-01-97-00050 (Boston: DHHS, 1999).
5. A. Flanagan, "Ensuring Health Care Quality: JCAHO’s Perspective," Clinical Therapeutics 19, no. 6 (1997): 1540–1544.[Medline]
6. Joint Commission on Accreditation of Healthcare Organizations, The Complete Guide to the 1996 Hospital Survey Process (Oakbrook Terrace, Ill.: JCAHO, 1996).
7. D.S. O’Leary, "Performance Measures. How Are They Developed, Validated, and Used?;" Medical Care 33, Suppl. 1 (1995): JS13–JS17.[Medline]
8. J.R. Griffith, S.R. Knutzen, and J.A. Alexander, "Structural versus Outcomes Measures in Hospitals: A Comparison of Joint Commission and Medicare Outcomes Scores in Hospitals," Quality Management in Health Care 10, no. 2 (2002): 29–38; and K.N. Barker et al., "Medication Errors Observed in Thirty-six Health Care Facilities," Archives of Internal Medicine 162, no. 16 (2002): 1897–1903.[Abstract/Free Full Text]
9. JCAHO, "Understanding the 1996 Hospital Performance Report," www.jcaho.org/1wapps/perfrep/undrstd/hap/1996.htm (4 February 2003); and JCAHO, The Complete Guide.
10. T.A. Marciniak et al., "Improving the Quality of Care for Medicare Patients with Acute Myocardial Infarction: Results from the Cooperative Cardiovascular Project," Journal of the American Medical Association 279, no. 17 (1998): 1351–1357.[Abstract/Free Full Text]
11. E.F. Ellerbeck et al., "Quality of Care for Medicare Patients with Acute Myocardial Infarction: A Four-State Pilot Study from the Cooperative Cardiovascular Project," Journal of the American Medical Association 273, no. 19 (1995): 1509–1514.[Abstract]
12. J. Daley et al., "Predicting Hospital-Associated Mortality for Medicare Patients: A Method for Patients with Stroke, Pneumonia, Acute Myocardial Infarction, and Congestive Heart Failure," Journal of the American Medical Association 260, no. 24 (1988): 3617–3624.[Abstract]
13. D.R. Thiemann et al., "The Association between Hospital Volume and Survival after Acute Myocardial Infarction in Elderly Patients," New England Journal of Medicine 340, no. 21 (1999): 1640–1648.[Abstract/Free Full Text]
14. DHHS, OIG, The External Review of Hospital Quality: The Role of Accreditation, Pub. no. OEI-01-97-00051 (Boston: DHHS, 1999).
15. J.D. Moore Jr., "JCAHO Drops a Survey Rating; Board Says ‘Commendation’ Award Has Several Weaknesses," Modern Healthcare 29, no. 46 (1999): 15.
16. DHHS, OIG, The External Review of Hospital Quality.
17. L. Dame and S.M. Wolfe, The Failure of "Private" Hospital Regulation: An Analysis of the Joint Commission on Accreditation of Health care Organizations’ Inadequate Oversight of Hospitals (Washington: Public Citizen Health Research Group, July 1996).
18. Ibid.
19. O’Leary, "Performance Measures"; D.S. O’Leary, "The Joint Commission Looks to the Future," Journal of the American Medical Association 258, no. 7 (1987): 951–952; and [Medline]D.S. O’Leary, "Reordering Performance Measurement Priorities," Health Affairs (July/Aug 1998): 38–39.
20. B.I. Braun, R.G. Koss, and J.M. Loeb, "Integrating Performance Measure Data into the Joint Commission Accreditation Process," Evaluation and the Health Professions 22, no. 3 (1999): 283–297.[Abstract/Free Full Text]
21. H.H. Schauffler, C. Brown, and A. Milstein, "Raising the Bar: The Use of Performance Guarantees by the Pacific Business Group on Health," Health Affairs (Mar/Apr 1999): 134–142.
22. K. Sandrick, "Raising the Bar: Purchasers and Providers Must Work Together to Meet the Quality-of-Care Challenge," Trustee 54, no. 9 (2001): 12–17.

                      What's this?