Health Affairs, 22, no. 2 (2003): 154-166 doi: 10.1377/hlthaff.22.2.154 © 2003 by Project HOPE	New Online   Pay Cuts For Medicare Docs   Access To Care Woes   Public Coverage More Efficient   Empowering Consumers

This Article Abstract Reprint (PDF) Submit a response to this article Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Services E-mail this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to My Personal Archive Download to Citation Manager Reprints & Permissions Citing Articles Citing Articles via HighWire Citing Articles via ISI Web of Science (45) Citing Articles via Google Scholar Google Scholar Articles by Romano, P. S. Articles by McDonald, K. M. Search for Related Content PubMed PubMed Citation Articles by Romano, P. S. Articles by McDonald, K. M. Related Collections Hospitals Minority Health Quality Of Care Rural Health Care

Research Challenge

A National Profile Of Patient Safety In U.S. Hospitals

Abstract

 
Measures based on routinely collected data would be useful to examine the epidemiology of patient safety. Extending previous work, we established the face and consensual validity of twenty Patient Safety Indicators (PSIs). We generated a national profile of patient safety by applying these PSIs to the HCUP Nationwide Inpatient Sample. The incidence of most nonobstetric PSIs increased with age and was higher among African Americans than among whites. The adjusted incidence of most PSIs was highest at urban teaching hospitals. The PSIs may be used in AHRQ’s National Quality Report, while providers may use them to screen for preventable complications, target opportunities for improvement, and benchmark performance.

Study Methods

Top Study Methods Results Discussion NOTES

 
Initial PSIs. The development of the initial PSIs was based on the Institute of Medicine’s (IOM’s) definition of patient safety, which is "freedom from accidental injury due to medical care, or medical errors."³ A more recent IOM report defined medical errors as "the failure of a planned action to be completed as intended or the use of a wrong plan to achieve an aim...[including] problems in practice, products, procedures, and systems."⁴ This definition excludes acts that do not achieve desired outcomes but are not the result of negligence, outcomes resulting from underlying or comorbid illnesses, and outcomes known to be unavoidable risks of a procedure. AHRQ researchers evaluated existing measures of adverse events, searched the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) for promising diagnosis codes, defined inclusion and exclusion criteria, grouped the resulting codes into twelve PSIs, and tested them using New York State data.⁵

Literature review. To expand and refine these initial PSIs, we searched Medline and EMBASE for articles that described, evaluated, or validated potential indicators of medical errors, patient safety, or preventable complications based on administrative data coded with ICD-9-CM codes. We also searched article bibliographies and queried team members to identify references. The Complications Screening Program (CSP) by Lisa Iezzoni and colleagues, which includes twenty-eight complications "that raise concern about the quality of care based on the rate...at individual hospitals," was an especially important source because of its extensive evaluation and its relevance to quality improvement.⁶

Indicator definition. We reviewed the initial AHRQ PSIs, adding codes for potential safety-related events from the 1998–2001 ICD-9-CM revisions and the CSP. This process was guided by conceptual considerations and recent evidence regarding preventability.⁷ The resulting list of more than 200 ICD-9-CM codes was then grouped into clinically coherent indicators. We specified a population at risk for each indicator, restricting the denominator to patients for whom that complication seemed more likely to be preventable. CSP-derived indicators were retained only if they met two validation criteria: (1) independent review of a random sample of 1,298 inpatient records from two states revealed that at least of one of three groups of reviewers (that is, coders, nurses, and physicians) confirmed the diagnosis in at least 75 percent of reported cases; and (2) "process of care failures" were identified more often by physician reviewers among cases flagged by the CSP indicator than among randomly sampled controls.⁸ All indicators were designed for use at the hospital level, but six were also adapted for use at the community level.⁹

Face/consensual validation. We subjected thirty-four indicators to face validation by an expert coding consultant and consensual validation by eleven panels of five to nine expert clinicians nominated by twenty-eight professional organizations or chapters thereof. Our methodology was adapted from the RAND/UCLA Appropriateness Method and included four steps: (1) an initial independent assessment of each indicator using a ten-item questionnaire covering several key domains (for example, "overall...usefulness," "likely to be preventable," "likely to represent true medical error or negligence"); (2) feedback of questionnaire responses, highlighting each individual’s rating relative to the group; (3) a conference call with facilitated discussion focusing on disagreements and suggestions for improvement; and (4) a final independent assessment using the same questionnaire.¹⁰

Panelists were allowed to suggest changes to indicator definitions before or during the conference call; these changes were adopted if supported by consensus. Each of fifteen surgical complication measures was independently reviewed by one of three multispecialty panels and one of three surgeon panels. Each of the other nineteen proposed indicators was reviewed by one of five other multispecialty panels. Their postdiscussion ratings of overall usefulness were used to select indicators for an "accepted" list. To qualify, an indicator had to achieve a median score of 7 or greater (on a 1–9 scale) without disagreement, where disagreement was defined as two or more panelist ratings at each extreme (that is, 1–3 and 7–9).¹¹ Several indicators that were accepted by the relevant panel were subsequently rejected by the research team because of concerns about coding or other operational issues, leaving twenty accepted indicators.¹²

Data analyses. We estimated the national incidence of potential safety-related events by applying "accepted" PSI definitions to the 1995–2000 Healthcare Cost and Utilization Project (HCUP) Nationwide Inpatient Sample (NIS). The NIS is the largest publicly available U.S. all-payer database, with data from nearly 1,000 hospitals in twenty-eight states, approximating a 20 percent stratified sample of nonfederal short-term, general, and other specialty hospitals.¹³ Based on the NIS sampling design, we weighted all results to generate estimates for the entire population of discharges from U.S. community hospitals. We then evaluated differences in the incidence of potential safety-related events across age/sex and racial/ethnic strata and across hospital strata defined using the contemporaneous American Hospital Association (AHA) survey.¹⁴

Analyses of PSI incidence by race/ethnicity and hospital characteristics were adjusted for age, sex, age-sex interactions, comorbidities, and diagnosis-related group (DRG) clusters. Because race/ethnicity was unknown for about 20 percent of discharges in the 2000 NIS, we repeated analyses of race/ethnicity using the 2000 HCUP State Inpatient Databases from the fourteen states with at least 93 percent reporting.¹⁵ Comorbidity adjustment was performed using an updated version of software developed by AHRQ researchers.¹⁶ We defined DRG clusters by aggregating adjacent DRGs with and without comorbidities and complications (and age-stratified DRGs), and by excluding certain DRGs to avoid adjusting for complications.

Results

Top Study Methods Results Discussion NOTES

 
National incidence. The twenty accepted PSIs represent a selective list of potential safety-related events deemed amenable to detection using administrative data, adequately coded in previous studies, and sensitive to the quality of care.¹⁷

We identified about 1.12 million potential safety-related events that occurred in 1.07 million hospitalizations at nonfederal acute care facilities in 2000 (Exhibit 1). Although our sample represented 36,318,000 hospitalizations, the denominators varied across indicators from 248,000 to 32,146,000. Approximately 34 percent of these events occurred in surgical hospitalizations, 31 percent in obstetric hospitalizations, and 35 percent in medical hospitalizations. About 24 percent of these events represented deaths, either affecting patients in low-mortality DRGs (that is, DRGs with less than 0.5 percent inpatient mortality in 1997) or reflecting failure to rescue after a major complication (such as pneumonia, thromboembolism, sepsis, acute renal failure, shock, cardiac arrest, or gastrointestinal hemorrhage/ulcer). Because these events were ascertained from unlinked administrative data, their timing and preventability, as well as the number of affected patients, cannot be determined without additional data. We also cannot estimate how many events involved patients who were excluded from the eligible population.

By contrast, most of the technical complications or errors, including iatrogenic pneumothorax, postoperative abdominopelvic wound dehiscence, and postoperative hemorrhage/hematoma, decreased in incidence between 1995 and 2000. Accidental punctures and lacerations represent an exception: Their incidence rose 7 percent during this period. The overall incidence of obstetric trauma decreased about 3 percent, apparently because of less frequent use of forceps and vacuum assistance. These options might decrease the need for cesarean delivery in some women with second-stage labor difficulties, but they are associated with an increased risk of perineal laceration.¹⁹ The incidence of birth trauma decreased 56 percent, from 1.51 percent to 0.67 percent, between 1997 and 2000.

The rarest sentinel-event PSIs also decreased steadily in incidence between 1995 and 2000: 18 percent for anesthesia reactions and complications, 7 percent for foreign bodies left during procedures, 10 percent for death in low-mortality DRGs, and 40 percent for transfusion reactions. Inpatient death rates following major complications (that is, failure to rescue) decreased by 6 percent (from 18.6 percent to 17.4 percent) between 1995 and 2000.

Patients at risk. The risk of hospital-reported, potential safety-related events varies across sex and age categories (Exhibit 2).²⁰ Among adults, the risk of most events increases with age. However, infants have the highest risk of anesthesia reactions and complications, postoperative sepsis, and postoperative hemorrhage/hematoma (among females). Several other indicators show a V-shaped relationship across age groups, with the lowest risk among older children and young adults (for example, failure to rescue or infection due to medical care).

For-profit hospitals had the lowest incidence of all obstetric indicators but the highest incidence of most other indicators for which incidence differed significantly across strata (for example, anesthesia reactions and complications, decubitus ulcer, foreign body left during procedure, infection due to medical care, postoperative respiratory failure, and sepsis). Nonprofit hospitals had the highest incidence of all obstetric indicators. The incidence of mortality-related events differed minimally across hospital control categories (Exhibit 4).

Discussion

Top Study Methods Results Discussion NOTES

 
There are several approaches to profiling inpatient safety. One approach involves reviewing medical records or querying physicians and other health care providers (through confidential reporting systems) to find serious errors and then looking forward in time to ascertain relevant outcomes.²² A related approach is to ascertain errors as they occur through ethnographic observation of health care teams.²³ The other general approach is to identify adverse outcomes that are often preventable and then to look backward to ascertain how and why they occurred. This approach was used successfully in the Harvard Medical Practice Study and follow-up studies in Utah and Colorado, but the screening criteria in those studies necessitated nurse review of every record.²⁴ By contrast, the AHRQ PSIs screen hospitalizations based only on computerized data that are collected by nearly all U.S. hospitals and are widely available for public use. The key advantages of this method are its low cost, unobtrusiveness, and universality. Using these data to monitor patient safety requires no additional effort by health professionals. The ultimate usefulness of these screening indicators, however, will result from selective review of flagged records and other efforts to identify and improve systems that contribute to the occurrence of safety-related events.

Of course, this event-based approach to identifying patient safety problems has several notable limitations. First, we can only find events for which there are corresponding ICD-9-CM codes. For example, no specific codes exist for iatrogenic injuries to nerves or urinary tract structures. Furthermore, our ascertainment of adverse events depends on the quality and completeness of coding. Although the principal diagnosis is generally well coded in administrative data, comorbidities and complications are often underreported.²⁵ ICD-9-CM coding guidelines actually preclude reporting complications that seem "integral to the disease process" or lack physician documentation indicating "their clinical significance."²⁶ Most important, the mere occurrence of a complication provides no information about its severity, timing, etiology, or preventability. Some events captured by the PSI logic may have been present at admission or may reflect the unalterable natural history of an illness. The former problem may be remedied by requiring hospitals to report whether each diagnosis was present at admission, as California and New York already do. We have limited data from prior studies on how often these events are related to errors in diagnosis or management.²⁷ Although our expert panels agreed that all of these PSIs are useful, there was some disagreement about the extent to which they result from negligence or medical error.

We generally emphasized specificity over sensitivity in designing these PSIs, by excluding subsets of patients for whom a complication seemed less likely to be preventable. As a result, several PSIs are limited to elective-surgery patients, even though these events occur as frequently or more so after emergency surgery. By restricting the eligible population, we sacrificed generalizability to the entire population of hospitalized patients. Compared with other screening methods based on billing data, which include broad criteria such as death, readmission, transfer to a special care unit or to another hospital, return to the operating room, cardiac arrest, and prolonged length-of-stay, the AHRQ PSIs seem likely to capture fewer preventable adverse events, but also fewer false positives.²⁸

Our results illustrate how the AHRQ PSIs may be used to improve our understanding of the epidemiology of patient safety. It is reassuring that the incidence of most PSIs has fallen since 1995, with the notable exception of postoperative medical complications, decubitus ulcer, and infection due to medical care. Although the number of states included in the NIS increased from nineteen in 1995–96 to twenty-eight in 2000, this change seems unlikely to have selectively affected certain PSIs. The increased incidence of postoperative medical complications probably reflects better documentation and coding, or the impact of shorter stays, increased acuity, nurse staffing shortages, and other stresses facing hospitals.

By comparing our results with those of studies based on detailed review of medical records, we hoped to learn more about indicator validity. For example, our results are consistent with previous studies that found less frequent use of effective cardiac medications, less timely use of antibiotics for pneumonia, and poorer overall quality of care among African American inpatients than among white inpatients, especially after hospital location and teaching status were adjusted for.²⁹ However, previous studies of preventable adverse events have not shown differences across racial/ethnic categories.³⁰

Prior studies have not found consistent effects of hospital ownership or control. For example, for-profit and nonteaching public hospitals in Colorado and Utah had significantly higher rates of preventable "operative adverse events" and "adverse events due to delayed diagnoses and therapies" than nonprofit hospitals had.³¹ By contrast, for-profit hospitals in New York had a lower percentage of adverse events due to negligence than nonprofit hospitals had.³² The AHRQ PSIs showed a mixed pattern, although rates for nonobstetric indicators were generally either highest at for-profit hospitals or similar across categories.

The high incidence of most PSIs in urban teaching hospitals is worrisome, although our findings are consistent with those reported by the developers of the initial PSI algorithms and the Complications Screening Program.³³ Differences in severity of illness and use of surgical interventions still confound these measures, given our limited ability to adjust risk using administrative data. In the Harvard Medical Practice Study, the proportion of adverse events resulting from negligence was lower at teaching hospitals than at nonteaching hospitals.³⁴ In a random sample of Medicare patients with acute myocardial infarction, effective medications were administered more often at major teaching hospitals than at minor teaching or nonteaching hospitals.³⁵ Hospitals with surgical housestaff had higher failure-to-rescue rates after prostatectomy and cholecystectomy than nonteaching hospitals had, but lower failure rates after coronary bypass surgery.³⁶ Studies in which physicians reviewed medical records to find quality problems have generally reported better care at teaching and large nonteaching hospitals than at small nonteaching hospitals, although differences in documentation practices may bias these results.³⁷

Given these limitations and concerns, the AHRQ PSIs will be useful primarily as screening tools for hospitals and hospital systems, medical groups, health plans, and purchasers to identify potential patient safety problems for further investigation. Providers can use them to flag suspicious cases for in-depth review, increasing the efficiency and yield of their quality improvement activities while focusing attention on system failures responsible for adverse events.

Public health agencies and provider associations can monitor PSI rates at the local, state, regional, and national levels. These community-level PSI rates should serve as useful benchmarks for organizations evaluating their own performance or tracking the impact of safety-enhancing interventions. Given the current paucity of tools to identify safety-related events, the AHRQ PSIs also could be used by the Centers for Medicare and Medicaid Services (CMS) and state agencies to track hospital-specific performance and guide quality improvement efforts. Because of variability in coding and uncertain preventability, these measures are not suitable for public reporting at the provider level. Ongoing and future research will establish whether and when these indicators are valid measures of safety-related hospital performance. A substantial portion of AHRQ’s investment of more than $50 million in patient safety focuses on reporting systems. As results from these studies become available, policymakers will be better able to design surveillance systems that triangulate chart review, administrative data, and self-reporting to achieve maximal identification of safety-related events.

Editor's Notes

 
Patrick Romano is an associate professor of medicine and pediatrics at the University of California, Davis, Division of General Medicine. Jeffrey Geppert is a research analyst at the National Bureau of Economic Research (NBER) in Stanford, California. Sheryl Davies is a research manager at the Center for Primary Care and Outcomes Research, also in Stanford, and Kathryn McDonald is the center’s executive director. Marlene Miller is director of quality and safety initiatives at the Johns Hopkins Children’s Center in Baltimore, Maryland. Anne Elixhauser is a senior research scientist at the Agency for Healthcare Research and Quality in Rockville, Maryland.

This work was supported by a contract from the Agency for Healthcare Research and Quality.

NOTES

Top Study Methods Results Discussion NOTES

 

1. M.R. Miller et al., "Patient Safety Indicators: Using Administrative Data to Identify Potential Patient Safety Concerns," Health Services Research 36, no. 6 (2001): 110–132.[Medline]
2. K. McDonald et al., Measures of Patient Safety Based on Hospital Administrative Data—The Patient Safety Indicators, Technical Review 5, Pub. no. 02-0038 (Rockville, Md.: Agency for Healthcare Research and Quality, 2002). Available at www.qualityindicators.ahrq.gov/data/hcup/qirefine.htm.
3. L.T. Kohn, J.M. Corrigan, and M.S. Donaldson, eds., To Err Is Human: Building a Safer Health System (Washington: National Academy Press, 1999).
4. Quality Interagency Coordination Task Force, Doing What Counts for Patient Safety: Federal Actions to Reduce Medical Errors and Their Impact (Washington: QuIC Task Force, 2000).
5. Miller et al., "Patient Safety Indicators."
6. L.I. Iezzoni et al., "A Method for Screening the Quality of Hospital Care Using Administrative Data: Preliminary Validation Results," Quality Review Bulletin 18, no. 11 (1992): 361–371; and L.I. Iezzoni et al., "Identifying Complications of Care Using Administrative Data," Medical Care 32, no. 7 (1994): 700–715.[Medline]
7. R.M. Wachter et al., Making Health Care Safer: A Critical Analysis of Patient Safety Practices, 20 July 2001, www.ahrq.gov/clinic/ptsafety (26 August 2002).
8. A. Lawthers et al., "Identification of In-Hospital Complications from Claims Data: Is It Valid?" Medical Care 38, no. 8 (2000): 785–795; [Medline]E.P. McCarthy et al., "Does Clinical Evidence Support ICD-9-CM Diagnosis Coding of Complications?" Medical Care 38, no. 8 (2000): 868–876; [Medline]S.N. Weingart et al., "Use of Administrative Data to Find Substandard Care: Validation of the Complications Screening Program," Medical Care 38, no. 8 (2000): 796–806; and [Medline]L.I. Iezzoni et al., "Does the Complications Screening Program Flag Cases with Process of Care Problems? Using Explicit Criteria to Judge Processes," International Journal of Quality in Health Care 11, no. 2 (1999): 107–118.
9. Community-level indicators differ from the hospital-level indicators described here only in that both principal and secondary diagnoses are used. Including the principal diagnosis in the definition of an indicator captures complications of outpatient procedures and late complications that necessitate readmission.
10. K. Fitch et al., The RAND/UCLA Appropriateness Method User’s Manual (Los Angeles: RAND, 2001).
11. The threshold for disagreement was raised to three or more panelist ratings at each extreme if the panel included more than seven members.
12. We labeled seventeen additional indicators as "experimental" because of lower panel ratings on usefulness or more serious concerns about validity: postangioplasty coronary bypass; decubitus ulcer in high-risk patients; in-hospital fractures possibly related to falls; intraoperative nerve compression injuries; malignant hyperthermia; postoperative aspiration pneumonia, acute myocardial infarction, iatrogenic cardiac complications and nervous system complications; reopening of surgical site; suture of iatrogenic laceration; obstetric wound complications (cesarean or vaginal); other obstetric complications; postpartum urinary tract infection; third- or fourth-degree perineal laceration; and uterine rupture. For definitions, see McDonald et al., Measures of Patient Safety Based on Hospital Administrative Data,
13. Agency for Healthcare Research and Quality, "Nationwide Inpatient Sample (NIS)," September 2002, www.ahrq.gov/data/hcup/hcupnis.htm (31 January 2003).
14. Hospital control was classified as public (nonfederal), private not-for-profit, or for-profit. Hospital type was classified as rural (outside a metropolitan statistical area), urban nonteaching, or urban teaching. A hospital is considered a teaching hospital if it has an American Medical Association (AMA)–approved residency program, is a member of the Council of Teaching Hospitals, or has a ratio of full-time-equivalent interns and residents to beds of 0.25 or higher. Bed-size cutpoints were chosen so that approximately one-third of the hospitals in each region and location/teaching combination are in each bed-size category. See AHRQ, "Table 5: Bed Size Categories, by Region," www.ahrq.gov/data/hcup/nistab5.htm (4 December 2002) for details.
15. AHRQ, "State Inpatient Databases (SID)," January 2002, www.ahrq.gov/data/hcup/hcupsid.htm (26 August 2002). The fourteen states were Arizona, California, Connecticut, Florida, Kansas, Massachusetts, Missouri, New Jersey, New York, South Carolina, Tennessee, Texas, Virginia, and Wisconsin.
16. A. Elixhauser et al., "Comorbidity Measures for Use with Administrative Data," Medical Care 36, no. 1 (1998): 8–27.[Medline]
17. For complete definitions of these potential safety-related events, see McDonald et al., Measures of Patient Safety, Appendix E; or send e-mail to support{at}qualityindicators.ahrq.gov.
18. Graphic displays of these temporal trends are available under "Publications" at www.qualityindicators.ahrq.gov.
19. V.L. Handa, B.H. Danielsen, and W.M. Gilbert, "Obstetric Anal Sphincter Lacerations," Obstetrics and Gynecology 98, no. 2 (2001): 225–230.[Abstract/Free Full Text]
20. Transfusion reaction data are not shown because their rates are less than 0.001 percent.
21. Data are available upon request from the authors.
22. U. Beckmann et al., "The Australian Incident Monitoring System in Intensive Care: AIMS-ICU, An Analysis of the First Year of Reporting," Anaesthesia and Intensive Care 24, no. 3 (1996): 320–329; [Medline]J.B. Battles et al., "The Attributes of Medical Event-Reporting Systems: Experience with a Prototype Medical Event-Reporting System for Transfusion Medicine," Archives of Pathology and Laboratory Medicine 122, no. 3 (1998): 231–238; and S.N. Weingart et al., "A Physician-Based Voluntary Reporting System for Adverse Events and Medical Errors," Journal of General Internal Medicine 16, no. 12 (2001): 809–814.[Medline]
23. L.B. Andrews et al., "An Alternative Strategy for Studying Adverse Events in Medical Care," Lancet (1 February 1997): 309–313.
24. T.A. Brennan et al., "Incidence of Adverse Events and Negligence in Hospitalized Patients: Results of the Harvard Medical Practice Study I," New England Journal of Medicine 324, no. 6 (1991): 370–376; and [Abstract]E.J. Thomas et al., "Incidence and Types of Adverse Events and Negligent Care in Utah and Colorado," Medical Care 38, no. 3 (2000): 261–271.[Medline]
25. J.M. Geraci et al., "International Classification of Diseases, 9th Revision, Clinical Modification Codes in Discharge Abstracts Are Poor Measures of Complication Occurrence in Medical Inpatients," Medical Care 35, no. 6 (1997): 589–602; [Medline]P.S. Romano, M.E. Schembri, and J.A. Rainwater, "Can Administrative Data Be Used to Ascertain Clinically Significant Postoperative Complications?" American Journal of Medical Quality 17, no. 4 (2002): 145–154; [Abstract]G.A. Hawker et al., "Accuracy of Administrative Data for Assessing Outcomes after Knee Replacement Surgery," Journal of Clinical Epidemiology 50, no. 3 (1997): 265–273; [Medline]T. Faciszewski et al., "Administrative Databases’ Complication Coding in Anterior Spinal Fusion Procedures. What Does It Mean?" Spine 20, no. 16 (1995): 1783–1788; [Medline]A.J. Hartz and E.M. Kuhn, "Comparing Hospitals That Perform Coronary Artery Bypass Surgery: The Effect of Outcome Measures and Data Sources," American Journal of Public Health 84, no. 10 (1994): 1609–1614; and [Abstract/Free Full Text]W.R. Best et al., "Identifying Patient Preoperative Risk Factors and Postoperative Adverse Events in Administrative Databases: Results from the Department of Veterans Affairs National Surgical Quality Improvement Program," Journal of the American College of Surgeons 194, no. 3 (2002): 257–266.[Medline]
26. "ICD-9-CM Official Guidelines for Coding and Reporting," Coding Clinic for ICD-9-CM 19, no. 2 (2002): 21–71.
27. E.L. Hannan et al., "A Methodology for Targeting Hospital Cases for Quality of Care Record Reviews," American Journal of Public Health 79, no. 4 (1989): 430–436; [Abstract/Free Full Text]Weingart et al., "Use of Administrative Data to Find Substandard Care"; Iezzoni et al., "Does the Complications Screening Program Flag Cases with Process of Care Problems?"; and L. Iezzoni et al., Project to Validate the Complications Screening Program, HCFA Contract no. 500-94-0055 (31 March 1998).
28. D.W. Bates et al., "Evaluation of Screening Criteria for Adverse Events in Medical Patients," Medical Care 33, no. 5 (1995): 452–462; and [Medline]A.M. Wolff, "Limited Adverse Occurrence Screening: An Effective and Efficient Method of Medical Quality Control," Journal of Quality Clinical Practice 15, no. 4 (1995): 221–233.
29. K.L. Kahn et al., "Health Care for Black and Poor Hospitalized Medicare Patients," Journal of the American Medical Association 271, no. 15 (1994): 1169–1174; [Abstract]J.Z. Ayanian et al., "Quality of Care by Race and Gender for Congestive Heart Failure and Pneumonia," Medical Care 37, no. 12 (1999): 1260–1269; [Medline]P.H. Stone et al., "Influence of Race, Sex, and Age on Management of Unstable Angina and Non-Q Wave Myocardial Infarction: The TIMI III Registry," Journal of the American Medical Association 275, no. 14 (1996): 1104–1112; and [Abstract]C.L. Pashos et al., "Trends in the Use of Drug Therapies in Patients with Acute Myocardial Infarction: 1988 to 1992," Journal of the American College of Cardiology 23, no. 5 (1994): 1023–1030,[Abstract]
30. H.R. Burstin, S.R. Lipsitz, and T.A. Brennan, "Socioeconomic Status and Risk for Substandard Medical Care," Journal of the American Medical Association 268, no. 17 (1992): 2383–2387; and [Abstract]E.J. Thomas and T.A. Brennan, "Incidence and Types of Preventable Adverse Events in Elderly Patients: Population Based Review of Medical Records," British Medical Journal (18 March 2000): 741–744.
31. E.J. Thomas, E.J. Orav, and T.A. Brennan, "Hospital Ownership and Preventable Adverse Events," Journal of General Internal Medicine 15, no. 4 (2000): 211–219.[Medline]
32. T.A. Brennan et al., "Hospital Characteristics Associated with Adverse Events and Substandard Care," Journal of the American Medical Association 265, no. 24 (1991): 3265–3269,[Abstract]
33. Miller et al., "Patient Safety Indicators"; and L.I. Iezzoni et al., "Using Administrative Data to Screen Hospitals for High Complication Rates," Inquiry 31, no. 1 (1994): 40–55.[Medline]
34. Brennan et al., "Hospital Characteristics."
35. J.J. Allison et al., "Relationship of Hospital Teaching Status with Quality of Care and Mortality for Medicare Patients with Acute MI," Journal of the American Medical Association 284, no. 10 (2000): 1256–1262,[Abstract/Free Full Text]
36. J.H. Silber et al., "Hospital and Patient Characteristics Associated with Death after Surgery: A Study of Adverse Occurrence and Failure to Rescue," Medical Care 30, no. 7 (1992): 615–629; and [Medline]J.H. Silber et al., "Evaluation of the Complication Rate as a Measure of Quality of Care in Coronary Artery Bypass Graft Surgery," Journal of the American Medical Association 274, no. 4 (1995): 317–323.[Abstract]
37. E.B. Keeler et al., "Hospital Characteristics and Quality of Care," Journal of the American Medical Association 268, no. 13 (1992): 1709–1714; [Abstract]E.M. Kuhn et al., "The Relationship of Hospital Characteristics and the Results of Peer Review in Six Large States," Medical Care 29, no. 10 (1991): 1028–1038; and [Medline]J.Z. Ayanian et al., "Quality of Care for Two Common Illnesses in Teaching and Nonteaching Hospitals," Health Affairs (Nov/Dec 1998): 194–205.

                      What's this?

This article has been cited by other articles:

				C Brown, T Hofer, A Johal, R Thomson, J Nicholl, B D Franklin, and R J Lilford An epistemology of patient safety research: a framework for study design and interpretation. Part 3. End points and measurement Qual. Saf. Health Care, June 1, 2008; 17(3): 170 - 177. [Abstract] [Full Text] [PDF]

				K. Van den Heede, W. Sermeus, L. Diya, E. Lesaffre, and A. Vleugels Adverse outcomes in Belgian acute hospitals: retrospective analysis of the national hospital discharge dataset Int. J. Qual. Health Care, June 1, 2006; 18(3): 211 - 219. [Abstract] [Full Text] [PDF]

				G. L. Daumit, P. J. Pronovost, C. B. Anthony, E. Guallar, D. M. Steinwachs, and D. E. Ford Adverse Events During Medical and Surgical Hospitalizations for Persons With Schizophrenia. Arch Gen Psychiatry, March 1, 2006; 63(3): 267 - 272. [Abstract] [Full Text] [PDF]

				C. Zhan and M. R. Miller Excess Length of Stay, Charges, and Mortality Attributable to Medical Injuries During Hospitalization JAMA, October 8, 2003; 290(14): 1868 - 1874. [Abstract] [Full Text] [PDF]

				S. N. Weingart and L. I. Iezzoni Looking for Medical Injuries Where the Light Is Bright JAMA, October 8, 2003; 290(14): 1917 - 1919. [Full Text] [PDF]