|
A National Profile Of Patient Safety In U.S. Hospitals
Measures based on routinely collected data would be useful to examine the epidemiology of patient safety. Extending previous work, we established the face and consensual validity of twenty Patient Safety Indicators (PSIs). We generated a national profile of patient safety by applying these PSIs to the HCUP Nationwide Inpatient Sample. The incidence of most nonobstetric PSIs increased with age and was higher among African Americans than among whites. The adjusted incidence of most PSIs was highest at urban teaching hospitals. The PSIs may be used in AHRQ’s National Quality Report, while providers may use them to screen for preventable complications, target opportunities for improvement, and benchmark performance.
Several years ago researchers at the Agency for Healthcare Research and Quality (AHRQ) developed a set of Patient Safety Indicators (PSIs) for identifying suspected instances of compromised patient safety based on hospital administrative data.1 This initial set of PSIs was designed to target events with a high likelihood of representing errors, such as foreign bodies left during procedures. Realizing the value of such indicators for institutional case-finding and patient safety improvement initiatives, AHRQ contracted with the University of California–Stanford Evidence-Based Practice Center to expand the initial set of indicators to address a broader range of patient safety concerns, to obtain clinical input about face validity, to review published evidence on coding and construct validity, and to revise indicators as appropriate. This final set of PSIs is now a component of AHRQ’s Quality Indicators.2 This paper presents national data on the incidence of these events over time and their association with patient and hospital characteristics.
Initial PSIs. The development of the initial PSIs was based on the Institute of Medicine’s (IOM’s) definition of patient safety, which is "freedom from accidental injury due to medical care, or medical errors."3 A more recent IOM report defined medical errors as "the failure of a planned action to be completed as intended or the use of a wrong plan to achieve an aim...[including] problems in practice, products, procedures, and systems."4 This definition excludes acts that do not achieve desired outcomes but are not the result of negligence, outcomes resulting from underlying or comorbid illnesses, and outcomes known to be unavoidable risks of a procedure. AHRQ researchers evaluated existing measures of adverse events, searched the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) for promising diagnosis codes, defined inclusion and exclusion criteria, grouped the resulting codes into twelve PSIs, and tested them using New York State data.5 Literature review. To expand and refine these initial PSIs, we searched Medline and EMBASE for articles that described, evaluated, or validated potential indicators of medical errors, patient safety, or preventable complications based on administrative data coded with ICD-9-CM codes. We also searched article bibliographies and queried team members to identify references. The Complications Screening Program (CSP) by Lisa Iezzoni and colleagues, which includes twenty-eight complications "that raise concern about the quality of care based on the rate...at individual hospitals," was an especially important source because of its extensive evaluation and its relevance to quality improvement.6 Indicator definition. We reviewed the initial AHRQ PSIs, adding codes for potential safety-related events from the 1998–2001 ICD-9-CM revisions and the CSP. This process was guided by conceptual considerations and recent evidence regarding preventability.7 The resulting list of more than 200 ICD-9-CM codes was then grouped into clinically coherent indicators. We specified a population at risk for each indicator, restricting the denominator to patients for whom that complication seemed more likely to be preventable. CSP-derived indicators were retained only if they met two validation criteria: (1) independent review of a random sample of 1,298 inpatient records from two states revealed that at least of one of three groups of reviewers (that is, coders, nurses, and physicians) confirmed the diagnosis in at least 75 percent of reported cases; and (2) "process of care failures" were identified more often by physician reviewers among cases flagged by the CSP indicator than among randomly sampled controls.8 All indicators were designed for use at the hospital level, but six were also adapted for use at the community level.9 Face/consensual validation. We subjected thirty-four indicators to face validation by an expert coding consultant and consensual validation by eleven panels of five to nine expert clinicians nominated by twenty-eight professional organizations or chapters thereof. Our methodology was adapted from the RAND/UCLA Appropriateness Method and included four steps: (1) an initial independent assessment of each indicator using a ten-item questionnaire covering several key domains (for example, "overall...usefulness," "likely to be preventable," "likely to represent true medical error or negligence"); (2) feedback of questionnaire responses, highlighting each individual’s rating relative to the group; (3) a conference call with facilitated discussion focusing on disagreements and suggestions for improvement; and (4) a final independent assessment using the same questionnaire.10 Panelists were allowed to suggest changes to indicator definitions before or during the conference call; these changes were adopted if supported by consensus. Each of fifteen surgical complication measures was independently reviewed by one of three multispecialty panels and one of three surgeon panels. Each of the other nineteen proposed indicators was reviewed by one of five other multispecialty panels. Their postdiscussion ratings of overall usefulness were used to select indicators for an "accepted" list. To qualify, an indicator had to achieve a median score of 7 or greater (on a 1–9 scale) without disagreement, where disagreement was defined as two or more panelist ratings at each extreme (that is, 1–3 and 7–9).11 Several indicators that were accepted by the relevant panel were subsequently rejected by the research team because of concerns about coding or other operational issues, leaving twenty accepted indicators.12 Data analyses. We estimated the national incidence of potential safety-related events by applying "accepted" PSI definitions to the 1995–2000 Healthcare Cost and Utilization Project (HCUP) Nationwide Inpatient Sample (NIS). The NIS is the largest publicly available U.S. all-payer database, with data from nearly 1,000 hospitals in twenty-eight states, approximating a 20 percent stratified sample of nonfederal short-term, general, and other specialty hospitals.13 Based on the NIS sampling design, we weighted all results to generate estimates for the entire population of discharges from U.S. community hospitals. We then evaluated differences in the incidence of potential safety-related events across age/sex and racial/ethnic strata and across hospital strata defined using the contemporaneous American Hospital Association (AHA) survey.14 Analyses of PSI incidence by race/ethnicity and hospital characteristics were adjusted for age, sex, age-sex interactions, comorbidities, and diagnosis-related group (DRG) clusters. Because race/ethnicity was unknown for about 20 percent of discharges in the 2000 NIS, we repeated analyses of race/ethnicity using the 2000 HCUP State Inpatient Databases from the fourteen states with at least 93 percent reporting.15 Comorbidity adjustment was performed using an updated version of software developed by AHRQ researchers.16 We defined DRG clusters by aggregating adjacent DRGs with and without comorbidities and complications (and age-stratified DRGs), and by excluding certain DRGs to avoid adjusting for complications.
National incidence. The twenty accepted PSIs represent a selective list of potential safety-related events deemed amenable to detection using administrative data, adequately coded in previous studies, and sensitive to the quality of care.17
We identified about 1.12 million potential safety-related events that occurred in 1.07 million hospitalizations at nonfederal acute care facilities in 2000 (Exhibit 1
Temporal trends. Temporal trends from 1995 through 2000 vary markedly for different hospital-reported, potential safety-related events.18 Postoperative medical and nursing-related adverse events, including respiratory failure (31 percent increase), infection due to medical care (14 percent), decubitus ulcer (19 percent), septicemia (41 percent), and thromboembolism (42 percent) steadily increased in incidence during this period; physiologic/metabolic derangements and hip fracture did not. The increase in the incidence of postoperative respiratory failure between 1997 and 1999 may have been attributable to the introduction of a new ICD-9-CM code for "acute and chronic respiratory failure" (518.84) in October 1998. However, the increases in the incidence of decubitus ulcer; infection "following infusion, injection, transfusion"; postoperative septicemia; and postoperative thromboembolism from 1995 to 2000 cannot be attributed to coding changes. By contrast, most of the technical complications or errors, including iatrogenic pneumothorax, postoperative abdominopelvic wound dehiscence, and postoperative hemorrhage/hematoma, decreased in incidence between 1995 and 2000. Accidental punctures and lacerations represent an exception: Their incidence rose 7 percent during this period. The overall incidence of obstetric trauma decreased about 3 percent, apparently because of less frequent use of forceps and vacuum assistance. These options might decrease the need for cesarean delivery in some women with second-stage labor difficulties, but they are associated with an increased risk of perineal laceration.19 The incidence of birth trauma decreased 56 percent, from 1.51 percent to 0.67 percent, between 1997 and 2000. The rarest sentinel-event PSIs also decreased steadily in incidence between 1995 and 2000: 18 percent for anesthesia reactions and complications, 7 percent for foreign bodies left during procedures, 10 percent for death in low-mortality DRGs, and 40 percent for transfusion reactions. Inpatient death rates following major complications (that is, failure to rescue) decreased by 6 percent (from 18.6 percent to 17.4 percent) between 1995 and 2000.
Patients at risk.
The risk of hospital-reported, potential safety-related events varies across sex and age categories (Exhibit 2
The risk of these events also varies across racial/ethnic categories after adjustment (Exhibit 3  ). White inpatients had a slightly higher risk of certain technical difficulties with procedures, such as anesthesia reactions and complications, iatrogenic pneumothorax, and obstetric trauma. By contrast, African American inpatients had a much higher risk of most medical and nursing-related postoperative complications, such as decubitus ulcer; infection following infusion, injection, or transfusion; and postoperative physiologic and metabolic derangements, thromboembolism, and sepsis. Mortality-related events and the rarest sentinel-event indicators were similarly frequent across racial/ethnic categories. Hispanic patients often had lower risk than either white or African American patients had. These results were confirmed using the fourteen state inpatient databases with the best reporting of race/ethnicity.
Hospital characteristics. In general, differences in the incidence of hospital-reported, potential safety-related events across hospital strata were small and often not meaningful after age, sex, age-sex interactions, comorbidities, and the reason for admission (that is, DRG cluster) were adjusted for. The differences across regions were especially small, with no consistent trends (data not shown).
For-profit hospitals had the lowest incidence of all obstetric indicators but the highest incidence of most other indicators for which incidence differed significantly across strata (for example, anesthesia reactions and complications, decubitus ulcer, foreign body left during procedure, infection due to medical care, postoperative respiratory failure, and sepsis). Nonprofit hospitals had the highest incidence of all obstetric indicators. The incidence of mortality-related events differed minimally across hospital control categories (Exhibit 4
Rural hospitals had the lowest incidence of most hospital-reported, potential safety-related events but the highest incidence of anesthesia reactions and complications, accidental puncture or laceration, postoperative hip fracture and abdominopelvic wound dehiscence, and birth trauma. Urban teaching hospitals had the highest incidence of most events but the lowest incidence of anesthesia reactions and complications, postoperative hip fracture, and birth trauma. Similarly, large hospitals had the highest incidence of most events but low incidence of anesthesia reactions and complications, postoperative hip fracture and respiratory failure, and abdominopelvic wound dehiscence.21
There are several approaches to profiling inpatient safety. One approach involves reviewing medical records or querying physicians and other health care providers (through confidential reporting systems) to find serious errors and then looking forward in time to ascertain relevant outcomes.22 A related approach is to ascertain errors as they occur through ethnographic observation of health care teams.23 The other general approach is to identify adverse outcomes that are often preventable and then to look backward to ascertain how and why they occurred. This approach was used successfully in the Harvard Medical Practice Study and follow-up studies in Utah and Colorado, but the screening criteria in those studies necessitated nurse review of every record.24 By contrast, the AHRQ PSIs screen hospitalizations based only on computerized data that are collected by nearly all U.S. hospitals and are widely available for public use. The key advantages of this method are its low cost, unobtrusiveness, and universality. Using these data to monitor patient safety requires no additional effort by health professionals. The ultimate usefulness of these screening indicators, however, will result from selective review of flagged records and other efforts to identify and improve systems that contribute to the occurrence of safety-related events. Of course, this event-based approach to identifying patient safety problems has several notable limitations. First, we can only find events for which there are corresponding ICD-9-CM codes. For example, no specific codes exist for iatrogenic injuries to nerves or urinary tract structures. Furthermore, our ascertainment of adverse events depends on the quality and completeness of coding. Although the principal diagnosis is generally well coded in administrative data, comorbidities and complications are often underreported.25 ICD-9-CM coding guidelines actually preclude reporting complications that seem "integral to the disease process" or lack physician documentation indicating "their clinical significance."26 Most important, the mere occurrence of a complication provides no information about its severity, timing, etiology, or preventability. Some events captured by the PSI logic may have been present at admission or may reflect the unalterable natural history of an illness. The former problem may be remedied by requiring hospitals to report whether each diagnosis was present at admission, as California and New York already do. We have limited data from prior studies on how often these events are related to errors in diagnosis or management.27 Although our expert panels agreed that all of these PSIs are useful, there was some disagreement about the extent to which they result from negligence or medical error. We generally emphasized specificity over sensitivity in designing these PSIs, by excluding subsets of patients for whom a complication seemed less likely to be preventable. As a result, several PSIs are limited to elective-surgery patients, even though these events occur as frequently or more so after emergency surgery. By restricting the eligible population, we sacrificed generalizability to the entire population of hospitalized patients. Compared with other screening methods based on billing data, which include broad criteria such as death, readmission, transfer to a special care unit or to another hospital, return to the operating room, cardiac arrest, and prolonged length-of-stay, the AHRQ PSIs seem likely to capture fewer preventable adverse events, but also fewer false positives.28 Our results illustrate how the AHRQ PSIs may be used to improve our understanding of the epidemiology of patient safety. It is reassuring that the incidence of most PSIs has fallen since 1995, with the notable exception of postoperative medical complications, decubitus ulcer, and infection due to medical care. Although the number of states included in the NIS increased from nineteen in 1995–96 to twenty-eight in 2000, this change seems unlikely to have selectively affected certain PSIs. The increased incidence of postoperative medical complications probably reflects better documentation and coding, or the impact of shorter stays, increased acuity, nurse staffing shortages, and other stresses facing hospitals. By comparing our results with those of studies based on detailed review of medical records, we hoped to learn more about indicator validity. For example, our results are consistent with previous studies that found less frequent use of effective cardiac medications, less timely use of antibiotics for pneumonia, and poorer overall quality of care among African American inpatients than among white inpatients, especially after hospital location and teaching status were adjusted for.29 However, previous studies of preventable adverse events have not shown differences across racial/ethnic categories.30 Prior studies have not found consistent effects of hospital ownership or control. For example, for-profit and nonteaching public hospitals in Colorado and Utah had significantly higher rates of preventable "operative adverse events" and "adverse events due to delayed diagnoses and therapies" than nonprofit hospitals had.31 By contrast, for-profit hospitals in New York had a lower percentage of adverse events due to negligence than nonprofit hospitals had.32 The AHRQ PSIs showed a mixed pattern, although rates for nonobstetric indicators were generally either highest at for-profit hospitals or similar across categories. The high incidence of most PSIs in urban teaching hospitals is worrisome, although our findings are consistent with those reported by the developers of the initial PSI algorithms and the Complications Screening Program.33 Differences in severity of illness and use of surgical interventions still confound these measures, given our limited ability to adjust risk using administrative data. In the Harvard Medical Practice Study, the proportion of adverse events resulting from negligence was lower at teaching hospitals than at nonteaching hospitals.34 In a random sample of Medicare patients with acute myocardial infarction, effective medications were administered more often at major teaching hospitals than at minor teaching or nonteaching hospitals.35 Hospitals with surgical housestaff had higher failure-to-rescue rates after prostatectomy and cholecystectomy than nonteaching hospitals had, but lower failure rates after coronary bypass surgery.36 Studies in which physicians reviewed medical records to find quality problems have generally reported better care at teaching and large nonteaching hospitals than at small nonteaching hospitals, although differences in documentation practices may bias these results.37 Given these limitations and concerns, the AHRQ PSIs will be useful primarily as screening tools for hospitals and hospital systems, medical groups, health plans, and purchasers to identify potential patient safety problems for further investigation. Providers can use them to flag suspicious cases for in-depth review, increasing the efficiency and yield of their quality improvement activities while focusing attention on system failures responsible for adverse events. Public health agencies and provider associations can monitor PSI rates at the local, state, regional, and national levels. These community-level PSI rates should serve as useful benchmarks for organizations evaluating their own performance or tracking the impact of safety-enhancing interventions. Given the current paucity of tools to identify safety-related events, the AHRQ PSIs also could be used by the Centers for Medicare and Medicaid Services (CMS) and state agencies to track hospital-specific performance and guide quality improvement efforts. Because of variability in coding and uncertain preventability, these measures are not suitable for public reporting at the provider level. Ongoing and future research will establish whether and when these indicators are valid measures of safety-related hospital performance. A substantial portion of AHRQ’s investment of more than $50 million in patient safety focuses on reporting systems. As results from these studies become available, policymakers will be better able to design surveillance systems that triangulate chart review, administrative data, and self-reporting to achieve maximal identification of safety-related events.
Patrick Romano is an associate professor of medicine and pediatrics at the University of California, Davis, Division of General Medicine. Jeffrey Geppert is a research analyst at the National Bureau of Economic Research (NBER) in Stanford, California. Sheryl Davies is a research manager at the Center for Primary Care and Outcomes Research, also in Stanford, and Kathryn McDonald is the center’s executive director. Marlene Miller is director of quality and safety initiatives at the Johns Hopkins Children’s Center in Baltimore, Maryland. Anne Elixhauser is a senior research scientist at the Agency for Healthcare Research and Quality in Rockville, Maryland. This work was supported by a contract from the Agency for Healthcare Research and Quality.
This article has been cited by other articles:
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Top
Study Methods
Results
