|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
The Impact Of Quality-Reporting Programs On Hospital Operations
We used data from the 2005–06 Community Tracking Study site visits to examine the impact of quality reporting on hospitals data collection and review processes, feedback and accountability mechanisms, quality improvement activities, and resource allocation. Individual hospitals participate in multiple, varied reporting programs with distinct effects on hospital operations. Reporting programs play complementary roles in encouraging quality improvement but are poorly coordinated and command sizable resources, in large part because of inadequate information technology. Policy should be directed at encouraging formal assessments of how individual and combinations of programs affect quality outcomes, and the development of adaptable information systems.
ASSESSING THE IMPACT OF QUALITY REPORTING on hospital operations can inform decisions about whether and how programs should be modified to best promote quality improvement (QI). During the past decade, programs for reporting hospital quality performance have proliferated, from local programs to broader initiatives such as Leapfrogs patient-safety program.1 In 2002 the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) began requiring such reporting.2 In 2003 the Centers for Medicare and Medicaid Services (CMS) launched its voluntary Hospital Quality Initiative (HQI). Hospitals response was anemic at first, until the Medicare Prescription Drug, Improvement, and Modernization Act (MMA) established that nonparticipating hospitals would not receive a 0.4 percent annual payment update. Participation then increased dramatically, and now nearly all eligible hospitals are reporting.3 Several studies document the benefits of providing hospitals feedback on their quality performance, particularly comparative information.4 An evaluation of the HQI pilot found that hospitals responded by placing higher priority on quality performance, improving data collection and documentation, and diverting resources from other priorities but rarely engaging in new QI activities.5 Other studies suggest that reporting results in intensification of QI activity, a greater focus on quality on the part of hospital management, and improved outcomes, especially when data on performance are publicly disclosed.6 Other studies found inconsistent effects on outcomes or QI activity.7 In this paper we build on earlier work by examining the interactions between different quality-reporting programs and their impact on operations at hospitals in diverse urban communities, from the perspectives of hospital executives and front-line staff, and focusing on care for patients with congestive heart failure (CHF). We defined "quality reporting" broadly to include programs that monitor quality performance, with the potential to influence hospital management through public reporting, private benchmarking, or specific incentives.
Since 1996 the Community Tracking Study (CTS) has conducted five rounds of site visits, each involving approximately 1,000 interviews in twelve representative U.S. metropolitan areas, to examine local changes in the organization, financing, and delivery of health care. CTS design details are available elsewhere.8 For this paper we drew on 111 Round Five interviews conducted between December 2004 and June 2005 with five hospital association leaders; representatives from JCAHO, the CMS, and six state reporting programs; and ninety-eight executives and staff (twenty-one chief executive officers, twenty-one vice presidents of nursing, thirty quality officers, and twenty-six clinical directors) at two to four of the largest hospitals or hospital systems in each market. The sample comprises thirty-six of the 139 total hospitals in the twelve markets. We focused on the largest hospitals to capture trends affecting the largest proportion of consumers. Each hospital contributed an average of 2.7 interviews (range, 1–4). "Clinical directors" were staff members identified by quality officers to be most responsible for QI activities for CHF. We focus on CHF because of its high prevalence and associated costs of care and because JCAHO and CMS initiatives target it.9 All but two hospitals had a program specific to CHF. Semistructured interviews included open-ended questions about specific reporting programs in which hospitals participated. We sought respondents perceptions of programs impact on the hospitals organizational culture around QI; its priorities, budget, staffing levels, and strategies; data collection and review activities; feedback and accountability mechanisms; and investments in and use of health information technology (IT). We also surveyed clinical directors on their use of eleven QI tools specifically targeting CHF. We selected these tools based on evidence supporting their effectiveness in QI and because they represent a range of intensity in the resources and coordination required to implement them.10 To better categorize hospitals responses to different reporting programs, we identified axes along which programs vary, besides their local versus national nature: (1) sponsorship—by purchaser (for example, Medicare), regulator (for example, state health departments), private insurer, professional groups (for example, Society of Thoracic Surgeons), or other private organizations (for example, hospital consortia); (2) program type—those requiring hospitals to submit primary data for public reporting (for example, JCAHO), those requiring primary data for private benchmarking (for example, hospital consortia), and those that rank performance based only on secondary data such as claims or patient surveys (for example, HealthGrades); (3) mandatory versus voluntary—a spectrum rather than distinct categories (most programs are voluntary), closely related to whether incentives are attached to participation; (4) incentives—incentives explicitly tied to participation perceived as rewards (for example, pay-for-performance bonuses) or punitive (for example, loss of accreditation); (5) quality improvement support—whether programs provide prescriptive information to guide hospitals QI activities (for example, Society of Thoracic Surgeons); and (6) inclusion of clinical outcome measures—whether programs include outcomes or only structural or process-of-care measures (for example, the 100,000 Lives Campaign). We corroborated hospitals participation in national reporting programs using data from JCAHOs Quality Check and the CMS Hospital Compare Web sites on hospital performance.11
Multiple programs We found that hospitals respond to multiple reporting programs: All thirty-six hospitals report to the CMS and JCAHO based on Hospital Compare and Quality Check. However, respondents at every hospital also reported participating in additional programs—a mean of 3.3 (range, 1–7), often starting years before CMS and JCAHO reporting. In total, respondents mentioned thirty-eight unique programs. Exhibit 1
Institutional support and attitudes Quality officers, CEOs, and hospital association leaders concurred that linkages to payment, JCAHO accreditation, and peer pressure from public benchmarking have made quality measurement and improvement higher priorities for hospital leadership. Some executives perceived the potential for purchasers and consumers to eventually make care choices based on performance data, but few believed that this was imminent. Heightened organizational attention to quality performance manifested in myriad ways, including (1) explicit inclusion of QI priorities on trustees agendas and in hospitals formal strategic planning; (2) boards and senior management accepting more regular, defined responsibilities for reviewing performance data and approving QI strategies; (3) restructuring of executive compensation to include quality performance–based incentives; (4) half of quality officers reporting that it was easier to lobby management for resources for quality measurement and QI; and (5) senior management actively exercising leadership with front-line staff. For example, at an Orange County (California) hospital, QI staff who met with resistance from physicians during a root-cause analysis of poor performance asked the CEO to come to the care unit and intervene in person, to immediate positive effect.12 Respondents similarly credited CMS and JCAHO programs with improving physicians attitudes toward quality measurement and improvement. Aside from stronger mandates from leadership, quality officers could leverage the payment and accreditation consequences in these working relationships. Quality officers and CEOs believed that physicians liked having focused sets of clinical priorities on which to work. They believed that physicians responded to performance feedback, including individual profiling at some hospitals, and peer pressure from public reporting of benchmarked performance data. Respondents found physicians more deferential to QI staff than they were before participating in quality reporting and more engaged in QI activities that some once dismissed. Specific clinical conditions. Reporting programs have also affected the specific clinical conditions and quality measures on which hospitals focus. This influence was not always viewed positively, and it varied by both program type and whether the program includes ongoing QI support. Many hospital staff who were engaged in long-standing QI activities dismissed CMS and JCAHO requirements as having a minor impact on QI priorities. But CEOs and quality officers at most hospitals contended that reporting programs, particularly those of the CMS and JCAHO, focus on an artificially limited number of objectives. The opportunity cost was a shift in attention and resources away from other important clinical areas. Response to incentives Hospitals respond to incentives, but they value ongoing support for QI. Respondents perceived the impetus to participate in JCAHO and CMS programs to be of a "push" nature, because these are effectively mandatory and involve public disclosure. In response, hospitals directed resources specifically to CMS and JCAHO core conditions—CHF, community-acquired pneumonia, heart attacks, and stroke—but without taking standardized approaches to improving performance. In contrast, respondents described the focus of some other programs as a "pull," because of the QI support they offered in the form of prescribed changes in care processes. Several quality officers cited as a reason for focusing on hospital-acquired pneumonia the Institute for Healthcare Improvements (IHIs) "ventilator bundle," which recommends a set of evidence-based practices, such as daily withdrawal of sedatives, to prevent ventilator-associated pneumonia. Respondents found these programs attractive both because they dont leave hospitals flailing about trying to identify evidence-based interventions on their own and because they encourage a culture of continuous QI. Respondents most often mentioned programs sponsored by IHI, state quality improvement organizations (QIOs), and professional organizations (Society for Thoracic Surgery, American College of Cardiology). Adequacy of resources Resources for quality measurement and improvement have increased, but they remain inadequate. Nearly all respondents, including representatives of reporting programs, believed that reporting increases hospitals costs, for both compliance and processes to improve performance. Half of quality officers reported staff increases of up to twelve full-time equivalents (FTEs) devoted to reporting and QI in the previous year. In markets such as northern New Jersey, where hospitals are financially less healthy, respondents considered reporting to be a particular cost burden. At many hospitals, management diverted staff from other tasks, such as financial reporting. Other hospitals simply gave existing staff more responsibilities. Half of clinical directors believed that reporting had resulted in a major increase in their workload; only a minority reported no change. Some respondents found it difficult to even assess the net cost burden of reporting because the associated costs are spread over a variety of hospital cost centers, reporting requirements change over time, and the impact of improved outcomes on finances is difficult to measure. Hospitals have committed more resources to reporting and QI activities largely in the form of more staff equivalents, new or upgraded software, and contracting with vendors to manage their data, but rarely in new hardware. Many hospitals committed more FTEs to chart abstraction and data review. CEOs and quality officers noted that staffing burdens increase with the number of programs they participate in, even for the same conditions, because technical formatting requirements still differ (for example, between JCAHO and CMS reports). The commonality underlying much of the staffing burden associated with reporting is the inadequacy of existing IT systems. Some hospitals still rely on paper records. Many that do have IT inherited patchwork systems that are poorly integrated across departments, which makes it difficult to generate cohesive reports on care for any given patient. At a Little Rock hospital with an electronic medical record (EMR) containing admission and drug data, staff still need to access separate laboratory, diagnostic, and discharge data systems to get complete data on a CHF patient. Even those with evolved IT support have not customized the software to reporting requirements, which makes it necessary to review individual charts for measures such as timing of aspirin for heart attack patients (staff had to sum the time patients spent in the emergency department and on the floor before aspirin was given). Many quality officers and CEOs spoke of plans to purchase new or upgraded IT hardware—higher-level capital investment decisions that are dependent on other complex factors. Respondents thus painted a picture of scrambling to meet immediate reporting requirements within their limited abilities and of engaging in longer-term strategic planning in anticipation of expanded quality reporting. Feedback and accountability Mechanisms for feedback and accountability have grown more sophisticated. At many hospitals, reporting resulted in greater frequency of review of performance data, particularly for CMS and JCAHO conditions. A third of hospitals formally review data at least monthly, more frequently than required for reporting. A few commit to even more rapid feedback to clinical staff, sometimes in real time, for informal review or root-cause analysis, especially for examining outlier cases. Respondents at hospitals that were focused on addressing outlier cases were especially appreciative of programs with rapid data turnaround times, such as the American Hospital Associations (AHAs) "Get with the Guidelines" program. Participation in reporting programs also influenced how hospitals disseminate performance data. Hospital systems tend to send individual facilities their disaggregated performance scores. Nearly all hospitals claimed that trustees review performance data, along with CEOs and clinical department leadership in many cases. Respondents credited the HQI in particular with propelling quality performance to higher levels of attention for their boards of trustees. At many hospitals, quality performance became an explicit priority incorporated into formal strategic goals, newly on par in importance with financial performance. Most hospitals share their overall performance scores with their entire staffs. At eight hospitals, reporting led to implementation of or planning for individual physician profiling, sometimes using benchmarks based on specialty peer groups, or written alerts identifying instances of missed care. But in nearly all cases, hospitals proved that they were sensitive to physicians concerns by adjusting profiles for case-mix severity and by reassuring physicians about data confidentiality. Respondents cited the HQI as the primary motivator for physician profiling; some hospitals plan to profile care only for CMS conditions. Hospitals restructured staff into multidisciplinary teams that were assigned to track and improve overall performance and performance for individual conditions, particularly those targeted by the CMS. These teams are more integrated across different service areas than in the past. And while a few hospitals make a single manager responsible for improvement in each clinical area, most are content to assign QI responsibility to teams, letting them select the best approaches. At least six hospitals formally tied executive or physician compensation to quality performance, again largely driven by the HQI. At hospitals in Miami, Orange County, and Phoenix, executives have 30–70 percent of their bonuses based on CMS scores. At an Indianapolis hospital, sixty top managers are subject to bonus withholds for poor performance. Even hospitals that instituted such incentives before CMS reporting began modified their pay structures afterward to incorporate CMS conditions. Adoption and modification of specific QI interventions Hospital respondents were divided on whether reporting as a whole had much effect on the quantity or type of specific process changes their hospitals made to improve care. Those who felt that reporting had little impact on QI interventions pointed out that their hospitals had been active in QI before participating in newer programs, or that their choices of interventions were largely internally driven, or both.
According to clinical directors, reporting had a moderate impact on hospitals use of specific QI interventions to improve care for CHF patients. Hospitals often adopted wholesale the recommendations of programs offering QI support. On average, specific reporting programs influenced the adoption or modification of half of the QI tools we examined (Exhibit 2
Two cases typify how reporting influenced use of QI tools. At an Orange County hospital, staff reevaluated an existing CHF program because of reporting, creating new standing-order sets and critical pathways. But they also modified an existing Web portal system to integrate data on patients from different hospital units and to improve coordination. Another hospital in Syracuse was already using critical pathways for CHF but did not collect or review data on staff compliance with pathways until the hospital began quality reporting. After switching to electronic data collection for reporting purposes, staff could analyze pathway compliance and direct interventions. Respondents disagreed on whether or not reporting had spillover effects on QI activities for unmeasured conditions. Many quality officers and CEOs who felt that reporting improved clinical care believed that those gains were limited to targeted conditions. But some hospitals generalized lessons learned from one program to other clinical areas. One quality officer reported that deliberately applying care processes recommended in the IHIs ventilator bundle to stroke care resulted in major improvement in stroke outcomes. More generally, some hospital leaders believed that reporting caused a shift in their staffs ability and eagerness to identify problem areas not related to measured conditions and to proactively address them. Reporting "raised their antennae for other patients," as one Orange County quality officer put it.
U.S. hospitals are actively engaged in a dizzying array of quality-reporting programs. We did not find persuasive evidence of systematic differences between hospitals engaged in more versus fewer programs in terms of system affiliation, academic health center status, or other characteristics, although hospitals participating in programs with QI support were more likely to consider themselves "leaders" in QI. Variation in the programs in which hospitals participate seemed largely idiosyncratic results of local market history (for example, influence of physician-leaders, presence of local programs). The thirty-eight programs we studied vary not only in clinical focus and reporting requirements but also in sponsorship, incentives, mandatory versus voluntary nature, inclusion of outcome measures, and whether or not they offer concrete QI support. Their impact on hospital operations is equally varied. Private health plan programs, disease registries, and public programs incorporate very different priorities and methods. But sponsors should appreciate that their variety presents an opportunity to learn from users about the benefits and drawbacks of each. Indeed, reporting programs play potentially complementary roles. National programs with deep payment or regulatory leverage can influence nearly all hospitals and can garner attention from those that would otherwise not prioritize QI highly. Smaller, specialized reporting programs can be more responsive than larger programs to the needs of individual organizations. Voluntary programs, particularly those offering ongoing QI support, help focus priorities at hospitals that are eager to take on the more challenging goals of continuous QI. JCAHO and CMS programs have clearly influenced how hospitals prioritize QI goals. They have improved organizational culture and staff attitudes toward QI and have resulted in hospitals devoting additional resources toward QI and honing feedback and accountability mechanisms. This is not surprising, given their near-mandatory nature. Evidence that changes in leaderships focus has rapid, trickle-down effects on front-line staff suggests that national programs have "raised the floor," stimulating hospitals to actively engage in QI that had not been doing so before. However, respondents claim that the regulatory and payment leverage of national programs artificially narrows the scope of QI in which hospitals might otherwise engage, especially for those with long institutional histories of QI. Whether that positively or negatively affects clinical care remains debatable, as hospitals that consider themselves QI leaders do not necessarily score well by objective measures such as JCAHOs benchmarks (data not shown); and because of formidable challenges in quantifying the effects of reporting on nontargeted conditions. Others have suggested that given limited resources, policymakers might consider shifting the clinical focus in these programs over time. Doing so would both stimulate improvement in new clinical areas as old ones are mastered and allow time to collect data on the sustainability of improved performance after the reporting spotlight has moved elsewhere.13 Conversely, leaders of other programs might consider that much of the perceived influence of JCAHO and the CMS derives from the "stick" nature of their incentives rather than from "carrots." All reporting programs could benefit from more deliberate coordination to decrease reporting burden. Coordination could foster more consistent or complementary sets of target conditions across programs and standardized technical reporting formats. Although JCAHO and CMS initiatives were tightly coordinated in design, this cooperation has not extended to most other programs, particularly private programs. Policymakers could support coordination between programs in tandem with wider adoption of and improvements in IT, including the design of systems adaptable enough to allow them to be customized for different reporting requirements. This will be challenging, given the variety of data required for standardized quality measures and likely shifts in reporting focus over time. Policymakers may face the daunting task of forcing a convergence between the evolution of IT, quality measures, and reporting requirements, or they must accept the continued inefficient allocation of hospitals resources. Finally, hospitals extensive participation in quality reporting raises questions about the ultimate effects of reporting on clinical outcomes. Recent studies suggest that performance has improved since JCAHO and CMS reporting, but they have not demonstrated a causal link.14 And, to our knowledge, no analyses have considered interactions between multiple programs as factors in observed improvement or lack thereof. More generally, hospitals have committed sizable amounts of resources to reporting without sound evidence on the effectiveness of even many individual programs. Policymakers and program sponsors should support research to fill this critical data gap. Our study has several limitations. Because we focused on the largest hospitals in a small number of urban communities, our findings might not be generalizable to smaller or rural hospitals. We also examined respondents perceptions of the impact of reporting, and we could not assess the actual effects on quality outcomes. Measures of impact on outcomes are best addressed quantitatively. Similarly, a general trend of hospitals paying more attention over time to quality issues might have contributed to the changes in operations we noted here. HOSPITAL STAKEHOLDERS HAVE ACCEPTED THE FACT that purchasers, insurers, and potentially consumers demand objective measures of their performance. They expect that quality reporting will grow in scope and become more closely aligned with payment and other incentives. The challenge for policymakers is to take advantage of this encouraging development and an attentive audience, to foster coordinated development of reporting and maximize its impact on the quality of patient care.
Hoangmai Pham (mpham{at}hschange.org) is a senior health researcher at the Center for Studying Health System Change (HSC) in Washington, D.C. Jennifer Coughlan is a research assistant at Mathematica Policy Research, also in Washington. Ann OMalley is a senior health researcher at HSC. The authors are grateful to Thomas Bodenheimer, Paul Ginsburg, and Cara Lesser for comments on earlier drafts. Community Tracking site visits are supported by a grant from the Robert Wood Johnson Foundation to the Center for Studying Health System Change.
This article has been cited by other articles:
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||