Health Affairs, 24, no. 3 (2005): 843-853 doi: 10.1377/hlthaff.24.3.843 © 2005 by Project HOPE	New Online   Obama's Success & Next Steps   3 Health Policy Options for President Obama   A Hybrid Proposal   Bipartisan Compromise

This Article Abstract Reprint (PDF) Supplemental Exhibits Submit a response to this article Alert me when this article is cited Alert me when Comments are posted Alert me if a correction is posted Services E-mail this article to a friend Similar articles in this journal Similar articles in ISI Web of Science Similar articles in PubMed Alert me to new issues of the journal Add to My Personal Archive Download to Citation Manager Reprints & Permissions Citing Articles Citing Articles via ISI Web of Science (44) Citing Articles via Google Scholar Google Scholar Articles by Audet, A.-M. J. Articles by Schoenbaum, S. C. Search for Related Content PubMed PubMed Citation Articles by Audet, A.-M. J. Articles by Schoenbaum, S. C. Related Collections Physicians Quality Of Care

DataWatch

Measure, Learn, And Improve: Physicians’ Involvement In Quality Improvement

Abstract

 
Payers, accreditors, and consumers are using quality improvement (QI) methods, but little is known about whether physicians do so. The results from this 2003 national physician survey indicate that most do not. Physicians do not routinely use data for assessing their performance and are reluctant to share those data. They infrequently participate in redesign activities. Physicians in larger and salaried groups are more likely to be engaged in QI. The science of QI has been "institutionalized" but not yet "professionalized."Accelerating physicians’ adoption of and participation in QI requires building the infrastructure to support quality and paying attention to professionalism, knowledge, and skills.

The extent of physicians’ resistance to QI is not well known. Few data exist to describe the degree of variation and the factors that might lead to greater buy-in from some groups of physicians. In an effort to close the data gap, this paper reports the results of a survey designed to gain a better understanding of physicians’ opinions about and involvement in QI and factors that are associated with their attitudes toward QI. The framework for the study consists of the basic QI model developed by W. Edwards Deming and Joseph Juran, which links measurement and feedback to learning that can lead to improvement—the so-called Shewhart Cycle or the plan-do-act cycle, which translated to fit clinical practice, becomes "practice-measure-improve."⁵ We describe how physicians have implemented the "measure" part of the cycle—what kinds of data they have access to—and the "improve" part of the cycle—whether they engage in QI activities. Given the healthy but at times heated public debate about measures of performance, and given their close association to professional accountability, we also explore physicians’ willingness to share these data with various parties.

Study Data And Methods

Top Study Data And Methods Study Results Results Of The Multivariate... Discussion NOTES

 
Data source and study population. Data are from the 2003 Commonwealth Fund National Survey of Physicians and Quality of Care conducted between March and May 2003. The self-administered questionnaire was mailed to 3,598 U.S. physicians, randomly selected from an American Medical Association (AMA) Physician Masterfile, a list including AMA members and nonmembers. All physicians in the sample were involved in direct care of adults and had been in practice at least three years after residency. Specialists unlikely to be involved in patient care long term (such as radiologists, anesthesiologists, pathologists, and dermatologists) were excluded. Identification of primary versus specialty care physicians was done using the AMA Masterfile. Data were weighted by sex, age, and practice setting to reflect the national distribution of physicians in the AMA Masterfile.

Study variables. We categorized physicians by number of years in practice; practice size (solo, small [2–9 physicians], medium [10–49], or large [50 or more]); mode of compensation (salaried or nonsalaried); hours per week involved in direct patient care; and routine or occasional use of electronic medical records (EMRs). The questionnaire surveyed physicians about access to practice-level data (physicians’ patients); access to and sources of quality-of-care data (physicians’ performance); involvement in redesign efforts; and views on sharing quality-of-care data (Exhibit 1).

A total of 1,837 surveys were returned, for a response rate of 52.8 percent.⁶ The majority of physicians completed the survey by mail (91 percent); the remaining 9 percent completed it online. There were no statistically significant differences between respondents and nonrespondents by sex, age, solo versus other practice size, specialty, or years in practice.

Study Results

Top Study Data And Methods Study Results Results Of The Multivariate... Discussion NOTES

 
Physician and practice characteristics. The majority of respondents were male and under age fifty-five (Exhibit 2). The most common practice type was small group and the least common, large group. More than half of respondents had been in practice for more than sixteen years or more, and nearly two-thirds performed more than forty hours of direct patient care per week. Overall, four-tenths were salaried. The majority were specialists, and nine-tenths had been certified in their specialty. More than one-third had been recertified, and another 20 percent planned to recertify within the next two years. Just over a quarter used an EMR routinely or occasionally.

Physicians involved in fewer than twenty hours of direct patient care per week were found to be less likely to easily generate any practice data, compared with physicians doing forty hours per week or more (p < .05) (Exhibit 2). Size of practice is relevant too: half of solo physicians could easily generate those types of data, compared with 61 percent of physicians in large groups (p < .05). Physicians who used EMRs routinely or occasionally were more likely than those who did not to say that data about their practices can easily be generated (p < .05).

Physicians’ access to quality-of-care data. Only one-third of physicians reported receiving any data (process, outcome, or patient surveys) about the quality of care they provide (Exhibit 2). The most common type of data received was patient surveys (25 percent) (Exhibit 1).

Physicians in larger practices and those who devote more hours per week to direct patient care reported being more likely than their peers in smaller practices or doing less direct patient care to receive quality-of-care data. About one-fifth of physicians in solo practice reported receiving data, compared with almost half of those who practiced in groups of fifty or more (p < .05). Also, salaried physicians were more likely than nonsalaried physicians to report having access to quality-of-care data (p < .05). Nearly half of primary care physicians reported getting quality-of-care data, compared with just one-quarter of specialists (p < .05).

Commercial insurance companies and health plans were by far the most common sources of data about quality of care: 25 percent of physicians reported receiving data from such groups.⁷ Thirteen percent reported generating their own quality-of-care data (Exhibit 2). Larger practices were more likely than smaller ones to generate performance data internally (p < .05). And nearly twice the percentage of salaried as nonsalaried physicians generated their own data (p < .05).

Participation in quality improvement activities. One-third of physicians reported having engaged in redesign efforts to improve the performance of the system of care in which they practice (Exhibit 2). Nearly twice the percentage of physicians in practices with more than fifty members said that they have engaged in redesign compared with solo physicians (p < .05). Salaried physicians also were more likely than nonsalaried physicians to be engaged in redesign, as were physicians who spent more hours providing direct patient care. Primary care physicians were engaged in redesign efforts more often than specialists were (p < .05).

Sharing performance data. Nearly three-quarters of physicians agreed (definitely or probably) that information about their clinical performance should be shared with the medical leadership of the health systems in which they work (Exhibit 1). Slightly more than half agreed that performance data should be shared with patients, but only 13 percent were in definite agreement. More than two-thirds said that the "general public" should probably or definitely not have access to such information. With the exception of willingness to share data with medical leadership, these opinions varied little by physician or practice characteristics.⁸

Results Of The Multivariate Analyses

Top Study Data And Methods Study Results Results Of The Multivariate... Discussion NOTES

 
Factors affecting data access. After all physician and practice characteristics were controlled for, three variables independently predict whether a physician can easily generate practice-level data: practice size, hours devoted to direct patient care, and salaried status (Exhibit 3). Physicians in practices of fifty or more were more likely than solo practitioners to generate such data easily (odds ratio = 1.68, p < .01); salaried physicians (OR = 0.79, p < .05), and physicians with twenty or fewer hours of direct patient care per week were less likely (OR = 0.59, p < .01).

Practice size, hours in direct patient care, salaried status, and use of EMRs remained significant independent predictors of whether information about the quality of care is generated from internal sources. Physicians in large versus solo practices, who are salaried versus nonsalaried, and who provide more than 40 hours versus 21–40 hours of direct patient care per week were more likely to generate data internally. Physicians using an EMR were more likely than those who did not to get data from internally generated sources (OR = 1.47, p < .05).

Factors affecting physicians’ involvement in clinical redesign efforts. Factors that independently increased the odds of physicians’ involvement in redesign efforts included larger practice size, being a primary care physician (as opposed to a specialist), longer hours per week devoted to direct patient care, and being recertified in one’s specialty. Physicians in practice for ten to fifteen years were more likely to be involved in redesign than those in practice for fewer than ten years or for more than fifteen years. Physicians in groups larger than fifty were more likely than solo physicians to have engaged in redesign (OR = 2.17, p < .01). Recertified physicians also were more likely than noncertified physicians or certified but not recertified physicians to be involved in redesign (OR = 1.31, p < .05).

Discussion

Top Study Data And Methods Study Results Results Of The Multivariate... Discussion NOTES

 
The 2003 Commonwealth Fund Survey of Physicians and Quality of Care suggests that as of mid-2003, physicians had not yet fully embraced QI principles and methods. Policies and proposals aimed at fostering the diffusion of QI must take into consideration the fact that the majority of U.S. physicians now provide care in the solo or small-group practice setting (2–9 physicians)—where, according to the results of our survey, the adoption of QI has been lowest.⁹

Study limitations. Our survey has some limitations. The response rate of 53 percent of physicians could bias the results. We did not find any basic demographic differences between respondents and nonrespondents; however, physicians who know more about or are more involved in QI might be more likely to respond than those who know less or are not involved. Data-driven physicians could more likely respond to surveys. On the other hand, physicians engaged in QI may have less time available to respond to surveys, compared with those who are not.

Engaging the medical profession. The medical profession’s long-standing resistance to embracing QI is unmistakable.¹⁰ Ernest Codman said in 1917: "The science of medicine, however sophisticated it may now be, is always in an experimental stage. We are all in the business of continuous quality improvement."¹¹ Thus, it has taken close to a century for this science to diffuse into clinical practice, and the process is not yet complete. Based on the results of this survey, we propose that to accelerate the pace of physicians’ involvement in QI, policies and incentives should focus on three areas: capacity, education, and professionalism.

System capacity and infrastructure. Collecting, analyzing, and transforming data into useful reports and then implementing changes require tools, staff, time, and money. Larger groups of physicians and those whose income is based on a salary might have more financial flexibility and access to capital and thus be in a better position to implement both the measurement and the improvement parts of the QI cycle. Organizational culture and management may play an independent role, given that large physician groups are more likely than solo physicians to have adopted a data-driven model of practice. More than a decade ago, the late John Eisenberg hypothesized that practice setting and an increased level of organization in health care delivery might affect the degree to which resources are dedicated to quality.¹² A variety of professional networks could help design practice models that support physicians’ obtaining and using data for improvement, and examples of such networks are being tested around the country.¹³

One hypothesis is that access to practice data would be easier for physicians who have invested in and used EMRs. In bivariate analyses, use of an EMR was found to be related to the ease of generating practice-level and quality-of-care data. But in multivariate analyses, EMR use was no longer a statistically significant predictor of the ease of generating practice-level data. Simply having an EMR does not mean that a physician can use it to its full capacity or reap all of the benefits it can bring regarding QI. Robert Miller and Ida Sim have shown that those who benefit most customize the EMR tool and reorganize workflow in their offices.¹⁴ And the extent to which those redesign efforts are possible might be different in solo practice compared with large groups.

Another surprising result is that after other practice characteristics were controlled for, the multivariate analyses revealed that nonsalaried physicians were more likely than salaried ones to have access to data about their own patients. Financial incentives might help explain these results. The incomes of nonsalaried physicians could be more closely linked to the volume of visits they generate. Thus, they might have greater incentives to monitor their patient panels more closely. In fact, nonsalaried physicians in our survey were significantly more likely than salaried physicians to send their patients reminders for follow-up appointments.¹⁵

For the great majority of the surveyed physicians, productivity remained the major factor determining compensation; clinical quality was cited as a major factor by less than 10 percent.¹⁶ Payment policies that appropriately reward quality or even involvement in QI work should be explored. It is encouraging that a number of performance-based payment programs are being implemented in the United States, and several of these are at the level of the individual physician. Ultimately, it will be important to evaluate their impact on quality.

The results indicate that one-quarter of surveyed physicians identified insurers and health plans as the most common source of their quality-of-care data. For many years, health plans have used the Health Plan Employer Data and Information System (HEDIS), a validated set of measures that assesses quality. But most data on performance, to date, have been measured at the hospital or health plan level. The National Committee for Quality Assurance (NCQA) is developing a set of measures that will target physician offices; the Centers for Medicare and Medicaid Services (CMS) has also launched a national project, "Doctor’s Office Quality," that will measure quality of care of physicians and their offices.¹⁷ Also, according to our survey, primary care physicians have access to quality-of-care data more often than specialists. The National Quality Forum, NCQA, and CMS should thus pay more attention to developing measures of specialty care.

Education: building knowledge and skills. Midcareer physicians reported the greatest level of engagement in QI activities. This is possibly because physicians trained more than fifteen years ago were not exposed to QI principles and because those just starting to practice have yet to implement them. Still, to accelerate the adoption of QI, it will be necessary to improve medical school curricula, residency training, and postgraduate medical education.¹⁸ The Association of American Medical Colleges (AAMC) has responded by appointing an expert panel within the Medical School Objectives Project (MSOP). Its charge is to make recommendations about QI within the context of undergraduate medical education.¹⁹ The AAMC aims to create ten medical school exemplars by year 2006 and sixty by 2009. In 1999 the American Council of Graduate Medical Education (ACGME) approved a new set of residency program training requirements, whereby residents need to reach competency in six areas, and two of these target quality improvement methods.²⁰

Professionalism. The Professionalism Charter of 2002 states that physicians should participate in "continuous improvement in the quality of health care."²¹ Since 1998 the American Board of Medical Specialties has mandated a program of ongoing maintenance of certification.²² Our survey suggests that recertification programs may be having a positive impact. A number of professional organizations and specialty societies are also getting involved in spreading knowledge and fostering implementation of QI among physicians.²³

Nonetheless, the degree to which physicians are unwilling to share information about the care they provide is unequivocal. Although doctors may be wary of the use of performance data for grading or ranking, many measures can be used for QI. At a minimum, physicians should be willing to share information with their peers. Such information sharing could help physicians refer patients to the most appropriate specialist for the patient’s condition.²⁴ Also, physicians should let patients know that they have performance information and that they use it for improving care. One-third of all surveyed physicians said that their patients were more likely to ask them about the quality of their care than they were two years ago.²⁵

If quality is to be rewarded, it will have to be measured; and the data will need to be more accessible than they are now. Physicians should take the lead in making care more transparent. This will mean balancing issues of ethics, fairness, accountability, and confidentiality. Physicians can engender increased trust between the public and the profession by allowing greater openness about the quality of care they provide. Ultimately, transparency could lead key stakeholders to align and coordinate their own QI activities with those of others, maintaining QI cycles in motion toward better care and better health outcomes.

Editor's Notes

 
Anne-Marie Audet (ama{at}cmwf.org) is assistant vice president in charge of quality improvement at the Commonwealth Fund in New York City. Michelle Doty is senior analyst, Jamil Shamasdin is program associate, and Stephen Schoenbaum is executive vice president for programs there.

The authors acknowledge Karen Davis for comments on earlier drafts of this paper, and Cathy Schoen, Kinga Zapert, Jordon Peugh, Karen Donelan, Dana Safran, and Kate Goonan for their help in developing the survey.

NOTES

Top Study Data And Methods Study Results Results Of The Multivariate... Discussion NOTES

 

1. D. Blumenthal and A.M. Epstein, "Quality of Health Care, Part 6: The Role of Physicians in the Future of Quality Management," New England Journal of Medicine 335, no. 17 (1996): 1328–1331.[Free Full Text]
2. Ibid.
3. M.D. Cabana et al., "Why Don’t Physicians Follow Clinical Practice Guidelines? A Framework for Improvement," Journal of the American Medical Association 282, no. 15 (1999): 1458–1465.[Abstract/Free Full Text]
4. A. Milstein and N.E. Adler, "Out of Sight, Out of Mind: Why Doesn’t Widespread Clinical Quality Failure Command Our Attention?" Health Affairs 22, no. 2 (2003): 119–127.[Abstract/Free Full Text]
5. T.A. Brennan, "Physicians’ Professional Responsibility to Improve the Quality of Care," Academic Medicine 77, no. 10 (2002): 973–980.[ISI][Medline]
6. Calculated using Response Rate 1. See American Association for Public Opinion Research, Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys, 3d ed. (Lenexa, Kans.: AAPOR, 2004).
7. See Supplemental Exhibit S1, content.healthaffairs.org/cgi/content/full/24/3/843/DC1.
8. See Supplemental Exhibit S4, content.healthaffairs.org/cgi/content/full/24/3/843/DC1.
9. C.K. Kane, "The Practice Arrangements of Patient Care Physicians," Physician Marketplace Report no. 2004–02, February 2004, www.ama-assn.org/ama1/pub/upload/mm/363/pmr-022004.pdf (24 March 2005).
10. M.L. Millenson, "The Silence," Health Affairs 22, no. 2 (2004): 103–112.[Abstract/Free Full Text]
11. E. Codman, "Re-Engineering Clinical Records for Production Control: 1917," Australian Health Reviews 24, no. 2 (2001): 71–73.
12. P.J. Greco and J.M. Eisenberg, "Changing Physicians’ Practices," New England Journal of Medicine 329, no. 17 (1993): 1271–1273.[Free Full Text]
13. See, for example, California HealthCare Foundation, "Santa Barbara County Care Data Exchange," www.chcf.org/documents/ihealth/SantaBarbaraFSWeb.pdf (15 February 2005); and C.J. McDonald et al., "The Regenstrief Medical Record System: A Quarter Century Experience," International Journal of Medical Informatics 54, no. 3 (1999): 225–253.[CrossRef][ISI][Medline]
14. R.H. Miller and I. Sim, "Physicians’ Use of Electronic Medical Records: Barriers and Solutions," Health Affairs 23, no. 2 (2004): 116–126.[Abstract/Free Full Text]
15. A.M. Audet et al., "Information Technologies: When Will They Make It into Physicians’ Black Bags?" Medscape General Medicine 6, no. 4 (2004).
16. Additional data are available in Supplemental Exhibit S1; see Note 7.
17. Centers for Medicare and Medicaid Services, "CMS Doctor’s Office Quality Project Overview," 15 July 2004, www.cms.hhs.gov/quality/doq/DOQOverview.pdf (12 December 2004).
18. See Institute of Medicine, Crossing the Quality Chasm: A New Health System for the Twenty-first Century (Washington: National Academies Press, 2001).
19. Association of American Colleges, Report V—Contemporary Issues in Medicine: Quality of Care, August 2001, www.aamc.org/meded/msop/msop5.pdf (12 December 2004).
20. See, for example, Accreditation Council for Graduate Medical Education, "Common Program Requirements," 1 February 2004, www.acgme.org/DutyHours/dutyHoursCommonPR.pdf (12 December 2004).
21. Medical Professionalism Project Members: ABIM Foundation, ACP-ASIM Foundation, and European Federation of Internal Medicine, "Medical Professionalism in the New Millennium: A Physician Charter," Annals of Internal Medicine 136, no. 3 (2002): 243–246 (published simultaneously in Lancet 359, no. 9305 [2002]: 520–522).[Free Full Text]
22. American Board of Medical Specialties, "Evaluating Practice Performance for MOC," www.abms.org/Downloads/Publications/4-HowWillPracPerfBeEvald.pdf (12 December 2004).
23. AMA, "Physician Consortium for Performance Improvement," 9 December 2004, www.ama-assn.org/ama/pub/category/2946.html (12 December 2004); and American College of Surgeons, "National Surgical Quality Improvement Program," www.nsqip.org/main/summary.asp (12 December 2004).
24. H. Fidler et al., "Changing Physicians’ Practices: The Effect of Individual Feedback," Academic Medicine 74, no. 6 (1999): 702–714.[ISI][Medline]
25. Additional data are available in Supplemental Exhibit S1; see Note 7.

                      What's this?