ASA recognizes the importance of collecting relevant data in today’s health care environment. Key changes in health care delivery have increased public awareness of patient safety and quality of care issues. National pressures continue to mount for physicians to demonstrate their competence and ability to deliver high-quality and cost-effective patient care services. Specialty societies need to provide measurement tools and feedback mechanisms to assist physicians in data collection and continuous quality improvement. Toward that end, ASA established the Committee on Performance and Outcomes Measurement (CPOM). According to ASA Bylaws, CPOM has broad responsibility for overseeing the initiatives of the Society that pertain to the measurement of clinical performance and outcomes.

In one of its earliest initiatives, CPOM developed “Guidelines for Database Management by the American Society of Anesthesiologists” that were approved by the 2000 House of Delegates. This document describes guidelines for data management (technical standards and organizational oversight) that would allow ASA to pool data into a comprehensive relational database. This collection of clinical data would have the potential to improve rather than merely to document a significant dimension of patient care. Such data might include, but not be limited to, clinical outcomes, patient satisfaction and resource utilization. Equally important would be the collection of demographic data that would permit identification of relationships between practice characteristics and clinical care. This document can be located on the ASA Web site at <www.ASAhq.org/publicationsAndServices/sgstoc.htm>.

CPOM’s recent review of this document suggests that it remains current and functional, though the committee does not recommend implementation at this time because the cost of data collection and analysis is too excessive for voluntary participation. The rates of most clinically significant adverse outcomes of anesthesia care are so low that data on large numbers of cases must be collected before statistically significant comparisons can be made to national benchmarks. Additionally, for benchmarks to be meaningful, most outcomes must be adjusted for risk. Effective risk-adjustment requires collecting as many as 10 risk-adjustment variables (e.g., data on coronary artery disease, chronic lung disease, diabetes mellitus, etc.) for every benchmarking data element (e.g., anesthesia-related death). The resulting data-gathering burden is considerable. Collecting denominator data on every anesthetic administered, as required to calculate risk-adjusted frequencies of events, is a burden that many anesthesiology departments are reluctant to take on voluntarily.

Other medical specialty societies have invested substantial resources into their benchmarking initiatives only to find that their databases were not sustainable. Among those unsuccessful efforts were MODEMS (Musculoskeletal Outcomes Data Evaluation and Management System), developed by the American Academy of Orthopaedic Surgeons, NEON (National Eyecare Outcomes Network), developed by the American Academy of Ophthalmology and DOCS (Documented Outcomes Collection System), developed by the American Urological Association. In contrast, the Veterans Administration (VA) National Surgical Quality Improvement Project (NSQIP) is a successful risk-adjusted perioperative outcomes database containing more than 1 million records.¹ This success is likely due to its mandatory participation requirement for VA hospitals. VA hospitals are therefore required to bear the cost of data collection, which is estimated at around $38 per patient record. Additionally the VA spends an equal amount on data analysis and reporting.

ASA should prepare itself for collection of performance data because several national initiatives suggest that reporting of adverse perioperative outcomes to a national database repository is likely to become mandatory in the future. Under these circumstances, data collection will be included as an unavoidable operating cost, as seen in the VA system. The VA’s NSQIP was developed after a 1986 congressional mandate required the VA to report risk-adjusted surgical outcomes and compare these with the national average. At that time, the VA had no information about their performance, and there was no national average. Since the NSQIP’s inception in 1994, the VA has reported consistent improvements in all surgical performance measures and demonstrated significant cost savings. In 1999 the VA began demonstrating the feasibility of implementing NSQIP in non-VA hospitals.²

Further evidence of the increasing likelihood of mandatory reporting follows. In its 1999 report “To Err Is Human: Building a Safer Health System,” the Institute of Medicine recommended “identifying and learning from errors through the immediate and strong mandatory reporting efforts, as well as the encouragement of voluntary efforts, both with the aim of making sure the system continues to be made safer for patients.”³ Also the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) continues to increase the scope of hospital collection and reporting of core performance measure data. Core measures are part of the JCAHO’s ORYX initiative, the principal objective of which is to integrate outcomes and other performance measurement data into the accreditation process. Additionally the division of the Centers for Disease Control and Prevention (CDC) most involved in quality activities (once called the Hospital Infection Program) has evolved to become the Division of Healthcare Quality Promotion. Over the past four years, CDC has moved beyond strictly focusing on the surveillance and guideline efforts, for which CDC is so well known, to putting guidelines into practice. CDC also is making progress toward a goal of expanding its focus beyond infections to other adverse events. It has recently established a goal to decrease surgical complications by 50 percent in five years through the Surgical Care Improvement Project (SCIP). Finally the American Medical Association (AMA) has convened the Physician Consortium for Performance Improvement (the Consortium) with the vision of becoming the leading source organization for evidence-based clinical performance measures and outcomes reporting tools for physicians. AMA holds the copyright in Physician Performance Measurement Sets (PPMS) developed or adopted by the Consortium but has recently resolved to let “lead organizations” use any PPMS in commercial or noncommercial products.

Like The Consortium, many organizations are hoping to become the leading source for clinical performance measures and outcomes reporting tools. Because of this, many of these organizations seem to be competing in the development of outcomes data sets, and little common terminology exists. One notable exception is the consistency between the VA’s NSQIP terminology and the CDC’s National Healthcare Safety Network (NHSN) terminology that relates to perioperative infections. Although the methodology being developed by the Anesthesia Patient Safety Foundation (APSF) Data Dictionary Task Force (DDTF) for comparing different data sets used by different automated anesthesia record system vendors may be applicable to this problem, it is still in development at this time. A more thorough description of the APSF efforts can be found here.

In an effort to determine the optimal timing for ASA’s development and/or participation in a national clinical anesthesia outcomes database, CPOM members continue to represent ASA on the AMA Consortium, the APSF DDTF and the CDC SCIP. Through the SCIP, ASA has formed a relationship with representatives of the VA’s NSQIP. As previously mentioned, NSQIP is a successful risk-adjusted perioperative outcomes database with the potential to become the model for the nation. John Steiner, M.D., a representative from NSQIP, attended the CPOM meeting at the ASA Annual Meeting in October to discuss technical aspects of developing the VA’s outcomes database and the business model for sustaining it.

While awaiting the proper timing for development of a national database repository for performance data, ASA has looked for other ways to allow its members to make use of performance measurement. The Committee on Quality Management and Departmental Administration (QMDA) and CPOM are working on a joint project to develop an ASA Quality Improvement Template that can be used “off the shelf” by departments of anesthesiology to improve quality of patient care and to meet various internal and external administrative requirements (e.g., accreditation). The need for such a template is apparent from requests submitted to QMDA by ASA members and from findings of the ASA Anesthesia Consultation Program. Additionally a template might form the foundation for a national benchmarking database if and when the goal of a national benchmarking database is achieved. Quality improvement data collected according to the template will be stored in local databases. Because the template will include a standardized format and definitions, the local data could subsequently be transferred to a national database and used for benchmarking. The quality improvement template is near completion, and electronic dissemination should begin in 2004.

References:

1. Khuri SF. The comparative assessment and improvement of quality of surgical care in the Department of Veterans Affairs. Arch Surg. 2002; 137:20-27.

2. Fink AS. The National Surgical Quality Improvement Program in non-veterans administration hospitals: Initial demonstration of feasibility. Ann Surg. 2002; 236:344-353.

3. Committee on Quality of Health Care in America of the IOM: To Err Is Human: Building a Safer Health System. Kohn L, Corrigan J, Donaldson M, eds. Washington, DC: National Academy Press; 1999:7.

Robert S. Lagasse, M.D., is Professor of Clinical Anesthesiology, Albert Einstein College of Medicine and Montefiore Medical Center, Bronx, New York.