| |
October 2000
Volume 64 |
Number 10
|
| |
|
| Aviation
Safety Reporting System: A Model for Reporting and Analyzing
Errors |
David M. Gaba, M.D.
Committee on Patient Safety and Risk Management.
Almost since its inception in 1976, the
Aviation Safety Reporting System (ASRS), which the National Aeronautics
and Space Administration (NASA) runs for the Federal Aviation
Administration (FAA), has been lauded as a key component of aviation
safety. As of this writing, the ASRS has collected more than 300,000
reports, of which over 50,000 have undergone a complete analysis.
Nine years ago, the Anesthesia Patient
Safety Foundation (APSF) looked into the ASRS when the APSF was
| "The ASRS experience suggests that, contrary
to traditional epidemiologic thinking, careful consideration
of the written encounters of others can generate useful safety
improvements even when 'counting' techniques falter." |
considering the creation of a reporting
system so that anesthesiologists could disclose their events or
errors for analyzation by others in order to derive patient safety
lessons. Nine years ago, the obstacles to such a system appeared
insurmountable. Today, the picture is different, and such systems
in health care may soon appear.
Why has the NASA-ASRS been such a powerful
force? What distinguishes it from other sorts of systems that
collect and analyze reports of safety related events? There are
a number of salient features of the ASRS that appear critical
to its success.
Key Features of the ASRS
A. Not a Regulatory Program: The
system is not managed by a regulatory agency but rather
by a third-party “honest broker (NASA) that has a reputation for
solid research and independence. In fact, the precursor to the
ASRS was run by the FAA itself. It generated few reports. In the
hands of NASA, it has generated hundreds of thousands of reports.
B. Voluntary Reports: The ASRS collects
voluntary reports of safety-related incidents from front-line
working personnel. By law, the ASRS does not collect accident
reports (if it receives one, it is forwarded) because accidents
are investigated by the National Transportation Safety Board (NTSB).
C. Confidential Reports: The ASRS
collects confidential reports, not anonymous reports.
A key benefit of collecting identified, confidential reports is
that the analyst can call back the reporter to obtain more information.
This is crucial because experience shows that many anonymous reports
have too little information to be of any use. After the analysis
of a report is complete, key identifying information is stripped
before it and the accompanying analysis are stored in the permanent
database. To date, courts have treated the de-identified data
as hearsay.
D. Domain Experts Analyze Narratives:
Unlike many reporting systems where the emphasis is on collecting
statistics on various items checked on a standardized form, the
ASRS reporting form (which includes some categorical data items)
emphasizes a written narrative of how the event transpired. Furthermore,
all reports are read by a domain expert analyst (pilots, mechanics,
air traffic controllers) who understands the meaning of the report,
and a significant fraction are subjected to full analysis. Ideally,
every report would be subject to complete analysis, but resource
limitations preclude doing so. Analysts choose reports for analysis
based on their salience, entry criteria for ongoing ASRS special
studies and on a random sampling system that guarantees that a
minimum fraction of all reports receive complete analysis. The
ASRS experience suggests that, contrary to traditional epidemiologic
thinking, careful consideration of the written encounters of others
can generate useful safety improvements even when counting techniques
falter.
E. Feedback to the Practicing Community:
A strength of the ASRS is its feedback to the aviation
community. When an immediate ongoing hazard is reported, ASRS
issues alert messages to facilities or agencies who can deal with
the hazard. Further, there are regular feedback vehicles to individuals
and organizations. There is a monthly newsletter (appropriately
enough named Callback) that goes out by mail free of charge
to anyone who subscribes. Callback is also available on the Internet.
Callback contains excerpts from reports and analyses that
can be of immediate use to the aviation community. The ASRS also
publishes a magazine, Directline, that is geared more to
organizations and managers in the aviation industry. The ASRS
database of de-identified information is actually available to
the public by CD-ROM. Report sets dealing with specific topics
are also assembled by the ASRS and made available over the Internet.
Thus, third parties can use the reports for self-education, training
and research. The ability of potential reporters to see the results
of others' reports is considered very important for generating
a willingness to submit reports.
F. Nationwide: A key advantage
to a nationwide program like the ASRS is that, rather than being
localized to a single region or a single company, similar events
occurring at disparate sites can be correlated to yield patterns
of occurrences that otherwise would each seem like just a single
unusual event.
Transition of VA PSRS
In May 2000, the Department of Veterans
Affairs and NASA announced a partnership to establish a patient
safety reporting system (PSRS) modeled closely after the ASRS.
Jeffrey B. Cooper, Ph.D., and David M. Gaba, M.D., served on a
planning committee for this effort in 1999. The Veterans Affairs
patient safety reporting system (VA PSRS) plans to collect reports
beginning in January 2001. Like the ASRS, the emphasis will be
on analysis by domain experts of narratives and other information
from confidential, voluntary reports from actual health care workers.
Feedback mechanisms are planned similar to those used in the ASRS,
including alert messages and a newsletter. The VA PSRS will cover
all of VA health care. Given the prominent emphasis on patient
safety in anesthesiology, however, it is expected that VA anesthesiologists
will be enthusiastic reporters and readers of VA PSRS materials.
Barriers
Heretofore, the major barriers to establishing
a meaningful safety event reporting and analysis system have been:
A. Medicolegal Concerns: Without
appropriate protection, the reports and/or database could be vulnerable
to discovery, to subpoena in civil and criminal litigation and
to disclosure under the Freedom of Information Act. Not only would
this have a chilling effect on reporters, it would also risk involving
the reporting system itself in costly litigation to attempt to
protect itself, to quash subpoenas and so forth. APSF, for example,
concluded in the mid-1990s that a thorough analysis of the legal
issues was required, for which APSF did not have sufficient resources.
The VA PSRS can proceed at this point because of unique characteristics
of the laws governing VA health care and clinicians that confer
protection to the reporter and institutions (e.g., 38 USC 5705).
Should certain pending legislation be enacted by Congress, the
medicolegal concerns would be rectified (see Recent
Legislation).
B. Cost: It sounds easy to put together
a reporting system: establish a Web site, obtain a telephone number
and a mailing address, etc. However, the ASRS experience shows
that collecting the reports is only the first step. Meaningful
analysis of the reports to make sense of what really happened
and why is the really useful part of a reporting system, but it
is also the most expensive. It takes time and effort by the analysts
to do this. Now that patient safety has become big news, however,
it is quite possible that funding may be forthcoming for either
large comprehensive programs or smaller more focused programs.
C. Incidents Versus Accidents: Health
care, of course, has no equivalent institution to the NTSB, which
conducts a rigorous independent analysis of catastrophic events.
In aviation, when an airplane crashes, it is apparent immediately.
It is easy to define a threshold for accidents versus nonaccidents
(in terms of injuries, deaths and cost of the damage). This is
much more difficult in health care, where events that seem to
end with no harm done may still be implicated in adverse outcomes
occurring days, weeks or months after the fact. In the VA PSRS,
for example, it is believed that it will not be possible or desirable
to distinguish between reports of incidents from those of accidents.
All events will be analyzed the same way.
Recent Legislation
Two bills recently introduced in the Senate
would set up or facilitate patient safety reporting systems analogous
to the ASRS. The Patient Safety and Errors Reduction Act (S. 2738),
introduced by Senators Jim M. Jeffords (R-VT), Michael B. Enzi
(R-WY) and Bill Frist (R-TN), would certify external medical event
analysis entities to collect and analyze information on medical
errors. It would extend to these programs substantial protections
of the reports or data from disclosure in litigation. The Voluntary
Error Reduction and Improvement in Patient Safety Act asrs.arc.nasa.gov/main
nf.htm (S. 2743), introduced by Senators Edward M. Kennedy (D-MA),
Jeff Bingaman (D-NM), Christopher J. Dodd (D-CT) and Patty Murray
(D-WA) would establish a National Patient Safety Reporting System
within the Agency for Healthcare Research and Quality (AHRQ) to
be structured and operated very much like the ASRS. This bill
would also extend substantial protection from disclosure of the
reports or data in litigation. S. 2743 calls for specific funds
for the reporting system, beginning with $25 million in fiscal
year 2001 and escalating to as much as $75 million in fiscal year
2005. Should either bill pass, it will be a strong shot in the
arm for the use of voluntary reports to yield important patient
safety information.
|
"Reporting of events by itself
means nothing. It is only a means to generate information
to guide improvements in patient care processes."
|
Conclusions
Reporting of events by itself means nothing.
It is only a means to generate information to guide improvements
in patient care processes. Devising these improvements and implementing
them is a daunting task that will be the next big challenge in
patient safety. It is highly likely that one or more protected
reporting and analysis systems will be operating within the next
few years, and anesthesiologists should avail themselves of the
opportunity to report their safety-related events. Should legislation
pass that extends legal protections to third-party reporting systems,
the anesthesia community (through ASA or APSF) might want to establish
such a program focused on perioperative events. Finally, anesthesiologists
can be proud that their interest in such systems will have eventually
led to their creation.
Additional information about the NASA ASRS
can be found at http://asrs.arc.nasa.gov/main_nf.htm.
Additional information about the Senate bills can be found at
http://thomas.loc.gov (enter
either S. 2738 or S. 2743 under Search Current Congress
for Text of Bills By Bill Number).
References:
1. Reynard WD, Billings CE, Cheaney ES,
Hardy R. The development of the NASA Aviation Safety Reporting
System. NASA Reference Publication. 1114. NASA; 1986.
2. Hardy R. Callback: NASA's Aviation Safety Reporting
System. Washington, DC: Smithsonian Institution Press; 1990.
3. Billings CE. Appendix B. Incident reporting systems in medicine
and experience with the aviation safety reporting system. In:
Cook RI, Woods DD, Miller C, eds. A Tale of Two Stories:
Contrasting Views of Patient Safety. Chicago: National
Patient Safety Foundation; 1998. Available at www.npsf.org/exec/billings.html
4. Barach P, Small SD. Reporting and preventing medical mishaps:
Lessons from non-medical near miss reporting systems. BMJ.
2000; 320:759-763.
| |
|
David
M. Gaba, M.D., is Director, Patient Safety Center of Inquiry
at VA Palo Alto Health Care System, and Professor of Anesthesia,
Stanford University School of Medicine, Palo Alto, California. |
|
return to top
|