June 1, 2013
Volume 77, Number 7
Policy Matters: Take Steps Now to Enhance Research on Future Value-Based Initiatives
Thomas R. Miller, Ph.D., M.B.A.
In an attempt to update an important systematic review,
I, too, was unable to find any randomized controlled trials
(RCTs) which studied the effectiveness of parachute use to prevent death and major trauma related to gravitational challenge.1 The referenced tongue-in-cheek 2003 article by Smith and Pell had an important message: sometimes randomized controlled trials, despite being the gold standard of health services research, are not appropriate or possible for the study of all questions of interest. Common sense, however, which may be sufficient to draw conclusions on parachute use, may not be adequate when studying initiatives aimed at enhancing value within a health care system. Fortunately, quasi-experimental designs that do not employ randomization still can be used effectively to study the impact of major initiatives aimed at reducing costs and improving the patient experience.2
In the “Practice Management” column of this NEWSLETTER, Dr. Schweitzer et al. provide an excellent overview of ASA’s vision of the Perioperative Surgical Home (PSH) model. So, suppose a hospital decides to implement the PSH, selected components thereof, or another program that may fall under the umbrella of the PSH model
(e.g., the Enhanced Recovery After Surgery [ERAS] program3). How can the organization “prove” that the PSH or related initiative(s) resulted in “a better medical outcome at a
Outside the RCT setting, it can be challenging to argue that quality improvements and cost reductions were a direct result of a new perioperative initiative and not the result of other hospital initiatives or external forces occurring at the same time, or simply the result of the natural maturation of perioperative services. In research terms, alternative plausible explanations to improvements in the measures of interest represent threats to the internal validity of the hypothesis that the PSH initiative, for example, was the intervention directly responsible for these improvements. In quasi-experimental designs, there are two important considerations to strengthen subsequent research: 1) using a phased implementation and
2) conducting periodic cost and quality measurements.
Using a Phased Implementation: Whether by type of surgical patient (e.g., abdominal versus orthopedic) or by site of care delivery (e.g., if within a hospital system, initial implementation at only one hospital), a phased implementation is a method to incorporate “controls” in quasi-experimental designs. Several organizations have taken this approach. For example, the University of Alabama Health System has implemented its PSH model in a single facility prior to evaluating the roll-out to other facilities.4 The department of anesthesiology and perioperative care developed an ERAS program at the University of California, Irvine and plans to subsequently implement it across the University of California system. In addition, this ERAS program initially targeted high-risk abdominal surgery and will be implemented for moderate and high-risk surgeries within the next two years.5 Kaiser has had a PSH in its Baldwin Park facility since 1998 and now has plans to implement this model at other facilities.6
Conducting Periodic Measurements: To help demon-strate the impact of these new models or programs, quality and cost measures need to be obtained at multiple points: prior to implementation, during implementation and after implementation. In addition, the measures need to be captured for all locations and surgical case types, both where the PSH model has been implemented, for example, and where (or for which surgical patient type) it is not in place (the “controls”). In the case of UC Irvine, the key measures for the ERAS implementation include improved patient experience, clinical outcomes, multi-disciplinary team collaboration, and reduction in length of stay and reduced risk of hospital-acquired infections.
Clearly, there are substantial challenges to defining and collecting quality and cost measures. However, regardless of the challenges, it is critically important to begin now the comprehensive measuring and reporting of cost and outcomes data relevant to perioperative services, even well before the implementation of PSH or related value-based initiatives. Getting a head start will facilitate discussions and help
engender plans for future improvements to the data. It is also clear that outcomes and cost measures are the pillars upon which health policy research and health delivery transformation will be built.
In summary, it is probably not possible, and certainly not practical, to conduct a double-blind RCT to assess the effectiveness of PSH and related models. However, quasi-experimental designs can inform providers and policymakers, and they can be strengthened by two important design features:
1. Implement a phased approach, by type of surgical patient, or by site of implementation, or both.
2. Start now to identify, define, develop, record, and routinely report quality and cost measures across all patient types and delivery sites.
There are many action items that can enable health services researchers to better demonstrate the effectiveness of a new program. For hospitals considering PSH or similar initiatives, early and deliberate planning of implementation timing and outcomes measurement will go a long way toward strengthening the base of evidence and facilitating the leadership role of anesthesiology in adding value to the health care system.
Thomas R. Miller, Ph.D., M.B.A.
is ASA’s Director of Health
1. Smith GCS & Pell JP. Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials. BMJ. 2003; 327, 20-27.
2. Shadish WR, Cook TD, & Campbell DT. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston, MA: Houghton Mifflin Company; 2002.
3. See http://www.erassociety.org/ for a description of ERAS as implemented in the U.K. and for an assessment of ERAS implementation in England, see Knott A, Pathak S, McGrath JS, et al. Consensus views on implementation and measurement of enhanced recovery after surgery in England: Delphi study. BMJ Open 2012;2:e001878. doi:10.1136/bmjopen-2012-001878.
4. Vetter TR, Goeddel LA, Boudreaux AM, Hunt III TR, Jones KA, & Pittet J-F. The Perioperative Surgical Home: how can it make the case to everyone wins? BMC Anesthesiology. 2013; 13:6.
5. Personal communications with Dr. Maxime Cannesson, Department of Anesthesiology & Perioperative Care, University of California - Irvine.
6. Trivedi NS, & Ghouri AF. Re-engineering the perioperative process: the Kaiser Baldwin Park Experience. Working Paper. 2003.