| |
October 2000
Volume 64 |
Number 10
|
| |
|
| ABA-ASA In-Training
Examination: Its Origin and Development |
C. Philip Larson, Jr., M.D.
Arthur S. Keats, M.D.
Each July for the past 25 years, customarily
the first or second Saturday of the month, virtually all residents
in training in anesthesiology in the United States and
Canada convene at a suitable testing site in their locale to take
the ABA-ASA In-Training examination. The candidates for the examination
span the full spectrum from those who are just starting in the
specialty to those who have completed their training and may be
in clinical practice. It is a rigorous, all-day examination that
leaves the examinees feeling challenged, perhaps humbled and usually
fatigued. Those taking the examination for the first time suddenly
realize the breadth and depth of the specialty of anesthesiology
and recognize how much they have yet to learn. About 70 percent
of those taking the examination are doing so to assess how their
training is progressing, while the others are taking it for credit
as the written examination component of the certification process
of the American Board of Anesthesiology (ABA). In general, the
examinees will take the examination two or three times as an in-training
evaluation and once (or sometimes up to three times) at the completion
of their training for board credit. How did this examination come
about? What were the objectives, and have they been met?
Written examinations in anesthesiology began with the creation
of the American Board of Anesthesiology in 1938. As part of the
certification process to achieve Diplomate status of the ABA,
a written examination was developed by the first directors of
the ABA. For more than 30 years it was given each July at the
conclusion of the then required two years of training in anesthesiology.
If the candidate passed the examination, he or she was then eligible
to take the oral examination. If not, the candidate waited a year
and took the written examination again the next July.
For many years, ASA also had its own certifying body in the form
of the American College of Anesthesiologists (ACA). The ACA sponsored
a written examination that could be taken after 18 months of formal
training and an oral examination that could be taken after completion
of 24 months. Candidates passing both the written and oral examinations
of the ACA were designated fellows. One could also become a fellow
by reciprocity after achieving Diplomate status from the ABA.
The impetus for the development of an in-training examination
came from three sources. First, by planning or happenstance (the
records are not clear on this issue), some program directors in
the late 1960s and early 1970s began using the ACA written examination
to test the progress of their residents. However, it did not function
as a legitimate in-training examination since there was no feedback
to the candidates or the program directors. Second, leaders in
the specialty increasingly questioned the need for two certifying
bodies in anesthesiology whose training requirements were different.
This arrangement caused confusion and criticism not only within
the specialty but also among hospital credentialing committees
and government agencies. Third, the Board of Governors of the
ACA recognized their gradually diminishing role as an examination
and certifying body as fewer candidates applied for fellow status,
opting instead for ABA certification. So the ACA redefined its
role as being the education arm of ASA, and the development of
an in-training examination was its first objective.
In 1971, under the leadership of ASA President Robert G. Hicks,
M.D., and ABA Secretary David M. Little, Jr., M.D., representatives
from the ABA and the ACA met to discuss this issue. This meeting
resulted in the creation of the Ad Hoc Committee on ABA-ACA Liaison.
This committee included Dr. Hicks, Dr. Little, G.W.N. Eggers,
Jr., M.D., and Arthur S. Keats, M.D., and was charged by each
of the parent organizations with the responsibility for determining
whether it was reasonable and feasible to merge both written examinations
into a single, jointly sponsored examination. The ad hoc committee
concluded that it was and formed an examinations subcommittee
in 1973. This subcommittee was asked to pursue the concept of
the development, administration and support of a mutually constructed
written examination that would be taken by all anesthesiology
residents in training. The subcommittee included Harry H. Bird,
Jr., M.D., Charles S. Coakley, M.D., and Dr. Eggers, representing
ASA, and William K. Hamilton, M.D., Robert T. Patrick, M.D., and
Dr. Keats, as chair representing the ABA. It was expected that
the subcommittee would develop a written examination of about
500 questions over the next two years, with a target date of June
1975 for the administration of the first in-training examination.
Both the ACA and the ABA planned to conduct their own written
examinations in 1974 and 1975 in case the subcommittee was not
able to prepare and conduct a suitable examination by 1975.
Table 1: ABA-ASA In-Training Examination Candidates
(1975-1999)
|
| Year |
#ABA Candidates Examined
|
#ASA Candidates Examined
|
Total # Examined
|
Examination Fees ($)
|
| 1975 |
0
|
1,649
|
1,649
|
25
|
| 1976 |
0
|
1,943
|
1,943
|
30
|
| 1977 |
1,542
|
1,672
|
3,214
|
40
|
| 1978 |
1,592
|
1,763
|
3,355
|
50
|
| 1979 |
1,693
|
1,882
|
3,575
|
50
|
| 1980 |
2,103
|
2,013
|
4,116
|
50
|
| 1981 |
2,073
|
2,442
|
4,515
|
50
|
| 1982 |
2,308
|
2,890
|
5,198
|
50
|
| 1983 |
2,453
|
3,161
|
5,614
|
50
|
| 1984 |
2,757
|
3,380
|
6,137
|
50
|
| 1985 |
2,666
|
3,469
|
6,135
|
50
|
| 1986 |
2,554
|
3,633
|
6,187
|
50
|
| 1987 |
2,621
|
3,643
|
6,264
|
50
|
| 1988 |
1,351
|
4,703
|
6,054
|
50
|
| 1989 |
2,109
|
4,974
|
7,083
|
65
|
| 1990 |
2,283
|
5,300
|
7,583
|
65
|
| 1991 |
2,317
|
5,706
|
8,023
|
65
|
| 1992 |
2,480
|
6,024
|
8,504
|
65
|
| 1993 |
2,508
|
6,082
|
8,590
|
65
|
| 1994 |
2,485
|
5,958
|
8,443
|
65
|
| 1995 |
2,659
|
5,171
|
7,830
|
65
|
| 1996 |
2,659
|
4,298
|
6,957
|
85
|
| 1997 |
2,465
|
3,802
|
6,267
|
85
|
| 1998 |
2,137
|
3,817
|
5,954
|
85
|
| 1999 |
1,602
|
4,095
|
5,697
|
85
|
|
An in-training examination is different from a certifying examination
in two major respects. First, an in-training examination must
cover the full scope of the specialty, while a certifying examination
need only sample selected knowledge within the scope. Second,
since the purpose of an in-training examination is primarily educational,
the candidates and their teachers must be given feedback on the
examination that will improve the educational process. In contrast,
a certifying examination need only provide a pass/fail report.
To identify the full scope of the specialty, Dr. Keats invited
all program directors of American training programs to name a
participant from their department who could assist the examination
subcommittee to define the body of knowledge that constituted
anesthesiology. This group of about 50 experts at first defined
the body of knowledge in outline form and then for each outline
section developed a statement of the information that was most
relevant to the practice of anesthesiology. Basic science or historical
information that had no direct relevance to clinical practice
was excluded. These experts were invited to consult with others
as they developed the areas of information so that the examination
would have the widest possible input from the specialty. These
area statements were then used to create questions for the first
in-training examination. Of this initial group of volunteers,
the best performers by interest and responsiveness became the
question writers group, and some ultimately became in-training
council members. At the outset, it was decided that the examination,
once created, would be compulsory for all trainees in anesthesiology.
The subcommittee contracted with the National Board of Medical
Examiners (NBME) to print, score and analyze the results of the
examination.
The first in-training examination was given on May 3, 1975, at
multiple locations in the United States. This examination consisted
of 350 questions, 140 of which were taken from prior ABA or ACA
written examinations, and had an established record of performance.
The other 210 questions were prepared by a group of question writers
utilizing the area statements prepared by anesthesiologists throughout
the country and then edited by the subcommittee. The cost of preparing
the examination was estimated to be $40,000, so a charge for each
of the estimated 2,000 examinees was set at $25. In reality, a
total of 1,649 residents completed the examination [Table
1], with the results being sent to the program directors in
mid-June 1975. In addition to the absolute score for each resident,
the program directors received normalized tables that indicated
the percentile ranking of each resident compared with the total
examinee pool and with all other examinees with the same number
of years of training. Finally, the reports provided an evaluation
of performance of the residents in the training program in the
three test areas, namely, physiological sciences, physical sciences
and clinical sciences.
Following completion of the first examination, the subcommittee
met with the NBME to review and critique the examination. The
data presented at that meeting indicated that the first examination
was of excellent quality, with a high degree of reliability and
discrimination between high and low performers and among those
with minimal and extensive years of training. It was also evident
that the data on performance in the three test areas were not
helpful to the program directors or the examinees, so this form
of feedback was discontinued.
Because of the enormous success of the first examination, the
Ad Hoc Committee on ABA-ACA Liaison recommended the creation of
a permanent Joint Council on In-Training Examinations and wrote
the Articles of Organization for the Council. The key features
of the new Joint Council were: 1) a council of eight members with
four being designated by the ABA and four by the president of
ASA in consultation with the chair of the Board of Governors of
the ACA; 2) all members must be either a diplomate of the ABA
or a fellow of the ACA; 3) after staggered terms ranging from
one to four years for the initial council members, all members
would be appointed to a four-year term, but not to exceed two
terms; 4) a council option to appoint adjunct members for one
year to meet specialized needs; 5) the fee structure set by the
council should make the examination self-sustaining; and 6) the
secretariat of the council shall reside in the offices of ASA.
The ASA House of Delegates approved these articles of organization
at its 1975 meeting, along with the proposal to allow residents
in Canadian training programs to participate in future in-training
examinations.
The 1976 examination was held in May at 159 sites (including,
for the first time, 14 sites in Canada as 194 Canadians who were
in training registered for the examination). With this examination
an attempt was made to improve the feedback to program directors
and candidates by not only providing standard scores of relative
performance and percent correct scores but also a list of key
words that represented in concise form the subject of the questions
that were answered incorrectly.
As a result of two years of experience with written examinations,
the ABA-ASA Joint Council made substantial changes in the 1977
examination. First, the date of the examination was moved from
May to early July so that those residents who were starting their
training as a PGY-1 would be able to take the examination. Second,
for the first time the ABA stopped conducting its own written
examination and instead used a subset of the in-training examination
as its written examination for certification. Third, two different
sets of examination booklets with identical content but varying
sequencing of questions were prepared to minimize any possibility
of cheating during the examination. Fourth, the listing of key
words provided for program directors and candidates was shortened
and refined to reflect more accurately and meaningfully the content
of the questions answered incorrectly. Finally, to maintain the
self-sustaining nature of the examination, the application fee
was increased to $40 per candidate.
The 1978 examination underwent additional changes. The most
important was the addition of 50 items (and the deletion of 50
others) that required clinical judgment to answer correctly, and
hence might distinguish knowledge gained during more advanced
training from that obtained early in training. Also, for the purpose
of designating the level of training, the PGY-0 category was added
to identify the new medical graduate. Finally, the application
fee was increased to $50 per candidate, a fee that was to be held
constant for the next 10 years.
Additional changes in the examination have occurred in the subsequent
22 years. The number of candidates taking the examination reflects
the fluctuations in residency pool that has occurred over the
years [Table 1]. Obviously, the content
of the examination is constantly revised to incorporate new knowledge,
new drugs and new anesthetic techniques. The source of questions
has remained broad and diverse through utilization of ABA oral
examiners as well as nominees for oral examiner status. The questions
submitted undergo rigorous review and revision and if deemed suitable
are ultimately approved by the Joint Council for inclusion in
a future examination. The members of the Joint Council, and especially
the chair, devote an extraordinary amount of time to this process.
In addition, it is traditional for the Joint Council to meet after
the results of the July examination are available to review problems
related to the content and the conduct of the examination. The
examination has not escaped the computer age. The questions, statistical
analysis of the examination and the follow-up reports to program
directors and candidates are all held on computer for easy access,
revision and reporting by the Joint Council.
What was thought by the governors of the ACA and the directors
of the ABA in the early 1970s to be a good idea has proven to
be an exceedingly valuable component in the overall education
of anesthesiologists in training. Training programs utilize the
keyword feedback to develop questions for clinical conferences
or to write brief reviews of the subject. Authors have used the
keywords to write books containing multiple choice questions and
discussion of the answers. Many training programs utilize the
in-training examination as a tool to assess whether residents
are eligible for advancement to the next level of training and/or
to develop specialized educational programs and goals that residents
must meet before advancement is considered. Finally, and most
importantly, the examination serves as a motivation for study
since all residents know that the examination is coming and that
the program director and faculty have certain expectations regarding
their performance.
| |
|
C.
Philip Larson, Jr., M.D., is Professor Emeritus, Anesthesia
and Neurosurgery, Stanford University School of Medicine,
and Professor of Clinical Anesthesiology, University of California-Los
Angeles School of Medicine. |
|
| |
|
Arthur
S. Keats, M.D., is Clinical Professor of Anesthesiology, University
of Texas Health Sciences Center, and Chief, Cardiovascular
Anesthesiology, Texas Heart Institute, Houston, Texas. |
|
return to top
|