Skip Ribbon Commands Skip to main content

The American Board of Pediatrics: Transitioning to Computerization

  abpLogo.gif

Overview of the American Board of Pediatrics

Founded in 1933, the American Board of Pediatrics (ABP) is one of twenty-four certifying boards that comprise the American Board of Medical Specialties, and is tasked with certifying general pediatricians and pediatric sub-specialists to ensure high quality care for infants, children and adolescents. The certification of medical providers by the APB, via the careful review of professional credentials and administration of continuous evaluation of practitioners' core competencies, provides a layer of protection for the American public in its efforts to assure the public that ABP board-certified physicians are maintaining their competence in the field.

The Challenge

As a way of continuously evaluating the skill sets of physicians, the ABP administers more than 5,000 recertification examinations each year as part of the Maintenance of Certification Process. In 2001, however, with decision to move to a secure, proctored examination and the need for global administration of the examination, the ABP decided that a more widespread and effective form of examination administration was needed.

With the dual requirements of scalability and international availability in mind, the ABP looked to computer-based testing (CBT) as a viable alternative to standard paper and pencil-based testing (PBT). Switching to computer-based testing would enable the administration of recertification exams from anywhere at any time, and eliminate the need for candidates to travel hundreds or even thousands of miles to reach a centralized, approved testing location. The ubiquity of a CBT would also increase the number of available seats for recertification tests and provide candidates with greater flexibility and control in examination scheduling.

Once the decision to migrate to CBT was made, the ABP identified three possible issues that could potentially impact the new format: test integrity, site security and delivery model. ABP examination content had never been compromised, and the ABP wanted to ensure that the migration to computerization did not change this fact. The Board also needed to monitor candidate identities to ensure adequate security during the examination process.

Lastly, because the examination is complex and scientific in nature, the ABP made the determination to develop its own test delivery driver, allowing it to maintain full control over the examination's content and format. Finding a testing partner that could work with the ABP's internal development team to support, enhance and integrate this approach was critical to the success of the transition.

The Solution

In 2002, the ABP selected Prometric, the world's leader in technology-enabled testing and assessment, as its partner for both the transition to CBT, and the administration of the maintenance of certification examinations. Prometric's global and secure network of testing centers provided the national and international backbone required by the ABP. Instead of having only one or two dates a year to take the PMCP examination at a given location, candidates could easily schedule their test via the Internet and take it in their hometown, in or outside of the United States, during an open window of availability.

Prometric's test center network also afforded the ABP the site security and content integrity that they required to ensure proper maintenance of certification. Through a combination of live proctors, closed-circuit television systems and biometric security procedures, candidates' identities were confirmed and suspected foul play could be quickly detected and eliminated. The computerized nature of the examination and its secure administration meant that the ABP could assure the public that the individual who appears at the Prometric testing center is the proper individual.

The unique delivery driver created by the ABP was enhanced by Prometric's willingness to work hand-in-hand with the organization's internal team during the development and transition process. Prometric quickly understood the significance of this examination not only to the candidates, but also to the health and well-being of the general public, and helped to seamlessly integrate the delivery driver and examination administration methods into a highly-functional testing environment.

Results

The first computer-based maintenance of certification examination was administered in early 2003, with an overwhelmingly positive candidate response. Effective communication between the ABP and the test-takers regarding the new format and added security measures, including biometric procedures, were announced well in advance of the testing window in order to head off unpleasant surprises. Candidates were also given an online demonstration of the new format, complete with a sign-on procedure and sample questions, to familiarize themselves with the format and software prior to the actual examination.

Additionally, surveys placed at the end of examinations helped the ABP consolidate candidate responses, which showed that any perceived negatives of the examination were offset by the convenience of self-scheduling and the ability to take the examination in a local test center. Prometric's hands-on involvement during the driver creation process greatly assisted the ABP to deal with the challenges inherent in the development and implementation of the new delivery system.

Next Steps

To further evolve its CBT program, the ABP is considering implementing Prometric's proprietary item banking technology, known as Intelitest. Intelitest would allow ABP to more efficiently manage its own test items, quickly refresh test content and enhance the examination development and delivery process for all of its CBT examinations.

ABP is also considering incorporating short, streaming video clips into some of its examinations to help foster a more "real-world" environment for complex subjects such as pediatric cardiology. This evolution would more accurately reflect those complex and critical situations facing pediatricians and subspecialists on a day-to-day basis.

An expansion of the CBT program to examinations other than the maintenance of certification examination is also being considered, as are upgrades to its existing delivery driver.

PROMETRIC'S KNOWLEDGE

Prometric is happy to share our knowledge and expertise.