Your exam content is your credentialing organization’s most valuable commodity. You spend a good deal of time, money, and resources to develop items that create a legally defensible exam that accurately measures a candidate’s ability to do a job. Are you taking the right steps to protect it?

For computer-based (CBT) exams, it’s crucial that you have contingencies put in place to help avoid content exposure and respond appropriately when exam content is found to have been exposed. In a bad-case scenario, failure to do so can result in administration disruptions and candidate rescheduling; in a worst-case scenario, you could end up having to completely redevelop your exam and go into damage control-and-repair mode to salvage your reputation.

Preventive Measures

There are several best practices that can be used to prevent exposure of exam content. Here are nine key strategies:

1. Create a strong culture around exam security within your organization. It’s important to ensure that all staff and contractors working with exam content are aware of the importance of exam security and enforce adherence to practices that protect exam content.
2. Maintain a large item bank from which multiple equivalent test forms (or LOFT forms) can be constructed, so candidates do not all receive the same questions on their exams.
3. Test in windows that are as short as possible to minimize opportunities for casual content sharing in person and online, and to minimize access for those looking to harvest content for personal or financial gain.
4. Limit how often each item is presented to avoid over exposure.
5. Build one or more backup test forms to be held dormant. These forms should have no overlapping items with the operational test forms so they can be used immediately, if needed, and should be updated frequently.
6. Develop and store item banks and tests on a secure server. Additionally, consider maintaining a limited-access bank of items that does not contain items from the main bank and store it on a separate server. This bank could contain the items for your backup test forms.
7. Deliver tests in reliable, secure testing centers.
8. Regularly conduct drift analyses to identify items that demonstrate significant difference in performance statistics over time. If your item content has been exposed, this can often be detected through changes in the performance of exposed items.
9. Passively (Google alert) and actively (webcrawl) monitor the Internet, specifically social media and test preparation sites, for content sharing.

While the development and implementation of some of these procedures require an up-front investment, they serve to reduce both real and reputational costs and decrease the amount of unplanned work that would need to be done should your exam content be compromised.

Follow-Up Measures

No matter how well an organization complies with good exam security practices, it’s impossible to prevent all breaches of security. An appropriate response to such incidents is key to protecting your program, organization, and reputation.

If a potential exam security incident is identified, your organization will need to take the following actions:

• Conduct a timely investigation of the incident;

• Identify the content that has been compromised;

• Take proper actions to prevent further dissemination of the material;

• Determine appropriate actions to take regarding the perpetrators; and

• Communicate with candidates and the public about the incident.

One of the most valuable components of an organization’s exam security program is an exam security incident response plan. In the flurry of activity that follows the discovery of a security breach, this plan provides your organization with a predetermined roadmap for managing the incident. Developing a comprehensive exam security incident response plan will give your organization the advantage of being prepared to act calmly and quickly when a potential exam security incident is discovered. 

Depending on your organization’s resources and strategic priorities, developing an effective exam security program might feel overwhelming. If this is the case for you, consider outsourcing to a third-party assessment development and delivery provider who can help you create plans and processes to safeguard your content, monitor the Internet for nefarious activity, and assist you in executing plans to respond to and recover from any possible breaches.

Author

Kathleen Champagne; Director, Content Development