Background
A pioneer in digital innovation, this industry leader has continuously evolved to meet the dynamic needs of content creators and developers worldwide. From its beginnings as a software company to becoming a top provider of digital content tools, it embodies a commitment to innovation and adaptability.
The Challenge
As the organization’s product portfolio expanded and its influence in the digital domain grew, so did the demand for certification programs. Since 2018, the company has scaled its certification offerings over fivefold to keep up with this demand. With over 170 exams and more than 18,000 items, maintaining the trajectory of exam development became nearly impossible.
This rapid expansion presented several challenges, such as:
- The need for a more agile and cost-effective solution to update exam content without overburdening teams.
- Enhanced security measures to address rising cybersecurity threats.
- The demand for additional exam items to support practice tests and recertification programs.
Recognizing the need for a robust solution, the organization sought a partner that could support these efforts with a commitment to collaboration, innovation, and achieving specific objectives:
- Enhanced item bank development with original, secure items.
- The ability to produce more exam forms to support practice test material.
- Establishing a process for incorporating AI technology.
Partnership Strategy
In late 2022, the organization responded to these challenges by forming a strategic partnership with Finetune by Prometric, a company specializing in AIenhanced test development solutions. Together, they set out to revolutionize the certification program by implementing Finetune Generate®, an AI hybrid item generation solution that combines advanced AI with expert subject matter expert (SME) review.
The teams collaborated to tailor Finetune Generate to meet the organization’s unique needs and found that the solution met all critical requirements.
Security was a primary concern, so Finetune Generate’s customizable, access-controlled capabilities enabled the company to train the model with its own questions, item references, exam blueprints, cognitive level descriptions, and item writing rules in a controlled environment, safeguarding against external interference. This approach gave the organization complete ownership of the content created within the platform, ensuring total confidence in security.
The Implementation Process
The partnership followed a four-step implementation approach, continuously refining the model with feedback from SMEs and item development workshops. Leveraging both Prometric’s expertise in item development and the company’s domain expertise, the team fine-tuned the AI model, ensuring psychometrically valid items.
As a result, the partnership produced 26 new AI models to support item development, reducing item creation time from 90 minutes to 30 minutes per item—a 67% decrease in development time. The organization anticipates further improvements as the tool is refined.
"This step is where the magic really happens. All content is handed over to Prometric, and they start building the model. A great team of AI and data scientists tags everything and builds the model based on our needs." - Senior Manager Exam and Content Development
Outcome
Collaborating with Prometric enabled the company to expedite the creation of unique exam content while ensuring alignment with the latest industry standards and product updates. Through Generate’s AI technology, the organization adopted a more agile approach to item development, allowing for quick adaptation to industry advancements.
This innovative approach extended across the entire certification ecosystem, including recertification exams, practice exams, and end-of-course assessments. This ensured a high-quality, consistent experience for all participants while addressing the program’s evolving needs.
Goals Achieved Through Prometric’s AI-Powered Finetune Generate
Optimized Time-to-Market: Accelerated development and release cycles for certification exams, enabling timely updates to meet industry standards.
1. Cost Savings: Adoption of AI technology led to over 60% cost savings per item, making the process more efficient.
2. Efficient Utilization of Expertise: By reducing item creation time by 67%, the company effectively optimized SME resources.
"Our investment in AI is crucial to handling the scale and complexity of our programs. Without Finetune Generate, sustaining this level of exam development would be unmanageable." - Global Head of Digital Certification Program
With Finetune by Prometric, the organization found a strategic partner capable of clearing roadblocks in efficient, secure exam development. The integration of AI transformed item creation from a manual process into a time- and cost-saving operation, enhancing productivity and allowing for further expansion.
As the AI system continues to improve with additional data, the organization has achieved a competitive edge in the certification space. Finetune’s ability to generate secure, robust items has helped optimize resources and demonstrated significant time and cost savings. With this success, plans are underway to integrate Generate into more offerings and courses.
The Global Head of Digital Certification Program anticipates further cost reductions as the Finetune system becomes more advanced. “Let’s embrace the potential of AI,” he concluded.
Tovább olvas