ISACA

Multi-Modal Testing helps ISACA deliver innovative certification testing and ensure integrity

Challenge

With 166,000+ certification holders and 140,000+ members in more than 180 countries, ISACA had growing concerns about the scalability, security, and availability of its certification test delivery. The leadership at ISACA knew that its paper-based testing model was outdated. “At ISACA, our value is all about integrity,” said Kim Cohen, director of certification. “That is my number one priority – not just for exams, but for the whole program. We convinced our members that we needed to modernize and upgrade test delivery. That’s when we decided to seek out a solution partner for computer-based testing.”

Outcome

PSI’s innovative solutions, willingness to provide services on demand, level of support, and high standards of security in item development and test delivery sum up why ISACA decided to partner with PSI. Today, PSI provides test development, psychometric services including job analysis and standard setting, and multi-modal test administration (computer-based/test center, remote proctored) for ISACA.

Solution

A commitment to customer service, value, and innovation:
Innovation and partnership were key drivers in the selection process. Being an international association with global reach, ISACA would not only be looking for perfect technology solutions, but also the perfect partner. The team launched an RFP process in 2016. “We had a scorecard, and it was a large and complex RFP to respond to, from item development and psychometrics to job practice analysis and cut score,” said Cohen. “PSI’s response was the most comprehensive of any solution provider.”

From initial RFP analysis to the final proposal delivery, ISACA was impressed with PSI’s performance throughout the process, demonstrating its strengths as a partner and a solution provider. As a proven and capable vendor, PSI became the clear choice and was selected by the internal taskforce.

Following selection in September 2016, a beta test was up and running in under six months. “Because we were able to work with PSI so efficiently, we were able to reduce our time to deployment. It was phenomenal,” Cohen explained.

During the early phases of the project, the migration of data and implementation of transactional connections was seamless, but there were challenges along the way. PSI offered a flexible “tell us what you need, and we’ll work to get it done” approach.

Commitment to Customer Service: ISACA chapters also served as customer service centers and needed to be centralized. During the implementation process, ISACA did the work to centralize their customer service, and PSI assisted to establish good customer service processes, procedures, and metrics. Customer satisfaction metrics, for example, have increased between seven and nine percentage points from the beginning of the new process.

Making Test Security a Priority: As an organization well-versed in risk management, ISACA puts security prioritization into practice by proactively managing security threats. ISACA, in conjunction with PSI’s Global Security Team, uses state-of-the-art data forensics to identify examinees who may have committed misconduct.

ISACA and PSI’s data forensics and security teams collaborate closely on an ongoing basis to monitor any potential security risks. Using sophisticated statistics, the teams identify patterns in pass rates, similarities between candidate responses, and trends in exam time and scores that may help diagnose irregularities and exam violations, even if nothing was discovered during the exam delivery process. This provides ISACA with the data it needs to determine if there has been a compromise, and if an audit or remediation is required at the testing center.

Innovation in Test Delivery: PSI developed a Test Center Request Process and KPIs for new test centers which were then communicated to their chapters. PSI was flexible and accommodating in terms of understanding ISACA’s relationships with their chapters. ISACA and PSI worked closely to identify ISACA’s priorities, such as adding more international test centers (especially in China) and introducing more kiosks.

PSI’s innovative kiosks were another integral reason that PSI was chosen as ISACA’s vendor; kiosks are easy to onboard and often used to fill the test center gaps in more remote locations internationally.

Result

PSI and ISACA continue to have a robust, mutually beneficial partnership. Performance results in areas including number of exam site locations, IT equipment, proctoring, and customer service continue to improve.

Key Deliverables Customer Satisfaction Metrics
Location Results Over 86% rated the location “Excellent” or “Good”
Computer Testing System Over 86% rated “Excellent” or “Good”
Scheduling Process Over 88% rated “Excellent” or “Good”
Examination Time & Date Over 90% felt Date & Time “Excellent” or “Good”
Candidate Information Guide 88% rated Guide “Excellent” or “Good”
Tutorial 90% rated “Excellent” or “Good”
Proctor 90% rated “Excellent” or “Good”

Contact Us

From 2016 when we met PSI to now, both ISACA and PSI have grown so much and have learned so much. We’ve had growing pains together. That’s being part of a family and a rewarding partnership.

Kim Cohen, Director of Credentials, ISACA

Read More Stories