A persistent demand has arisen in the IT space for certifications that accurately reflect the real-life circumstances that candidates encounter on the job. Practical examinations already on the scene come much closer than standard multiple-choice at mirroring the day-to-day experiences of test takers, but at an inordinate cost to candidates and issuing organizations. As a pioneer in the IT sector, Hewlett Packard Enterprise sought a trusted partner with a deep understanding of the industry to help bring their vision for a new Hybrid IT Master ASE Certification exam to life.
“We’re very familiar with practical exams that require candidates to travel to a set location. They’re in a room with the hardware, and a proctor is in the room with them. None of that experience is scalable,” says Janet Lehr, Certification Exam Manager at HPE. With a clear mission to modernize the existing, highly valued practical exams that are a critical part of HPE’s partner-ready compliance training, an outline for the new, industry-leading hybrid exam took shape.
A key part of the framework for this new exam is the shift from shipping and standing up hardware in a classroom or testing facility to permanently housing the hardware in a data center, which can be remotely accessed by test takers. When the time came to select a flexible partner that was willing and able to integrate with HPE’s learning management system and the aggregate solutions used in this project, PSI stood out. As a pioneer of secure online proctoring platforms and best-in-class exam delivery, PSI’s core partnership approach delivers real value to HPE’s project.
“It became clear very early on that there were architecture and design concepts that we needed to test which couldn’t just be done by APIs on real hardware,” explains Dane Erickson, Exam Development Manager at HPE. The hybrid examination, which went live on November 1st, 2019, is delivered in three unique parts:
- Discreet Option Multiple Choice testing: Candidates answer multiple choice questions using content protective item types.
- Discreet Option Multiple Choice testing inside a live environment: Candidates answer questions based on their experiences in the environment.
- Practical Examination: Candidates access and manipulate the live environment remotely.
This innovative exam structure was only made possible through the integration of PSI and Caveon Test Security’s SEI test engine. This union was developed with the highest level of security in mind. Discreet Option Multiple-Choice, dynamic SmartItems, and multiple task scenarios ensure that no two candidates will see or complete the same exam. Even if information about what is on the exam is shared, every candidate must have the skills and competencies to conclusively pass all three sections.
Picking the Right Partners
“We knew that it would take a village to make this performance-based exam work,” says Lehr of the intrepid team that was assembled to make this exam a reality. “Without having really excited, motivated, adventure-loving partners, we couldn’t have done this.” PSI worked very closely with the accomplished and talented partners – Caveon Test Security, Computer Data Inc., and Platinum Training Solutions – that were selected to deliver vital, customized parts of HPE’s program architecture. To complete the project in the six-month timeframe allotted, it was crucial that each of the partners understood HPE’s needs, and that they would come together securely and harmoniously to make the examination work to the highest standards.
Throughout the rigorous development timeline leading to an enthusiastic UAT and go-live, PSI crafted exceptional, secure candidate scheduling, registration, and online proctoring solutions that were customized to meet HPE’s specific requirements. “The partnerships were really magic,” says Michael O’Leary, Technology Support Manager at Caveon. “Whether or not you have a really good plan, or all the money it takes to make a project happen, if you don’t have the people coming together in a meaningful and truthful way, you just fail.”
The technical requirements behind HPE’s Hybrid IT Master ASE Certification examination are highly specialized and precisely calibrated to provide a seamless candidate experience. “In all of software you’re seeing this specialization, and companies coming into a niche where they can perform at really high levels and shine individually,” O’Leary explains. As a key contributor, it was vital that PSI understood exactly how to communicate securely to effectively ensure that the right teams were getting the right information outside of their own development ecosystems.
“The scheduling process for this hybrid IT exam is very different,” Lehr explains. “When candidates are selecting a date and time for their exam, not only are they reserving their seat with the remote proctor, but they’re also reserving CDI’s hardware lab and technical proctor to monitor the lab kits at the same time.” To accomplish this, PSI worked across teams to develop a secure experience with faceted mechanisms in place for communication between the vendors and the candidates. “We worked with PSI and CDI’s API to make the scheduling experience really pop,” says O’Leary. “We thought forward with it. The beautiful thing about these integrations is that we can make easy adjustments to the scheduling cadence of the exams in the future.” The result is an ideal scenario for ensuring candidate accessibility and operational ease on the back end.
It is critical from a candidate experience point of view that HPE offers single sign-on as part of their hybrid exam. “We needed single-sign on to be able to move that learner around seamlessly from our learning management system to the testing environment,” says Jim Lucari, Senior Manager Certification Development at HPE.
Candidates will only need to create one account on the host site where HPE’s LMS is located. After logging in, they will automatically be taken to PSI to launch their exam. “When they complete the exam, they will be logged back out to the HPE site, and they’re done,” says O’Leary. “Single sign-on is most important from a security standpoint,” he adds. “When you have candidates with accounts all over the place, you’re opening up areas for PII to leak.” Only pertinent candidate information is transferred between the environments, and that data transfer is completely protected and inaccessible.
Live Online Proctoring
Traditionally, a performance-based exam using hardware would occur over the course of several days and require candidates and proctors to travel to a testing location, compounding anxiety and expense. In an age of accessibility, this conventional methodology greatly limits exam availability for candidates, and offers little or no flexibility in terms of choosing a testing location.
“We have found with our other exams that live online proctoring is more secure than it is in many testing centers around the world,” says Lucari. His team knew very early on that they wanted to include live online proctoring with PSI Bridge as part of the new hybrid exam. “We’ve had online proctors report to us that they’ve heard people in the background while the exam is going on. They see eye movement and they know someone is reading something and cheating is going on, and they’ll stop the exam.”
“When we send candidates into traditional in-person testing environments it affects their psychology. It affects what they know and their ability to express that,” adds O’Leary. There has never been a better time to beef up security and provide accessibility for remote candidates in high-stakes testing. As a nod to the forward-thinking nature of HPE’s hybrid exam, O’Leary notes, “Folks taking exams outside brick-and-mortar exam centers is not only the future, it’s the ethical future.”
I’ll never forget it. There were a lot of us on the call where we were watching it actually happen inside of all the different environments, and it just worked. And it was really cool.
“We have created a new level of skill with this exam that we are now able to validate and confirm,” says Erickson. “People are excited to show that we are beyond that bar.” HPE’s Hybrid IT Master ASE Certification exam is the first of its kind with international scope, and the opportunity to shape the future of high-stakes testing is enormous.
Stellar Candidate Experiences
“This new performance-based exam will have a very positive impact on our candidates,” says Lehr. Virtually every aspect of the candidate experience has been designed to surpass current industry standards. “We’re aiming for no unnecessary frustration,” says O’Leary. “Candidates should be frustrated by the written items if they don’t know the answers and by the tasks if they don’t know how to do them, they should not be frustrated in any way by getting to the items or getting to the tasks.”
“This certification absolutely has meaning,” says Erickson. “You cannot go and buy a brain dump somewhere and get through this exam.” In practical terms, there is real value for candidates who earn this significant credential. “It is a key part of our Partner Ready compliance requirements,” says Lehr, “Gold and Platinum partners who meet these requirements will receive higher financial benefits.”
Foundation for the Future
In terms of assembling an outstanding team, surpassing key project milestones, and receiving enormously positive candidate feedback since launch, the new Hybrid IT Master ASE Certification exam is already a success.
“We now have the foundation to expand this capability to other levels of our program,” says Lehr. Noting the global and multi-industry scope of this exam, O’Leary adds, “People are going to be looking at this exam all over the world and saying, ‘Is this an exam we can replicate inside of our industry?’” The innovation behind PSI’s leading technology perfectly aligns with HPE’s mission as a pathfinder in the certification space. To encapsulate this groundbreaking approach Erickson concludes, “We want to be the lead of a movement where credentials and certifications actually mean that candidates have the skills that they need.”
We now have the foundation to expand this capability to other levels of our program.