Zach Irwin can create your AI generation tool and officiate your wedding. 

When most people think of test development and certification, they picture the final product. But long before a candidate ever logs into an exam, an intricate, highly scientific process takes place behind the scenes so that every single question is fair, valid, and reliable.

At the heart of this process is the psychometrics team. Often misunderstood as a solitary, numbers-only profession, psychometrics is actually a dynamic, deeply collaborative field. No one embodies this better than Zach Irwin, Senior Psychometrician at Kryterion.

In this month’s Employee Spotlight, we sit down with Zach to discuss his career journey, the revolutionary impact of artificial intelligence on test development, his dedication to client success, and the surprising ways he unplugs when he steps away from the data.

Building a Foundation in Measurement and Psychology

Zach’s journey to becoming a Senior Psychometrician at Kryterion was paved with a deep curiosity about human behavior and workplace dynamics. His academic foundation began at Missouri State University, where he earned a B.S. in Psychology, followed by an M.S. in Industrial/Organizational Psychology.

His professional career launched at Applied Measurement Professionals (AMP) as a Research Associate. When PSI acquired AMP, Zach transitioned into the role of Associate Psychometrician, working as a consulting professional for a wide variety of clients. This consulting experience proved invaluable.

After moving to the University of Kansas Center for Educational Testing & Evaluation, the draw of client consulting in the certification and licensure space brought him back to his roots. He joined Kryterion as a Psychometrician, quickly proving his expertise and elevating to his current role as Senior Psychometrician, where he helps guide the strategic testing initiatives of our diverse client base.

Pioneering AI in Test Development

If there is one area where Zach’s impact is felt most acutely at Kryterion today, it’s in the realm of technological innovation. Zach has been instrumental in keeping Kryterion at the cutting edge of industry advancements, particularly regarding Artificial Intelligence.

When asked about a notable project he is especially proud of, Zach points directly to his work in developing the AI-assisted item generation tool currently integrated within Webassessor.

“From my research with early adopters of the tool, I have seen substantial item writing benefits for clients,” Zach explains. “It allows for faster item drafting and even faster item review to validate those items for use in live administration, where we can then gather data on their performance.”

With increased adoption and ongoing enhancements, Zach sees this tool as a massive leap forward in test development efficiency. But he isn’t stopping there. He continues to explore other AI applications that can streamline the entire test development lifecycle.

“I continue to work on keeping up with the rapid pace of change AI has introduced to the test development and psychometrics processes,” he says. “This is an overarching set of knowledge, skills, and abilities that bridges almost all professions. It continually challenges my notions on traditional test development and psychometric activities and encourages me to think of new applications to modernize those activities.”

To stay at the forefront of this evolution, Zach remains highly active in professional groups and associations dedicated to the intersection of AI and psychometrics, ensuring Kryterion’s solutions are not just current, but visionary.

Debunking the Myth About Psychometricians

There is a persistent stereotype about data scientists and psychometricians that they spend their days locked in windowless rooms, crunching numbers in absolute isolation. Zach is quick to laugh this off and correct the record.

“Psychometrics is often viewed as a stuffy profession, cloistered away in solitude, crunching numbers and writing technical reports,” Zach notes. “But the majority of my time is actually spent directly working with people, including clients and other Kryterion staff, to best serve the goals of all of our examination programs.”

Whether he is conducting job analyses, developing test blueprints, writing items, evaluating exam results, or setting cut scores to ensure valid and fair delivery, his day is highly interactive.

“All of it requires huge amounts of interpersonal time with Kryterion staff, clients, and subject matter experts,” he says. 

Championing Client Success and “One Team”

That collaborative spirit extends to how Zach views Kryterion’s overarching philosophy. When asked what he considers to be the key to Kryterion’s success, his answer highlights a commitment to genuine partnership and ethical consulting.

“I believe the key to Kryterion’s success is our flexibility and sincere intentions to meet client needs, while convincingly deterring them from making critical mistakes in the name of expedience or reduced cost,” Zach explains.

In the testing industry, organizations are frequently tempted to cut corners. “We see a lot of clients looking for the cheap and fast solution to their needs,” he says. “Kryterion has always been very good at arguing the case for solutions that actually meet their desired goals.”

This dedication to doing things the right way is mirrored in the Kryterion core value that resonates most with him: We are one team.

“I never feel the need to say ‘that’s not my job,’” Zach shares. “Working at Kryterion, I’m not only pulling for all of our other departments and willing to assist in their activities wherever I can, but I also feel like a member of each and every one of our clients’ teams as their dedicated psychometric support. We’re all here to make sure that the exams we develop and deliver serve the candidates and the public interests they serve.”

Unplugging, Officiating, and Fostering Empathy

For someone who spends his days pioneering AI tools and analyzing complex data sets, Zach’s preferred method of unwinding is surprisingly analog.

“I love the outdoors,” he says. “I make it a point to unplug and go camping one or two times a year. Canoeing, stoking the campfire, cooking without a stove or electricity, sleeping under the stars. I love the simplicity and pace of days spent outside of modernity.”

Beyond his love for the wilderness, Zach has a few other unique attributes. For one, he is a certified minister. “Not that it is all that hard to achieve if you have 10 minutes to spare and an internet connection,” he jokes, “but if you need a wedding officiant, I can accommodate.”

His empathy extends deeply to animals as well. While his household currently consists of a fish tank and his daughter’s corn snake (which he admits are “not the most exciting or interactive pets”), Zach’s history with animals is touching. After recently losing their two remaining dogs prior to a move, the family is currently taking a break from furry friends. But their past is filled with incredible generosity.

“At one point, my wife and I would regularly foster animals. They were mostly dogs but also kittens and rabbits,” he recalls. More remarkably, they spent years doing hospice fostering for elderly or critically ill dogs, providing comfort and care during their final days.

Whether he is protecting the validity of a professional exam, pioneering new ways to leverage AI, or offering a warm home to a dog in need, Zach Irwin brings a rare blend of analytical brilliance and profound empathy to everything he does. We are incredibly proud to have him on the Kryterion team.