Services
- - - Talk to an Expert
      
      Our platform, solutions, and services are perfect for building modern credentialing programs. Let us help you get off to a flying start.
      
      Schedule Meeting
Use Cases
- - - Professional & Corporate
  - - Associations
  - - Talk to an Expert
      
      Our platform, solutions, and services are perfect for building modern credentialing programs. Let us help you get off to a flying start.
      
      Schedule Meeting
Our Partners
Resources
- - - Podcasts
      
      Kryterion's podcast explores the rise of technology, artificial intelligence, machine learning and the global demand for credentials.
      
      Listen Now
  - - AI
      
      4 Ways AI is Redefining Customer Experience and Operational Efficiency
      Read More →
  - - Talk to an Expert
      
      Our platform, solutions, and services are perfect for building modern credentialing programs. Let us help you get off to a flying start.
      
      Schedule Meeting
About Us

Top Mistakes New Certification Programs Make And How To Avoid Them

Certification programs have proven to be an incredible tool for fueling business growth. The powerful benefits they generate impact all participants: certification program sponsors, certified professionals, and organizations hiring certified professionals.

Still, acceptance of a newly created certification in its target market is rarely automatic. Credibility is a precious commodity. Significant forethought and exacting effort are generally required to earn and keep it.

The certification development missteps we identify below have become quite familiar to Kryterion over the course of 20 years of psychometric consulting work!

We share these insights in the hope that they can help your program sidestep errors that can cost it dearly. Sometimes, those costs are financial. In other cases, the cost is reputational, and the damage done can be lasting, even before the program itself is well established.

Mistake #1: Not Conducting Market Research

A variety of market forces can spur discussion about the feasibility of developing a new certification program. We list and describe some of the most common reasons below.

Take heed. The marketplace you seek to serve with a new certification will decide for itself which reasons to certify are most compelling.

Too often, new certification developers don’t conduct enough market research to grasp their markets’ most pressing concerns. They compound the error by failing to determine if the market will pay for the value that the certification program intends to offer.

There is more to market research than assessing target market interests and requirements. The long-term viability of a certification program depends on developing a complete picture of the competitive landscape in which it will operate. As part of that process, new program managers must also achieve an honest understanding of their own competitive advantage(s) or lack thereof.

Will the certification program you envision address one or more of these common market interests?

Elevation of A Profession’s Status

Today’s innovative world economy is creating demand for new types of skilled professions daily. For similar reasons, respect for a variety of established professions has diminished.

In either case, advanced certifications that help those professions establish or expand their value may also increase the status of their respective professions.

However, in the absence of valid market data indicating widespread support, this objective may not prove a winner. The challenge is to clearly identify and deliver the high-level skills and expertise the marketplace genuinely appreciates.

Increased Brand Loyalty

Companies that want their brand to succeed must foster loyalty. How will your certification enable those firms to boost availability of the deliverables that their clients care about most? Quality, efficiency, reliability, expertise, and value.

Career Advancement

As Forbes points out, “By ensuring employees are continuing to develop new skills, businesses not only reskill their current employee population, but they also attract new workers who are eager to learn and advance within the company.”

A good certification program will provide all the components necessary to chart an employee’s professional path(s), while quantifying their skills.

Professional Recognition

Members of your target market are ambitious. They’re driven to grow their careers, to achieve, to expand their horizons. Their desire for professional respect and recognition can be a powerful motivator.

Building a certification that the marketplace values and validates can be as significant to your program’s long-term success as it is to the success of its certificants.

Mistake #2: Underestimating What It Takes

Developing a certification program constitutes a significant challenge. For many organizations, including those whose due diligence was excellent, the responsibilities and requirements prove far more complex than expected.

Credible

According to the National Skills Coalition, credible certification programs provide professionals with the means to “equitably achieve their informed employment and educational goals.”

This means there must be “valid, reliable, and transparent evidence that the certification satisfies the criteria that constitute quality,” the group says.

NSC has identified 3 criteria that organizations should include as part of a credible program:

Evidence of ample job opportunities associated with the credential. There must also be quantitative data and direct communication with employers.
Evidence of the competencies mastered by learners; they must align with expected job opportunities.
Evidence of both employment and earnings outcomes of individuals after acquiring the credential.

Legally Defensible

Certification programs, whether run for profit or not, face real marketplace risks. It is not uncommon for individuals who failed to earn a high-stakes certification to challenge that process. In some cases, a challenge can become a lawsuit!

The repercussions of a successful lawsuit can negatively impact both the programs financial stability and its brand.

To protect your program, consider consulting an attorney familiar with credentialing law before test development begins to avoid potential issues:

Failure to follow certification best practices
Establishment of unfair or unsound eligibility requirements
Failure to establish written policies for handling appeals and disciplinary actions
Failure to establish standard operating procedures for administration of documented policies and procedures

Professional Testing Standards

Professional testing standards, including best practices for certification development, involve a great deal more expertise than is commonly understood. The subject matter experts (SMEs) you recruit to help develop test questions are not, typically, trained credential development specialists.

That responsibility typically falls to the highly trained professionals known as psychometricians.

They’re the experts on developing credentialing exams that are fair, reliable, legally defensible, and consistent with testing standards.

What can go wrong? A lot. A legally defensible examination must be based on research that

accurately defines the underlying knowledge and skills of a job role, and
validates the level of minimum competence required to perform adequately in that role

Widely-accepted development processes and techniques have evolved to establish exam content and passing standards. Significant expertise is required to select and apply the appropriate methodologies needed to create a defensible exam.

Programs that create exams unsupported by such expertise can be forced to start over from scratch, if successfully challenged.

The process is typically rigorous. It requires careful and complete documentation.

The good news is that a well-designed, carefully implemented test development process helps enhance program credibility. Employers will quickly associate certificants who deliver superior results with the programs that certified them and draw conclusions.

It Takes Resources

There is a lot to do. Market research. Competitive analyses. Policy and procedure development. Ongoing SME recruitment and management. Job task analyses. Item writing workshops. Beta testing. Test form development. Standard setting.

Dr. Leslie Thomas, Kryterion’s Chief Strategy and Product Officer and a psychometrician with over 20 years of industry experience, states that the process required to build a professional certification exam typically requires nine to 12 months. For healthcare and finance industry-related certifications, it can take even longer! Certifications for the IT industry typically take less time, reflecting the rapid pace of product development and narrower scope of assessment.

And the cost? It varies depending on the type of exam (written vs. performance-based) and number of exams. According to Dr. Thomas, development of a single written exam can cost around $50K, while a performance-based exam can cost over $100K.

So much for the development costs. But what about the program’s maintenance costs? For more about that, see Mistake #4, below.

Mistake #3: Choosing the Wrong Subject Matter Experts

As Indeed.com explains, subject matter experts have an in depth understanding of a particular job. This includes knowledge, skill, and ability requirements, as well as other relevant information. SME’s expertise qualifies them to assist with various phases in the development and validation of tests and assessments.

It is also important that your SMEs are representative of the target population for your certification.

In fact, SMEs—even those with similar professional credentials— frequently have different strengths. An SME with academic and product knowledge will likely have a much different perspective than an SME with daily, on-the-job experience. Look to the latter for insights into “real world” scenarios. Since certifications validate the applied knowledge needed to perform the job, SMEs with real world insights are absolutely needed.

In other words, the important thing is to develop the right mix of SMEs to ensure a solid range of experience and expertise. Depending on the development process, your mix may also include at least one individual whose expertise is a near match for what professionals call the minimally qualified candidate (MQC). That’s a person who passes the certification exam with the lowest level of acceptable performance.

This SME serves as a reality check for more experienced SMEs. Without this individual’s insights and input, they might expand certification scope or expertise beyond that appropriately expected of a minimally qualified candidate.

To maintain the security of your examination(s), the SMEs involved in development of your certification must be willing to sign and abide by non-disclosure agreements.

Additionally, they should agree to waive any claim to intellectual property (IP) that they help develop for the certification program. They should also agree not use this IP for their own professional purposes. The IP then becomes the exclusive property of your program, as it should be.

SMEs that won’t agree to these terms are unlikely to have the best interests of your program at heart.

Top Mistakes New Certification Programs
Make And How To Avoid Them

DOWNLOAD THE GUIDE

Mistake #4: Thinking It’s A "One and Done" Project

Certification program management sometimes forgets that its project isn’t done when a new certification exam makes its successful – and gratifying – debut. Nothing could be further from the truth. Implementation is just the first step in a certification program’s life cycle.

The need to maximize your program’s ROI makes implementation of an ongoing test maintenance plan and budget essential.

An annual exam “health check” ensures that the exam continues to perform as expected, year after year. This typically involves review of performance statistics with the goal of identifying test questions that must be replaced. Subsequent steps include identifying suitable replacements, modifying test forms, and adjusting passing scores accordingly.

Ongoing monitoring of pass rates can help determine whether test content has been exposed and may require replacement.

In many industries rapid innovation is the new normal. New skill sets develop. Existing protocols evolve. More efficient tools and processes replace previously accepted methods. Program management is responsible for ensuring that its exam content reflects current standards of professional practice and performance.

Inclusion of outdated exam content can quickly lead to loss of marketplace credibility. Plus, the inability to defend results that rely on outdated content can put program viability at serious risk.

Keeping test content fresh serves another, equally vital purpose. It enhances test security. The more popular your certification becomes, the more test content will be exposed, and the more likely it will be openly discussed and shared. Updating test content on a recurring basis protects a credential’s reputation and the underlying investment it represents.

Mistake #5: Insufficient Test Security Measures

As noted in Mistake #2 above, development of a single written exam can cost around $50K. The absence of a comprehensive and effective test security program puts that hefty investment at significant risk!

Failure to understand and address the full range of threats to exam content can severely damage a certification’s appeal, credibility, and value. The more valuable a certification appears, the more likely it is to attract unscrupulous profiteers. The risk to “high-stakes” exams is particularly acute.

A certification exam is considered high-stakes when its outcome qualifies or disqualifies the candidate for a highly-valued qualification or opportunity. Examples include a job, promotion, or licensure.

Consider the following list of common and potentially damaging test security shortcomings:

Failure to ensure high-stakes exams are supervised by proctors unincentivized by test outcome.
- Trained proctors can authenticate test candidates’ identities, ensuring that the person who earns the credential actually sat for the exam.
- Trained proctors deter and prevent use of prohibited reference materials.
- Trained proctors can recognize disguised recording and spying equipment and act to prevent their use.
- Use of dedicated proctors, hired and trained by a third party, reduces the likelihood of proctor+candidate collusion by removing potential conflicts of interest.
Failure to supplement the efforts of trained proctors by recording audio-video of test sessions. Such recordings can help substantiate proctors’ assertions of prohibited behaviors.
Failure to draft a comprehensive candidate agreement (CCA) establishing terms of the program and exam. CCAs describe required and prohibited candidate behaviors and the consequences to candidates for violating the agreement. Program expectations of all candidates are firmly established on an equitable basis.
Failure to obtain test candidates’ formal acceptance of a CCA before access to the exam is permitted. Allowing access to your exam without a signed agreement allows bad actors free reign.
Failure to obtain signed Non-Disclosure Agreements (NDAs) from all SMEs, program staff, and others with access to your exam content. Without these agreements in place your exam content may wind up widely exposed.

Mistake #6: Confusing Certificates with Certifications

The terminology is confusing. But the distinctions between the two are real and have meaningful legal implications.

The more formal terms, assessment-based certificate and professional certification will help clarify the differences between them.

The objective of an assessment-based certificate is to build competence. It starts with a training program that prepares participants to achieve a certain level of performance. Participants’ whose subject matter mastery is subsequently confirmed by an assessment receive a “certificate.” And thus, the label: assessment-based certificate.

A professional certification validates existing competencies. It verifies that the recipient has the minimum qualifications required to competently perform a certain professional responsibility based on predetermined and standardized criteria. A formal training leading up to or preparing candidates for certification is not required.

There is another, equally important distinction between these two approaches.

Assessment-based certificates are typically considered low stakes. The goal of a certificate program is to ensure that a specific level of competence is achieved. When it isn’t, additional training and remediation are given until competence is reached.

Assessment-based certificates build or expand an existing professional role. Once a certificate is earned, there is no continuing education requirement. The certificate’s validity may or may not be limited to a specified date range.

Professional certifications, on the other hand, are frequently prerequisites for employment outcomes. These include hiring, promotions, raises, or permission to perform a role. For this reason, they are considered high-stakes exams.

Another important distinguishing feature is that their validity is typically limited to a specified date range. If not renewed, certifications will expire.

Top Mistakes New Certification Programs
Make And How To Avoid Them

DOWNLOAD THE GUIDE

Mistake #7: Forgetting About Renewal Requirements

As just noted, professional certifications are typically only valid for a limited time and expire if not renewed. This means program management also has a responsibility to define – and communicate – certification renewal requirements before launching them. Setting candidate expectations for the program from the very start is an important step in establishing overall program credibility.

There is another important factor to consider. Professional certifications confirm a minimum level of professional competence. The disruption generated by the technological and scientific advances makes it essential to consistently re-evaluate and update the criteria your certification assess.

The skills and knowledge that constituted minimum competence last year or the year before may not be applicable next year! Don’t think that employers and your competitors won’t notice when your certificants lack the expected expertise.

You also have a continuing responsibility to track and verify renewal documentation submitted by certificants. This is necessary to ensure that each person using your designation has earned that privilege.

Depending on the complexity of the requirements and the volume of renewal applicants, process management can become quite complex. If your program offers multiple certifications, the management challenge can grow exponentially.

Nonetheless, efficient handling of these processes is an essential element of your brand image maintenance.

Closing Thoughts

Creating a new certification program comes with a steep learning curve. Kryterion hopes that this summary will help you anticipate and proactively avoid the Top Mistakes Certification Programs Make during the development and implementation process.

Want to learn more about best practices and cost-effective certification program development procedures? Contact us. Kryterion would be happy to help you explore your credentialing options!

Talk to an Expert

Talk to an Expert

Podcasts

4 Ways AI is Redefining Customer Experience and Operational Efficiency

Talk to an Expert

Top Mistakes New Certification Programs Make And How To Avoid Them

Table of Contents