The NAVLE®: From Survey to Blueprint

The third part of our educational series explains how items are determined for inclusion on the North American Veterinary Licensing Examination (NAVLE^®).

The NAVLE is based on a blueprint, which indicates the requirements for building the proper test that evaluates the knowledge, skills, and competencies needed by veterinarians entering private clinical practice. The blueprint is established by using the results of a practice analysis process (including verification by subject matter experts (SMEs) in veterinary medicine) and is periodically updated to reflect changes in the practice of veterinary medicine.

How is a blueprint update determined, and what does the process look like?

Since the International Council for Veterinary Assessment (ICVA) recently completed a practice analysis study and recoding project, let’s take a closer look at how the process from survey to blueprint unfolds.

Step 1: Practice Analysis

A practice analysis begins with a survey that is distributed to thousands of veterinarians throughout the U.S. and Canada. The survey is used to obtain information about the species that veterinarians routinely see in practice and the diagnoses they are required to manage in their work.

Based on the results of the most recent practice analysis, an updated blueprint for NAVLE was created. This new blueprint was developed based on “competencies” (i.e., the ability to do something and understand why it should be done), rather than “activities” (i.e., a thing that a person does).

This shift from activities to competencies meant that current NAVLE items had to be recoded to reflect the new competency codes.

Step 2: Recoding

Recoding all of the NAVLE items was a significant undertaking. There are 300 questions and 60 pre-test items (un-scored questions) on each NAVLE, plus a pool of items ready for future administrations – all of which had to be individually evaluated and recoded.

In 2018 volunteers began the process of reviewing and recoding NAVLE items to align with the NAVLE Competency Coding Guidelines. The final batch of item review and recoding was completed in January 2019.

Step 3: Item Writing and Pre-Testing

In order to ensure that items available to the NAVLE are current, new items must be written annually. The editors and psychometricians at the National Board of Medical Examiners^® (NBME^®) analyze the item pool and identify the content areas for which new NAVLE items should be written. Specific writing assignments are given to NAVLE Item Writers, who are veterinarians recruited from across the U.S. and Canada trained to write viable test questions.

After the item is written, reviewed, and approved by a team of SMEs, the item appears on the NAVLE but does not impact students’ scores. Known as “pre-testing,” this process provides an opportunity to gather statistical information about the items, including the item’s difficulty and its ability to discriminate between high- and low-scoring test-takers.

The pre-tested items and their respective statistics are then reviewed by NAVLE SMEs who represent all the major species. Members of the Assessment Development Committee (ADC), as well as other veterinarians recruited by the ADC, review the new items, their statistics, and determine whether the items should be deleted, rewritten, or moved to the pool of potential live items to be used and scored on a future NAVLE.

Step 4: Final Blueprint

With the recoding exercise complete and newly written items coded to competencies, all NAVLE items are now in alignment with the updated blueprint, beginning with the November-December 2019 test administration.

Missed the first two parts? Read A History Lesson: Creation of the NAVLE® and Practice Analysis: Incorporating Veterinary Trends into the NAVLE^®.

For more information about the NAVLE recoding project or updated blueprint, contact us.