Transition Recommendations For The Reporting Of USMLE Step 1 Scores As Pass/Fail

On February 12, 2020, the United States Medical Licensing Examination (USMLE) announced that, “the USMLE program will change score reporting for Step 1 from a three-digit numeric score to reporting only a pass/fail outcome” as early as January 1, 2022. This change will have a major impact on medical education. USMLE has not announced a complete plan for the transition to pass/fail score reporting. USMLE’s careful management of this transition is vital to ensure the fair and ethical treatment of all medical students. As noted in the Summary Report and Preliminary Recommendations from the Invitational Conference on USMLE Scoring (InCUS), “proposed solutions should do no harm and should be fair to all examinees.”

Upcoming decisions for USMLE

One frequently asked question is, “If I received a 3-digit score on my Step 1 examination, will my 3-digit score be visible on paper and Electronic Residency Application Service (ERAS) transcripts after Step 1 outcomes are reported as pass/fail?” USMLE has said, “The USMLE program has not yet determined how the policy change will impact score reports and transcripts.” This creates the possibility that current students will have preexisting numeric scores converted to pass/fail outcomes if they apply to residency programs after January 1, 2022.

USMLE must decide how the policy change will impact score reports and transcripts for students who have received or will receive a three-digit numeric score before the transition to pass/fail score reporting. One option is to permit students with numeric scores to include them in their residency applications. There is an alternative proposal to convert numeric scores to pass/fail outcomes on all paper and ERAS transcripts after the transition date. This would ensure that all students have pass/fail score reporting on transcripts after the transition date. However, this would amend the rules and policies governing Step 1 score reporting. For students who have already taken the exam, it would be unfair to amend the rules and policies governing their score reporting after the exam has been taken. Such an amendment could do harm to students with high preexisting numeric Step 1 scores, as they would lose the current value of their scores for residency applications.

Additionally, the conversion of numeric scores to pass/fail scores on transcripts could cause a significant change in the training plans of current students. Some students would likely prefer to have their numeric scores converted to pass/fail outcomes and may elect to delay their training to ensure that their numeric scores are converted to pass/fail outcomes on transcripts. Alternatively, some students may seek to accelerate their training to ensure that they maintain the current value of high numeric Step 1 scores. For example, based on the current announcement, students in MD-PhD programs would potentially need to discontinue their PhD programs and return to medical school before July 1, 2020, if they hope to ensure that their numeric Step 1 scores are visible on transcripts when they apply for residency. This would undermine the stated goals and mission of MD-PhD programs and other combined degree programs.

In summary, the mandatory conversion of three-digit numeric scores to pass/fail outcomes on transcripts would: 1) amend the rules and policies governing Step 1 score reporting after the exam has been taken; 2) do harm to students with preexisting high scores, 3) disrupt the training timeline of current students; and, 4) undermine the stated goals of combined degree programs. Therefore, USMLE and ERAS should not mandate the conversion of preexisting three-digit numeric Step 1 scores to pass/fail outcomes on transcripts.

USMLE must also decide when Step 1 score reporting will become pass/fail. USMLE has announced that the change will occur no earlier than January 1, 2022. If the policy change is enacted on January 1, 2022, it would, at a minimum, likely have a direct impact on current first-year medical students in the Class of 2023 who plan to take the Step 1 examination during their third year of medical school. First-year students have already started preparing for the Step 1 examination under the assumption that they will receive a 3-digit numeric score. To change Step 1 score reporting for current students would be an injustice as the rules governing Step 1 score reporting would be changed while students have already started preparing for the exam. Therefore, USMLE should consider converting Step 1 score reporting to pass/fail no earlier than January 1, 2023, to minimize the impact of the scoring change on enrolled students. Future students should be informed of all transition details prior to matriculation.

Lastly, USMLE must decide how to accommodate students at institutions that currently take Step 1 at different times in their medical school curriculums. Without oversight, there would be a year when students at institutions that take Step 1 during their second year of medical school would have numeric scores for their residency applications, while students at institutions that take Step 1 during their third year of medical school would have pass/fail scores. Step 1 scores are currently used as a primary metric for residency selection. It would be unfair for students at some schools to have numeric scores and students at other schools to have pass/fail scores. However, for the reasons described above, mandating the conversion of preexisting numeric scores to pass/fail scores is also not fair.

The most equitable way forward may be for USMLE to work with their affiliates and stakeholders to mandate that future medical students are not eligible to take Step 1 until their third year of medical school. If this policy is adopted, it should be enacted for students matriculating in 2020 or later. However, it should only be considered if students and schools are informed of the change prior to student matriculation. If the mandate begins with students matriculating in 2020, USMLE, along with their affiliates and stakeholders, could mandate that members of that class are not eligible to take Step 1 until after January 1, 2023, at which point exams would receive a pass/fail score. This solution addresses the aforementioned concerns and provides a clear roadmap for the transition to pass/fail score reporting that accommodates the interests of students at medical schools that currently have different curriculums and timelines.

The implementation of USMLE’s decision to change Step 1 score reporting to pass/fail will have a significant impact on medical students. Some students will need to make education and career decisions based on USMLE’s current announcement before July 1, 2020. Without detailed transition plans, it will be difficult for students to make informed decisions. Therefore, we feel that USMLE should make additional transition decisions and announcements as judiciously and expeditiously as possible.

Recommendations

• USMLE and ERAS should not mandate the conversion of preexisting three-digit numeric Step 1 scores to pass/fail outcomes on transcripts.
• USMLE should consider converting Step 1 score reporting to pass/fail no earlier than January 1, 2023, to minimize the impact of the scoring change on enrolled students. Future students should be informed of all transition details prior to matriculation.
• USMLE should consider working with their affiliates and stakeholders to mandate that future medical students are not eligible to take Step 1 until their third year of medical school. This change should be considered for students matriculating in 2020, or later, but should only be considered if students and schools are informed of the change prior to student matriculation.

Closing remarks

It is essential that USMLE considers all viewpoints when developing a plan for the transition to pass/fail score reporting. We hope that this commentary will raise awareness and stimulate the discussions necessary to ensure that USMLE’s transition plan acknowledges the interests of all members of the medical community. Through these discussions, we hope that these recommendations will receive the endorsement of medical schools, USMLE, and other major organizations. Alternatively, we hope that these recommendations will inspire the publication of alternative recommendations that protect the interests of all members of the medical community.

David F. Havlicek and Ian B. Winthrop are medical students.

Source