“Are you OK?” a medical school classmate asked when I snuck back into the lecture hall. I had just learned that I bombed arguably the most important test of my life. I was not OK. I was employing every technique I could muster so I wouldn’t break down in public. I had just checked my score on the United States Medical Licensing Examination (USMLE) Step 1 test. The first of three “Step” exams that physicians-in-training must pass to become licensed to practice independently, the computer-based test focuses on the basic sciences. It’s an eight-hour grind done in a single sitting. This test, once completed in paper and pencil by students en masse and viewed mostly as a tedious hurdle towards licensure, has since become an obsessive focus among medical students due to its elevation in the residency application process, fueling anxieties, ambitions, and evaluations of self-worth. I will never forget the sensation of shock that followed the big reveal: I had passed, but performed worse than my lowest expectations. I was already beginning to reimagine my future, whole careers disappearing before my eyes. Now that I’m nearing the completion of my residency in internal medicine and pediatrics at the Harvard combined program at Brigham and Women’s Hospital and Boston Children’s Hospital, it all seems so melodramatic. But it is a familiar story for the thousands of students each year who land in the grey area between passing and the cutoff scores many residency programs use to choose whom to interview, unable to retake the test (which can only be done if you fail) and uncertain about what comes next. I have felt different ways about Step 1 since that day. The first was shame, as I tried to understand how I had let such a thing happen to me and hid my poor performance from others. Then came redemption, as I worked hard to compensate, over-performing on my clinical clerkships and subsequent Step examinations. Self-doubt followed, as a handful of residency program directors could not look past my Step 1 score to extend me an offer to interview. Finally, conquest, after I opened my envelope on Match Day to learn I had matched at what had always been my top-choice program. But the feeling I share today is anger: At the unnecessary anguish an exam that was designed to be pass/fail has caused so many. At the harmful impact the test has had on learning in medical school. At the rejection of common sense, evidence, and empathy that so many in leadership positions have fallen prey to by reducing residency applicants into three-digit numbers. At the laziness of this approach and disingenuous claims that the test “levels the playing field” for applicants. And at the greed of a multimillion-dollar industry that has exploited the opportunity to extract money from already overly indebted students (registering for the test costs $630, while Step 1 preparation materials and courses can run much higher). On March 11, leaders from the parent organizations of USMLE — the Federation of State Medical Boards and the National Board of Medical Examiners — along with the American Medical Association, the Association of American Medical Colleges, and the Educational Commission for Foreign Medical Graduates will convene in Philadelphia for the Invitational Conference on USMLE Scoring, also known as InCUS. The main objective of the conference is to reach consensus on the issue of whether the USMLE should continue to report numeric scores on its licensing exams, including Step 1. I see only two acceptable paths forward. One is to completely reform Step 1 (and undergraduate medical education alongside it, standardizing the way curricula are structured and how student performance is reported) to make the Step 1 score truly meaningful. This is unlikely because it would require massive coordinated effort. The other is to return the test to what it was originally intended to do: provide a benchmark assessment of competency for the purpose of professional licensure (like the bar examination for lawyers) by eliminating numeric scoring and making it pass/fail. That would put the onus back on residency programs and medical schools to agree on more suitable ways to evaluate students. Academic and evidence-based arguments abound against the continued “Step 1 climate,” which is directly engendered by its numeric scoring system. In recent issues of the journal Academic Medicine and its podcast, a rich exchange has highlighted the evidence that Step 1 is a poor predictor of future clinical performance; the negative impact of the test on students’ mental health and engagement with institutional curricula; and the discriminatory effect of the test on underrepresented minorities, women, and those from lower socioeconomic backgrounds. I am sympathetic to the challenges residency program directors face in reviewing ever-mounting numbers of applications on a tight schedule with diminishing numerical or comparative data for use in decision-making. However, the common practice of using Step 1 cut-off scores and its corresponding notion that Step 1 is some sort of equalizer across applicants — and therefore has singular value to the resident selection process — can’t reasonably be justified. First, there are wide discrepancies across medical schools that challenge the notion that the test is “standardized.” Some schools teach to the test, while others focus on the exam’s many blind spots, knowing full well that students will study the Step 1 material on their own. The amount of time off students are given to independently prepare for the test varies from a few weeks in some medical schools to several months in others. The test’s flexible timing is also nonstandard: Many schools require students to pass the test before starting rotations in the hospital, while others push it back until after clinical clerkships, knowing students will benefit from the additional experience. Even the version of the test a student sees is not standard, requiring convoluted statistical corrections that make direct comparisons of scores between test sittings (and applicants) even more problematic. On an individual level, students’ personal, family, and health circumstances, as well as their financial ability to afford the services of the growing and predatory Step 1 prep industry, have an enormous impact on achievement on the exam. And the capriciousness of assessing a person’s performance on a single day is unavoidable, particularly when the Prometric centers that administer Step 1 impose steep penalties on test takers for changing test dates and applicants cannot retake the test if they do poorly. A number of medical schools have been inadvertently complicit in the growing importance Step 1 has assumed in the resident selection process by adopting pass-fail grading for students in their coursework. Often well-intended and couched in the language of wellness, this transition to pass/fail grading has precluded schools from making meaningful claims on the relative strengths of their students to residencies, leaving Step 1 as a pillar of objectivity to fill a perceived vacuum. Grading fairly is always a challenging task, but it must not be one that schools shy away from tackling, as they are certainly better positioned to evaluate the performance of their students over four or more years than a nameless licensing exam completed in one day. I was extraordinarily lucky. I benefited from going to a prestigious medical school with a rigorous grading system that allowed me to distinguish myself when it came time to apply to residency programs. I had the time and the capability to prove myself on future USMLE exams. I worked with faculty members who knew me and could advocate for me. My desired specialty was not dermatology, orthopedics, or otolaryngology, competitive specialties that are notorious for their emphasis on Step 1 scores in resident selection decisions. And my top choice residency program did not rely on Step 1 cut-off scores and invests in a laborious, holistic process when deciding who to interview. My Step 1 score did not ultimately limit my professional ambitions. But it does limit others’. Given the lack of standardization across medical school curricula and the misappropriation of test scores for screening medical students for residency programs, the USMLE should no longer report numerical scores for Step 1. There are many possible paths beyond a numerically scored Step 1 that have been presented by others and merit further discussion. The time has come for the medical profession to apply the same thoughtfulness and rigor that physicians aspire to in clinical practice into deciding how it evaluates trainees. We must not contract out this essential responsibility to licensing boards and rely on Step 1 numbers for simplicity’s sake. Source
