Students Urge USMLE Step 1 Fixes, Physician Leaders Reply With Insult

The response of physician leaders to a student proposal to overhaul Step 1 of the United States Medical Licensing Examination (USMLE) seems to indicate a generational divide in the medical community.



Step 1, which medical students take after the second year of undergraduate studies, and which plays a pivotal role in residency selection, has become such a dreaded event that thousands of memes have been created expressing the anxiety that goes along with studying for and taking the test.

It has also given rise to a test-prep industry that gobbles up time and money — for those who have the financial resources — and, increasingly, a mindset that studying for Step 1 is more important than attending medical school classes.

The pressure to succeed on Step 1 "has gotten much worse" in recent years, said Jed Wolpaw, MD, MEd, assistant professor and residency program director at the Johns Hopkins Department of Anesthesia and Critical Care Medicine in Baltimore, Maryland. It was bad enough a decade or so ago when he was taking the exam, he told Medscape Medical News. Now, "it is really all-consuming," with students disappearing for months while they prepare.

That hyper-charged environment is what led a group of medical students and a resident to write an article in Academic Medicine proposing to eliminate numerical scoring on Step 1 and make it pass/fail.

The article, and a patronizing response, published in the same journal, by Peter Katsufrakis, MD, MBA, president and CEO of the National Board of Medical Examiners (NBME) and Humayun Chaudhry, DO, MS, president and CEO of the Federation of State Medical Boards, ignited a firestorm on Twitter.

Hundreds of tweeters said that Katsufrakis and Chaudhry were tone deaf and had an antiquated view of learning.

One passage in particular incensed the authors and tweeters. The CEOs suggested that if students had more time away from studying, they could theoretically take up other activities that would help them become good doctors. But, they went on, "If students were to devote more time to activities that make them less prepared to provide quality care, such as binge-watching the most recent Netflix series or compulsively updating their Instagram account, this could negatively impact residency performance and ultimately patient safety," they wrote.

A Slap in the Face

The remark "is gratuitous and highly disrespectful of medical students; the overwhelming majority work incredibly hard to become an MD. An unfortunate choice of words," tweeted Eric Holmboe, MD (@boedudley), senior vice president for Quality Research and Academic Affairs at the American Board of Internal Medicine, and former associate program director at Yale University's Primary Care Internal Medicine Residency Program.

"I have concluded that it is one of the most thoughtless and arrogant statements about #MedEd that I have encountered in a long, long time," said Art Kellermann, MD, MPH, dean of the school of medicine at the Uniformed Services University, Bethesda, Maryland, in a tweet.

Wolpaw said he was flabbergasted. "It's an incredibly insulting thing to say. It's the worst of what I'll call old-school medicine culture," Wolpaw said in an interview.

It's an example of the hierarchical — and patronizing — thinking that is increasingly being challenged in medicine, he said. Educators have come to the realization that medical students and residents are adults, not children, Wolpaw continued. "And they very much do know what is important for their personal learning."

Ironically, it would probably be a good thing if students and residents did take time to binge-watch Netflix, said Wolpaw, citing what he called "sky-high burnout rates." He added, "I would love it if my trainees were able to step away a little bit from training and spend it working out or having dinner with a friend."


Comment Draws Attention to Issue

"It is unfortunate that they responded to our article with an antagonistic tone," said lead author David Chen, a third-year medical student at the University of Washington School of Medicine, Seattle.

The response "revealed how disconnected NBME leadership is from the realities of medical students," Chen told Medscape Medical News. "In a profession with twice the national suicide rate, this conversation needs to be about more than taking jabs at modern pastimes," he said.

But, he added, it was helpful in a way, because it drew attention to the students’ plea. Their article took aim at what they saw as a myriad of problems with Step 1, an initial step on the path to securing a residency.

Too much time is devoted to scoring well on Step 1, at the cost of time spent on other aspects of being a doctor, they said. "The emphasis placed on Step 1 indirectly devalues the five other competencies — patient care, practice-based learning and improvement, professionalism, interpersonal skills and communication, and systems-based practice — by reinforcing their position as 'soft,' rather than quantifiable, skills," they wrote.

What they call the "Step 1 climate" — a hard focus on getting the best score, with $1,000 or more spent on outside prep, plus many hours locked away studying — also hinders diversity in medicine, Chen and colleagues wrote. "Under-represented minorities, women, and those with less parental income on average have lower Step 1 scores," they noted.

Scores have become the key criteria to assist residency directors in selecting candidates for highly competitive slots.

To slow down the escalating Step 1 arms race, the students suggested eliminating scoring and moving to pass/fail, though they acknowledged that was an imperfect and partial solution.

Chaudhry and Katsufrakis argued that residency programs have found that numerical scores help them differentiate among the thousands of applicants for a limited number of slots. "Does Step 1 performance predict residency success? To our knowledge, no study has been done to answer this question," they said. But, they added, "performance on Step 1 has been shown to correlate highly with similar licensing exams," they said.

Wolpaw agreed that Step 1 numerical scores are necessary to help winnow down applicants and do correlate with later performance on specialty board tests. "My program gets 1000 applications for 25 spots," said Wolpaw. "There's no way I can conduct 1000 interviews," he said. Instead, the program conducts 160 interviews to get to that final 25 residents.

But he also agrees with the students. "There need to be changes," Wolpaw said. "A Step 1 score doesn't give me any idea of who will fit into my program," he said, adding that it also doesn't tell him if someone will become a good doctor.

But moving to pass/fail could have a ripple effect. "If we make Step 1 pass/fail and don't do anything with the boards, we'll set up a problem where people will be at high risk for not passing the boards," he said.

Mona M. Signer, president and CEO of the National Residency Matching Program (NRMP), said that "program directors use USMLE scores because they are one of the few objective data points available during the residency selection process," even though the NRMP has encouraged them to consider other characteristics.

"The most successful matches are based on the overall 'goodness of fit' between an applicant and program, and that requires an honest exchange of information," Signer told Medscape Medical News. The NRMP is working with the American Association of Medical Colleges and NBME, among others, to develop a tool to give applicants more information about programs in their specialty of interest, with a goal of helping them send fewer applications.

That, in turn, could reduce the application burden on individual programs, Signer said. "If that goal can be achieved, programs should be able to review applicants in a more wholistic way, with less reliance on USMLE scores," she said.


Conflicts, Lack of Transparency

Within a day or so of the Twitter explosion, Katsufrakis and Chaudhry issued a statement of apology about their "Netflix" remark. "As leaders of the USMLE, we believe that students, medical educators, and the public deserve our respect," they said. "Our statement was inconsistent with that belief, and we are deeply sorry."

They said that their "poor choice of words detracted from the intentions of our article and misrepresented the confidence and respect we have always had for America’s future physicians. We sincerely apologize and look forward to working with Academic Medicine to correct our mistake."

The CEOs reiterated that they planned to organize a conference in 2019 to explore Step 1 concerns, and to continue conversations with stakeholders "including state medical boards, residency program directors, medical students, residents, and medical educators."

Chen was not satisfied. "What I want, and what other people want, is actual change," he told Medscape Medical News, adding that the conference seems to be just window-dressing. "I didn't see any mention of transparency on who's invited to the conference or on how decisions will be made," he said.

Furthermore, Chen noted that USMLE has a vested financial interest in Step 1, which might be preventing the leaders from changing the status quo.

Bryan Carmody, MD, an assistant professor of pediatrics at Eastern Virginia Medical School in Norfolk, tweeted evidence about what he said were NBME's inherent conflicts of interest. Carmody alleged that NBME revenues had tripled from $45 million in 2001 to $154 million in 2017, and that at least a third goes to executive compensation.

"Top executives at the NBME are handsomely compensated," said Carmody, in a tweet. Moving to a pass/fail system would "ultimately lead to a substantial loss of revenue for the NBME (as fewer students and schools would be willing to pay for their nonmandatory services)," he said. "Does this strike anyone else as a potential financial conflict of interest?"

For his part, Chen is hoping the Step 1 issue will gain wider attention. "As a medical student, I can say that we are in dire need of national leadership — NBME, program directors, and deans — that understand the current situation at a deeper level," he said.

"If they were able to recognize the Step 1 climate as a crisis for medical students and the medical profession as a whole, I have little doubt that we would see change."

Source