ChatGPT Successfully Passes U.S. Medical Exam

ChatGPT, which produces texts using artificial intelligence, has achieved a score close to that required to pass a difficult medical exam in the United States, according to a study published Thursday.

Sensation

California-based startup OpenAI launched a conversation robot in November that caused a sensation and was a huge success because it was easy to use, capable of engaging in human-like conversation, answering questions fairly accurately, and even writing novels or poems in seconds.

As part of the study, published in the journal PLOS Digital Health, researchers from Ancibel Health tested the program's performance in an exam typically taken for medical students in the United States, which includes questions in various areas, such as scientific knowledge, clinical reasoning, bioethics, and more.

Exam Parts

According to AFP, this exam, called the "Medical Licensing Examination in the United States", is divided into 3 parts, the first of which students undergo after about two years of study, the second after 4 years, while they must pass the third in order to be able to practice medicine.

ChatGPT was tested on 350 of the 376 questions posted on the U.S. Medical Licensing Examination website, which were included in the June 2022 exam. Those based on images were excluded from the questions.

The questions were divided into 3 types, one of which was the open-ended questions "What is the diagnosis for this patient in light of the information provided?" , and others that can be answered from a set of suggestions without the need to explain "What is the most appropriate next step to proceed out of the following?" , and a third category can be chosen to answer from a set of suggestions with the explanation "What is the most likely cause of the patient's nocturnal symptoms? Explain what you based your answer on."

Examiner corrected ChatGPT's answers and labeled, while a third chapter on the differences between them.

Percentage of answers

The correct ChatGPT response rate ranged from 52.4% to 75%, with the overall requirement for success in the test being 60%.

The study concluded that "ChatGPT is approaching the margin of success."

Lucia Ortiz de Zaraté, a researcher at the Autonomous University of Madrid, said: "This study shows the potential of artificial intelligence in the medical field."

"It can be useful for doctors when they are diagnosing and prescribing treatment," she said.

Ability to pass other exams

Another study in January showed that ChatGPT could pass exams at a U.S. law school, even if it ranked last in the ranking of successful students.