GabyA team of researchers from the University of California, San Diego, compared the written responses of doctors and the renowned ChatGPT to health-related questions found on the Reddit forum AskDoc. The goal of the scientists was to determine whose advice was better – that of the medical professionals or the artificial intelligence.
Researchers have concluded that ChatGPT provides higher quality responses and demonstrates more understanding and empathy than qualified doctors.
Although ChatGPT is often criticized for its imperfections, a new study has shown it in a positive light. In particular, the AI’s responses outperformed those of doctors on the same questions in 79 percent of cases.
What they learned about scientists
About 452,000 internet users post messages about their medical issues on the Reddit forum AskDocs. Verified medical professionals offer solutions to these problems, the publication reported. Daily Mail .
Researchers selected 195 random questions at the forum for qualified doctors to answer. At the same time, the team posed the same questions to ChatGPT, asking it to provide its own versions of the answers.
While reviewing the results of the experiment, the expert group of three doctors did not know which response belonged to the physician and which to the chatbot.
Later, Jessica Kelly, the co-author of the study, summarized that the messages from ChatGPT contained more detailed and accurate information than the responses from doctors.
One of the examples of the competition “doctor” vs AI
Question user forum : “What is the risk of becoming blind after bleach gets into the eyes, causing irritation and dryness?”
Answer doctor : “It seems you will recover. You should rinse your eyes every time a chemical substance or foreign object gets into them. You can also contact Poison Control at 1-800-222-1222.”
Answer ChatGPT : “I’m sorry to hear that bleach got in your eyes. It’s important to rinse your eyes as soon as possible after chemical splashes to remove as much of it as you can and prevent further irritation or damage. If you haven’t done so already, you should rinse your eyes with clean water or saline solution as soon as possible. You can do this by keeping your eyes open and directing a gentle stream of water from the inner corner of your eye to the outer corner for at least 15 minutes. If you don’t have access to water, you can use saline solution or artificial tears to rinse your eyes. If you experience severe pain, redness, or changes in vision, or if you cannot rinse your eyes properly, you should seek medical attention immediately. However, if your eye just feels irritated and dry, you can treat it at home using artificial tears or eye ointment to keep it moist and comfortable. It’s unlikely that you will go blind from bleach getting in your eyes, but it’s important to take care of them and seek medical help if necessary to prevent further irritation or damage.”
All the experts preferred the responses from ChatGPT. Its average score for consultation quality was 4, while the empathy level was 4.67. In contrast, the doctor’s score for quality was 3.33, and for empathy, it was 2.33.
Professor of Information Technology James Davenport from the University of Bath (UK) had a mixed response to the conclusions of the study. He noted that ChatGPT’s answers were, on average, four times longer than those of the doctors. The team claimed they selected answers blindly, not knowing which belonged to a doctor and which to ChatGPT, the scientist reminded. While this may have been technically true, the length and style certainly revealed the authorship. Professor Davenport also pointed out that “teams of clinicians often rely on prepared responses,” whereas “a stochastic parrot like ChatGPT has a much broader range of answers than even the largest library of prepared responses.”
Recently, ChatGPT caused a genuine stir in the medical community. It turned out that the chatbot is capable of passing the gold standard exam required for medical practice in the United States. This increases the likelihood that it may one day replace human doctors.
