Result of clinical trial and warnings
A randomized clinical trial was conducted in the U.S to understand LLM influence on diagnostic reasoning. It was discovered AI chatbots outperformed physicians on average score >90%.
The study involved 50 licensed U.S physicians assigned to never-known 6 cases each. They were randomly divided into intervention and control groups. The intervention group used AI Chatbots for their diagnosis. The control group were restricted to just conventional resources alone.
Result shows that the intervention group did slightly well above the control group by 2% average margin. Why didn’t the intervention group do better? Insufficient know-how of course.
Lately, many physicians haven’t yet mastered the use of LLMs in disease diagnosis, hence their performance. They take pride in their clinical judgement which is obviously reasonable. LLMs should be a doctor’s assistant, and so can be consulted for second opinion.
While there are still problems attached with using LLMs, I still believe we can still use them to our own advantage. Caution is needed during our reliance on them.
Mistakes from over reliance on Dr. Google shouldn’t be replicated when using LLMs. We have seen from examples how people searched on google for their own diagnosis. To avoid worse scenarios from happening, these tools should be used to guide physicians rather than as our chief consultants.