Educational content on VJNeurology is intended for healthcare professionals only. By visiting this website and accessing this information you confirm that you are a healthcare professional.

Share this video  

EAN 2025 | The limitations and concerns associated with using AI models in clinical decision-making

Alberto De Lorenzo, MD, University of Milan, Milan, Italy, discusses the limitations and concerns associated with using artificial intelligence (AI) models, such as ChatGPT, in clinical decision-making. Dr De Lorenzo highlights the need for advanced prompting to achieve accurate results and the importance of critical interpretation of model outputs. He stresses that GPT models should be viewed as an aid, not a substitute, for human clinical reasoning. This interview took place at the 11th Congress of the European Academy of Neurology (EAN 2025) in Helsinki, Finland.

These works are owned by Magdalen Medical Publishing (MMP) and are protected by copyright laws and treaties around the world. All rights are reserved.

Transcript

There are many for sure. Practical considerations are first, clinicians need to be aware of the techniques to improve the accuracy and the performance of ChatGPT. Normal zero-shot prompting, which is when we ask just a simple question, has lower accuracy and lower performance, whereas more advanced prompting is important, so this is important for people to know how to ask questions to GPT or to any other language model...

There are many for sure. Practical considerations are first, clinicians need to be aware of the techniques to improve the accuracy and the performance of ChatGPT. Normal zero-shot prompting, which is when we ask just a simple question, has lower accuracy and lower performance, whereas more advanced prompting is important, so this is important for people to know how to ask questions to GPT or to any other language model. This is very important. Second, it is very important that clinicians should be aware that models can hallucinate, they can make mistakes, they do, and so they should be aware that they need to be critical, they still need to rely on their clinical knowledge to interpret the results and envision it in the clinical context of the patient, which is something only the human clinician can do at the moment. I think another important limitation is ethical, of course, because the patients come to us as doctors and we need to provide our expertise. We should not see GPT as a substitution of ourselves or our clinical reasoning, just as an aid, as I said before. And also, as the future approaches, I think one main important point is that these technologies should not lead to, for example, a reduction in the doctor-patient relationship just because we have another tool that can substitute us. The main point is to use it as an aid so that our relationship with the patient, our core clinical reasoning, remains at the center of the clinical work but is enhanced by some model, which, of course, we know we have knowledge limitations in real life, and these models may help to bridge this gap. Of course, there is also, I mean, privacy and relevant limitations, because, of course, we are supplying this information to models which need to be standardized and approved for that, which is something we currently don’t have, and, of course, we also need to find which is the best model, which is the best prompt, something we currently don’t have to be sure that we have the best possible solution. So, at the moment, it’s something very exploratory, the clinician may rely on in their clinical practice if they really need help for complex cases, but it’s not something we can use for everyday clinical practice for every case. It is not the way we are right now, I think, at least for these complex cases of polyneuropathies.

 

This transcript is AI-generated. While we strive for accuracy, please verify this copy with the video.

Read more...