ESOC 2024 | AI to improve recognition of stroke symptoms in calls to emergency medical services

What I do in my research, along with my research group, I look at whether or not the patients with stroke are recognized when they are calling into emergency medical services whether that be the 911 equivalent in Denmark which is called 112 or the medical helpline which is sort of the out of hours service, although in the data I look at it’s a medical helpline that’s up 24/7 so people can always call. What we look at is whether the call handlers, do they recognize that this is a stroke case and do they get the patient the help they need? Do they send an ambulance, get them fast to the hospital? Working with that data, we’re trying to show that you can use artificial intelligence, or machine learning more specifically, to actually enhance this and make a supportive tool for these call handlers to help them recognize stroke in these audio calls.

So what we showed was that for the 911 equivalent line, there was a lot of stroke cases not being recognized -about a third- and for the medical helpline it depends on how you look at the data. If you look at it broadly and say that well, they have to say that this is a stroke, it was only 1 in 4 that was recognized. If you look at it more narrowly and say okay well as long as they have given a specific diagnosis and then it can’t be stroke then it’s about half that they recognize. So that’s a lot that they miss and worse on the medical helpline, which is not the one we advise patients to use but in our data we can see that on all calls to the EMS, it’s not half but it’s getting up there; it’s about 11,000 that call the 911 line and about 9,000 that called the medical helpline about strokes. So a lot of people are not getting to the hospital on time and we can see that reflected in our other registry data where over half the stroke patients are not getting to the hospital in time for, for example, thrombolysis.

It helps to think about, what’s the challenge for the call handler? Because the call handlers are not generally stroke experts. The call handlers are, in our case, paramedics and nurses and sometimes physicians, but not necessarily neurologists or in other way cardiovascular experts. They have to triage everything. They have to triage the broken legs, the pregnancies, the poisonings, and the strokes… so they have a great deal to sort between and so it’s difficult for them to handle strokes. For example, in the data set we used for the machine learning, only 4 in 1000 calls were a stroke. So that tells you how difficult it is for them and so we’re thinking that with an automated detection of stroke using this machine learning technology we might be able to enhance their recognition and to give them that boost to say “oh this could be a stroke… should I act on this and send the ambulance?” and then get more stroke patients to the hospital fast.

So what we did was we took this data set… we’re fortunate in Denmark to have very robust registries, so we took all the calls to the EMS service and we linked that with the dispatcher suspicion of stroke (whether or not they suspected it, just a yes or no) and then we linked that with our stroke registry where all strokes treated in hospital stroke units are registered. So now we had a data set where we could say “okay this is a call that was made, it was made by a patient with a stroke yes or no, and it was recognized by the dispatcher yes or no”. Using that data set, we could get a machine learning algorithm to first transcribe the call and then to analyze the call to see whether there were patterns related to whether it was a stroke call or not. Then, using that, make a prediction on if it was given a new call whether this is a stroke yes or no. That’s the basis of it really. What’s important is we did this on data from 2015 to 2020, so it’s about 1.5 million calls in the data set, and then we tested it on data from 2021, which was about 344,000 calls. Respectively, for the training data set we had about 7500 strokes and then 750 strokes in the test data set. What’s good about that is that since we use a data set that’s completely different from the training data set for testing, we’re certain that it doesn’t just learn which of these specific cases are strokes, but it learns patterns.

We can also see that when we sort of try to open the black box of the algorithm… So usually you can’t really say what is the prediction on this specific patient from. It’s an advanced neural network so there’s a lot of processes when you when you crack open the code. But, we can see on the large scale which words are highly predictive of it being stroke or not stroke. We were really encouraged by this because we can see that the words used for a stroke case that are highly predictive are things like which side of the body they’re talking about, having double vision, slurred speech, feeling weak, needing an ambulance… words we would normally associate with stroke. Whereas the words that are highly associated with it not being a stroke call, those are things like having pneumonia, pregnancy, needing over-the-counter medication… things we wouldn’t really associate with an acute stroke call. So that was really encouraging for us.

We have the results and in these results we were very conservative in saying when we were bit large when we’re saying well how many are recognized by the human medical call handler. They recognize about 53% of strokes in the case where we were fairly inclusive. But for the machine learning algorithm, it recognizes 63% so that’s 10% higher recognition. Notably we achieve a positive predictive value (which is when it’s suspected stroke how often is it right), we achieve for the human call handlers 17.1% but for the for the machine learning we achieve almost 25%. It does better on both parameters and that’s important because usually if you get a very high recognition or sensitivity but then the positive predicted value tanks, well then you just have an algorithm that that overloads the medical system later. So that was important for us.

What we did, we tried to make it as simple as possible because this is the first time that anyone has done any of this so we wanted to make sure. You could do different things. You could do things with audio, so you could you could potentially look at speech patterns. This requires a very robust base recognition of speech patterns because if you have a very heterogeneous group where there’s a lot of differences in how people speak it would be more difficult to get the slurring of speech, the stilted pattern of speaking for patient with aphasia for example. But you could look into that or you could look into… well get some more data. If you have a system where you have the patient’s age, the patient’s medical history, the patient’s sex, then you could maybe get this to the algorithm and that might improve the algorithm as well so there are possibilities for enhancing this but all of it requires robust data and a robust infrastructure digitally.