AAN 2025 | Developing an accurate seizure forecasting tool: advances and challenges

Seizure forecasting, really exciting field and the goal here of course is to know when is the next seizure, how high is my risk for seizure soon so that I can plan my life. Should I go driving? Can I miss my drugs today? Should I take extra drugs today? Should I stay home today? Right? So do I want to take low-risk activities? Do I want to take high-risk activities or just normal activities. And of course, no one has built a successful, deployable, proper seizure forecasting tool that really works, that’s been proven to work at this point. There’s been a lot of really cool research that says that maybe we could. And my lab is one of them that said maybe we could. Our question was as low-tech as you could get. We said, could you use seizure diaries all by themselves and use that to predict the risk for tomorrow? And several years ago, we published a study that says that, yep, you actually could. And we think that’s true because we took thousands of historical diaries and we blanked out tomorrow from those historical diaries and we said, what’s the risk? And then we got a risk and it worked out that it was better than chance. So we’re very proud of ourselves. And we said, we’re onto something. And other labs have done similar types of things with different types of data. So because we had that exciting success, We then collected some prospective data from patients. So that means that we actually signed people up and we took their history and we said, OK, we’re going to predict tomorrow. We won’t tell you so that we don’t have you drive your car and crash into a wall. But instead, we’re just going to pretend like we’re going to give it to you. But instead, we’ll just know and you’ll tell us when are the events. So we did all that and we used our same technique as before. As before, so we didn’t change the rules and to our surprise, it wasn’t very effective and we looked into a lot of detail about why and it turns out that the way we measured success the first time and the way many many labs are measuring success the first time is not very good it’s very permissive it allows for a lot of things that look seemingly good to be declared success when in fact it’s not that good. So I started calling this thing a napkin method so here’s the napkin method. You take a napkin and you write down, how many seizure days have I had in the last 90 days? And then you divide by 90. So if I had three seizure days, then my risk for a seizure tomorrow is one out of 30, or like once a month, right? So this is like the easiest thing that you could do if you have a grade school education you can handle, the napkin method. That method is more powerful than our fancy AI using deep learning. And it was very sad to learn that, but it’s also very important because we compared it now to several other very famous methods. And I won’t call any specific lab out, but many famous methods have been tried on our historical data of thousands of patients, and none of them beat the napkin method. So we said, okay, wait a second. So the napkin method’s that good. Maybe there’s some mathematical reason why the napkin method is that good. And so we actually recently published a paper that says the napkin method is just as good or better than just doing the random shuffling of forecasts that most labs are using to make sure that they’re better than chance. So we basically said, look, you should not be asking if you’re better than chance, you should be asking, are we better than the napkin method? Why? Because the napkin method is super powerful and it appears to be really, really good according to the standard measures. Another thing that we noticed is that the measures of success don’t take into account people’s individual seizure frequency. And if your seizure frequency is really high, then it’s easy to forecast tomorrow. Your risk is high. If your risk is low, then it’s very easy also. Your risk is low. And any method, no matter how sophisticated, if it says your risk is low, when you have a low seizure frequency, it’s going to be right most of the time. And so this seemingly obvious point gets missed in these measures of AUC and fancy statistical measures. It’s not included. And so we said, look, you have to put your measure of success as a function of what’s your seizure frequency. And so these two little innovations using an appropriate control and also watching out for whose seizure frequency you’re talking about when you make these analyses makes a huge difference. And all of the famous algorithms for seizure forecasting that we’ve seen so far, they’re not using these two innovations. So we published this. We hope everyone uses it. And we hope that forecasting becomes a reality as soon as possible. But it’s a lot harder than we originally thought. And so we’re excited to see better things in the future and we’re disappointed that things that seem promising really aren’t better than that napkin method.

This transcript is AI-generated. While we strive for accuracy, please verify this copy with the video.