by Carnegie Mellon Portugal Program

Examples of detection and crop, when the bed and the patient were properly detected, surrounding scenery removed, with enough space left to capture the full scale of seizures (a,b 2D Mask R-CNN crop is more effective, c,d depth crop is more effective, dotted red line—detection box, straight red line—crop box). Credit: Scientific Reports (2022). DOI: 10.1038/s41598-022-23133-9

A team of researchers at INESC TEC and the University of Munich, including Carnegie Mellon Portugal (CMU Portugal) Ph.D. student Tamás Karácsony, tested an innovative solution to classify seizures, the main symptom of epilepsy, using infrared radar and 3-D videos. Scientific Reports recently published the results of this work, coordinated by Karácsony's supervisor and CMU Portugal Scientific Director João Paulo Cunha, a researcher at INESC TEC and Professor at FEUP.

Despite a vast amount of video material available on seizure classification, studies on the subject are still rare, and even rarer are approaches for automated, AI-supported solutions. This new study presents a new approach, which is the first to consider near-real-time classification from two-second samples, showing the feasibility of a system to support the diagnosis and monitoring process (based on action recognition) that uses deep learning. This technique allows for distinguishing between frontal and temporal lobes seizures (the two most common classes of epilepsy) or non-epileptic events.

Epilepsy is a chronic neurological disease that affects 1% of the world population, with seizures as one of the main symptoms—whose semiology is crucial to diagnose potential occurrences. The analysis of seizures is usually done using 2-D video-EEG (electroencephalogram) at epilepsy monitoring units (EMUs) by specialized health care professionals. "During clinical diagnosis, the clinicians utilize these videos to visually recognize movements of interests defined by movement features (semiology)," explained Karácsony.

However, the semiology assessment is limited by a high inter-rater variability among said professionals, and despite being promising, the automatic and semi-automatic approaches using computer vision still depend on considerable "human in the loop" efforts. "A patient is usually monitored for several days, which has to be fully reviewed afterward for the seizures. This requires a lot of time and effort from the clinical staff," added the researcher.

To overcome this, the team of researchers has developed a deep learning-based approach for the automatic and near real-time classification of epileptic seizures. According to Karácsony, "We present a new contribution inspired by the way experts analyze the semiology of seizures, taking into account not only the presence of specific movements of interest in different parts of the patient's bodies, but also their dynamics and their biomechanical aspects, such as speed or acceleration patterns, or range of motion."

The team turned to the world's largest 3-D video-EEG database and extracted videos of 115 seizures, first developing a semi-specialized and automatic pre-processing algorithm to remove unnecessary environments from the videos. In practical terms, two image cropping methods are combined—depth and Mask R-CNN—providing a clean scenario and, consequently, improving the extraction of relevant information from the available videos, minimizing unrelated variations, and improving the seizure classification process.

In a further explanation about the process used, Tamás explained, "Our solution uses an action recognition approach with an intelligent 3-D cropping of the scene to remove unrelated information, such as clinicians moving around the patients. By removing it, our method significantly improved classification performance. This research has also proven the feasibility of our action-recognition approach to distinguish two classes of epilepsy and the non-epileptic class, with only two seconds of samples, making it useful for near-real-time monitoring. In addition, the solution we propose can be used in other datasets of 3-D video for analysis and monitoring of seizures."

Therefore, in translating this knowledge to improved diagnosis and treatment, the approach serves two purposes: "It can be used for monitoring and alarms—which can alarm staff; or, if the approach is transferred to an ambulatory setting, a caregiver, when a seizure is ongoing, resulting in a faster response, which might decrease associated risks and Sudden Unexpected Death in Epilepsy (SUDEP). Without a near real-time approach, this would not be feasible", said Karácsony.

More research is required before this system can be implemented in clinical practice. Nevertheless, in the long-term the system is expected to benefit clinicians, clinics, and patients. "With automated diagnosis support, the clinicians have to spend less time reviewing the videos, thus can treat more patients, hopefully make better decisions, which reduces associated costs (material and health) for clinics and society," he concluded.

More information: Tamás Karácsony et al, Novel 3D video action recognition deep learning approach for near real time epileptic seizure classification, Scientific Reports(2022). DOI: 10.1038/s41598-022-23133-9

Journal information: Scientific Reports 

Provided by Carnegie Mellon Portugal Program