by Beijing Institute of Technology Press Co.

Cross-subject emotion recognition framework based on fNIRS, including data collection, data partitioning, model training and evaluation. Credit: Cyborg and Bionic Systems

Emotion recognition based on neural signals is leading the revolution of affective science. By analyzing brain activity, we can glimpse the secrets of the human inner world. This technology not only helps to understand the nature of emotions, but also offers new possibilities for the field of mental health. Emotion recognition using neural signals will be crucial for addressing anxiety, depression, or enhancing emotional intelligence. It enables us to comprehend how emotions affect our lives and choices.

The previous approaches to emotion recognition mostly depended on sources like questionnaires, facial expressions, voice, and so on. These approaches suffer from issues such as high subjectivity and susceptibility to disturbance. With the advancement and refinement of brain imaging technology, researchers can employ these methods to measure neural activity signals without harming the brain during emotion elicitation processes, enabling emotion types to be distinguished by neural activity traits.

Among them, functional near-infrared spectroscopy (fNIRS) is a novel non-invasive neural imaging technique, which is not only resistant to motion artifacts, but also has high spatial resolution. A Chinese research group conducted a cross-subject emotion recognition study based on fNIRS signals, and achieved high decoding performance, promoting the development of affective brain-computer interface.

They published their recommendations in Cyborg and Bionic Systems.

"Cross-subject emotion recognition based on fNIRS has important prospects and significance. fNIRS technology is non-invasive, easy to operate, and can measure brain activity. It helps us to understand the emotional experiences of different individuals, and provides new possibilities for the field of mental health. Through cross-subject studies, we can establish emotion models, and identify the emotional characteristics of different groups of people."

"This will help with personalized treatment, emotion regulation, and psychological assistance. Cross-subject emotion recognition based on fNIRS provides us with a more in-depth and comprehensive way to understand emotions, and promotes the development of affective science," said corresponding author Xiaopeng Si, associate professor in the Academy of Medical Engineering and Translational Medicine, Tianjin University.

"By designing an emotion induction experiment with Chinese videos as natural stimuli, and constructing an fNIRS emotion recognition database, we laid the foundation for exploring a new type of BCI for cross-subject emotion recognition based on fNIRS," Si and co-author He Huang said.

In addition, Si and Huang et al. introduced deep learning technology in this work, which significantly improved the emotion decoding performance. "Based on the signal data characteristics of fNIRS, a dual-branch joint network (DBJNet) was specially constructed. Experimental results show that the model can be generalized to new subjects and exhibits excellent cross-subject emotion recognition performance," Si and co-author He Huang said.

More information: Xiaopeng Si et al, Cross-Subject Emotion Recognition Brain–Computer Interface Based on fNIRS and DBJNet, Cyborg and Bionic Systems (2023). DOI: 10.34133/cbsystems.0045

Journal information: Cyborg and Bionic Systems 

Provided by Beijing Institute of Technology Press Co.