GenEmo-Net: Generalizable Emotion Recognition Using Brain Functional Connections Based Neural Network

Abstract

The aim of this research is to construct a generalizable and biologically-interpretable emotion recognition model utilizing complex electroencephalogram (EEG) signals for realizing emotional state of human brain. In this paper, the spatial-temporal information of EEG signals is used to extract brain connectivity-based feature, i.e., phase-locking value (PLV), that incorporates phase information between a pair of signals. These functional features are then fed as input to our proposed model (GenEmo-Net), which encompasses of Graph Convolutional Neural Network (GCNN) and Long Short-Term Memory Network (LSTM). It is able to dynamically learn the adjacency matrix that resembles functional connections in the brain, and are combined with the temporal features learnt by LSTM. To validate the generalization ability of our model, the experimental setup combines three emotion databases, namely DEAP, DREAMER, and AMIGOS, which increases variability and reduces biasness among subjects and trials. We evaluated the performance of our proposed model on the combined dataset, which achieved a classification accuracy of 70.98 ± 0.73, 65.47 ± 0.56, and 70.09 ± 0.37 for discrimination of valence, arousal, and dominance, respectively. Notably, our generalized model gives more robust results for emotion recognition tasks when compared to other methods. In addition, the biological interpretation of GenEmo-Net is tested via the final adjacency matrix, learnt at the end of training, for VAD processing units. Above results demonstrate the efficacy of the GenEmo-Net for recognizing human emotions and also highlight substantial variations in the spatial and temporal brain characteristics across distinct emotional states. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.