A Transformer Based Emotion Recognition Model for Social Robots Using Topographical Maps Generated from EEG Signals

Abstract

Emotions are an integral part of living beings which influence their thoughts, actions, and interactions with other beings. Understanding human emotions is very important in communicating with others. Developing an emotion recognition model that can be implemented in robots is a critical step in human-robot interaction (HRI). With the rise of artificial intelligence, many techniques are available in machine learning and deep learning to solve this problem, one such technique is Transformers. Transformers, are used in trending technologies like BERT, ChatGPT, DALL-E-2, etc., We used transformers in this study as they have an edge over other by providing flexibility, adaptability, transfer learning, multimodality, parallelization etc., The dataset used is GAMEEMO, which contains EEG signals which are collected from 28 subjects while they were playing four computer-based games which emulate emotions like boring, calm, horror, and funny. Using EEG signal for emotion recognition have advantages like direct measure of brain activity, non-invasiveness, good temporal resolution etc., First, we preprocessed the raw EEG signal using bandpass filtering then created a 5-s epoch out of signal. Next, we converted the 1D EEG signal to a 2D topographical image using independent component analysis by taking 10 principal components out of 14 by persevering at least 95% of the variance in the data. From 9 h and 20 min of GAMEEMO EEG signal, we generated 82, 880 topographical images. Finally, these images were fed to a deep learning-based visual transformers model for the classification of emotions, the best accuracy of the model is 84.71%, our model performed better when compared with the other state-of-the-art models. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.