Designing a Hybrid System based on between Deep learning Network with Face Detection algorithm for Emotion Recognition through Facial Features Analysis

Abstract

Objectives: In recent years, the demand for automatic emotion recognition systems has increased for use in various fields, including efforts to modify an individual's mood to improve mental health, as well as assisting in identifying the emotions of children with autism spectrum disorder who struggle to express their emotional states. Deep learning networks are linked with face Detection algorithms. If a deep learning network is used alone, any object detected in the image will be considered as a face and will be processed to determine whether it has emotions or not. This results in high computational complexity, very high response time, and low accuracy if there is more than one face in the same image. Methods:  In this research, images were first introduced into the Fer-Net convolutional neural network after performing some preprocessing operations. The Fer-Net was selected after experimenting with several other CNN. Facial features were then extracted from the network, and these extracted features were classified into four basic emotions. Additionally, several standard databases were tested individually, such as FER-2013 and AffectNet, for training and evaluating the network. Subsequently, the previous databases were merged with other databases like RAF-DB and CK+ to increase the number of training samples and evaluation samples in order to avoid the issue of overfitting. Finally, we linked facial detection with the classification network obtained from the trained model using the MTCNN algorithm to identify the faces present in the image before analyzing the facial features and determining the emotions extracted from them. Results: First, Data Augmentation was implemented on the data in standard database (Fer-2013), and the overfitting problem was resulted.