Murdoch University Research Repository

Welcome to the Murdoch University Research Repository

The Murdoch University Research Repository is an open access digital collection of research
created by Murdoch University staff, researchers and postgraduate students.

Learn more

EMOTENET: Deep neural network for facial emotion recognition using image set classification

Tahir, Sharjeel (2021) EMOTENET: Deep neural network for facial emotion recognition using image set classification. Other thesis, Murdoch University.

PDF - Whole Thesis
Download (5MB) | Preview


With the growing use of robots and intelligent machines in our day-to-day as well as in-dustrial practices, it has become immensely important for machines to understand human feelings and emotions. This work addresses one of the contemporary issues in computer vi-sion i.e, Facial Emotion Recognition (FER). Many researchers have used machine learning methods to perform FER from video, image, text and audio data. Image set classification, which is more versatile than single images, has gained popularity in recent years. Although several works have incorporated image set classification with geometrical representation methods, deep learning methods have not been yet used for facial emotion recognition. We propose a novel methodology that pioneers the implementation of deep neural net-works for FER using image set classification. The proposed network is pretrained with VGGFace weights, and is built on VGG16 network with added layers as per the task’s requirement. Predictions are based on a combination of two different voting strategies including, Majority Voting and Weighted Voting. Besides reaching superior accuracy for one of the most challenging datasets for facial analysis i.e., SFEW (Static Facial Emotion in the Wild), the proposed approach achieves state-of-the-art accuracy on FER2013 (Fa-cial Emotion Recognition 2013) and FERG (Facial Emotion Recognition Group) datasets, without data augmentation, feature extraction and any additional training data. Our ex-perimental results demonstrate that the proposed image set classification provides more accurate results than the existing state-of-the-art FER approaches. In addition, an au-tonomous robot is set up and configured that can perform tasks, such as remote view, tele-operation, and 3D-mapping.

Item Type: Thesis (Other)
Murdoch Affiliation(s): IT, Media and Communications
Notes: This is a Research Masters with Training thesis
Supervisor(s): Shah, Afaq, Sohel, Ferdous and Laga, Hamid
Item Control Page Item Control Page


Downloads per month over past year