Murdoch University Research Repository

Welcome to the Murdoch University Research Repository

The Murdoch University Research Repository is an open access digital collection of research
created by Murdoch University staff, researchers and postgraduate students.

Learn more

A joint Deep Boltzmann Machine (jDBM) model for person identification using mobile phone data

Alam, M., Bennamoun, M., Togneri, R. and Sohel, F. (2016) A joint Deep Boltzmann Machine (jDBM) model for person identification using mobile phone data. IEEE Transactions on Multimedia, 19 (2). pp. 317-326.

Link to Published Version:
*Subscription may be required


We propose an audio-visual person identification approach based on a joint deep Boltzmann machine (jDBM) model. The proposed jDBM model is trained in three steps: a) learning the unimodal DBM models corresponding to the speech and facial image modalities, b) learning the shared layer parameters using a joint Restricted Boltzmann Machine (jRBM) model and c) the fine-tuning of the jDBM model after the initialization with the parameters of the unimodal DBMs and the shared layer. The activation probabilities of the units of the shared layer are used as the joint features and a logistic regression classifier is used for the combined speech and facial image recognition. We show that by learning the shared layer parameters using a jRBM, a higher accuracy can be achieved compared to the greedy layer-wise initialization. The performance of our proposed model is also compared with state-of-the art support vector machine (SVM), deep belief network (DBN), and the deep auto-encoder (DAE) models. In addition, our experimental results show that the joint representations obtained from the proposed jDBM model are robust to noise and missing information. Experiments were carried out on the challenging MOBIO database, which includes audio-visual data captured using mobile phones.

Item Type: Journal Article
Murdoch Affiliation(s): School of Engineering and Information Technology
Publisher: IEEE
Copyright: © IEEE 2017
Item Control Page Item Control Page