Murdoch University Research Repository

Welcome to the Murdoch University Research Repository

The Murdoch University Research Repository is an open access digital collection of research
created by Murdoch University staff, researchers and postgraduate students.

Learn more

SubICap: Towards Subword-informed Image Captioning

Sharif, N., Bennamoun, M., Liu, W. and Shah, S.A.A. (2021) SubICap: Towards Subword-informed Image Captioning. In: IEEE Winter Conference on Applications of Computer Vision (WACV) 2021, 3 - 8 January 2021, Waikoloa, HI, USA

Link to Published Version: https://doi.org/10.1109/WACV48630.2021.00358
*Subscription may be required

Abstract

Existing Image Captioning (IC) systems model words as atomic units in captions and are unable to exploit the structural information in the words. This makes representation of rare words very difficult and out-of-vocabulary words impossible. Moreover, to avoid computational complexity, existing IC models operate over a modest sized vocabulary of frequent words, such that the identity of rare words is lost. In this work we address this common limitation of IC systems in dealing with rare words in the corpora. We decompose words into smaller constituent units `subwords' and represent captions as a sequence of subwords instead of words. This helps represent all words in the corpora using a significantly lower subword vocabulary, leading to better parameter learning. Using subword language modeling, our captioning system improves various metric scores, with a training vocabulary size approximately 90% less than the baseline and various state-of-the-art word-level models. Our quantitative and qualitative results and analysis signify the efficacy of our proposed approach.

Item Type: Conference Paper
Murdoch Affiliation(s): IT, Media and Communications
URI: http://researchrepository.murdoch.edu.au/id/eprint/62398
Item Control Page Item Control Page