Murdoch University Research Repository

Welcome to the Murdoch University Research Repository

The Murdoch University Research Repository is an open access digital collection of research
created by Murdoch University staff, researchers and postgraduate students.

Learn more

WEmbSim: A simple yet effective metric for image captioning

Sharif, N., White, L., Bennamoun, M., Liu, W. and Shah, S.A.A. (2020) WEmbSim: A simple yet effective metric for image captioning. In: 2020 Digital Image Computing: Techniques and Applications (DICTA), 29 Nov. - 2 Dec. 2020, Melbourne, Australia

Link to Published Version: https://doi.org/10.1109/DICTA51227.2020.9363392
*Subscription may be required

Abstract

The area of automatic image caption evaluation is still undergoing intensive research to address the needs of generating captions which can meet adequacy and fluency requirements. Based on our past attempts at developing highly sophisticated learning-based metrics, we have discovered that a simple cosine similarity measure using the Mean of Word Embeddings (MOWE) of captions can actually achieve a surprisingly high performance on unsupervised caption evaluation. This inspires our proposed work on an effective metric WEmbSim, which beats complex measures such as SPICE, CIDEr and WMD at system-level correlation with human judgments. Moreover, it also achieves the best accuracy at matching human consensus scores for caption pairs, against commonly used unsupervised methods. Therefore, we believe that WEmbSim sets a new baseline for any complex metric to be justified.

Item Type: Conference Paper
Murdoch Affiliation(s): IT, Media and Communications
URI: http://researchrepository.murdoch.edu.au/id/eprint/60173
Item Control Page Item Control Page