Enhanced transfer learning with ImageNet trained classification layer

Shermin, T., Teng, S.W., Murshed, M., Lu, G., Sohel, F. and Paul, M. (2019) Enhanced transfer learning with ImageNet trained classification layer. Lecture Notes in Computer Science, 11854 . pp. 142-155.

Abstract

Parameter fine tuning is a transfer learning approach whereby learned parameters from pre-trained source network are transferred to the target network followed by fine-tuning. Prior research has shown that this approach is capable of improving task performance. However, the impact of the ImageNet pre-trained classification layer in parameter fine-tuning is mostly unexplored in the literature. In this paper, we propose a fine-tuning approach with the pre-trained classification layer. We employ layer-wise fine-tuning to determine which layers should be frozen for optimal performance. Our empirical analysis demonstrates that the proposed fine-tuning performs better than traditional fine-tuning. This finding indicates that the pre-trained classification layer holds less category-specific or more global information than believed earlier. Thus, we hypothesize that the presence of this layer is crucial for growing network depth to adapt better to a new task. Our study manifests that careful normalization and scaling are essential for creating harmony between the pre-trained and new layers for target domain adaptation. We evaluate the proposed depth augmented networks for fine-tuning on several challenging benchmark datasets and show that they can achieve higher classification accuracy than contemporary transfer learning approaches.

Item Type:	Journal Article
Murdoch Affiliation(s):	Information Technology, Mathematics and Statistics
Publisher:	Springer Verlag
Copyright:	© 2019 Springer Nature Switzerland AG
Other Information:	Conference paper from Pacific-Rim Symposium on Image and Video Technology (PSIVT) 2019; Sydney, NSW. 18 - 22 November 2019
URI:	http://researchrepository.murdoch.edu.au/id/eprint/54074

Item Control Page