Leveraging auxiliary tasks with affinity learning for weakly supervised semantic segmentation

Xu, L., Ouyang, W., Bennamoun, M., Boussaid, F., Sohel, F. and Xu, D. (2021) Leveraging auxiliary tasks with affinity learning for weakly supervised semantic segmentation. In: IEEE/CVF International Conference on Computer Vision (ICCV) 2021, 10 - 17 2021, Montreal, QC, Canada pp. 6964-6973.

Abstract

Semantic segmentation is a challenging task in the absence of densely labelled data. Only relying on class activation maps (CAM) with image-level labels provides deficient segmentation supervision. Prior works thus consider pre-trained models to produce coarse saliency maps to guide the generation of pseudo segmentation labels. However, the commonly used off-line heuristic generation process cannot fully exploit the benefits of these coarse saliency maps. Motivated by the significant inter-task correlation, we propose a novel weakly supervised multi-task framework termed as AuxSegNet, to leverage saliency detection and multi-label image classification as auxiliary tasks to improve the primary task of semantic segmentation using only image-level ground-truth labels. Inspired by their similar structured semantics, we also propose to learn a cross-task global pixellevel affinity map from the saliency and segmentation representations. The learned cross-task affinity can be used to refine saliency predictions and propagate CAM maps to provide improved pseudo labels for both tasks. The mutual boost between pseudo label updating and cross-task affinity learning enables iterative improvements on segmentation performance. Extensive experiments demonstrate the effectiveness of the proposed auxiliary learning network structure and the cross-task affinity learning method. The proposed approach achieves state-of-the-art weakly supervised segmentation performance on the challenging PASCAL VOC 2012 and MS COCO benchmarks.

Item Type:	Conference Paper
Murdoch Affiliation(s):	IT, Media and Communications
URI:	http://researchrepository.murdoch.edu.au/id/eprint/65469

Item Control Page