Towards efficient feature sharing in MIMO architectures - CNRS - Centre national de la recherche scientifique Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Towards efficient feature sharing in MIMO architectures

Résumé

Multi-input multi-output architectures propose to train multiple subnetworks within one base network and then average the subnetwork predictions to benefit from ensembling for free. Despite some relative success, these architectures are wasteful in their use of parameters. Indeed, we highlight in this paper that the learned subnetwork fail to share even generic features which limits their applicability on smaller mobile and AR/VR devices. We posit this behavior stems from an ill-posed part of the multi-input multi-output framework. To solve this issue, we propose a novel unmixing step in MIMO architectures that allows subnetworks to properly share features. Preliminary experiments on CIFAR 100 show our adjustments allow feature sharing and improve model performance for small architectures.
Fichier principal
Vignette du fichier
Sun_Towards_Efficient_Feature_Sharing_in_MIMO_Architectures_CVPRW_2022_paper.pdf (2.93 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03951737 , version 1 (23-01-2023)

Identifiants

Citer

Rémy Sun, Alexandre Ramé, Clément Masson, Nicolas Thome, Matthieu Cord. Towards efficient feature sharing in MIMO architectures. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Jun 2022, New Orleans, United States. pp.2696-2700, ⟨10.1109/CVPRW56347.2022.00303⟩. ⟨hal-03951737⟩
28 Consultations
13 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More