Understanding the computational demands underlying visual reasoning - CNRS - Centre national de la recherche scientifique Accéder directement au contenu
Article Dans Une Revue Neural Computation Année : 2022

Understanding the computational demands underlying visual reasoning

Résumé

Visual understanding requires comprehending complex visual relations between objects within a scene. Here, we seek to characterize the computational demands for abstract visual reasoning. We do this by systematically assessing the ability of modern deep convolutional neural networks (CNNs) to learn to solve the "Synthetic Visual Reasoning Test" (SVRT) challenge, a collection of twenty-three visual reasoning problems. Our analysis leads to a novel taxonomy of visual reasoning tasks, which can be primarily explained by both the type of relations (same-different vs. spatial-relation judgments) and the number of relations used to compose the underlying rules. Prior cognitive neuroscience work suggests that attention plays a key role in human's visual reasoning ability. To test this, we extended the CNNs with spatial and feature-based attention mechanisms. In a second series of experiments, we evaluated the ability of these attention networks to learn to solve the SVRT challenge and found the resulting architectures to be much more efficient at solving the hardest of these visual reasoning tasks. Most importantly, the corresponding improvements on individual tasks partially explained the taxonomy. Overall, this work advances our understanding of visual reasoning and yields testable Neuroscience predictions regarding the need for feature-based vs. spatial attention in visual reasoning.
Fichier principal
Vignette du fichier
2108.03603.pdf (4.72 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03318448 , version 1 (10-08-2021)

Identifiants

Citer

Mohit Vaishnav, Remi Cadene, Andrea Alamia, Drew Linsley, Rufin Vanrullen, et al.. Understanding the computational demands underlying visual reasoning. Neural Computation, 2022, pp.1-25. ⟨10.1162/neco_a_01485⟩. ⟨hal-03318448⟩
31 Consultations
5 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More