What Does It Take to Reproduce Experiments? Evidences from the Neuro-Symbolic Domain - Laboratoire d'Informatique de Grenoble Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2023

What Does It Take to Reproduce Experiments? Evidences from the Neuro-Symbolic Domain

Résumé

Reproducibility is a desirable property of scientific experiments, which is gaining relevance in computer science. Although guidelines exist, it is not immediately obvious what is necessary for making an experiment reproducible and what is necessary for reproducing it. In this work, we address the task of independently reproducing results published in the neuro-symbolic domain. We describe the methodology used for reproducing (using the available code) and replicating (using an alternative implementation) the experiments conducted with Knowledge Enhanced Neural Networks. We also extend them by applying the KENN to additional datasets. For each of these steps, we identify issues that may arise. This work shows that reproducibility should not be taken for granted. We discuss solutions that have proven effective in overcoming the encountered problems. This can be used as a guide for further reproducibility studies and generally improve reproducibility in machine learning.
Fichier principal
Vignette du fichier
nsrep.pdf (363.01 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04035305 , version 1 (09-06-2022)
hal-04035305 , version 2 (17-03-2023)
hal-04035305 , version 3 (20-03-2023)
hal-04035305 , version 4 (13-12-2023)

Identifiants

  • HAL Id : hal-04035305 , version 2

Citer

Luisa Sophie Werner, Nabil Layaïda, Pierre Genevès, Jérôme Euzenat. What Does It Take to Reproduce Experiments? Evidences from the Neuro-Symbolic Domain. 2023. ⟨hal-04035305v2⟩
370 Consultations
227 Téléchargements

Partager

Gmail Facebook X LinkedIn More