Unifying Parsing and Tree-Structured Models for Generating Sentence Semantic Representations - CNRS - Centre national de la recherche scientifique Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Unifying Parsing and Tree-Structured Models for Generating Sentence Semantic Representations

Résumé

We introduce a novel tree-based model that learns its composition function together with its structure. The architecture produces sentence embeddings by composing words according to an induced syntactic tree. The parsing and the composition functions are explicitly connected and, therefore, learned jointly. As a result, the sentence embedding is computed according to an interpretable linguistic pattern and may be used on any downstream task. We evaluate our encoder on downstream tasks, and we observe that it outperforms tree-based models relying on external parsers. In some configurations, it is even competitive with Bert base model. Our model is capable of supporting multiple parser architectures. We exploit this property to conduct an ablation study by comparing different parser initializations. We explore to which extent the trees produced by our model compare with linguistic structures and how this initialization impacts downstream performances. We empirically observe that downstream supervision troubles producing stable parses and preserving linguistically relevant structures.

Fichier principal
Vignette du fichier
2022.naacl-srw.33.pdf (626.52 Ko) Télécharger le fichier
Origine : Fichiers éditeurs autorisés sur une archive ouverte

Dates et versions

hal-03992330 , version 1 (17-02-2023)

Identifiants

Citer

Antoine Simoulin, Benoît Crabbé. Unifying Parsing and Tree-Structured Models for Generating Sentence Semantic Representations. 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics "Human Language Technologies", Jul 2022, Seattle, WA, United States. pp.267-276, ⟨10.18653/v1/2022.naacl-srw.33⟩. ⟨hal-03992330⟩
36 Consultations
17 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More