Communication Dans Un Congrès Année : 2024

Evaluating Knowledge Graph-to-text Generation Models for English and Russian on Out Of Domain Data

Résumé

While the WebNLG dataset has prompted much research on generation from knowledge graphs, little work has examined how well models trained on the WebNLG data generalise to unseen data and work has mostly been focused on English. In this paper, we introduce novel benchmarks for both English and Russian which contain various ratios of unseen entities and properties. These benchmarks also differ from WebNLG in that some of the graphs stem from Wikidata rather than DBpedia. Evaluating various models for English and Russian on these benchmarks shows a strong decrease in performance while a qualitative analysis highlights the various types of errors induced by non i.i.d data.
Fichier principal
Vignette du fichier
2024.inlg-main.11.pdf (417.08 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04854968 , version 1 (24-12-2024)

Licence

Identifiants

  • HAL Id : hal-04854968 , version 1

Citer

Anna Nikiforovskaya, Claire Gardent. Evaluating Knowledge Graph-to-text Generation Models for English and Russian on Out Of Domain Data. 17th International Natural Language Generation Conference, Sep 2024, Tokyo, Japan. ⟨hal-04854968⟩
8 Consultations
5 Téléchargements

Partager

More