Conference Papers Year : 2024

Evaluating Knowledge Graph-to-text Generation Models for English and Russian on Out Of Domain Data

Abstract

While the WebNLG dataset has prompted much research on generation from knowledge graphs, little work has examined how well models trained on the WebNLG data generalise to unseen data and work has mostly been focused on English. In this paper, we introduce novel benchmarks for both English and Russian which contain various ratios of unseen entities and properties. These benchmarks also differ from WebNLG in that some of the graphs stem from Wikidata rather than DBpedia. Evaluating various models for English and Russian on these benchmarks shows a strong decrease in performance while a qualitative analysis highlights the various types of errors induced by non i.i.d data.
Fichier principal
Vignette du fichier
2024.inlg-main.11.pdf (417.08 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04854968 , version 1 (24-12-2024)

Licence

Identifiers

  • HAL Id : hal-04854968 , version 1

Cite

Anna Nikiforovskaya, Claire Gardent. Evaluating Knowledge Graph-to-text Generation Models for English and Russian on Out Of Domain Data. 17th International Natural Language Generation Conference, Sep 2024, Tokyo, Japan. ⟨hal-04854968⟩
0 View
0 Download

Share

More