What functions can Graph Neural Networks compute on random graphs? The role of Positional Encoding - Laboratoire Jean-Alexandre Dieudonné Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

What functions can Graph Neural Networks compute on random graphs? The role of Positional Encoding

Résumé

We aim to deepen the theoretical understanding of Graph Neural Networks (GNNs) on large graphs, with a focus on their expressive power. Existing analyses relate this notion to the graph isomorphism problem, which is mostly relevant for graphs of small sizes, or studied graph classification or regression tasks, while prediction tasks on nodes are far more relevant on large graphs. Recently, several works showed that, on very general random graphs models, GNNs converge to certains functions as the number of nodes grows. In this paper, we provide a more complete and intuitive description of the function space generated by equivariant GNNs for node-tasks, through general notions of convergence that encompass several previous examples. We emphasize the role of input node features, and study the impact of node Positional Encodings (PEs), a recent line of work that has been shown to yield state-of-the-art results in practice. Through the study of several examples of PEs on large random graphs, we extend previously known universality results to significantly more general models. Our theoretical results hint at some normalization tricks, which is shown numerically to have a positive impact on GNN generalization on synthetic and real data. Our proofs contain new concentration inequalities of independent interest.
Fichier principal
Vignette du fichier
gnn_characterization.pdf (807.29 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04103771 , version 1 (23-05-2023)

Licence

Paternité

Identifiants

Citer

Nicolas Keriven, Samuel Vaiter. What functions can Graph Neural Networks compute on random graphs? The role of Positional Encoding. NeurIPS 2023 - 37th Annual Conference on Neural Information Processing Systems, Dec 2023, New-Orleans, United States. pp.1-28. ⟨hal-04103771⟩
51 Consultations
30 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More