More Communication Does Not Result in Smaller Generalization Error in Federated Learning - CNRS - Centre national de la recherche scientifique Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

More Communication Does Not Result in Smaller Generalization Error in Federated Learning

Résumé

We study the generalization error of statistical learning models in a Federated Learning (FL) setting. Specifically, there are $K$ devices or clients, each holding an independent own dataset of size $n$. Individual models, learned locally via Stochastic Gradient Descent, are aggregated (averaged) by a central server into a global model and then sent back to the devices. We consider multiple (say $R \in \mathbb N^*$) rounds of model aggregation and study the effect of $R$ on the generalization error of the final aggregated model. We establish an upper bound on the generalization error that accounts explicitly for the effect of $R$ (in addition to the number of participating devices $K$ and dataset size $n$). It is observed that, for fixed $(n, K)$, the bound increases with $R$, suggesting that the generalization of such learning algorithms is negatively affected by more frequent communication with the parameter server. Combined with the fact that the empirical risk, however, generally decreases for larger values of $R$, this indicates that $R$ might be a parameter to optimize to reduce the population risk of FL algorithms. The results of this paper, which extend straightforwardly to the heterogeneous data setting, are also illustrated through numerical examples.

Dates et versions

hal-04457317 , version 1 (14-02-2024)

Identifiants

Citer

Romain Chor, Milad Sefidgaran, Abdellatif Zaidi. More Communication Does Not Result in Smaller Generalization Error in Federated Learning. IEEE Symposium on Information Theory (ISIT), 2023, Taipei, Taiwan. ⟨hal-04457317⟩
16 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More