Pré-Publication, Document De Travail Année : 2025

Refined Analysis of Federated Averaging's Bias and Federated Richardson-Romberg Extrapolation

Résumé

In this paper, we present a novel analysis of FedAvg with constant step size, relying on the Markov property of the underlying process. We demonstrate that the global iterates of the algorithm converge to a stationary distribution and analyze its resulting bias and variance relative to the problem's solution. We provide a first-order bias expansion in both homogeneous and heterogeneous settings. Interestingly, this bias decomposes into two distinct components: one that depends solely on stochastic gradient noise and another on client heterogeneity. Finally, we introduce a new algorithm based on the Richardson-Romberg extrapolation technique to mitigate this bias.
Fichier principal
Vignette du fichier
paper_fedavg_richardson_romberg.pdf (866.52 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04878343 , version 1 (10-01-2025)

Identifiants

  • HAL Id : hal-04878343 , version 1

Citer

Paul Mangold, Alain Durmus, Aymeric Dieuleveut, Sergey Samsonov, Eric Moulines. Refined Analysis of Federated Averaging's Bias and Federated Richardson-Romberg Extrapolation. 2025. ⟨hal-04878343⟩
0 Consultations
0 Téléchargements

Partager

More