Stochastic Quasi-Newton Langevin Monte Carlo - CNRS - Centre national de la recherche scientifique Accéder directement au contenu
Communication Dans Un Congrès Année : 2016

Stochastic Quasi-Newton Langevin Monte Carlo

Résumé

In the past few years, Stochastic Gradient Markov Chain Monte Carlo (SG-MCMC) methods have been proposed to scale up MCMC to the large-scale scheme. Whilst these methods have proven useful in many applications, they might suffer from poor mixing rates when the target densities exhibit high correlations or large scale differences. In this study, we propose a novel SG-MCMC method that takes the local correlations and scale into account by using ideas from Quasi-Newton optimization methods that directly approximate the inverse Hessian by using a limited history of samples and their gradients. Our method is both computationally efficient and powerful since it uses dense approximations of the inverse Hessian while keeping the time and memory complexities linear with the dimension of the problem. We provide formal theoretical analysis where we show that the proposed method is asymptotically unbiased and consistent with the posterior expectations. We illustrate the effectiveness of the approach on both synthetic and real datasets. Our experiments on two challenging applications (speech enhancement and distributed link prediction) show that our method achieves fast convergence rates similar to Riemannian approaches while at the same time having low computational requirements similar to diagonal preconditioning-based approaches.
Fichier principal
Vignette du fichier
hamcmc_icml.pdf (1.22 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01306596 , version 1 (21-09-2016)

Identifiants

  • HAL Id : hal-01306596 , version 1

Citer

Umut Şimşekli, Roland Badeau, Taylan Cemgil, Gael Richard. Stochastic Quasi-Newton Langevin Monte Carlo. International Conference on Machine Learning (ICML), Jun 2016, New York, NY, United States. ⟨hal-01306596⟩
194 Consultations
55 Téléchargements

Partager

Gmail Facebook X LinkedIn More