Abstract
This paper introduces the Quotient Bayesian Learning Rule, an extension of natural-gradient Bayesian updates to probability models outside the exponential family. Building on the observation that many heavy-tailed and otherwise non-exponential distributions arise as marginals of minimal exponential families, we prove that such marginals inherit a unique Fisher-Rao information geometry through a quotient-manifold construction. Exploiting this geometry, we derive the Quotient Natural Gradient algorithm, which takes steepest-descent steps in the well-structured covering space and guarantees parameterization-invariant optimization in the target space. Empirical results on the Student-t distribution show faster convergence and higher-quality solutions than previous variants of the Bayesian Learning Rule.
Mykola Lukashchuk
PhD student
I am a PhD candidate at the Electrical Engineering department, Eindhoven University of Technology.
Raphaël Trésor
PhD student
Raphaël Trésor is a PhD candidate in the SPS group at Eindhoven University of Technology’s Electrical Engineering department.
Wouter Nuijten
PhD Student
PhD student studying active inference as variational inference; core contributor to RxInfer.jl.
İsmail Şenöz
Chief Scientist
LazyDynamics
Ismail Senoz is a co-founder & chief scientist of Lazy Dynamics
