Nikolai Zaki: Flow Integration for Bayesian Evidence Estimation in Stein Variational Gradient Descent

BCCN Berlin / Technische Universität Berlin

Abstract

At AABI 2021 we introduced KL flow integration for flow based variational inference as a method to estimate the normalization constant of the target in methods such as SVGD. At the time we were able to obtain reasonable results for Gaussian targets but failed on more challenging problems. This thesis investigates how the algorithm can be improved to better work these problems, such as Bayesian logistic regression and multi modal sampling. In doing so, a series of issues are identified and problem agnostic solutions are proposed and themselves investigated. In particular we show how the time derivative of the KL divergence can be computed under various flows, allowing us to find ones better suited to integration. We investigate the effects of posterior approximation, alternative gradient descent steps and of annealing of the target distribution and show that individually and together they significantly improve the performance of flow integration.

 

Additional Information

Master Thesis Defense

 

Organized by

Prof. Manfred Opper   & Prof. Klaus Obermayer   / Lisa Velenosi