A moment-matching metric for latent variable generative models

Beaulac, Cédric (2023). « A moment-matching metric for latent variable generative models ». Machine Learning.

Fichier(s) associé(s) à ce document :
Télécharger (2MB)


It is difficult to assess the quality of a fitted model in unsupervised learning problems. Latent variable models, such as variational autoencoders and Gaussian mixture models, are often trained with likelihood-based approaches. In the scope of Goodhart's law, when a metric becomes a target it ceases to be a good metric and therefore we should not use likelihood to assess the quality of the fit of these models. The solution we propose is a new metric for model comparison or regularization that relies on moments. The key idea is to study the difference between the data moments and the model moments using a matrix norm, such as the Frobenius norm. We first show how to use this new metric for model comparison and then for regularization. We show that our proposed metric is faster to compute and has a smaller variance than the commonly used procedure of drawing samples from the fitted distribution. We conclude this article with a demonstration of both applications and we discuss our findings and future work.

Type: Article de revue scientifique
Mots-clés ou Sujets: Moment estimators, Latent variable models, Goodness-of-fit, Frobenius norm
Unité d'appartenance: Faculté des sciences > Département de mathématiques
Déposé par: Cédric Beaulac
Date de dépôt: 13 juin 2023 08:50
Dernière modification: 13 juin 2023 08:50
Adresse URL : http://archipel.uqam.ca/id/eprint/16686


Voir les statistiques sur cinq ans...