Metric Space Magnitude for Evaluating the Diversity of Latent Representations
DOKPE
- Limbeck, Katharina Technical University of Munich, Germany
- Andreeva, Rayna University of Edinburgh, UK
- Sarkar, Rik University of Edinburgh, UK
- Rieck, Bastian ORCID University of Fribourg
- 2024
Published in:
- 38th Conference on Neural Information Processing Systems (NeurIPS 2024). - 2024, p. 1-43
English
The magnitude of a metric space is a novel invariant that provides a measure of the
‘effective size’ of a space across multiple scales, while also capturing numerous
geometrical properties, such as curvature, density, or entropy. We develop a family
of magnitude-based measures of the intrinsic diversity of latent representations,
formalising a novel notion of dissimilarity between magnitude functions of finite
metric spaces. Our measures are provably stable under perturbations of the data,
can be efficiently calculated, and enable a rigorous multi-scale characterisation and
comparison of latent representations. We show their utility and superior performance across different domains and tasks, including (i) the automated estimation of
diversity, (ii) the detection of mode collapse, and (iii) the evaluation of generative
models for text, image, and graph data.
‘effective size’ of a space across multiple scales, while also capturing numerous
geometrical properties, such as curvature, density, or entropy. We develop a family
of magnitude-based measures of the intrinsic diversity of latent representations,
formalising a novel notion of dissimilarity between magnitude functions of finite
metric spaces. Our measures are provably stable under perturbations of the data,
can be efficiently calculated, and enable a rigorous multi-scale characterisation and
comparison of latent representations. We show their utility and superior performance across different domains and tasks, including (i) the automated estimation of
diversity, (ii) the detection of mode collapse, and (iii) the evaluation of generative
models for text, image, and graph data.
- Faculty
- Faculté des sciences et de médecine
- Department
- Département d'Informatique
- Language
-
- English
- Classification
- Computer science and technology
- Other electronic version
- License
- CC BY
- Open access status
- green
- Identifiers
- Persistent URL
- https://folia.unifr.ch/unifr/documents/331216
Other files
Statistics
Document views: 8
File downloads:
- limbeck24a.pdf: 2
- neurips-2024-metric-space-magnitude-for-evaluating-the-diversity-of-latent-representations-supplemental-conference.zip: 1