Florentin Guth

Research Fellow

Flatiron Institute

Biography

I’m an incoming Assistant Professor at MIT EECS and a Research Fellow in the Center for Computational Neuroscience at the Flatiron Institute. I am part of the Learning Mechanics collaboration as well as the Simons Collaboration on the Physics of Learning and Neural Computation. I am an organizer of the Sci4DL workshop series. During 2023-2026 I was also a Faculty Fellow in the Center for Data Science at NYU. I completed my PhD at École Normale Supérieure in Paris (dissertation and defense slides).

Research Areas

Science of Deep Learning
Probabilistic Models of Data
Information Theory and Geometry

Graphical Models
Signal Processing
High-Dimensional Statistics

Research Interests

My main research interest is to uncover the inductive biases of deep learning: what are the assumptions about the structure of natural data implicitly encoded in the architecture and optimizer that enables it to generalize? Once made explicit, these inductive biases will turn into fundamental principles of probabilistic inference and learning with natural data. This problem can be tackled both theoretically, by proposing mathematical models of data or studying toy models of feature learning, and empirically, by measuring data statistics or key properties of the learned weights and representations. I expand on applying the scientific method to understanding deep learning in this essay. See also the other posts on the Learning Mechanics website as well as our perspective paper “There Will Be a Scientific Theory of Deep Learning”. Specific research themes include probabilistic models of images, generalization in diffusion models, and empirical studies of learned weights, see more details below.

Featured Publications

(see Google Scholar for a complete list)

Learning normalized image densities via dual score matching

We show how to estimate normalized probabilities by modifying the objective and architecture of diffusion models, and report several surprising observations on the properties of the learned probabilistic model.

Generalization in diffusion models arises from geometry-adaptive harmonic representations

We show that diffusion models can enter a generalization regime where the generated images are independent of the samples in the training set by relying on inductive biases towards geometric regularity in images.

A rainbow in deep network black boxes

We show how deep network weights can be compressed to a reduced set of summary statistics (the learned “features”) that (i) all networks share no matter their initialization and (ii) that allow regenerating weights with an equivalent performance.

Phase collapse in neural networks

We show that deep networks rely on phase collapses to classify images, rather than sparse coding as assumed in previous works, and introduce a simplified architecture with minimal learning while maintaining high accuracy.