We show that diffusion models can enter a generalization regime where the generated images are independent of the samples in the training set by relying on inductive biases towards geometric regularity in images.
We show how deep network weights can be compressed to a reduced set of summary statistics (the learned “features”) that (i) all networks share no matter their initialization and (ii) that allow regenerating weights with an equivalent performance.
We show that deep networks rely on phase collapses to classify images, rather than sparse coding as assumed in previous works, and introduce a simplified architecture with minimal learning while maintaining high accuracy.
