2

Why do diffusion models generalize long before they memorize? We trace these two stages to distinct parts of the Neural Tangent Kernel spectrum, and show how spectral regularization can suppress memorization.

We explain why overfitting in diffusion models leads to memorization despite the familiar double-descent picture from regression. Theory and U-Net experiments show how regularization restores the benefits of larger models.

We explore how diffusion models represent the cosmic web through their self-attention maps. Different layers learn distinct environments across spatial scales, capturing coherent structures and information beyond two-point statistics.

We map the geometry of high-dimensional learning landscapes by counting and characterizing their minima and saddles. For phase retrieval, this reveals transitions in local stability that help explain the behavior of gradient descent.

Diffusion models learn to generate new samples before they memorize their training data. We show that larger datasets delay memorization, creating a wider window in which training itself provides an implicit form of regularization.

We study how the evolving curvature of a phase retrieval landscape guides gradient descent. At finite dimension, information present at initialization can help the dynamics escape bad minima and recover the signal.

Generative models, with at its forefront diffusion models, are becoming central in both science and industry with many relevant applications. We however still lack a theoretical understanding of how they are able to create new samples. In this work, we provide the first quantitative analytical analysis of the behaviour of the generative backward dynamics using methods from statistical physics.

This paper exposes the reachable accuracies of the redshift-space constraints provided by the several environments of the cosmic web with respect to the matter monopole and quadrupole statistics. By splitting the density field into its cosmic web components, one can tighten down the constraints by factors up to 5.5 for the summed neutrino mass.

We use a new approach based on self-supervised deep learning networks originally applied to transparency separation in order to …

This work aims at extracting the cosmological information content of the several cosmic web environments. While we know that the matter power spectrum is not containing all the information about hte underlying cosmological model, we can wonder wether the environments are enclosing different types of information that one can use to break some of the degeneracies among parameters of the model. In particular, we show that a simple two-point correlator becomes sensitive to higher-order features when we have a look at the environments instead of the full matter distribution.