2

We consider the landscape of empirical risk minimization for high-dimensional Gaussian single-index models (generalized linear models). The objective is to recover an unknown signal $\boldsymbol{\theta}^\star \in \mathbb{R}^d$ (where $d \gg 1$) from a loss function $\hat{R}(\boldsymbol{\theta})$ that depends on pairs of labels $(\mathbf{x}_i \cdot \boldsymbol{\theta}, \mathbf{x}_i \cdot \boldsymbol{\theta}^\star)_{i=1}^n$, with $\mathbf{x}_i \sim \mathcal{N}(0, I_d)$, in the proportional asymptotic regime $n \asymp d$. Using the Kac-Rice formula, we analyze different complexities of the landscape -- defined as the expected number of critical points -- corresponding to various types of critical points, including local minima. We first show that some variational formulas previously established in the literature for these complexities can be drastically simplified, reducing to explicit variational problems over a finite number of scalar parameters that we can efficiently solve numerically. Our framework also provides detailed predictions for properties of the critical points, including the spectral properties of the Hessian and the joint distribution of labels. We apply our analysis to the real phase retrieval problem for which we derive complete topological phase diagrams of the loss landscape, characterizing notably BBP-type transitions where the Hessian at local minima (as predicted by the Kac-Rice formula) becomes unstable in the direction of the signal. We test the predictive power of our analysis to characterize gradient flow dynamics, finding excellent agreement with finite-size simulations of local optimization algorithms, and capturing fine-grained details such as the empirical distribution of labels. Overall, our results open new avenues for the asymptotic study of loss landscapes and topological trivialization phenomena in high-dimensional statistical models.

Diffusion models have achieved remarkable success across a wide range of generative tasks. A key challenge is understanding the mechanisms that prevent their memorization of training data and allow generalization. In this work, we investigate the role of the training dynamics in the transition from generalization to memorization. Through extensive experiments and theoretical analysis, we identify two distinct timescales; an early time τgen at which models begin to generate high-quality samples, and a later time τmem beyond which memorization emerges. Crucially, we find that τmem increases linearly with the training set size n, while τgen remains constant. This creates a growing window of training times with n where models generalize effectively, despite showing strong memorization if training continues beyond it. It is only when n becomes larger than a model-dependent threshold that overfitting disappears at infinite training times. These findings reveal a form of implicit dynamical regularization in the training dynamics, which allow to avoid memorization even in highly overparameterized settings. Our results are supported by numerical experiments with standard U-Net architectures on realistic and synthetic datasets, and by a theoretical analysis using a tractable random features model studied in the high-dimensional limit.

The reasons of why local iterative algorithms like gradient descent are able to converge towards good solutions in high-dimensional and non-convex landscapes currently remain a complete mystery in several cases. In this work, we focus on a single-layer neural network with a quadratic activation function, a reminiscence of a common problem called phase retrieval. Leveraging methods and intuitions from statistical physics, we show the success of gradient descent is conditioned by a transition in the Hessian of some peculiar states in the loss landscapes stopping the dynamics when the dimension is infinite. When it is finite, we show this picture changes drastically and the system exploits the initial information to slip away from the bad minima and find back the good solution before reaching the bad ones.

Generative models, with at its forefront diffusion models, are becoming central in both science and industry with many relevant applications. We however still lack a theoretical understanding of how they are able to create new samples. In this work, we provide the first quantitative analytical analysis of the behaviour of the generative backward dynamics using methods from statistical physics.

This paper exposes the reachable accuracies of the redshift-space constraints provided by the several environments of the cosmic web with respect to the matter monopole and quadrupole statistics. By splitting the density field into its cosmic web components, one can tighten down the constraints by factors up to 5.5 for the summed neutrino mass.

We use a new approach based on self-supervised deep learning networks originally applied to transparency separation in order to …

This work aims at extracting the cosmological information content of the several cosmic web environments. While we know that the matter power spectrum is not containing all the information about hte underlying cosmological model, we can wonder wether the environments are enclosing different types of information that one can use to break some of the degeneracies among parameters of the model. In particular, we show that a simple two-point correlator becomes sensitive to higher-order features when we have a look at the environments instead of the full matter distribution.

The extraction of patterns from spatially structured point-cloud datasets is ubiquitous in many fields of science. In this work, we address the case of extracting one-dimensional structure from such data by formulating the problems in terms of a regularised version of a mixture model.

The way clusters are embedded in the cosmic web is linked with their intrinsic structure and dynamics. In this study, we carry an analysis of the links between the connectivity, the number of filaments a cluster is connected to, and their shape and dynamics (ellipticity, accretion rate etc.). In particular, we report a correlation between the connectivity and the assembly history of clusters with young and perturbed clusters being more connected than the old and relaxed ones.

The task of clustering point-cloud data is nowadays believed to be either easy to carry or uninformative because the lack of knowledge (number of clusters, sizes, etc.) on the underlying pattern. This work proposes to use a statistical physics formulation of the clustering performed by means of a Gaussian Mixture Model to alleviate some of the drawbacks of the clustering task. In particular, it shows that we can explore the dataset to obtain several key information on the number of clusters, their size and how they are embedded in space, even in high dimensions.