A critical initialization for biological neural networks

Thousands of neurons

Neural modeling

Neural recordings resemble linear dynamical systems governed by a symmetric connectivity matrix.

Author

Marius Pachitariu, Lin Zhong, Alexa Gracias, Amanda Minisi, Crystall Lopez, Carsen Stringer

Published

Jan 2025

Abstract

Artificial neural networks learn faster if they are initialized well. Good initializations can generate high-dimensional macroscopic dynamics with long timescales. It is not known if biological neural networks have similar properties. Here we show that the eigenvalue spectrum and dynamical properties of large-scale neural recordings in mice (two-photon and electrophysiology) are similar to those produced by linear dynamics governed by a random symmetric matrix that is critically normalized. An exception was hippocampal area CA1: population activity in this area resembled an efficient, uncorrelated neural code, which may be optimized for information storage capacity. Global emergent activity modes persisted in simulations with sparse, clustered or spatial connectivity. We hypothesize that the spontaneous neural activity reflects a critical initialization of whole-brain neural circuits that is optimized for learning time-dependent tasks.

preprint | code | bluesky thread

Thread:

What if… spontaneous neural activity 🧠 reflects the baseline rumblings of a brainwide dynamical system initialized for learning? We find that the rumblings have macroscopic properties like those emerging from linear symmetric, critical systems 🧵 #neuroscience #neuroAI

schematic of neural recordings from mouse V1, whole-brain, and hippocampus; neural activity traces from the population, showing more correlated activity in V1 and whole-brain recordings versus more decorrelated activity in hippocampus

Long timescales and large principal components (PCs) can be produced by a dynamical system with random connectivity and independent stochastic inputs, if the connectivity matrix is critically-normalized.

Simulations of a linear dynamical system with symmetric random connectivity, and non-symmetric random connectivity, produce covariance structure which has an eigenspectrum with a powerlaw decay

Furthermore, the principal components in the model decay as a power-law, a phenomenon we have previously reported in large-scale neural recordings: https://www.science.org/doi/10.1126/science.aav7893; https://www.nature.com/articles/s41586-019-1346-5
In V1 and brainwide ephys recordings we observed that the PC variances decayed as a power-law with exponents of 0.7-0.85, consistent with symmetric, critically-normalized simulations.

neural recordings from V1 and whole-brain, along with the estimated variance of the principal components of the neural activity, which decay with a power-law of exponent 0.7-0.85, which is close to the exponents from symmetric random matrices.

But in hippocampus, we observed exponents around 0.5 that did not change after shuffling, suggesting that hippocampal activity is closer to completely independent neurons.

hippocampal neural activity with a power-law decay of 0.5, which is smaller than the exponent from a symmetric matrix => the variances are more similar across PCs, suggesting more independent neural activity

The model also predicted that higher PCs have longer timescales, which was true in the data.

auto-correlograms of principal components in V1, brainwide and hippocampus recordings; in each the timescales decay as a function of the PC index

An estimate of the dynamics matrix of the data (using DMD) revealed mostly real eigenvalues, further supporting symmetric dynamics.

schematic of dynamic mode decomposition (DMD), a way to estimate the dynamics matrix empirically from data. DMD finds real eigenvalues for the dynamics from a symmetric random matrix and complex eigenvalues from a non-symmetric random matrix, as expected. In the neural data, we see primarily real eigenvalues, similar to the symmetric random matrix.

Global emergent activity modes persisted in simulations with sparse, clustered or spatial connectivity.

Simulations of connectivity with sparse binary connections, clustered connections, and locally-structured connections. These simulations resulted in power-law exponents around 0.7 when the global connectivity was sufficiently high.

Simulations with spatial connectivity replicated several of the properties we observed in the neural recordings, such as a spatial dependence of top correlated neuron pairs, and top PCs which were globally spread across cortex.

Simulation with local connectivity and neural recordings had similar properties such as: little-to-no relationship of average correlations of neurons with distance; top-correlated neurons are nearby spatially; top PCs have weights which are approximately uniformly spread out throughout the simulation / recording window.

We hypothesize that the spontaneous neural activity reflects a critical initialization of whole-brain neural circuits that is optimized for learning tasks that are time-dependent and working-memory dependent. More details in the paper by @marius10p.bsky.social.