Statistical inference of finite-rank tensors

Chen, Hongbin; Mourrat, Jean-Christophe; Xia, Jiaming

Annales Henri Lebesgue, Tome 5 (2022), pp. 1161-1189.

Nous considérons un modèle d’inférence statistique général de produits tensoriels de rang fini. Pour toute structure d’interaction et tout ordre de produits tensoriels, nous identifions l’énergie libre limite du modèle en termes d’une formule variationnelle. Notre approche consiste à montrer d’abord que l’énergie libre limite doit être la solution de viscosité d’une certaine équation de Hamilton–Jacobi.

We consider a general statistical inference model of finite-rank tensor products. For any interaction structure and any order of tensor products, we identify the limit free energy of the model in terms of a variational formula. Our approach consists of showing first that the limit free energy must be the viscosity solution to a certain Hamilton–Jacobi equation.

Classification : 82B44, 82D30
Mots clés : inference problem, Hamilton–Jacobi equation, tensor

[ACCM21] Alberici, Diego; Camilli, Francesco; Contucci, Pierluigi; Mingione, Emanuele The solution of the deep Boltzmann machine on the Nishimori line, Commun. Math. Phys., Volume 387 (2021) no. 2, pp. 1191-1214 | DOI | MR | Zbl

[BDBG10] Barra, Adriano; Di Biasio, Aldo; Guerra, Francesco Replica symmetry breaking in mean-field spin glasses through the Hamilton–Jacobi technique, J. Stat. Mech. Theory Exp., Volume 2010 (2010) no. 09, P09006 | MR | Zbl

[BDFT13] Barra, Adriano; Dal Ferraro, Gino; Tantari, Daniele Mean field spin glasses treated with PDE techniques, Eur. Phys. J. B, Volume 86 (2013) no. 7, 332 | DOI | MR

[BDM + 16] Barbier, Jean; Dia, Mohamad; Macris, Nicolas; Krzakala, Florent; Lesieur, Thibault; Zdeborová, Lenka Mutual information for symmetric rank-one matrix estimation: A proof of the replica formula, Advances in Neural Information Processing Systems (NIPS) (Lee, D.; Sugiyama, M.; Luxburg, U.; Guyon, I.; Garnett, R., eds.) (NeurIPS Proceedings), Volume 29, Curran Associates, Inc. (2016), pp. 424-432

[BM19a] Barbier, Jean; Macris, Nicolas The adaptive interpolation method: a simple scheme to prove replica formulas in Bayesian inference, Probab. Theory Relat. Fields, Volume 174 (2019) no. 3-4, pp. 1133-1185 | DOI | MR | Zbl

[BM19b] Barbier, Jean; Macris, Nicolas The adaptive interpolation method for proving replica formulas. Applications to the Curie–Weiss and Wigner spike models, J. Phys. A, Math. Theor., Volume 52 (2019) no. 29, 294002 | DOI | MR

[BMM17] Barbier, Jean; Macris, Nicolas; Miolane, Léo The layered structure of tensor estimation and its mutual information, 2017 ${55}^{\mathrm{th}}$ Annual Allerton Conference on Communication, Control, and Computing (Allerton), IEEE (2017), pp. 1056-1063 | DOI

[Che20] Chen, Hong-Bin Hamilton–Jacobi equations for nonsymmetric matrix inference (2020) (https://arxiv.org/abs/2006.05328)

[CX20] Chen, Hong-Bin; Xia, Jiaming Hamilton–Jacobi equations for inference of matrix tensor products (2020) (https://arxiv.org/abs/2009.01678)

[CX22] Chen, Hong-Bin; Xia, Jiaming Hamilton–Jacobi equations from mean-field spin glasses (2022) (https://arxiv.org/abs/2201.12732)

[GB09] Genovese, Giuseppe; Barra, Adriano A mechanical approach to mean field spin models, J. Math. Phys., Volume 50 (2009) no. 5, 053303 | MR | Zbl

[Gue01] Guerra, Francesco Sum rules for the free energy in the mean field spin glass model, Fields Institute Communications, Volume 30 (2001) no. 11 | MR | Zbl

[KG18] Kadmon, Jonathan; Ganguli, Surya Statistical mechanics of low-rank tensor decomposition, Advances in Neural Information Processing Systems (Bengio, S.; Wallach, H.; Larochelle, H.; Grauman, K.; Cesa-Bianchi, N.; Garnett, R., eds.) (NeurIPS Proceedings), Curran Associates, Inc. (2018), pp. 8201-8212

[LBM21] Luneau, Clément; Barbier, Jean; Macris, Nicolas Mutual information for low-rank even-order symmetric tensor estimation, Inf. Inference, Volume 10 (2021) no. 4, pp. 1167-1207 | DOI | MR | Zbl

[LM19] Lelarge, Marc; Miolane, Léo Fundamental limits of symmetric low-rank matrix estimation, Probab. Theory Relat. Fields, Volume 173 (2019) no. 3-4, pp. 859-929 | DOI | MR | Zbl

[LMB20] Luneau, Clément; Macris, Nicolas; Barbier, Jean High-dimensional rank-one nonsymmetric matrix decomposition: the spherical case (2020) (https://arxiv.org/abs/2004.06975)

[LML + 17] Lesieur, Thibault; Miolane, Léo; Lelarge, Marc; Krzakala, Florent; Zdeborová, Lenka Statistical and computational phase transitions in spiked tensor estimation, 2017 IEEE International Symposium on Information Theory (ISIT), IEEE (2017), pp. 511-515 | DOI

[Mio17] Miolane, Léo Fundamental limits of low-rank matrix estimation: the non-symmetric case (2017) (https://arxiv.org/abs/1702.00473)

[Mou20] Mourrat, Jean-Christophe Hamilton–Jacobi equations for finite-rank matrix inference, Ann. Appl. Probab., Volume 30 (2020) no. 5, pp. 2234-2260 | DOI | MR | Zbl

[Mou21a] Mourrat, Jean-Christophe Free energy upper bound for mean-field vector spin glasses (2021) (https://arxiv.org/abs/2010.09114)

[Mou21b] Mourrat, Jean-Christophe Hamilton–Jacobi equations for mean-field disordered systems, Ann. Henri Lebesgue, Volume 4 (2021), pp. 453-484 | DOI | MR | Zbl

[Mou21c] Mourrat, Jean-Christophe Nonconvex interactions in mean-field spin glasses, Probability and Mathematical Physics, Volume 2 (2021) no. 2, pp. 281-339 | DOI | MR | Zbl

[Mou22] Mourrat, Jean-Christophe The Parisi formula is a Hamilton–Jacobi equation in Wasserstein space, Can. J. Math., Volume 74 (2022) no. 3, pp. 607-629 | DOI | MR | Zbl

[MP20] Mourrat, Jean-Christophe; Panchenko, Dmitry Extending the Parisi formula along a Hamilton–Jacobi equation, Electron. J. Probab., Volume 25 (2020), 23 | DOI | MR | Zbl

[MR19] Mayya, Vaishakhi; Reeves, Galen Mutual information in community detection with covariate information and correlated networks, 2019 ${57}^{\mathrm{th}}$ Annual Allerton Conference on Communication, Control, and Computing (Allerton), IEEE (2019), pp. 602-607 | DOI

[Ree20] Reeves, Galen Information-Theoretic Limits for the Matrix Tensor Product (2020) (https://arxiv.org/abs/2005.11273)

[RMV19] Reeves, Galen; Mayya, Vaishakhi; Volfovsky, Alexander The geometry of community detection via the mmse matrix, 2019 IEEE International Symposium on Information Theory (ISIT), IEEE (2019), pp. 400-404 | DOI

[Roc70] Rockafellar, R Tyrrell Convex Analysis, Princeton Mathematical Series, 36, Princeton University Press, 1970 | DOI | Zbl

[Zha11] Zhang, Fuzhen Matrix theory, Universitext, Springer, 2011 | DOI | MR | Zbl