On fine properties of mixtures with respect to concentration of measure and Sobolev type inequalities

Chafaï, Djalil; Malrieu, Florent

Annales de l'I.H.P. Probabilités et statistiques, Tome 46 (2010) no. 1, pp. 72-96.

Les mélanges dont il est question ici sont des combinaisons convexes de lois de probabilité. Malgré cette définition simple, un mélange peut être beaucoup plus subtil que ses composants. Un mélange de lois gaussiennes par exemple peut donner lieu à des potentiels à profonds puits multiples. Dans ce travail, nous étudions les propriétés fines des mélanges vis à vis de la concentration de la mesure et des inégalités de type Sobolev. Nous proposons des bornes sur la transformée de Laplace faisant intervenir le diamètre de la famille mélangée pour une distance de transport. Notre analyse des inégalités de type Sobolev pour les mélanges à deux composants révèle des relations naturelles avec une forme d'isopérimétrie pour les bandes, ainsi qu'avec le transport optimal sous contrainte de support. Nous établissons que la constante de Poincaré peut rester bornée lorsque la proportion du mélange tend vers 0 tandis que la constante de Sobolev logarithmique peut exploser. Ce phénomène contre intuitif n'est pas réductible à un problème de support et peut être vu comme une trace de la comparaison variance-entropie sur l'espace à deux points. Pour les mélanges, la propriété de concentration de la mesure sous-gaussienne et l'inégalité de Poincaré sont plus stables que l'inégalité de Sobolev logarithmique. Nous illustrons nos résultats avec une collection d'exemples à deux composants concrets. Ce travail conduit à plusieurs questions ouvertes.

Mixtures are convex combinations of laws. Despite this simple definition, a mixture can be far more subtle than its mixed components. For instance, mixing gaussian laws may produce a potential with multiple deep wells. We study in the present work fine properties of mixtures with respect to concentration of measure and Sobolev type functional inequalities. We provide sharp Laplace bounds for Lipschitz functions in the case of generic mixtures, involving a transportation cost diameter of the mixed family. Additionally, our analysis of Sobolev type inequalities for two-component mixtures reveals natural relations with some kind of band isoperimetry and support constrained interpolation via mass transportation. We show that the Poincaré constant of a two-component mixture may remain bounded as the mixture proportion goes to 0 or 1 while the logarithmic Sobolev constant may surprisingly blow up. This counter-intuitive result is not reducible to support disconnections, and appears as a reminiscence of the variance-entropy comparison on the two-point space. As far as mixtures are concerned, the logarithmic Sobolev inequality is less stable than the Poincaré inequality and the sub-gaussian concentration for Lipschitz functions. We illustrate our results on a gallery of concrete two-component mixtures. This work leads to many open questions.

Classification : 60E15, 49Q20, 46E35, 62E99
Mots clés : transportation cost distances, Mallows or Wasserstein distance, mixtures of distributions, finite gaussian mixtures, concentration of measure, gaussian bounds, tails probabilities, deviation inequalities, functional inequalities, Poincaré inequalities, Gross logarithmic Sobolev inequalities, band isoperimetry, transportation of measure, mass transportation

[1] C. Ané, S. Blachère, D. Chafaï, P. Fougères, I. Gentil, F. Malrieu, C. Roberto and G. Scheffer. Sur les inégalités de Sobolev logarithmiques. Société Mathématique de France, Paris, 2000. (Preface by D. Bakry and M. Ledoux.) | MR | Zbl

[2] D. Bakry and M. Émery. Diffusions hypercontractives. In Séminaire de probabilités (XIX, 1983/84) 177-206. Lecture Notes in Math. 1123. Springer, Berlin, 1985. | Numdam | MR | Zbl

[3] D. Bakry, M. Ledoux and F.-Y. Wang. Perturbations of functional inequalities using growth conditions. J. Math. Pures Appl. (9) 87 (2007) 394-407. | MR | Zbl

[4] F. Barthe, P. Cattiaux and C. Roberto. Interpolated inequalities between exponential and Gaussian, Orlicz hypercontractivity and isoperimetry. Rev. Mat. Iberoamericana 22 (2006) 993-1067. | MR | Zbl

[5] F. Barthe and C. Roberto. Sobolev inequalities for probability measures on the real line. Studia Math. 159 (2003) 481-497. (Dedicated to Professor Aleksander Pełczyński on the occasion of his 70th birthday (Polish).) | MR | Zbl

[6] S. G. Bobkov. Concentration of normalized sums and a central limit theorem for noncorrelated random variables. Ann. Probab. 32 (2004) 2884-2907. | MR | Zbl

[7] S. G. Bobkov. Generalized symmetric polynomials and an approximate de Finetti representation. J. Theoret. Probab. 18 (2005) 399-412. | MR | Zbl

[8] S. G. Bobkov and F. Götze. Exponential integrability and transportation cost related to logarithmic Sobolev inequalities. J. Funct. Anal. 163 (1999) 1-28. | MR | Zbl

[9] F. Bolley and C. Villani. Weighted Csiszár-Kullback-Pinsker inequalities and applications to transportation inequalities. Ann. Fac. Sci. Toulouse Math. 14 (2005) 331-352. | Numdam | MR | Zbl

[10] L. A. Caffarelli. Monotonicity properties of optimal transportation and the FKG and related inequalities. Comm. Math. Phys. 214 (2000) 547-563. | MR | Zbl

[11] L. A. Caffarelli. Erratum: [Comm. Math. Phys. 214 (2000) 547-563]. Comm. Math. Phys. 225 (2002) 449-450. | Zbl

[12] E. A. Carlen. Superadditivity of Fisher's information and logarithmic Sobolev inequalities. J. Funct. Anal. 101 (1991) 194-211. | MR | Zbl

[13] S. Chatterjee. Spin glasses and Stein's method. Preprint, 2007. Available at arXiv:0706.3500v2 [math.PR]. | Zbl

[14] P. Diaconis and L. Saloff-Coste. Logarithmic Sobolev inequalities for finite Markov chains. Ann. Appl. Probab. 6 (1996) 695-750. | MR | Zbl

[15] H. Djellout, A. Guillin and L. Wu. Transportation cost-information inequalities and applications to random dynamical systems and diffusions. Ann. Probab. 32 (2004) 2702-2732. | MR | Zbl

[16] B. S. Everitt and D. J. Hand. Finite Mixture Distributions. Chapman & Hall, London, 1981. | MR | Zbl

[17] V. P. Fonf, J. Lindenstrauss and R. R. Phelps. Infinite dimensional convexity. In Handbook of the Geometry of Banach Spaces 599-670. North-Holland, Amsterdam, 2001. | MR | Zbl

[18] S. Frühwirth-Schnatter. Finite Mixture and Markov Switching Models. Springer, New York, 2006. | MR | Zbl

[19] I. Gentil. Inégalités de Sobolev logarithmique et de Poincaré pour la loi uniforme. Unpublished 2004. Available on the author's web page.

[20] I. Gentil and C. Roberto. Spectral gaps for spin systems: Some non-convex phase examples. J. Funct. Anal. 180 (2001) 66-84. | MR | Zbl

[21] N. Gozlan. A characterization of dimension free concentration in terms of transportation inequalities. Preprint, 2008. Available at arXiv: 0804.3089 [math.PR]. | MR | Zbl

[22] M. Gromov and V. D. Milman. A topological application of the isoperimetric inequality. Amer. J. Math. 105 (1983) 843-854. | MR | Zbl

[23] L. Gross. Logarithmic Sobolev inequalities. Amer. J. Math. 97 (1975) 1061-1083. | MR | Zbl

[24] L. Gross. Hypercontractivity, logarithmic Sobolev inequalities, and applications: A survey of surveys. In Diffusion, Quantum Theory, and Radically Elementary Mathematics 45-73. Math. Notes 47. Princeton Univ. Press, Princeton, NJ, 2006. | MR

[25] B. Helffer. Semiclassical Analysis, Witten Laplacians, and Statistical Mechanics. Series in Partial Differential Equations and Applications 1. World Scientific Publishing, River Edge, NJ, 2002. | MR | Zbl

[26] W. Hoeffding. Probability inequalities for sums of bounded random variables. J. Amer. Statist. Assoc. 58 (1963) 13-30. | MR | Zbl

[27] R. Holley and D. Stroock. Logarithmic Sobolev inequalities and stochastic Ising models. J. Statist. Phys. 46 (1987) 1159-1194. | MR | Zbl

[28] M. Jerrum, J.-B. Son, P. Tetali and E. Vigoda. Elementary bounds on Poincaré and log-Sobolev constants for decomposable Markov chains. Ann. Appl. Probab. 14 (2004) 1741-1765. | MR | Zbl

[29] O. Johnson. Convergence of the Poincaré constant. Teor. Veroyatnost. i Primenen. 48 (2003) 615-620. | MR | Zbl

[30] I. Kontoyiannis and M. Madiman. Measure concentration for compound Poisson distributions. Electron. Comm. Probab. 11 (2006) 45-57. | MR | Zbl

[31] L. Kontoyiannis and M. Madiman. Entropy, compound Poisson approximation, log-Sobolev inequalities and measure concentration. In Information Theory Workshop, 24-29 Oct. 2004. IEEE 71-75, 2004.

[32] R. Latała. On some inequalities for Gaussian measures. In Proceedings of the International Congress of Mathematicians, Vol. II Beijing, 2002 813-822. Higher Ed. Press, Beijing, 2002. | MR | Zbl

[33] M. Ledoux. Concentration of measure and logarithmic Sobolev inequalities. In Séminaire de Probabilités, XXXIII 120-216. Springer, Berlin, 1999. | Numdam | MR | Zbl

[34] M. Ledoux. The Concentration of Measure Phenomenon. Mathematical Surveys and Monographs 89. Amer. Math. Soc., Providence, RI, 2001. | MR | Zbl

[35] N. Madras and D. Randall. Markov chain decomposition for convergence rate analysis. Ann. Appl. Probab. 12 (2002) 581-606. | MR | Zbl

[36] K. Marton. A simple proof of the blowing-up lemma. IEEE Trans. Inform. Theory 32 (1986) 445-446. | MR | Zbl

[37] K. Marton. Bounding ̅d-distance by informational divergence: A method to prove measure concentration. Ann. Probab. 24 (1996) 857-866. | MR | Zbl

[38] V. G. Maz'Ja. Sobolev Spaces. Springer, Berlin, 1985. (Translated from the Russian by T. O. Shaposhnikova.) | MR | Zbl

[39] G. Mclachlan and K. Basford. Mixture Models. Statistics: Textbooks and Monographs 84. Marcel Dekker Inc., New York, 1988. | MR | Zbl

[40] G. Mclachlan and D. Peel. Finite Mixture Models. Wiley, New York, 2000. | MR | Zbl

[41] L Miclo. Quand est-ce que des bornes de Hardy permettent de calculer une constante de Poincaré exacte sur la droite?. Preprint, 2005. Available at http://hal.archives-ouvertes.fr/hal-00017875/en/. | Numdam | MR

[42] F. Otto and M. G. Reznikoff. A new criterion for the logarithmic Sobolev inequality and two applications. J. Funct. Anal. 243 (2007) 121-157. | MR | Zbl

[43] R. R. Phelps, Lectures on Choquet's Theorem, 2nd edition. Lecture Notes in Math. 1757. Springer, Berlin, 2001. | MR | Zbl

[44] S. T. Rachev. Probability Metrics and the Stability of Stochastic Models. Wiley, Chichester, 1991. | MR | Zbl

[45] L. Saloff-Coste. Lectures on finite Markov chains. In Lectures on Probability Theory and Statistics (Saint-Flour, 1996) 301-413. Lecture Notes in Math. 1665. Springer, Berlin, 1997. | MR | Zbl

[46] A. J. Stam. Some inequalities satisfied by the quantities of information of Fisher and Shannon. Information and Control 2 (1959) 101-112. | MR | Zbl

[47] V. N. Sudakov. Geometric problems in the theory of infinite-dimensional probability distributions. Proc. Steklov Inst. Math. (1979) 1-178. (Cover to cover translation of Trudy Mat. Inst. Steklov 141 (1976).) | MR | Zbl

[48] A. Takatsu. On Wasserstein geometry of the space of Gaussian measures, 2008. Available at arXiv:0801.2250 [math.DG].

[49] M. Talagrand. Transportation cost for Gaussian and other product measures. Geom. Funct. Anal. 6 (1996) 587-600. | MR | Zbl

[50] D. M. Titterington, A. F. M. Smith and U. E. Makov. Statistical Analysis of Finite Mixture Distributions. Wiley, Chichester, 1985. | MR | Zbl

[51] C. Villani. Topics in Optimal Transportation. Graduate Studies in Mathematics 58. Amer. Math. Soc., Providence, RI, 2003. | MR | Zbl

[52] C. Villani. Optimal Transport, Old and New. Springer, Berlin, 2009. | MR | Zbl