In this paper we focus on the problem of estimating a bounded density using a finite combination of densities from a given class. We consider the Maximum Likelihood Estimator (MLE) and the greedy procedure described by Li and Barron (1999) under the additional assumption of boundedness of densities. We prove an bound on the estimation error which does not depend on the number of densities in the estimated combination. Under the boundedness assumption, this improves the bound of Li and Barron by removing the factor and also generalizes it to the base classes with converging Dudley integral.
Mots clés : mixture density estimation, maximum likelihood, Rademacher processes
@article{PS_2005__9__220_0,
author = {Rakhlin, Alexander and Panchenko, Dmitry and Mukherjee, Sayan},
title = {Risk bounds for mixture density estimation},
journal = {ESAIM: Probability and Statistics},
pages = {220--229},
publisher = {EDP-Sciences},
volume = {9},
year = {2005},
doi = {10.1051/ps:2005011},
mrnumber = {2148968},
zbl = {1141.62024},
language = {en},
url = {http://archive.numdam.org/articles/10.1051/ps:2005011/}
}
TY - JOUR AU - Rakhlin, Alexander AU - Panchenko, Dmitry AU - Mukherjee, Sayan TI - Risk bounds for mixture density estimation JO - ESAIM: Probability and Statistics PY - 2005 SP - 220 EP - 229 VL - 9 PB - EDP-Sciences UR - http://archive.numdam.org/articles/10.1051/ps:2005011/ DO - 10.1051/ps:2005011 LA - en ID - PS_2005__9__220_0 ER -
%0 Journal Article %A Rakhlin, Alexander %A Panchenko, Dmitry %A Mukherjee, Sayan %T Risk bounds for mixture density estimation %J ESAIM: Probability and Statistics %D 2005 %P 220-229 %V 9 %I EDP-Sciences %U http://archive.numdam.org/articles/10.1051/ps:2005011/ %R 10.1051/ps:2005011 %G en %F PS_2005__9__220_0
Rakhlin, Alexander; Panchenko, Dmitry; Mukherjee, Sayan. Risk bounds for mixture density estimation. ESAIM: Probability and Statistics, Tome 9 (2005), pp. 220-229. doi : 10.1051/ps:2005011. http://archive.numdam.org/articles/10.1051/ps:2005011/
[1] , Universal approximation bounds for superpositions of a sigmoidal function. IEEE Trans. Inform. Theory 39 (1993) 930-945. | Zbl
[2] , Approximation and estimation bounds for artificial neural networks. Machine Learning 14 (1994) 115-133. | Zbl
[3] and, Rates of convergence for minimum contrast estimators. Probab. Theory Related Fields 97 (1993) 113-150. | Zbl
[4] , Uniform Central Limit Theorems. Cambridge University Press (1999). | MR | Zbl
[5] , A simple lemma on greedy approximation in Hilbert space and convergence rates for Projection Pursuit Regression and neural network training. Ann. Stat. 20 (1992) 608-613. | Zbl
[6] and, Probability in Banach Spaces. Springer-Verlag, New York (1991). | MR | Zbl
[7] and, Mixture density estimation, in Advances in Neural information processings systems 12, S.A. Solla, T.K. Leen and K.-R. Muller Ed. San Mateo, CA. Morgan Kaufmann Publishers (1999).
[8] , Estimation of Mixture Models. Ph.D. Thesis, The Department of Statistics. Yale University (1999).
[9] , On the method of bounded differences. Surveys in Combinatorics (1989) 148-188. | Zbl
[10] , On the size of convex hulls of small sets. J. Machine Learning Research 2 (2001) 1-18. | Zbl
[11] and, Generalization bounds for function approximation from scattered noisy data. Adv. Comput. Math. 10 (1999) 51-80. | Zbl
[12] , Rates of convergence for the maximum likelihood estimator in mixture models. Nonparametric Statistics 6 (1996) 293-310. | Zbl
[13] , Empirical Processes in M-Estimation. Cambridge University Press (2000).
[14] and, Weak Convergence and Empirical Processes with Applications to Statistics. Springer-Verlag, New York (1996). | MR | Zbl
[15] and, Probability inequalities for likelihood ratios and convergence rates for sieve mles. Ann. Stat. 23 (1995) 339-362. | Zbl
Cité par Sources :
