Estimating the conditional density by histogram type estimators and model selection

Sart, Mathieu

ESAIM: Probability and Statistics, Tome 21 (2017), pp. 34-55.

We propose a new estimation procedure of the conditional density for independent and identically distributed data. Our procedure aims at using the data to select a function among arbitrary (at most countable) collections of candidates. By using a deterministic Hellinger distance as loss, we prove that the selected function satisfies a non-asymptotic oracle type inequality under minimal assumptions on the statistical setting. We derive an adaptive piecewise constant estimator on a random partition that achieves the expected rate of convergence over (possibly inhomogeneous and anisotropic) Besov spaces of small regularity. Moreover, we show that this oracle inequality may lead to a general model selection theorem under very mild assumptions on the statistical setting. This theorem guarantees the existence of estimators possessing nice statistical properties under various assumptions on the conditional density (such as smoothness or structural ones).

N. Akakpo, Adaptation to anisotropy and inhomogeneity via dyadic piecewise polynomial selection. Math. Methods Stat. 21 (2012) 1–28. | DOI | MR | Zbl

N. Akakpo and C. Lacour, Inhomogeneous and anisotropic conditional density estimation from dependent data. Electr. J. Stat. 5 (2011) 1618–1653. | MR | Zbl

Y. Baraud, Estimator selection with respect to Hellinger-type risks. Probab. Theory Relat. Fields 151 (2011) 353–401. | DOI | MR | Zbl

Y. Baraud, Estimation of the density of a determinantal process. Confluentes Math. 5 (2013) 3–21. | DOI | Numdam | MR | Zbl

Y. Baraud and L. Birgé, Estimating composite functions by model selection. Ann. Inst. Henri Poincaré, Probab. Stat. 50 (2014) 285–314. | DOI | Numdam | MR | Zbl

Y. Baraud and L. Birgé, Estimating the intensity of a random measure by histogram type estimators. Probab. Theory Relat. Fields 143 (2009) 239–284. | DOI | MR | Zbl

Y. Baraud, L. Birgé and M. Sart, A new method for estimation and model selection: $\rho$-estimation. Invent. Math. (2016) 1–93. | MR

K. Bertin, C. Lacour and V. Rivoirard, Adaptive estimation of conditional density function. Ann. Inst. Henri Poincaré, Probab. Stat. 52 (2016) 939–980. | DOI | MR | Zbl

L. Birgé, Model selection via testing: an alternative to (penalized) maximum likelihood estimators. Ann. Inst. Henri Poincaré, Probab. Stat. 42 (2006) 273–325. | DOI | Numdam | MR | Zbl

L. Birgé, Model selection for poisson processes. In Asymptotics: particles, processes and inverse problems. Vol. 55 of IMS Lect. Notes Monogr. Ser. IMS, Beachwood, OH (2007) 32–64. | MR | Zbl

L. Birgé, Robust tests for model selection. In From Probability to Statistics and Back: High-Dimensional Models and Processes. A Festschrift in Honor of Jon Wellner. Vol. 9. IMS Collections (2012) 47–64. | MR

A.-K. Bott and M. Kohler, Adaptive estimation of a conditional density. Inter. Stat. Rev. (2015). | MR

E. Brunel, F. Comte and C. Lacour, Adaptive estimation of the conditional density in the presence of censoring. Sankhyā: Indian J. Stat. (2007) 734–763. | MR | Zbl

G. Chagny, Warped bases for conditional density estimation. Math. Methods Stat. 22 (2013) 253–282. | DOI | MR | Zbl

S. Cohen and E. Le Pennec, Conditional Density Estimation by Penalized Likelihood Model Selection and Applications. Preprint (2011). | arXiv

J.G. De Gooijer and D. Zerom, On conditional density estimation. Stat. Neerlandica 57 (2003) 159–176. | DOI | MR | Zbl

L. Devroye and G. Lugosi, A universally acceptable smoothing factor for kernel density estimates. Ann. Stat. (1996) 2499–2512. | MR | Zbl

S. Efromovich, Conditional density estimation in a regression setting. Ann. Stat. 35 (2007) 2504–2535. | DOI | MR | Zbl

J. Fan, L. Peng, Q. Yao and W. Zhang, Approximating conditional density functions using dimension reduction. Acta Math. Appl. Sin. Engl. Ser. 25 (2009) 445–456. | DOI | MR | Zbl

J. Fan, Q. Yao and H. Tong, Estimation of conditional densities and sensitivity measures in nonlinear dynamical systems. Biometrika 83 (1996) 189–206. | DOI | MR | Zbl

J. Fan and T. Yim, A data-driven method for estimating conditional densities. Biometrika 91 (2004) 819–834. | DOI | MR | Zbl

O.P. Faugeras, A quantile-copula approach to conditional density estimation. J. Multivariate Anal. 100 (2009) 2083–2099. | DOI | MR | Zbl

L. Györfi and M. Kohler, Nonparametric estimation of conditional distributions. IEEE Trans. Inform. Theory 53 (2007) 1872–1879. | DOI | MR | Zbl

P. Hall and Q. Yao, Approximating conditional distribution functions using dimension reduction. Ann. Stat. 33 (2005) 1404–1421. | DOI | MR | Zbl

R.J. Hyndman, D.M. Bashtannyk and G.K. Grunwald, Estimating and visualizing conditional densities. J. Comput. Graphical Stat. 5 (1996) 315–336. | MR

R.J. Hyndman and Q. Yao, Nonparametric estimation and symmetry tests for conditional density functions. J. Nonparametric Stat. 14 (2002) 259–278. | DOI | MR | Zbl

P. Massart, Concentration inequalities and model selection. École d’été de Probabilités de Saint-Flour. Vol. 1896 of Lect. Notes Math. Springer Berlin/Heidelberg (2003). | MR | Zbl

M. Rosenblatt, Conditional probability density and regression estimators. In Multivariate Analysis, II. Proc. of Second Internat. Sympos., Dayton, Ohio, 1968. Academic Press, New York (1969) 25–31. | MR

M. Sart, Estimation of the transition density of a Markov chain. Ann. Instit. Henri Poincaré, Probab. Stat. 50 (2014) 1028–1068. | Numdam | MR | Zbl

M. Sart, Model selection for poisson processes with covariates. ESAIM: PS 19 (2015) 204–235. | DOI | Numdam | MR | Zbl