Investigations particulaires pour l'inférence statistique et l'optimisation de plan d'expériences

Parent, Éric; Amzal, Billy; Girard, Philippe

Journal de la Société française de statistique & Revue de statistique appliquée, Tome 149 (2008) no. 1, pp. 27-51.

Les algorithmes particulaires sont des techniques de Monte-Carlo qui associent des étapes d'échantillonnage pondéré, de rééchantillonnage bootstrap, de régénérescence markovienne et de recuit simulé. Grâce à trois exemples de complexité croissante, nous décrivons leurs implémentations pour l'estimation du maximum de vraisemblance, l'évaluation de la distribution a posteriori pour un modèle à variables latentes et la recherche du plan d'expérience optimal. Les solutions de ces exemples pédagogiques illustrent les performances et les limites de ces algorithmes, promis à une place de choix dans la trousse à outils du statisticien.

Particle algorithms are Monte Carlo techniques that put together steps of importance sampling, bootstrap resampling, markovian rejuvenating and simulated annealing. We develop three examples of increasing complexity and explain how to implement such algorithms for maximum likelihood search, for inference of a model with latent variables and for optimal design. Since we believe that particle algorithms will soon become tools of choice for statistical practitioners, their results are compared with the known solutions of these rather common examples so as to test the algorithms' performances and to show their limits.

[1] Amzal B., Bois F., Parent E., Robert C.P. 2006. Bayesian Optimal Design Via Interacting Particle Systems. Journal of the American Statistical Association, 101(474), 773-785. | MR | Zbl

[2] Andrieu C., Doucet A. 2000. Simulated Annealing for Maximum a posteriori Parameter Estimation of Hidden Markov Models. IEEE Trans. Information Theory, 46(3), 994-1004. | MR | Zbl

[3] Andrieu C., Doucet A., Singh S., Tadic V. 2004. Particle Methods for Change Detection, System Identification, and Control. Proceedings of the IEEE, 92(3), 423-438.

[4] Arulampalam M. S., Maskell S., Gordon N., Clapp T. 2002. A Tutorial on Particle Filters for Online Nonlinear/Non-Gaussian Bayesian Tracking. IEEE Transaction on Signal Processing, 50(2), 174-188.

[5] Barnett V. 1973. Bayesian and Decision Theoric Methods Applied to Industrial Problems. The statistician, 22(3), 199-226.

[6] Berger J. O. 1985. Statistical Decision Theory and Bayesian Analysis. Springer Verlag, New York. | MR | Zbl

[7] Bernier J., Parent E., Boreux J-J. 2000. Statistique de l'Environnement. Traitement Bayésien de l'Incertitude. Paris : Lavoisier.

[8] Brooks S P. 1998. Markov Chain Monte Carlo Method and its Application. The Statistician, 47, 69-100.

[9] Brooks S.P. 2003. Bayesian Computation : A Statistical Revolution. Trans. Roy. Statist. Soc., series A, 15, 2681-2697. | MR | Zbl

[10] Brooks S.P., Morgan B.J.T. 1995. Optimization Using Simulated Annealing. The Statistician, 44, 241-257.

[11] Cappé O., Guilin A., Marin J.M., Robert C.P. 2004. Population Monte Carlo. Journal of Computational and Graphical Statistics, 13(4), 907-929. | MR

[12] Cappé O., Moulines E., Rydèn T. 2005. Inference in Hidden Markov Models. Springer. | MR | Zbl

[13] Carlin B.P., Kadane J.B., Gelfand A.E. 1998. Approaches for Optimal Sequential Decision Analysis in Clinical Trials. Biometrics, 54(3), 964-975. | Zbl

[14] Chopin N. 2002. A Sequential Particle Filter Method for Static Models. Biometrika, 89, 539-552. | MR | Zbl

[15] Chopin N. 2004. Central Limit Theorem for Sequential Monte Carlo Methods and its Application to Bayesian Inference. Ann. Stat., 32(6), 2385-2411. | MR | Zbl

[16] Del Moral P. 2004. Feynman-Kac Formulae : Genealogical and Interacting Particle Systems with Applications. Springer-Verlag. | MR | Zbl

[17] Del Moral P., Guionnet A. 1999. Central Limit Theorem for Nonlinear Filtering and Interacting Particule Systems. Ann. Appl. Prob., 155-194. | MR | Zbl

[18] Del Moral P., Kallel L., Rowe J. 2001. Natural Computing Series : Theoretical Aspects of Evolutionary Computing. Springer-Verlag, Berlin. Chap. Modelling Genetic Algorithms with Interacting Particle Systems, pages 10-67.

[19] Douc R., Cappé O. 2005. Comparison of Resampling Schemes for Particle Filtering. Pages 64-69 of : Proceedings of the 4th International Symposium on Image and Signal Processing and Analysis.

[20] Douc R., Cappé O., Moulines E., Robert C.P. 2002. On the Convergence of the Monte Carlo Maximum Likelihood Method for Latent Variable Models. Scandinavian Journal of Statistics, 29(4), 615-635. | MR | Zbl

[21] Douc R., Guilin A., Marin J.M., Robert C.P. 2007. Convergence of Adaptive Mixtures of Importance Sampling Schemes. Ann. Stat., 35(1), 420-448. | MR | Zbl

[22] Doucet A., De Freitas N., Gordon N. 2001. Sequential Monte Carlo Methods in Practice. Springer-Verlag. | MR | Zbl

[23] Doucet A., Godsill J.M., Robert C.P. 2002. Population Marginal maximum a posteriori Estimation Using Markov Chain Monte Carlo. Statistics and Computing, 12, 77-84. | MR | Zbl

[24] Doucet A., Del Moral P., Peters G.W. 2004. Sequential Monte Carlo Samplers. Tech. rept. Cambridge University.

[25] Gelman A., Rubin D. B. 1992. Inference from Iterative Simulation Using Multiple Sequences. Statistical Science, 457-472.

[26] Gelman A., Carlin J. B., Stern H. S., Rubin D. B. 1995. Bayesian Data Analysis. London : Chapman and Hall. | MR | Zbl

[27] Geman S., Geman D. 1984. Stochastic Relaxation, Gibbs Distribution and the Bayesian Restoration of Images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6(6), 721-741. | Zbl

[28] Gilks W.R., Richardson S., Spiegelhalter D. 1996. Markov Chain Monte Carlo in Practice. London : Chapman and Hall. | MR | Zbl

[29] Kass R. E., Carlin B. P., Gelman A., Neal R. M. 1998. Markov Chain Monte Carlo in Practice : A Roundtable Discussion. The American Statistician, 52(2), 93-100. | MR

[30] Kuhn E., Lavielle M. 2004. Coupling a Stochastic Approximation Version of EM with an MCMC Procedure. ESAIM, Probab. Stat., 8, 115-131. | EuDML | Numdam | MR | Zbl

[31] Künsch H. 2001. State Space and Hidden Markov Models. Pages 109-173 of : Barndor-Nielsen, O. E., Cox, D. R., Klüppelberg, C. (eds), Complex Stochastic Systems. Chapman and Hall. | MR | Zbl

[32] Künsch H. 2005. Recursive Monte Carlo Filters. Ann. Stat., 1983-2021. | Zbl

[33] Liu J.S., Chen R. 1998. Sequential Monte Carlo Methods for Dynamic Systems. Journal of American Statistical Association, 93(443), 113-119. | MR | Zbl

[34] Mc Cullagh P., Nelder J.A. 1989. Generalized Linear Models. second edn. Chapman and Hall/CRC.

[35] Mengersen K.L., Robert C.P., Guihennec-Jouyaux C. 1999. MCMC Convergence Diagnostics : A Reviewww (with Discussion). Pages 415-440 of : Bernardo, J.M., Berger, J.O., Dawid, A.P., Smith, A.F.M. (eds), Bayesian Stat., vol. 6. Oxford University Press. | MR | Zbl

[36] Metropolis N., Rosenbluth A.W., Rosenbluth M.N., Teller A.H, Teller E. 1953. Equations of State Calculations by Fast Computing Machines. J. Chem. Phys., 21, 1087-1091.

[37] Müller P. 1999. Simulation-Based Optimal Design. Pages 459-474 of : Bernardo, J.M., Berger, J.O., Dawid, A.P., Smith, A.F.M. (eds), Bayesian Stat., vol. 6. Oxford University Press. | MR | Zbl

[38] Munier B., Parent E. 1998. Le Développement Récent des Sciences de la Décision : Un Regard Critique sur la Statistique Décisionnelle Bayésienne. In : Parent, E., Hubert, P., Bobée, B., Miquel, J. (eds), Bayesian Methods in Hydrol. Sci. UNESCO Publishing.

[39] Oujdane N., Ruberthaler S. 2005. Stability and Uniform Particle Approximation of Nonlinear Filters in case of non Ergodic Signals. Stochastic Analysis and Applications, 23, 421-448. | MR | Zbl

[40] Parent E., Bernier J. 2007. Le Raisonnement Bayésien : Modélisation et Inférence. Paris : Springer Verlag France. | Zbl

[41] Parent, E., Chaouche A., Girard P. 1995. Sur l'Apport des Statistiques Bayésiennes au Contrôle de la Qualité par Attribut, partie 1 : Contrôle Simple. Rev. Statistique Appliquée, XLIII(4), 5-18. | EuDML | Numdam

[42] Parent E., Lang G., Girard P. 1996. Sur l'Apport des Statistiques Bayésiennes au Contrôle de la Qualité par Attribut, partie 2 : Contrôle Séquentiel Tronqué. Rev. Statistique Appliquée, XLIV(1), 37-54. | EuDML | Numdam

[43] Pitt M., Shephard N. 1999. Filtering Via Simulation : Auxiliary Particle Filters. Ecological Applications, 94(446), 590-599. | MR | Zbl

[44] Reeves C.R., Wright C.C. 1996. Genetic Algorithms and the Design of Experiments. In : Proc. IMA Fall Workshop on Evolutionary Algorithms.

[45] Robert C. P. 2006. Le Choix Bayésien : Principes et Pratique. Paris : Springer Verlag France.

[46] Robert C.P., Casella G. 2004. Monte-Carlo Statistical Methods. Springer-Verlag. | MR | Zbl

[47] Rubin D.B. 1988. Using the SIR Algorithm to Simulate Posterior Distributions. Pages 395-402 of : Bayesian Statistics 3, vol. 3. Oxford University Press. | Zbl

[48] Smith A.F.M., Gelfand A.E. 1992. Bayesian Statistics without Tears : a Sampling-Resampling Perspective. The American Statistician, 46, 84-88. | MR

[49] Tagaras G. 1986. Economic Design of Acceptance Sampling and Process Control Procedures for Quality Assurance in Complex Production Systems. Ph.D. thesis, Stanford University.

[50] Tanner M. H. 1992. Tools for Statistical Inference : Observed Data and Data Augmentation Methods. New York : Springer-Verlag. | MR | Zbl

[51] Van Laarhoven P.J.M., Aarts E.H.L. 1987. Simulated Annealing : Theory and Applications. Reider Pub. and Kluwer Dordrecht, Holland. | MR | Zbl

[52] Von Neumann J., Morgenstern O. 1944. Theory of Games and Economic Behavior. Princeton Univ. Press, Princeton NJ. | MR | Zbl