Conditional inference in parametric models

Broniatowski, Michel; Caron, Virgile

Journal de la société française de statistique, Tome 160 (2019) no. 2, pp. 48-66.

Cet article propose une nouvelle approche d’inférence statistique, fondée sur la simulation d’échantillons conditionnés par une statistique des données. L’approximation de la vraisemblance conditionnelle de longues séries d’échantillons sachant la statistique des données admet une forme explicite qui est présentée. Lorsque la statistique de conditionnement est exhaustive par rapport à un paramètre fixé, on montre que la densité approchée est également invariante par rapport à ce même paramètre. Une nouvelle procédure de Rao-Blackwell est proposée et les simulations réalisées montrent que le théorème de Lehmann Scheffé reste valide pour cette approximation. L’inférence conditionnelle sur les familles exponentielles avec paramètre de nuisance est également étudiée, menant à des tests de Monte Carlo, dont les performances sur échantillonnage conditionnel sont comparées à celles sur bootstrap paramétrique. Enfin, on s’intéresse à l’estimation du paramètre d’intérêt par la vraisemblance conditionnelle.

This paper presents a new approach to conditional inference, based on the simulation of samples conditioned by a statistics of the data. Also an explicit expression for the approximation of the conditional likelihood of long runs of the sample given the observed statistics is provided. It is shown that when the conditioning statistics is sufficient for a given parameter, the approximating density is still invariant with respect to the parameter. A new Rao-Blackwellisation procedure is proposed and simulation shows that Lehmann Scheffé Theorem is valid for this approximation. Conditional inference for exponential families with nuisance parameter is also studied, leading to Monte Carlo tests; comparison with the parametric bootstrap method is discussed. Finally the estimation of the parameter of interest through conditional likelihood is considered.

Keywords: Conditional inference, Rao Blackwell Theorem, Lehmann Scheffé Theorem, Exponential families, Nuisance parameter, Simulation
Mot clés : Inférence conditionnelle, Théorème de Rao Blackwell, Théorème de Lehmann Scheffé, Familles exponentielles, Paramètre de nuisance, Simulation

[1] Barndorff-Nielsen, O.E. (1978). Information and exponential families in statistical theory. Wiley Series in Probability and Mathematical Statistics. Chichester: John Wiley & Sons. | MR | Zbl

[2] Bardnörff-Nielsen, O.E. and Cox, R.R. (1994). Inference and Asymptotics. Chapman & Hall, London. | MR | Zbl

[3] Basu, D. (1977). On the elimination of nuisance parameters. J. Amer. Statist. Assoc. 72 (1977), no. 358, 355–366. | MR | Zbl

[4] Broniatowski M. and Caron, V. (2011). Long runs under a conditional limit distribution. Ann. Appl. Probab. 24 (2014), no. 6, 2246–2296. | MR

[5] Broniatowski M. and Caron, V. (2011). Small variance estimators for rare event probabilities. ACM Trans. Model. Comput. Simul. 23 (2013), no. 1, Art. 7, 23 pp. | MR

[6] Casella, G. and Robert, C. P. (1996). Rao-Blackwellisation of sampling schemes. Biometrika 83 , no. 1, 81–94. | MR | Zbl

[7] Casella, G. and R. C. P. (1998). Post-processing accept-reject samples: recycling and rescaling. J. Comput. Graph. Statist. 7 , no. 2, 139–157 | MR

[8] Cheng, R.C.H. (1984). Generation of inverse Gaussian variates with given sample mean and dispersion. Appl. Statist. 33, 309–16 | MR | Zbl

[9] Dembo, A. and Zeitouni, O. (1996) Refinements of the Gibbs conditioning principle. Probab. Theory Related Fields 104 1–14. | MR | Zbl

[10] Diaconis, P. and Freedman, D.A. (1988) Conditional limit theorems for exponential families and finite versions of de Finetti’s theorem. J. Theoret. Probab. 1 381–410. | MR | Zbl

[11] Efron, B. (1975). Defining the curvature of a statistical problem (with applications to second order efficiency) (with discussion). Ann. Statist. 3, 1189-1242. | MR | Zbl

[12] Efron, B. (1978). The geometry of exponential families. Ann. Statist. 6, 362-376. | MR | Zbl

[13] Efron, B. (1979), Bootstrap methods: another look at the jackknife. Ann. Statist. 7 , no. 1, 1–26. | MR | Zbl

[14] Engen, S. and Lillegard, M. (1997). Stochastic simulations conditioned on sufficient statistics. Biometrika, 84, 235–240. | MR | Zbl

[15] Fraser, D. A. S. (2004). Ancillaries and conditional inference. With comments by Ronald W. Butler, Ib M. Skovgaard, Rudolf Beran and a rejoinder by the author. Statist. Sci. 19 , no. 2, 333–369 | MR | Zbl

[16] Iacobucci, A., Marin, J.-M., Robert, C. (2010). On variance stabilisation in population Monte Carlo by double Rao-Blackwellisation. Comput. Statist. Data Anal. 54 , no. 3, | MR

[17] Jöckel, K.-H. (1986). Finite sample properties and asymptotic efficiency of Monte Carlo tests. Ann. of Stat., 14, 336–347. | MR | Zbl

[18] Lehmann, E.L. (1986). Testing Statistical Hypotheses. Springer. | MR

[19] Lindqvist, B.H., Taraldsen, G., Lillegärd, M. and Engen, S. (2003). A counterexample to a claim about stochastic simulations. Biometrika 90 , no. 2, 489–490. | MR | Zbl

[20] Lindqvist, B.H. and Taraldsen, G. (2005). Monte Carlo conditioning on a sufficient statistic. Biometrika 92 , no. 2, 451–464. | MR | Zbl

[21] Lockhart, R. and O’Reilly, F. (2005) A note on Moore’s conjecture. Statist. Probab. Lett. 74 , no. 2, 212–220. | MR | Zbl

[22] Lockhart, R A., O Reilly, F J. and Stephens, A. (2007) Use of the Gibbs sampler to obtain conditional tests, with applications. Biometrika 94 , no. 4, 992–998. | MR | Zbl

[23] Lockhart, R.A. and Stephens, M. A. (1994). Estimation and tests of fit for the three–parameter Weibull distribution. J. Roy. Statist. Soc.. B. 56:491–500. | MR | Zbl

[24] O’Reilly, F. and Gravia-Medrano, L. (2006). On the conditional distribution of goodness-of-fit tests. Commun. Statist. A, 35:541–9. | MR | Zbl

[25] Moore, David S. (1973). A note on Srinivasan’s goodness-of-fit test. Biometrika 60 , 209–211. | MR | Zbl

[26] Pedersen, B.V., (1979). Approximating conditional distributions by the mixed Edgeworth-saddlepoint expansion. Biometrika, 66(3), 597–604. | Zbl

[27] Pace L. and Salvan A., (1992). A note on conditional cumulants in canonical exponential families. Scand. J. Statist., 19, 185–191. | MR | Zbl

[28] Perron, F. (1999). Beyond accept-reject sampling. Biometrika 86 , no. 4, 803–813 | MR | Zbl

[29] Reid, N. (1995). The roles of conditioning in inference. With comments by V. P. Godambe, Bruce G. Lindsay and Bing Li, Peter McCullagh, George Casella, Thomas J. DiCiccio and Martin T. Wells, A. P. Dawid and C. Goutis and Thomas Severini. With a rejoinder by the author. Statist. Sci. 10 , no. 2, 138–157, 173–189, 193–196. | MR | Zbl

[30] Douc, R. and Robert, C.P.(2011). A vanilla Rao-Blackwellization of Metropolis-Hastings algorithms. Ann. Statist. 39 , no. 1, 261–277 | MR | Zbl

[31] Sundberg, R. (2009). Flat and multimodal likelihoods and model lack of fit in curved exponential families. Research Report 2009:1, http://www.math.su.se/matstat. | MR | Zbl