Plans d’expériences numériques d’information de Kullback-Leibler minimale

Jourdan, Astrid; Franco, Jessica

Journal de la société française de statistique, Tome 150 (2009) no. 2, pp. 52-64.

Les utilisateurs de codes numériques onéreux en temps de calcul souhaitent réduire le coût en limitant le nombre de simulations suivant un choix judicieux fondé sur l’utilisation de plans d’expériences adaptés au contexte numérique et appelés “space-filling designs ”. Afin de remplir au mieux l’espace des variables d’entrée du simulateur, nous proposons une méthode de construction de plans dont les points sont le plus uniformément répartis dans l’hypercube unité. Pour mesurer l’écart entre la fonction de densité associée aux points du plan et celle de la loi uniforme, nous utilisons l’information de Kullback-Leibler, ce qui revient par ailleurs à utiliser l’entropie de Shannon. Celle-ci est estimée par une méthode de Monte Carlo dans laquelle la fonction de densité est remplacée par son estimation par noyaux gaussiens.

Experimental designs are tools which can dramatically reduce the number of simulations required by time-consuming computer codes. Because we don’t know the true relation between the response and inputs, designs should allow one to fit a variety of models and should provide information about all portions of the experimental region. One strategy for selecting the values of the inputs at which to observe the response is to choose these values so they are spread evenly throughout the experimental region, according to “space-filling designs”. In this article, we suggest a new method based on comparing the empirical distribution of the points in a design to the uniform distribution with the Kullback-Leibler information. The considered approach consists of estimating this difference or, reciprocally, the Shannon entropy. The entropy is estimated by a Monte Carlo method where the density function is replaced by its kernel density estimator.

[1] Beirlant, J.; Dudewicz, E.J.; Györfi, L.; Van Der Meulen, E. C. Nonparametric entropy estimation : an overview., Int. J. Math. Stat. Sci., Volume 6(1) (1997), pp. 17-36 | MR | Zbl

[2] Bettinger, R.; Duchêne, P.; Pronzato, L.; Thierry, E. Design of experiments for response diversity., Proc. 6th International Conference on Inverse Problems in Engineering (ICIPE), Volume Journal of Physics : Conference Series, Dourdan (Paris), June (2008)

[3] Barron, A. R.; Györfi, L.; Van Der Meulen, E. C. Distribution estimation consistent in total variation and two types of information divergence, IEEE Trans. Inform. Theory, Volume 5 (1992), pp. 1867-1883 | MR | Zbl

[4] Currin, C.; Mitchell, T.; Morris, M.; Ylvisaker, D. A bayesian approach to the design and analysis of computer experiments. (1988) no. 6498 (Technical report available from the national technical information service, Springfield, Va. 22161)

[5] Chaloner, K.; Verdinelli, I. Bayesian Experimental Design : A review., Statis. Sci., Volume 10 (1995), pp. 237-304 | MR | Zbl

[6] Dudewicz, E. J.; Van Der Meulen, E. C. Entropy-Based tests of uniformity., J. Amer. Statist. Assoc., Volume 76 (1981), pp. 967-974 | MR | Zbl

[7] Franco, Jessica. Planification d’expériences numériques en phase exploratoire pour la simulation des phénomènes complexes., Ecole Nationale Supérieure des Mines de Saint-Etienne (2008) (Ph. D. Thesis)

[8] Gunzburger, M.; Burkardt, J. Uniformity measures for point sample in hypercubes., https ://people.scs.fsu.edu/$\tilde{}$burkardt/pdf/ptmeas.pdf, 2004

[9] Hero, A.; Bing Ma; Michel, O.; Gorma, J. Applications of entropic spanning graphs., Signal Processing Magazine, IEEE, Volume 19 (2002), pp. 85-95

[10] Johnson, M. E.; Moore, L. M.; Ylvisaker, D. Minimax and maximin distance design., J. Statist. Plann. Inf., Volume 26 (1990), pp. 131-148 | MR

[11] Joe, H. Estimation of entropy and other functional of multivariate density., Ann. Int. Statist. Math., Volume 41 (1989), pp. 683-697 | MR | Zbl

[12] Kullback, S.; Leibler, R. A. On information and sufficiency., Ann. Math. Statist., Volume 22 (1951), pp. 79-86 | MR | Zbl

[13] Kosachenko, L. F.; Leonenko, N. N. Sample estimate of entropy of a random vector., Problem of Information Transmission, Volume 23 (1987), pp. 95-101 | Zbl

[14] Koehler, J. R.; Owen, A. B. Computer Experiments., Handbook of statistics, Volume 13 (1996), pp. 261-308 | MR | Zbl

[15] Leonenko, N. N.; Pronzato, L.; Savani, V. A class of Rényi information estimators for multidimensional densities., Annals of Statistics (2009) (to appear) | Zbl

[16] Niederreiter, H. Point sets and sequences with small discrepancy., Monasth. Math., Volume 104 (1987), pp. 273-337 | MR | Zbl

[17] Park, J. S. Optimal Latin Hypercube Designs for Computer Experiments., J. Stat. Plan. Inf., Volume 39 (1994), pp. 95-111 | MR | Zbl

[18] Scott, D. W. Multivariate Density Estimation : Theory, practice and visualization, John Wiley & Sons, New York, Chichester, 1992 | MR | Zbl

[19] Silverman, B. W. Density estimation for statistics and data analysis., Chapman & Hall, London, 1986 | MR | Zbl

[20] Stein, M. Large sample properties of simulations using latin hypercube sampling., Technometrics, Volume 29 (1987), pp. 143-151 | MR | Zbl

[21] Sebastiani, P.; Wynn, H. P. Maximum entropy sampling and optimal Bayesian experimental design., J. Royal Statist. Soc., Volume 62 (2000), pp. 145-157 | MR | Zbl

[22] Shewry, M. C.; Wynn, H. P. Maximum Entropy Sampling., J. Appl. Statist., Volume 14 (1987), pp. 165-170

[23] Thiémard, E. Sur le calcul et la majoration de la discrépance à l’origine., École polytechnique fédérale de Lausanne (2000) (Ph. D. Thesis)