Gestion des données manquantes en Analyse en Composantes Principales

Josse, Julie; Husson, François; Pagès, Jérôme

Journal de la société française de statistique, Tome 150 (2009) no. 2, pp. 28-51.

Une solution classique pour réaliser une Analyse en Composante Principale (ACP) sur données incomplètes consiste à chercher les axes et les composantes qui minimisent l’erreur de reconstitution sur les données présentes. Plusieurs algorithmes ont été proposés dans la littérature comme NIPALS, une approche par moindres carrés alternés pondérés et une approche par ACP itérative. Cette dernière consiste en une imputation itérative des données au cours du processus d’estimation et s’apparente à un algorithme EM d’un modèle particulier. Ces algorithmes sont décrits dans le cadre commun de la minimisation du critère. Cette présentation unifiée permet de mieux comprendre leurs propriétés et les difficultés qu’ils rencontrent. Nous nous focalisons ensuite sur le problème principal du surajustement et montrons comment la formulation probabiliste de l’ACP (Tipping & Bishop, 1997) offre un terme de régularisation adapté pour pallier à ce problème. Les performances de l’algorithme finalement proposé sont comparées à celles des autres algorithmes à partir de simulations.

An approach commonly used to handle missing values in Principal Component Analysis (PCA) consists in ignoring the missing values by optimizing the loss function over all non-missing elements. This can be achieved by several methods, including the use of NIPALS, weighted regression or iterative PCA. The latter is based on iterative imputation of the missing elements during the estimation of the parameters, and can be seen as a particular EM algorithm. First, we review theses approaches with respect to the criterion minimization. This presentation gives a good understanding of their properties and the difficulties encountered. Then, we point out the problem of overfitting and we show how the probabilistic formulation of PCA (Tipping & Bishop, 1997) offers a proper and convenient regularization term to overcome this problem. Finally, the performances of the new algorithm are compared to those of the other algorithms from simulations.

[1] Bartholomew, D.J. Latent Variable Models and factor Analysis, Griffin, 1987 | Zbl

[2] Buchanan, A M.; Fitzigibbon, A W. Damped Newton algorithms for matrix factorization with missing data, Computer vision and pattern recognition, Volume 2 (2005), pp. 316-322

[3] Bishop, Christopher M. Pattern recognition and machine learning, Springer, 2006 | MR | Zbl

[4] Bishop, Christopher M. Bayesian PCA, Proceedings of the 1998 conference on Advances in neural information processing systems II, MIT Press, Cambridge, MA, USA (1999), pp. 382-388

[5] Bro, R. Multi-way analysis in the food industry. Models, algorithms and applications (1998) (Ph. D. Thesis)

[6] Canny, J F. Collaborative filtering with privacy via factor analysis, Proceding IEEE Symposium on Security and Privacy (2002), pp. 45-57

[7] Caussinus, Henri Models and uses of principal component analysis (with discussion), Multidimensional Data Analysis (de Leeuw, J; Heiser, W; Meulman, J; Critchley, F, eds.), DSWO Press, 1986, pp. 149-178

[8] Christofferson, A The one-component model with incomplete data, Uppsala University, Institute of statistics (1969) (Ph. D. Thesis)

[9] Denis, Jean-Baptiste Ajustements de modèles linéaires et bilinéaires sous contraintes linéaires avec données manquantes, Revue de statistique appliquée, Volume 39 (1991), pp. 5-24

[10] Droesbeke, Jean-Jacques; Fichet, B; Tassi, P Modèles pour l’analyse des données multidimensionelles, Economica, 1992

[11] de Ligny, C L.; Niewdorp, G H E.; Btrderode, W K.; Hammers, W E.; van Houweling, J C. An application of Factor Analysis with missing data, Technometrics, Volume 23 (1981), pp. 91-95

[12] Dempster, A P.; Laird, N M.; Rubin, D B. Maximum likelihood from incomplete data via the EM algorithm, Journal of the Royal Statistical Society B, Volume 39 (1977), pp. 1-38 | MR | Zbl

[13] Escoufier, Yves Le traitement des variables vectorielles, Biometrics, Volume 29 (1973), pp. 751-760 | MR

[14] Funk, S Netflix Challenge, 2008 http://sifter.org/~simon/journal/20061211.html

[15] Grung, B; Manne, R Missing values in principal component analysis, Chemiometrics and intelligent laboratory systems, Volume 42 (1998), pp. 125-139

[16] Greenacre, Michael Theory and applications of correspondence analysis, Acadamic Press, 1984 | MR | Zbl

[17] Golub, G H.; Van Loan, C F. Matrix computations, Johns Hopkins University Press, 1996 | MR | Zbl

[18] Gabriel, K R.; Zamir, S Lower Rank Approximation of Matrices by Least Squares with Any Choice of Weights, Technometrics, Volume 21 (1979), pp. 236-246 | Zbl

[19] Heiser, W J. Convergent computation by iterative majorization : theory and applications in multidimensional data analysis, Recent Advances in Descriptive Multivariate Analysis (Krzanowski, W J, ed.), Oxford University Press, 1995, pp. 157-189 | MR

[20] Husson, F; Josse, J; Lê, S FactoMineR R package version 1.11 , 2009 http://factominer.free.fr

[21] Hastie, Trevor; Tibshirani, Robert; Friedman, Jerome The elements of statistical learning. Data Mining, Inference and Prediction, Springer series in statistics, 2001 | Zbl

[22] Healy, M J R.; Wesmacott, M Missing values in experiments analyzed on automatic computers, Applied statistics, Volume 5 (1956), pp. 203-206

[23] Josse, J; Pagès, J; Husson, F Testing the significance of the RV coefficient, Computational Statististics and Data Analysis, Volume 53 (2008), pp. 82-91 | MR | Zbl

[24] Kiers, H A L. Weighted least squares fitting using ordinary least squares algorithms, Psychometrica, Volume 62 (1997), pp. 251-266 | Zbl

[25] Kroonenberg, P M. Applied Multiway data analysis (chap.7), Wiley series in probability and statistics, 2008 | MR | Zbl

[26] Little, R J A.; Rubin, D B. Statistical analysis with missing data, Wiley series in probability and statistics, New-York, 1987, 2002 | MR | Zbl

[27] Marlin, B M Missing data problems in machine learning, University of Toronto (2008) (Ph. D. Thesis)

[28] Nora-Chouteau, C Une méthode de reconstitution et d’analyse de données incomplètes, Université Pierre et Marie Curie (1974) (Ph. D. Thesis)

[29] Pearson, K On lines and plane of closest fit to systems of points in space, Phil. Mag., Volume 2 (1901), pp. 559-572 | JFM

[30] Raiko, T; Ilin, A; Karhunen, J Principal Component Analysis for sparse High-Dimensional Data, Neural Information Processing (2007), pp. 566-575

[31] Roweis, S EM algorithms for PCA and Sensible PCA, Advances in Neural Information Processing Systems, Volume 10 (2008), pp. 626-632

[32] Rubin, D B; Thayer, D T EM algorithms for ML factor analysis, Psychometrica, Volume 47 (1982), pp. 69-76 | MR | Zbl

[33] Rubin, D B Multiple imputation for non-response in survey, Wiley, 1987 | MR | Zbl

[34] Rubin, D B Multiple imputation after 18+ years, Journal of the American Statistical Association, Volume 91 (1996), pp. 473-489 | Zbl

[35] Schafer, J L Analysis of incomplete multivariate data, Chapman & Hall/CRC, 1997 | MR | Zbl

[36] Schafer, J L; Graham, J W Missing Data : Our View of the State of the Art, Psychological Methods, Volume 7 (2002), pp. 147-177

[37] Srebro, N Learning with Matrix Factorizations, Massachusetts institute of technology (2004) (Ph. D. Thesis) | MR

[38] Stacklies, W; Redestig, H; Scholz, M; Walther, D; Selbig, J pcaMethods-a bioconductor package providing PCA methods for incomplete data, Bioinformatics, Volume 23 (2007), pp. 1164-1167

[39] Tipping, M; Bishop, C M Mixture of probabilistic principal component analysers, Neural Computation, Volume 11 (1999), pp. 443-482

[40] Tipping, M; Bishop, C M Probabilistic Principal Component Analysis, Journal of the Royal Statistical Society B, Volume 61 (1999), pp. 611-622 | MR | Zbl

[41] Tenenhaus, M La régression PLS théorie et pratique, Technip, 1998 | MR | Zbl

[42] Verbeek, J J Notes on Probabilistic PCA with missing values (2009) (Technical report)

[43] Wentzell, P D; Andrews, D T; Hamilton, D C; Faber, K; Kowalski, B R Maximum likelihood principal component analysis, J. Chemom., Volume 11 (2002), pp. 339-366

[44] Walczak, B; Massart, D L Dealing with missing data Part I, Chemiometrics and Intelligent Laboratory System, Volume 58 (2001), pp. 15-27

[45] Wold, H Estimation of principal components and related methods by iterative least squares, Multivariate Analysis (Krishnaiah, P R, ed.), Academic Press, 1966, pp. 391-420 | MR | Zbl

[46] Wold, H Nonlinear estimation by iterative least squares procedures, Research Papers in Statistics : Festschrift for Jerzy Neyman (David, F N, ed.), Wiley, 1966, pp. 411-444 | MR | Zbl