Inference in Mixed Hidden Markov Models and Applications to Medical Studies

Delattre, Maud

Journal de la société française de statistique, Tome 151 (2010) no. 1, pp. 90-105.

Cet article veut montrer la nécessité d’étendre la définition des modèles de Markov cachés (HMM), ainsi que leurs méthodes d’estimation au cadre des études de population. Nous motivons alors les intérêts des modèles de Markov cachés à effets mixtes (MHMM) au travers d’un état de l’art succinct sur les modèles de Markov cachés et leurs nombreuses applications. Nous nous limiterons à des problématiques médicales. Dans ce cadre, les modèles de Markov cachés supposent que l’évolution des maladies peut s’interpréter à travers différents états. En effet, la distinction de divers stades évolutifs dans la maladie justifie l’application de modèles de Markov cachés à certaines pathologies, comme cela a déjà été le cas pour la migraine, ou encore l’épilepsie.

La définition des modèles de Markov cachés à effets mixtes est très récente. Ces nouveaux modèles sont des candidats intéressants pour la modélisation de symptômes. Les données utilisées sont complexes par leur structure et nécessitent toutefois une démarche d’analyse particulière. En outre, les méthodes d’apprentissage pour les MHMM restent peu nombreuses. Notre démarche est la suivante. Pour commencer, nous proposons d’estimer les paramètres de population au moyen de l’algorithme SAEM (Stochastic Approximation EM), dont la convergence est rapide. La procédure forward développée pour les HMM permet ici un calcul simple de la vraisemblance complète à chaque étape de la procédure MCMC de l’algorithme. Ensuite, les paramètres individuels sont obtenus par maximisation a posteriori de leur distribution. Enfin, les séquences d’états les plus probables pour chaque individu sont estimées par l’algorithme de Viterbi. Une étude par simulations Monte-Carlo illustre les propriétés de nos algorithmes.

The aim of the present paper is to document the need for adapting the definition of hidden Markov models (HMM) to population studies, as well as for corresponding learning methodologies. In this article, mixed hidden Markov models (MHMM) are introduced through a brief state of the art on hidden Markov models and related applications, especially focusing on disease related problems. Making the main assumption that a given pathology can be considered at different stages, hidden Markov models have for example already been used to study epileptic activity or migraine.

Mixed-effects hidden Markov models have been newly introduced in the statistical literature. The notion of mixed hidden Markov models is particularly relevant for modeling medical symptoms, but the data complexity generally requires specific care and the available methodology for MHMM is relatively poor. Our new approach can be briefly described as follows. First, we suggest to estimate the population parameters with the SAEM (Stochastic Approximation EM) algorithm, which has the property to converge quickly. The well-known forward recursions developed for HMM allow to compute easily the complete likelihood at each step of the MCMC procedure used within SAEM. Then, for dealing with the individuals, we suggest to estimate each set of individual parameters with the MAP (Maximum A Posteriori) of the parameter distributions. Finally, the hidden state sequences are decoded using the Viterbi algorithm. Some Monte-Carlo experiments are presented to illustrate the accuracy of our algorithms.

[1] Albert A two state Markov mixture model for a time series of epileptic seizure counts, Biometrics, Volume 37 (1991), pp. 1371-1381

[2] Altman Mixed hidden Markov models : an extension of the hidden Markov model to the longitudinal data setting, Journal of the American Statistical Association (2007) | Zbl

[3] Anisimov; Maas; Danhof; Pasqua, Della Analysis of responses in migraine modelling using hidden Markov models, Statistics in Medicine, Volume 26 (2007), pp. 4163-4178

[4] Albert; McFarland; Smith; Frank Time series for modelling counts from relapsing remitting disease : application to modelling disease activity in multiple sclerosis, Statistics in Medicine, Volume 13 (1994), pp. 453-466

[5] Altman; Petkau Application of hidden Markov models to multiple sclerosis lesion count, Statistics in Medicine, Volume 24 (2005), pp. 2335-2344

[6] Baum; Eagon An inequality with applications to statisical estimation for probabilistic functions of Markov processes and to a model for ecology (1966) | Zbl

[7] Baum; Petrie Statistical inference for probabilistic functions of finite state Markov chains, The Annals of Mathematical Statistics (1966) | Zbl

[8] Baum; Petrie; Soules; Weiss A maximization technique occuring in the statistical analysis of probabilistic functions of Markov chains, The Annals of Mathematical Statistics, Volume 41 (1970), pp. 164-171 | Zbl

[9] Cooper; Lipsitch The analysis of hospital infection data using hidden Markov models, Biostatistics, Volume 5 (2004), pp. 223-237 | Zbl

[10] Detilleux The analysis of disease biomarker data using a mixed hidden Markov model, Genet. Sel. Evol. (2008), pp. 491-509

[11] Douc; Matias Asymptotics of the maximum likelihood estimator for general hidden Markov models, Bernoulli, Volume 7 (2001), pp. 381-420 | Zbl

[12] Douc; Moulines; Olsson; Handel, Van Consistency of the Maximum Likelihood Estimator for general hidden Markov models, The Annals of Statistics (to appear) | Zbl

[13] Ip; Jones, Snow; Zhang; Rijmen Mixed effects hidden Markov models, Statistics in Medicine (2007)

[14] Kuhn; Lavielle Coupling a Stochastic Approximation Version of EM with an MCMC Procedure, ESAIM : Probability and Statistics, Volume 8 (2004), pp. 115-131 | Numdam | Zbl

[15] Leroux Maximum-likelihood estimation for hidden Markov models, Stochastic Processes and their Applications, Volume 40 (1992), pp. 127-143 | Zbl

[16] Rabiner A tutorial on Hidden Markov Models and selected applications in speech recognition, Proceedings of the IEEE, Volume 77 (1989), pp. 257-286

[17] Strat, Le; Carrat Monitoring epidemiologic surveillance data using hidden Markov models, Statistics in Medicine, Volume 18 (1999), pp. 3463-3478