Identification in Causal Models With Hidden Variables
Journal de la société française de statistique, Volume 161 (2020) no. 1, pp. 91-119.

Targets of inference that establish causality are phrased in terms of counterfactual responses to interventions. These potential outcomes operationalize cause effect relationships by means of comparisons of cases and controls in hypothetical randomized controlled experiments. In many applied settings, data on such experiments is not directly available, necessitating assumptions linking the counterfactual target of inference with the factual observed data distribution. This link is provided by causal models. Originally defined on potential outcomes directly (Rubin, 1976), causal models have been extended to longitudinal settings (Robins, 1986), and reformulated as graphical models Spirtes et al., 2001; Pearl, 2009). In settings where common causes of all observed variables are themselves observed, many causal inference targets are identified via variations of the expression referred to in the literature as the g-formula (Robins, 1986), the manipulated distribution (Spirtes et al., 2001), or the truncated factorization (Pearl, 2009).

In settings where hidden variables are present, identification results become considerably more complicated. In this manuscript, we review identification theory in causal models with hidden variables for common targets that arise in causal inference applications, including causal effects, direct, indirect, and path-specific effects, and outcomes of dynamic treatment regimes. We will describe a simple formulation of this theory (Tian and Pearl, 2002; Shpitser and Pearl, 2006b,b; Tian, 2008; Shpitser, 2013) in terms of causal graphical models, and the fixing operator, a statistical analogue of the intervention operation (Richardson et al., 2017).

Classification: 62H99, 60E05
Keywords: identification, graphical models, causal inference, hidden variable models
Shpitser, Ilya 1

1 Department of Computer Science, Johns Hopkins University, 3400 N Charles St, Baltimore, MD 21218
     author = {Shpitser, Ilya},
     title = {Identification in {Causal} {Models} {With} {Hidden} {Variables}},
     journal = {Journal de la soci\'et\'e fran\c{c}aise de statistique},
     pages = {91--119},
     publisher = {Soci\'et\'e fran\c{c}aise de statistique},
     volume = {161},
     number = {1},
     year = {2020},
     mrnumber = {4125250},
     zbl = {1445.62129},
     language = {en},
     url = {}
AU  - Shpitser, Ilya
TI  - Identification in Causal Models With Hidden Variables
JO  - Journal de la société française de statistique
PY  - 2020
SP  - 91
EP  - 119
VL  - 161
IS  - 1
PB  - Société française de statistique
UR  -
LA  - en
ID  - JSFS_2020__161_1_91_0
ER  - 
%0 Journal Article
%A Shpitser, Ilya
%T Identification in Causal Models With Hidden Variables
%J Journal de la société française de statistique
%D 2020
%P 91-119
%V 161
%N 1
%I Société française de statistique
%G en
%F JSFS_2020__161_1_91_0
Shpitser, Ilya. Identification in Causal Models With Hidden Variables. Journal de la société française de statistique, Volume 161 (2020) no. 1, pp. 91-119.

[1] Avin, Chen; Shpitser, Ilya; Pearl, Judea Identifiability of Path-Specific Effects, Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence (IJCAI-05), Volume 19, Morgan Kaufmann, San Francisco (2005), pp. 357-363

[2] Chakraborty, Bibhas; Moodie, Erica E. M. Statistical Methods for Dynamic Treatment Regimes (Reinforcement Learning, Causal Inference, and Personalized Medicine), Springer, New York, 2013 | DOI | MR

[3] Drton, Mathias Discrete chain graph models, Bernoulli, Volume 15 (2009) no. 3, pp. 736-753 | MR | Zbl

[4] Haavelmo, Trygve The statistical implications of a system of simultaneous equations, Econometrica, Volume 11 (1943), pp. 1-12 | DOI | MR | Zbl

[5] Huang, Yimin; Valtorta, Marco Pearl’s Calculus of Interventions is Complete, Twenty Second Conference On Uncertainty in Artificial Intelligence (2006)

[6] Lauritzen, Steffan L. Graphical Models, Oxford, U.K.: Clarendon, 1996 | MR

[7] Miles, Caleb; Shpitser, Ilya; Kanki, Phyllis; Melone, Seema; Tchetgen Tchetgen, Eric J. Quantifying an Adherence Path-Specific Effect of Antiretroviral Therapy in the Nigeria PEPFAR Program, Journal of the American Statistical Association (2017) | MR

[8] Malinsky, Daniel; Shpitser, Ilya; Richardson, Thomas S. A Potential Outcomes Calculus for Identifying Conditional Path-Specific Effects, Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics (2019)

[9] Neyman, Jerzy Sur les applications de la thar des probabilities aux experiences Agaricales: Essay des principle. Excerpts reprinted (1990) in English, Statistical Science, Volume 5 (1923), pp. 463-472 | MR

[10] Pearl, Judea Direct and Indirect Effects, Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence (UAI-01), Morgan Kaufmann, San Francisco (2001), pp. 411-420

[11] Pearl, Judea Causality: Models, Reasoning, and Inference, Cambridge University Press, 2009 | MR

[12] Pearl, Judea The causal mediation formula – a guide to the assessment of pathways and mechanisms (2011) no. R-379 ( Technical report ) | DOI

[13] Pearl, Judea Probabilistic Reasoning in Intelligent Systems, Morgan and Kaufmann, San Mateo, 1988 | MR

[14] Richardson, Thomas S.; Evans, Robin J.; Robins, James M.; Shpitser, Ilya Nested Markov Properties for Acyclic Directed Mixed Graphs, 2017 (Working paper)

[15] Robins, James M.; Greenland, Sander Identifiability and Exchangeability of Direct and Indirect Effects, Epidemiology, Volume 3 (1992), pp. 143-155 | DOI

[16] Richardson, Thomas S. Markov properties for acyclic directed mixed graphs, Scandinavial Journal of Statistics, Volume 30 (2003) no. 1, pp. 145-157 | DOI | MR | Zbl

[17] Robins, James M. A new approach to causal inference in mortality studies with sustained exposure periods – application to control of the healthy worker survivor effect, Mathematical Modeling, Volume 7 (1986), pp. 1393-1512 | DOI | MR | Zbl

[18] Robins, James M. Marginal Structural Models versus Structural Nested Models as Tools for Causal Inference, Statistical Models in Epidemiology: The Environment and Clinical Trials, NY: Springer-Verlag (1999) | MR | Zbl

[19] Robins, James M. Testing and estimation of direct effects by reparameterizing directed acyclic graphs with structural nested models, Computation, Causation, and Discovery (Glymour, C.; Cooper, G., eds.), Menlo Park, CA, CAmbridge, MA: AAAI Press/The MIT Press (1999), pp. 349 - 405 | MR

[20] Robins, James M.; Richardson, Thomas S. Alternative graphical causal models and the identification of direct effects, Causality and Psychopathology: Finding the Determinants of Disorders and their Cures (2010)

[21] Richardson, Thomas S.; Robins, Jamie M. Single World Intervention Graphs (SWIGs): A Unification of the Counterfactual and Graphical Approaches to Causality, preprint: (2013)

[22] Richardson, Thomas; Spirtes, Peter Ancestral graph Markov models, Annals of Statistics, Volume 30 (2002), pp. 962-1030 | MR | Zbl

[23] Rubin, D. B. Causal Inference and Missing Data (with discussion), Biometrika, Volume 63 (1976), pp. 581-592 | DOI | MR | Zbl

[24] Spirtes, Peter; Glymour, Clark; Scheines, Richard Causation, Prediction, and Search, Springer Verlag, New York, 2001

[25] Shpitser, Ilya Counterfactual Graphical Models for Longitudinal Mediation Analysis With Unobserved Confounding, Cognitive Science (Rumelhart special issue), Volume 37 (2013), pp. 1011-1035 | DOI

[26] Shpitser, Ilya Segregated Graphs and Marginals of Chain Graph Models, Advances in Neural Information Processing Systems 28, Curran Associates, Inc. (2015)

[27] Shpitser, Ilya Identification in Graphical Causal Models, Handbook of Graphical Models (2017) | MR | Zbl

[28] Shpitser, Ilya; Pearl, Judea Identification of Conditional Interventional Distributions, Proceedings of the Twenty Second Conference on Uncertainty in Artificial Intelligence (UAI-06), AUAI Press, Corvallis, Oregon (2006), pp. 437-444

[29] Shpitser, Ilya; Pearl, Judea Identification of Joint Interventional Distributions in Recursive Semi-Markovian Causal Models, Proceedings of the Twenty-First National Conference on Artificial Intelligence (AAAI-06), AAAI Press, Palo Alto (2006)

[30] Shpitser, Ilya; Sherman, Eli Identification of Personalized Effects Associated With Causal Pathways, Proceedings of the 34th Annual Conference on Uncertainty in Artificial Intelligence (UAI-18) (2018)

[31] Shpitser, Ilya; Tchetgen Tchetgen, Eric J. Causal Inference with a Graphical Hierarchy of Interventions, Annals of Statistics, Volume 44 (2016) no. 6, pp. 2433-2466 | arXiv | MR | Zbl

[32] Tian, Jin Identifying Dynamic Sequential Plans, Proceedings of the Twenty-Fourth Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI-08), AUAI Press, Corvallis, Oregon (2008), pp. 554-561

[33] Tian, Jin; Pearl, Judea On the Testable Implications of Causal Models with Hidden Variables, Proceedings of the Eighteenth Conference on Uncertainty in Artificial Intelligence (UAI-02), Volume 18, AUAI Press, Corvallis, Oregon (2002), pp. 519-527

[34] Verma, Thomas S.; Pearl, Judea Equivalence and Synthesis of Causal Models (1990) no. R-150 ( Technical report )

[35] Wright, Sewall Correlation and Causation, Journal of Agricultural Research, Volume 20 (1921), pp. 557-585