Ways to do causal inference
There are a few primary ways to do causal inference analysis properly. All rely on varying degrees of assumptions, otherwise nothing would ever get done. None of these descriptions are complete, but click on the links in each for more detail. Addtionally, these are not all the ways to do causal inference, but rather common "families" of methods that are often employed. Here is a good link to read Causal Design patterns
Instrumental Variables (IV)
Instrumental variables are a really powerful tool and are fun to think about. However, it is easy to violate assumptions we make in an IV analysis, which can contaminate our results. Additionally, finding instruments is not always easy.
Regression Discontuinity Design (RDD)
RDD's are another example of an intuitive but tricky tool ( a common theme in the field).
Difference in Difference (Dif-n-Dif)
Again, a powerful, intuitive tool that is hindered by the assumptions required.
Synthetic Control Methods
Essentially, a better version of difference in difference. Really cool idea, with some of the technical details still being worked out.
Sensitivity Analyses
This is a broader class of methods that relax assumptions we make when modeling. While these methods never lead to an exact answer, they allow us to paint a broad picture of what the true causal impact really is.
Regression Adjustment
Often people do this and think they are done :(.
Propensity Score Methods
Probably the most important and ubiquitious method (maybe?) Here is a great blog primer by Lucy D'Agostino McGowan, who has a cool causal inference podcast. Propensity Score Weighting<
Matching
Similar to propensity score methods in that we try to deconfound our observations.
Randomized Control Trials (RTC)
In one sense, these are the gold standards. Whenever we have an experiment where subjects (people, animals, objects etc.) take some treatment at random, any difference between control and treatment groups can be attributed causally. Which solves the main problem of causal inference, doesn't it? Well yes. But, experimental subjects may not truly represent a random population. The conditions of the experiment may not mirror the real world at all. Even in a random experiment, if we want to look at effects of sub-populations, such as differenes by gender for example, we can no longer interpret differences between the treatment and control groups causally, because of mediating and moderating variables. Additionally, RTC can be very expensive and difficult to run. While they are certainly useful in say the effect of drugs or vaccines, their broad impact is not quite as clear (despite the 2019 Nobel prize in economics).
Graphs and Potential Outcomes
Graphs and potential outcomes are not necessarily methods in their own right, but they represent ways of setting up your problem/model. Judea Pearl is largely responsible for the graph based approach, whereas the potential outcome approach is often coined the "Rubin causal model" after Donald Rubin. Strangely, Pearl and Rubin are nemesis. In fact, one of the unique aspects of causal inference is the fueds in the field. For example, two pioneers, Pearl and Rubin, do not exactly see eye to eye. In fact, Judea Pearl does not exactly see eye to eye with a lot of people, including Andrew Gelman, a very good causal researcher who maintains an interesting blog. Here is an example of an interaction between Pearl and Gelman. This collaborative research framework has served as a model for setting up the COVID-19 Forecast Hub.
Academic Papers
Here is a list of important academic papers that are also well-written and clearly make an effort to be accessible (as possible as that is at the academic research level).
- Papakostas, D., Hahn, P.R., Murray, J. , Zhou, F. , and Gerakos, J. "DO FORECASTS OF BANKRUPTCY CAUSE BANKRUPTCY? A MACHINE LEARNING SENSITIVITY ANALYSIS.". 2021. Arxiv
- Herren, A. and Hahn, P.R. "Semi-supervised learning and the question of true versus estimated propensity scores". 2020. Arxiv,
Have you ever wondered if auditors predicting a company going bankrupt will cause that bankruptcy? If so, this is your paper.
A straightforward application of semi-supervised machine learn- ing to the problem of treatment effect estimation would be to consider data as “unlabeled" if treatment assignment and covariates are observed but out- comes are unobserved. According to this formulation, large unlabeled data sets could be used to estimate a high dimensional propensity function and causal inference using a much smaller labeled data set could proceed via weighted estimators using the learned propensity scores. In the limiting case of infinite unlabeled data, one may estimate the high dimensional propensity function exactly. However, longstanding advice in the causal inference com- munity suggests that estimated propensity scores (from labeled data alone) are actually preferable to true propensity scores, implying that the unlabeled data is actually useless in this context. In this paper we examine this para- dox and propose a simple procedure that reconciles the strong intuition that a known propensity functions should be useful for estimating treatment effects with the previous literature suggesting otherwise. Further, simulation studies suggest that direct regression may be preferable to inverse-propensity weight estimators in many circumstances.
Podcasts and Websites
Casual Inference
Casual Inference | 2020
Fun podcast about causal inference, statistics, and data science. Hosted by
Live free or dichotimize
Live free or dichotimize | Blog
Fun podcast about causal inference, statistics, and data science. Hosted by Lucy D'Agostino and Ellie Murrary.