Resources for Learning Causal Inference
Resources for Learning Causal Inference
  •  
Causal Inference Resources

Had the pandemic happened last year, would there be so many awesome and new resources on Causal Inference?

That statement is a counterfactual: a question we ask about the current state of affairs in the world if something would have been different in the past.

According to Judea Pearl, a leading figure in causal inference, there are three rungs in the ladder of causation (more and more I see this model referred to as “Pearl’s Causal Hierarchy”).

The first (lowest) rung deals only with associations (sometimes, wrongly bounded to only correlations—which are only linear) in the data. This is the level where ALL machine learning works.

The second rung entails interventions. In this second level, we can know what will happen when we change (force) something. Babies and some animals deal with this level when they interact with the world. In order to deal formally with this level, Pearl developed an extension to probability theory called “do-calculus”.

Lastly, the third (and highest) rung in the ladder is the counterfactual. At this level, we need imagination to guess what the world would be had something been different in the past. As far as we know, we humans are the only living animals capable of doing this. (I will have much more to say about this in a future post relating this ability to human language and its computational properties).

Apparently, it is not so easy to define what is a cause even though we all have an intuition for it. Philosophers have taken a stab at it for centuries (if not millennia), so I won’t say that the issue is settled, but I do like one definition I have come across by Prof. Pearl himself and it goes something like this:

Having both A and B occurred, we can say that A is a cause for B, if and only if, had not A happened, B would not have happened.

I like this definition over others that deal with interventions precisely because it relies on counterfactuals—the highest and probably uniquely human level in the ladder. Let’s see an example that for some reason I have present whenever causality comes up in a conversation.

Let’s say that we see that whenever we have soup for supper, it is followed by steak. One of the earliest attempts to define causality dealt with this kind of situations: A is always followed by B, so A must cause B. Well, our common sense tells us that there is no way that the soup is the cause of the steak, but all the past empirical evidence shows that there is, at least, an association. So this definition does not suffice.

Now, let’s apply our preferred definition. Let’s imagine that we already ate. Would have we eaten steak if there had not been any soup? Yes! So the soup cannot be the cause for the steak. Another example, had the sun risen if the rooster hadn’t crowed? Yes! So the rooster does not cause the sun to rise every morning even though we see that first the rooster crows and then the sun rises.

After all this digression about the three rungs in the ladder of causality, the main objective of this post is to collect (mainly) free resources to learn causal inference. Most of these resources are brand new! Amazing timing for me who just picked up the subject last year as my research area to get my doctoral degree in computer science.

Causal inference is able to answer causal questions from observational data 🤯. For me, this was mind-blowing when I first learned it. By the way, that’s less than two years ago, in 2019 when Amazon suggested me The Book of Why: The New Science of Cause and Effect by Judea Pearl and Dana Mackenzie.

I couldn’t believe that I hadn’t heard about it before. I mean, I was getting a master’s in computer science and Pearl himself is a Turing Award winner for his work in causal inference and artificial intelligence. I got to grad school to learn about machine learning and AI in general; I was IN the AI lab of the research center and yet not only me but no one I asked knew about all this amazing work. Unbelievable.

So back to the topic again. This last year I have been following the right people on Twitter. That has been a gold mine. Apparently, people can be productive and giving during a pandemic! I will update the following list whenever I come across with more stuff. But for now, there’s plenty. And if by any chance any of the following authors reads this: Thank you so much for your time and generosity!

Books

MOOCs

  • Causal Diagrams: Draw Your Assumptions Before Your Conclusions: An edX MOOC by Prof. Miguel Hernán. It is a comprehensive introduction to the causal diagrams.

  • Causal Data Science with Directed Acyclic Graphs: A Udemy MOOC by Prof. Paul Hünermund. It is a hands-on introduction to causal inference and its applications to data science.

  • Statistical Rethinking: Properly not a course on causal inference (and not a MOOC) but a Bayesian Course by Prof. Richard McElreath that touches on causal inference. This GitHub repo holds materials for the course including links to videolectures and slides.

  • Causal Generative Machine Learning Minicourse: Minicourse by Prof. Robert O. Ness that goes quickly from DAGs to generative models in machine learning. Specially interesting in this minicourse is the generalization of causal systems to use the language of probabilistic programming. This is more general than causal DAGs.

  • Introduction to Causal Inference: Complete introduction to causal inference from a machine learning perspective by Brady Neal, a PhD student in causal inference and machine learning at Mila-Quebec Institute. This is a collection of YouTube lectures, recommended readings, and accompanying lecture notes soon to be published as a book.

Blog Posts

Minicourses or Tutorials

Programming Frameworks

For the R language

  • CausalImpact: An R package by Google for causal inference using Bayesian structural time-series models
  • DAGitty: An R package AND browser-based environment for creating and analyzing causal diagrams.
  • causaleffect: An R package for causal effect identification and transportation.

For the Python language

  • tfcausalimpact: Implementation of CausalImpact on top of TensorFlow for Python.
  • CausalML: A Python package by Uber for uplift modeling and causal inference with machine learning
  • DoWhy: A Python library by Microsoft for end-to-end causal inference
  • Ananke: A Python library for causal inference using grpahical models.

For the Julia language

  • CausalInference.jl: A Julia package for causal inference, graphical models and structure learning.
  • Omega.jl: A Julia package for causal and probabilistic inference in Julia.

Conferences and Workshops