How the (Econometric) Sausage is Made

In these trying times, we take comfort where we can find it.  And, with everything else that's wrong in the world, one would think it cruel to take away a source of comfort to others. Nonetheless, this is exactly what happened this week on #EconTwitter, thanks to  Paul Hünermund  and Beyers Louw. In a just released working paper, Hünermund and Louw (2020) discuss a point that, while not new, certainly merits repeating for the benefit of researchers. Keele et al. (2020) discuss the same point as well. Reminding us of this point serves to effectively remove a source of comfort relied upon by researchers. Not that I am calling Paul and Beyers cruel! The point of this post is not simply to re-iterate the aforementioned papers. But, it is motivated by the above papers. And, to get there we must first talk about the above papers.  The setting is one in which the researcher wishes to estimate the causal effect of a treatment, D, on an outcome, Y, using observational data.  A common way to

While certain individuals on #EconTwitter have gotten me hooked on an undisclosed Netflix show that my teenage daughter loves and my teenage son makes fun of me for watching, my kids and I all agree on one truly fantastic show: The main characters are Shawn and Gus. Shawn helps the Santa Barbara police department solve crimes given his unique skill. He is hyper-observant. His skill was developed during childhood by his father, a police detective. However, Shawn convinces the police department he is psychic. Together, Shawn and Gus, best friends since childhood, run Psych, a psychic detective agency. As a result of Shawn's psychic-ness, he often "sees" the future and helps the police nab the criminal. One of Shawn's catch phrases in the show is "Waaaaaaiiiiiittttt for it!". Seeing the future and waiting for it aren't just common themes in Psych. They are also common themes -- either explicitly or implicitly -- in econometrics. Specifically, continuing on

To say that difference-in-differences (DID) as a strategy to estimate causal effects has seen a resurgence in the past decade would be an understatement. A search for the term on Google Scholar produces nearly 2,000 hits. In 2020. About 22,000 since 2010. And, to say that I am not much of a fan of DID would also be a bit of an understatement. In case you are curious, I feel this way for two reasons. First, the lack of originality , as suggested by Google Scholar. Admittedly, this is an unfair critique. If it works, then it should be popular. But, I have my suspicions. I also have inner demons that make me detest the popular. I'm sure it has nothing to with my childhood and being, well, less than popular.  Second, most DID papers refer to the policy change or intervention as a  natural experiment . I equally detest that phrase in the majority of its usages. However, if one is going to pursue the DID strategy to examine the causal effect of a policy change or other intervention, then