How the (Econometric) Sausage is Made

Apparently Taylor Swift was in the news this week. While I can't say I am very up on Taylor Swift songs (or most pop culture for that matter), I did find this line in one of her older songs, Mean , from 2010: " But you don't know what you don't know " Seems like very relevant words for empirical researchers. There is so much to know when trying to be a top notch empirical researcher. You have to know the application and all the associated institutional details. You have to know the data and all its quirks (including measurement errors, but that's a topic for another day). And, yes, you have to know the econometrics. But, there is so much on the econometric side, it's hard sometimes for researchers to know what to ask as they don't know what they don't know .  If you are fortunate enough to at least know what you don't know , often times we are afraid to ask. We are afraid of people's response...  Well, sometimes people aren't being mea

The signs are everywhere these days.  Unfortunately, most of them are depressing and infuriating as they foretell even more depressing and infuriating times ahead.  But, today is the day to focus on the positive. Sometimes signs are actually a  good thing. One such example of this is with instrumental variables (IV). In one of the earliest posts  on this blog, I wrote about the finickiness  of IV. In particular, I wrote about the finite sample properties of IV in exactly identified models (i.e., where the number of endogenous variables is equal to the number of instrumental variables). I talked about how while two-stage least squares estimates of over-identified models (i.e., where the number of endogenous variables exceeds the number of instrumental variables) are biased, the IV/2SLS estimator in an exactly identified model fares even worse because the finite sample moments of the estimator do not even exist .  Now, researchers typically ignore this fact and focus on the fact that

In Act II, Scene II of Shakespeare’s  Romeo and Juliet , Juliet says,  “ What’s in a name? That which we call a rose by any other name would smell as sweet. ”  I, however, am not Shakespeare. I am a bear of very little brain. I did awful in all my humanities classes.  So, I googled exactly what the heck Juliet was talking about. It says here " The importance of a person or thing is the way it is; not because of what it is called. Simply, it means the names of things cannot affect what they actually are. This line is, in fact, very profound, suggesting that a name is just a label to distinguish one thing from another. It neither has any worth, nor gives true meaning. " I love this explanation! It offers a perfect segue to the topic of causality . Causality is the goal of much (most?) empirical work in economics, especially in microeconomics. Yes, there is definitively a role for non-causal empirical work and I am, increasingly, sensitive to this point. But, that's not the

Empirical work is hard. Damn hard. Particularly when it comes to causal inference. And that's without humans getting involved. Once humans enter the fray, an insanely hard problem can become downright impossible. The human element has led to a well-publicized replication crisis in many disciplines. Economics is not immune. A forthcoming paper by Ferraro & Shukla (2020) foretell a looming replication crisis in environmental and resource economics. The empirical practices giving rise to this forthcoming crisis are not new, nor are they unique to researchers in this field. But, they do bear repeating. And repeating. And repeating. Ferraro & Shukla (2020) focus on a number of "questionable research practices," two of which I shall focus on here: p-hacking and multiple testing. Let's start with p-hacking. Adam McCloskey wrote a guest blog this week on a working paper of his that I will discuss below. In it, he defines p-hacking, stating " Broadly speaking, p-