The Great Divide
I applaud the fact that the American Economic Association, the European Economic Association, the Econometric Society, and the Royal Economic Society have established an ad-hoc joint committee tasked with proposing ways to improve the publication process in economics. I encourage everyone to fill out the survey (limited to members of the AEA, EEA, ES, and RES, however) even though I am quite cynical about what may come of it.
Although only tangentially related, I decided to write a post about a topic that vexes me (and others to whom I have spoken, but I will leave it to them to out themselves) at the intersection of the Venn diagram containing publishing (editing and refereeing), teaching, and econometrics.
What lies at this intersection is the great divide between theoretical and applied econometrics, both in the classroom and in the wild.
As I have progressed in my career, I have enjoyed writing more econometrics papers and fewer economics papers per se. However, when I present, I usually nervously start my talks by saying that "I am not an econometrician." Recently, I presented in front of a very, very famous econometrician that I have known a long, long time. He has always been super nice to me. I joke that he is probably the most famous person that actually likes me.
When I said "I am not an econometrician," he replied, "Oh stop that." Nice!
After my talk concluded, he spoke up in the room, complimenting the paper, and then telling me it won't publish in an econometrics journal. While I don't disagree, this exchange gets at the heart of what vexes me about the current state of econometrics.
On the one hand, applied econometrics has become so monotone, with difference-in-differences (DID) dominating the landscape. In case one has been in hibernation the past decade, this is now documented in a new working paper by Paul Goldsmith-Pinkham. Two figures from his paper that he tweeted out:
First, the massive growth across all NBER groups in reliance on DID. Regression discontinuity (RD) is not too far behind.
Not only do I find this incredibly boring, but it is also bad science. As economists, we should better appreciate the value of diversification.
On the other hand, theoretical econometrics has become, well, very theoretical. Even a journal which has been my favorite up until now, Journal of Applied Econometrics, is no longer "applied". As an example, I went to the journal's homepage and clicked on the first forthcoming empirical micro paper: "Quantiles of the gain distribution of an early childhood intervention." Sounds innocent enough. Then you look at the paper. And, if you are really brave, you check out the supplemental material. Is it any wonder that the vast majority of applied econometricians then return to the safety of DID?
[As an aside, I know and like Carlos very much. This is not in any way a knock on the paper. In fact, I did not read the paper and this is precisely my point. It's even about measurement error!]
It is not an understatement to say that every paper I have published in JAE in my career would not be published there today. How do I know? Because everything I send there is now is deemed "not rigorous".
Thank you for indulging me. This blog post is really not supposed to be me complaining. I thoroughly enjoy my career. But... I hate the current state of the profession.
The econometric journals, including the more traditional "applied" journals (JAE, but also Econometric Reviews, REStat, JBES to some extent, etc.) have created a great divide between new methodologies and insights and the ability (and desire) of applied researchers to engage. To be fair, applied researchers also deserve blame as many seem uninterested in learning new econometric methods and referees have disdain for papers that try something "different" instead of "different-in-differences".
I don't have an answer to this. And, if I did, those in power wouldn't listen. It is interesting, however, that this same debate has been raging for quite some time. If you haven't read it, all theorists and applied people should read Kennedy (2002) and the comments and rebuttal in the same issue. In it, Kennedy quotes Magnus (1999):
For a while now, theoretical and applied econometricians have operated in separate corners of the professional universe. Despite this, there were at least some journals that previously allowed some from each camp to interact. In my experience, this door has now closed.
But what if there is no window???
I applaud and have so much respect for the difficulty and importance of theoretical econometric research. There is indeed great value in clearly spelling out assumptions, deriving asymptotic properties, proper inference, etc. However, the value does not always translate into meaningful improvements in applied work. Moreover, the extreme technical nature creates a barrier to entry for most applied researchers, even as consumers rather than producers of such knowledge.
Kennedy describes applied researchers working in the basement, while theorists occupy the top (third floor). He quotes Leamer (1978):
So, I guess we need an elevator not a window, eh?
In a comment on Kennedy, Hendry argues that this great divide between theorists and applied people permeates all the way down to introductory econometrics textbooks:
"There is a considerable gulf between the elementary -- and inappropriate -- methodology inherent in many such textbooks, and what applied econometricians need to do in practice to model economic data."
While I am often cynical and insist that the reason why many applied people don't venture beyond DID is because they don't actually enjoy econometrics -- it's just a means to an end -- the truth of the matter is that applied work is also very difficult and applied researchers have limited time and bandwidth. Kennedy quotes Bodkin & Hsiao (1996):
Theorists that care about their research filtering into the applied researcher's toolkit need to internalize this quote. Applied researchers spend 20% of their time thinking about econometrics. It is not then feasible, let alone worth it, to slog through technical assumptions, asymptotic proofs, etc. to figure out if a new method is useful or not and code-able or not.
In particular, journals that purport to be "applied" should heed the advice from another of the comments on Kennedy. Magnus states that:
"If one has the choice between usefulness and truth, go for usefulness."
Econometric research these days seems to be about every excruciating little truth. And what does all this truth do for applied researchers? Nothing useful.
Moving forward, there must be a place (and outlet) for research that brings new methods to applied people without forcing such papers to be written in a manner that clears the bar for theorists. In my experiences of late, many applied researchers are clamoring for this type of work, many non-theorists love to write such papers, but the publication process has closed the door on such papers. And, as what unites us all is Economics, we all believe in the power of incentives. So, let's fix this.
