It's Latin
Listening to my Spotify playlist this week, several songs by the Gipsy Kings have come up. Clearly my musical preferences have not evolved over time. One peppy song is Bamboleo. Part of the chorus is
The distinction between de jure and de facto treatments is not only relevant when assessing the effects of policies, laws, or regulations. It is also frequently relevant when analyzing individual-level treatments, where individuals are assigned to one treatment arm ("by law"), but end up in a different treatment arm ("in fact"). In the case of randomized control trials (RCTs), researchers are fortunate to observe both de jure and de facto treatment in many cases and thus both the de jure and de facto effects are, in principle, identified.
The distinction between de jure and de facto treatment came up -- implicitly -- this week on Twitter in discussion of a new NBER working paper by Cronin et al. (2020). In the paper, the authors note the de facto effect of psychotherapy on the treatment of mild-to-moderate depression and anxiety relative to drug therapy is positive. However, when individuals are given the opportunity to live it their way, individuals do not opt for psychotherapy. Thus, in the absence of an enforceable mandate that individuals undergo psychotherapy, the de jure effect of merely having psychotherapy available is negligible at best ... because individuals don't utilize it.
Bamboleo, bambolea
Porque mi vida, yo la prefiero vivir asi
In English, this translates to (according to Google anyway)
Bamboleo, bambolea
Because in my life, I prefer to live it that way
You can almost hear the music!
Well, there is something else to the song besides the uplifting beat; there are the lyrics. As the song says, individuals typically live life the way they prefer, not the way dictated from on high. Heeding this message is quite useful when it comes to ... the analysis of program or policy effects.
What does this have to do with the empirical analysis of programs or policies? I'm glad you asked. Well, it brings to the fore the important point that just because things are dictated in a certain manner does not preclude a different reality from emerging. Agents may, after all, may prefer to live it a different way.
In research, the terms de jure and de facto are used to capture these concepts. De jure is Latin for "by law," while de facto means "in fact." The difference between de jure and de facto is fairly well known, although perhaps not always given the important attention it deserves. The terms are invoked to highlight the conceptual difference between policies or regulations on paper ("by law") and policies or regulations as implemented ("in fact").
Understanding the causal effects of a policy by law and in fact are interesting and important, but answer distinct questions. The causal effect of a policy by law captures an intent-to-treat (ITT) effect. What is the effect of having on paper a particular policy, law, or regulation? This is arguably a policy relevant parameter since a policymaker can readily change the policy on paper. In addition to being perhaps more policy relevant, the de jure effect of a policy is also likely to be an easier parameter to estimate since it is typically easier to collect data on policies, laws, or regulations on paper since they are, you know, written down.
That said, the ITT effect of a policy may be of limited use for two reasons. First, it may lack external validity since the in fact implementation of a policy, law, or regulation in one location may not carryover to a new location. Second, and related, absence of an effect of a policy, law, or regulation by law may mask the fact that the policy, law, or regulation may be useful under a different in fact implementation.
The causal effect of the in fact implementation of a policy captures something different; the effect of the policy, law, or regulation actually put into practice. What is the effect of implementing a particular policy, law, or regulation? This is also a policy relevant parameter if a policymaker can influence the in fact implementation of a policy. However, in the absence of this ability -- because agents prefer to live it their way -- the effect of the policy, law, or regulation as implemented may be of little practical value. In either case, the de facto effect is likely to differ from the de jure effect. The de facto effect is also likely to be a difficult parameter to estimate in many situations in that implementation may be difficult to measure.
Note, while some discussion on Twitter devolved into which of these two effects is "more" important, the only sensible conclusion, in my view, is that they are both important because they answer different questions. The evidence of a positive effect of psychotherapy informs us that psychotherapy is beneficial when utilized ("in fact"). However, the structural evidence provided by the authors informs us that a policy of making psychotherapy readily available ("by law") has little to no effect since it is under-utilized.
To further highlight the important distinction between de jure and de facto effects, I conducted a small Monte Carlo simulation. The code is available here. The DGP is given by
D = I(u>0.5)
D* = D⋅I(w≥θ) + Z⋅I(w<θ),
Y = b0 + b1⋅D* + e
where u,w,Z~Unif(0,1) and e~N(0,1). Here, D is a binary indicator of the de jure presence or absence of a particular policy, law, or regulation (e.g., county-level nonattainment status under the Clean Air Act (CAA), which implies that more stringent environmental regulations are mandatory). D* is a continuous variable over the unit interval measuring the de facto implementation of the policy, law, or regulation. With probability θ, the de facto implementation deviates from the de jure mandate. Note, the deviation is allowed to occur in both directions (e.g., nonattainment counties may implement less stringent regulation than mandated under the CAA and in-attainment counties may implement more stringent regulation despite the lack of mandate). The outcome, Y, depends on the de facto implementation.
In my simulation, I treat D* as a latent variable and D as observed. I then estimate b0 and b1 by OLS, regressing Y on D. Note, since D* and D both span the unit interval, the OLS coefficient on D is comparable to the true value of the marginal effect of D*, b1.
I conduct 10,000 repetitions with N=1,000 for each value of θ = {0,0.1,0.2,0.3,0.4,0.5}. These are referred to as DGPs 0-5. The true values of b0 and b1 are set to one. The mean squared error (MSE), mean absolute error (MAE), and bias are reported below.
Not surprisingly, the greater the deviation between D and D*, the greater the deviation between the estimated de jure effect and the true, de facto marginal effect of D*.
The bottom line: music is to be appreciated on many, many levels. Oh yeah, and be sure to be explicit about the question being answered by your analysis. While both de jure and de facto information is useful, to make precise policy recommendations requires we know which we have estimated.
References
Cronin, C.J., M.P. Forsstrom, and N.W. Papageorge (2020), "What Good Are Treatment Effects without Treatment? Mental Health and the Reluctance to Use Talk Therapy," NBER WP #27711
