http://masteringmetrics.com/ ]]>

I’m reading the bad control section again and having trouble connecting the argument with the common practice of including intermediate outcomes as a way to test for the mechanism. Many papers (Nunn and Qian 2014, Chen, Kung and Ma 2017, etc) use the intermediate outcomes as controls: if the coefficient of the treatment variable changes in magnitude or significance when including these intermediate outcome, they claim the intermediate outcome is a potential channel of the effect. For example, by adding occupation to the regression between education and earnings, the coefficient is reduced. This would imply that the occupation is a potential explanation of education’s effect on earnings.

The method makes sense intuitively but the papers using them do not provide econometric theories to support. I’m wondering what’re your thoughts on this. Thanks!

Jie

]]>-JA ]]>

It’s worthwhile to note that you can end up with significant biases when you have in mismeasurement of the independent variable as well as the dependent variable.

]]>In a completely randomized experiment with a binary outcome, if you want to adjust for covariates to improve precision, you can use either logit (with an average marginal effect calculation) or OLS to consistently estimate the average treatment effect, even if your model’s “wrong”. Probit doesn’t enjoy this robustness property.

The first-order conditions for OLS and the logit MLE imply a nice property: if you compute an “untreated” predicted probability for each person, using her actual covariate values but setting the treatment dummy to 0, then the average “untreated” prediction in the control group equals the raw control mean. In large enough samples, this will be very similar to the average “untreated” prediction in the full sample (since the distribution of covariates in the control group will resemble the distribution in the full sample). The latter is a regression-adjusted control mean. So we have an adjusted control mean that enjoys the same consistency properties as the raw control mean.

Similarly, we can compute a “treated” predicted probability for each person, and the resulting adjusted treatment group mean enjoys the same consistency properties as the raw treatment group mean. So the difference between the adjusted treatment and control group means is consistent for ATE. None of this depends on the model being correct.

The probit MLE first-order conditions don’t imply the same nice property.

David Freedman gave a rigorous proof for logit in “Randomization does not justify logistic regression”. (The negative message is that you can’t just use predictions at the mean covariate values, and the coefficient on treatment doesn’t estimate anything meaningful if the model’s wrong. But diehard MHE fans already know that.)

Freedman also briefly discussed probits:

“On the other hand, with the probit, the plug-in estimators are unlikely to be consistent, since the analogs of the likelihood equations (16–18) below involve weighted averages rather than simple averages. In simulation studies … Numerical calculations also confirm inconsistency of the plug-in estimators [average marginal effect estimates from the probit], although the asymptotic bias is small.”

A couple other references:

D. Firth and K. Bennett (1998). “Robust models in probability sampling.” JRSSB 60: 3-21.

J. Wooldridge (2007). “Inverse probability weighted estimation for general missing data problems.” J. Econometrics 141: 1281-1301. (See section 6.2.)

]]>