We learn something new (and useful!) every day . . .
Macartan Humphreys of Columbia University has shown why regression estimates of treatment effects can often be expected to fall between the average effect on the treated and the average effect on controls. His theorem goes like this: Let D denote treatment, let p(X) denote the propensity score E[D|X], and let M(X) denote the covariate-specific treatment effects, E[Y1-Y0|X]. Suppose that M(X) varies in a monotone way with p(X) (either weakly increasing or weakly decreasing). Then OLS estimates of the treatment effect in model using saturated control for covariates (i.e., the sort of regression discussed in Section 3.3.1 of MHE) will lie between E[Y1 – Y0| D=1] and E[Y1-Y0| D=0]. Read all about it in his working paper.
Why is a treatment effect likely to be monotone in the propensity score? This happens in the Angrist (1998) study of the effects of military service because those who benefit the most from military service are least likely to be qualified and therefore least likely to be treated. In other cases, where self-selection is more important than qualifications (as in the Roy [1951] model), those most likely to benefit from treatment may be the most likely to get treated. Either case is fine as long as it’s one or the other.
Why is this useful? It’s one more reason why OLS is a good summary statistic for program impact. Check out this figure from Macartan’s paper, which illustrates the OLS-is-in-between property using the Angrist (1998) data:
Figure 3 from Humphreys (2009)
The figure shows how OLS estimates of the effects of voluntary military service are almost always between matching estimates of effects on veterans and matching estimates of effects on non-veterans. This happens because covariate-specific estimates of veteran effects are either unrelated to the propensity score or they are a weakly decreasing function of the propensity score.
OLS is between the effect on the treated and the effect on controls
We learn something new (and useful!) every day . . .
Macartan Humphreys of Columbia University has shown why regression estimates of treatment effects can often be expected to fall between the average effect on the treated and the average effect on controls. His theorem goes like this: Let D denote treatment, let p(X) denote the propensity score E[D|X], and let M(X) denote the covariate-specific treatment effects, E[Y1-Y0|X]. Suppose that M(X) varies in a monotone way with p(X) (either weakly increasing or weakly decreasing). Then OLS estimates of the treatment effect in model using saturated control for covariates (i.e., the sort of regression discussed in Section 3.3.1 of MHE) will lie between E[Y1 – Y0| D=1] and E[Y1-Y0| D=0]. Read all about it in his working paper.
Why is a treatment effect likely to be monotone in the propensity score? This happens in the Angrist (1998) study of the effects of military service because those who benefit the most from military service are least likely to be qualified and therefore least likely to be treated. In other cases, where self-selection is more important than qualifications (as in the Roy [1951] model), those most likely to benefit from treatment may be the most likely to get treated. Either case is fine as long as it’s one or the other.
Why is this useful? It’s one more reason why OLS is a good summary statistic for program impact. Check out this figure from Macartan’s paper, which illustrates the OLS-is-in-between property using the Angrist (1998) data:
Figure 3 from Humphreys (2009)
The figure shows how OLS estimates of the effects of voluntary military service are almost always between matching estimates of effects on veterans and matching estimates of effects on non-veterans. This happens because covariate-specific estimates of veteran effects are either unrelated to the propensity score or they are a weakly decreasing function of the propensity score.