Evaluating policy changes that occur everywhere at the same time is difficult because of the lack of a clear counterfactual. To address this problem, researchers often proxy for differential exposure using some observed characteristic. In an influential paper, Melissa Kearney and Philip Levine examine the effect of the MTV program ``16 and Pregnant’’ on teen birth rates, using the pre-treatment levels of MTV viewership across media markets as an instrument. This is essentially a difference-in-differences approach. We show that controlling for differential time trends in birth rates by a market’s pre-treatment racial/ethnic composition or unemployment rate cause Kearney and Levine’s results to disappear, invalidating the parallel trends assumption necessary for a causal interpretation. Extending the pre-treatment period and estimating placebo tests, we find evidence of an “effect” long before 16 and Pregnant started broadcasting. We also reassess Kearney and Levine’s social media evidence and show that it is fragile and inconsistent, casting substantial
doubt on the hypothesized causal mechanism. Our results strongly suggest that the identifying assumptions necessary for a causal interpretation of the effect of ``16 and Pregnant’’ on teen birth rates are not met and highlight the difficulty of drawing causal inference from national point-in-time policy changes.
|
