David Roodman's Re-analysis Finds No Link Between Temperature and Judge Decisions, Criticizes Academic Publishing

Economist David Roodman has published a re-analysis challenging a prominent 2019 study that claimed a causal link between outdoor temperature and U.S. immigration judges' asylum decisions. His findings, detailed in a "replication opinion" and highlighted by Paul Novosad on social media, suggest the original correlation disappears upon correcting errors and expanding the dataset, prompting Roodman to assert that "consumers of economic research are sometimes more truth-seeking than the producers." This re-evaluation raises significant questions about academic rigor and the peer-review process.

The original paper, "Temperature and Decisions: Evidence from 207,000 Court Cases" by Anthony Heyes and Soodeh Saberian, published in the American Economic Journal: Applied Economics, concluded that a 10°F increase in case-day temperature reduced decisions favorable to asylum applicants by 6.55 percent. This finding suggested that warmer weather negatively influenced judicial leniency, even for judgments made indoors. The study garnered considerable attention for its implications on cognitive output and decision-making.

Roodman's re-analysis meticulously addressed methodological issues, including the original study's use of Greenwich Mean Time instead of local temperatures and its limited dataset. After correcting these discrepancies and incorporating a broader range of data, Roodman found that the purported link between temperature and asylum grants vanished. He dismissed the original authors' defense—that their conclusion held if Chinese asylum-seekers were excluded—as unpersuasive and potentially indicative of "p-hacking."

Paul Novosad's tweet, which sparked Roodman's deeper dive, expressed "pique with economics journals over how they handle challenges to published papers." Roodman echoed this sentiment, criticizing the American Economic Journal: Applied Economics for publishing the original authors' rejoinder as a "Correction," which he argued obscured the fundamental invalidation of the initial finding. This incident, according to Roodman, exemplifies a broader issue where journals may struggle to prioritize truth-seeking amidst other considerations like citation metrics or avoiding controversy.

Roodman advocates for the regular production of "replication opinions" to enhance post-publication review and provide independent assessments of disputed research. He contends that such external scrutiny could compel academic journals to improve their handling of research challenges, ultimately serving the public and decision-makers who rely on accurate scientific findings. The debate underscores the ongoing challenges in ensuring the robustness and reproducibility of economic research.