Publications
Embrace Your Fallibility: Thoughts on Code Integrity
The Political Methodologist, June 2016
PDF Download
It’s natural to think that the reason we find problems in the code behind published papers is carelessness or inattention on behalf of authors, and that the key to minimizing problems in our code is to be more careful. The truth, I have come to believe, is more subtle: humans are effectively incapable of writing error-free code, and that if we wish to improve the quality of the code we write, we must start learning and teaching coding skills that help maximize the probability our mistakes will be found and corrected. In this piece I present an overview of specific programming practices we can all implement, both individually and in collaborative projects. I also offer a few reflections on the implications of this idea for third-party review of code by academic journals and for how the discipline should respond to errors in published work.
Lessons from a Decade of Replications at the Quarterly Journal of Political Science
PS: Political Science and Politics, April 2016
Based on post for The Political Methodologist awarded Most Viewed Post of 2015
To allow researchers to investigate not just whether a paper’s methods are theoretically sound, but also whether they have been properly implemented and are robust to alternative specifications, it is necessary that published papers be accompanied by their underlying data and code. This paper details the experience and lessons learned at the Quarterly Journal of Political Science since it began requiring authors to provide this type of replication code in 2005. It finds that of the 24 empirical papers subject to in-house replication review since September 2012, only 4 packages required no modifications. Most troubling, 14 of these packages (58 percent) had results in the paper that differed from those generated by the author’s own code. Based on these experiences, this paper presents a set of guidelines for authors and journals for how to improve the reliability and usability of replication packages.
Resources
Defensive Programming Presentation and Exercises
Beamer presentation on defensive programming with associated exercises and solutions.
Writing Tests for Stata: A Tutorial
Examples of the types of integrity tests suggested in Embrace Your Fallibility for Stata.
Writing Tests for R: A Tutorial
Examples of the types of integrity tests suggested in Embrace Your Fallibility for R. Note I have more experience in Stata, so if you see problems with the R tutorial, please flag it for me!
Exporting Results from Stata to LaTeX
Tips on exporting results directly from Stata for import into LaTeX with no transcription or hand copying.