Lessons on Empirical Legal Research Through a Twombly/Iqbal Lens

Through the Civil Procedure and Federal Courts Blog, I learned of the existence of this article by David Freeman Engstrom a while back, but due to the hectic schedule of wrapping up work I was not able to read it until my flight back to Iowa.  Once I was finally able to read it, I found the article to be very informative and interesting, and I recommend that anybody who is interested in Civil Procedure, the ramifications of Twombly and Iqbal, or empirical legal research in general to read it.  The full citation is: David Freeman Engstrom, The Twiqbal Puzzle and Empirical Study of Civil Procedure, 65 Stan. L. Rev. 1203 (2013).

Here is the (mercifully short) abstract:

This essay, written for a Stanford Law Review issue exploring “The Empirical Revolution in Law,” offers a critical assessment of the large body of empirical scholarship examining the effect of the Supreme Court’s decisions in Bell Atlantic Corp. v. Twombly and Ashcroft v. Iqbal on judicial and litigant behavior and then uses the critique to make some broader observations about the past, present, and future of empirical study of civil procedure.

The article offers some good examples of empirical legal studies of the impact of Twombly and Iqbal -- the two cases that raised the standard of proof required in civil pleadings.  While many commentators theorize and lament that these cases would create a sea change in civil procedure, the satisfactory studies apparently show that while some changes have occurred, they are not as substantial as the changes forecasted or supposedly demonstrated by less-satisfactory studies.

I found the most informative portion of the article to be that beginning on page 1213 which detailed the missteps of commentators discussing Twombly and Iqbal's impact.  Among other problems, the article discusses sampling bias, failure to include control groups, and the danger of using Westlaw or LexisNexis in generating samples of cases for legal research.  I mentioned in this earlier post that I had come across some scholarship that appeared to make these errors, but Engstrom's article uncovers the depth and widespread nature of these errors in many of the published studies of these cases.

I think that this article, or in the very least, the portions of the article discussing the systematic errors made by scholars studying Twombly and Iqbal are very helpful for law journals' articles editors who may not be entirely familiar with what good empirical legal research looks like.  Engstrom tells a good cautionary tale, and provides an approachable list of missteps that should be identified and avoided, both by authors and by law journals.