Citations, Hierarchies, and Algorithms in Legal Scholarship

I read Brian Frye's Techdirt article, "It's the End of Citation As We Know It & I Feel Fine," where he makes the bold claim that the "worst thing about legal scholarship is the footnotes." In a field where article lengths are expanding to 100 pages and beyond, where costly submission software crowds out students and professionals who are outside of the academy, and where journals will publish anything--even ridiculous articles about shooting fish with guns, there is serious competition for the title of what aspect of legal scholarship is "the worst."

I won't deny that footnotes in legal scholarship can be a bit overwhelming for the uninitiated, and that some editors demand citations for everything under the sun. But I'm not sure that the footnote craze is as horrible as Frye makes it out to be. To start, I suspect that the focus on footnotes originates, at least in part, with the legal writing that many law students will go on to prepare in practice, in which arguments referencing cases and statutes require frequent citations to support the claims being made. To the extent that law review articles--particularly those with a more doctrinal focus like 50-state surveys of laws governing the shooting of fish with guns--include citations to case law or statutes, those footnotes should be encouraged to confirm that the legal claim being made has a basis in legal authority. 

Even when citations are to scholarly, rather than legal, authorities, frequent footnotes can be helpful. They maybe a resource for those doing research in the area to find related scholarship on particular issues. They can serve as a substitute for literature reviews, reducing the length of what may already be a too-long piece. Footnotes to scholarship may also provide a barometer as to the legitimacy of claims being made. Overreliance on single sources, or--God forbid--citations to one's own work may undermine claims that are presented as well-established. Finally, numerous, repetitive, and useless footnotes, (I'm thinking especially of introduction footnotes beginning with "See infra Section __"), should be dealt with by the author, who can refuse to include such footnotes in the initial draft and who can (and should) push back on editors who demand such useless additions.

Frye turns to a discussion of ScholarSift, a platform created by Rob Anderson and Trent Wenzel, that purports to analyze legal scholarship to "identify the most relevant articles." From what little I can find out about ScholarSift, people can upload an article (either a draft, completed, or already-published article) and the system locates "relevant" articles based on analysis of the article's text and citations. 

Frye suggests that ScholarSift could be used as a substitute for footnotes by finding sources that are similar or relevant to the text being analyzed. This does not seem feasible, as the software appears to be built around connecting authors to similar, or "relevant" sources based on the whole of a draft. It does not appear that the system is designed to connect one particular statement or proposition in an article to a source (or sources) that support that statement--instead, it generates a list of "related" articles (and, I think, cases, laws, and maybe books) that are "relevant" to the article as a whole. Replacing footnotes with this program would be similar to a law review article listing a bibliography at the end and telling the reader to look through all the sources to confirm whether the article's contents are accurate. As much work as sorting through footnotes may be, this approach sounds like much more of a burden.

(I admit that I do not have a ScholarSift account--which you apparently can only get by submitting a request by email. If my description of how the system's use of submissions to generate results is therefore incorrect, I welcome corrections.)

But Frye's discussion of ScholarSift raises some interesting notions about how it may assist in legal research help legal scholarship as a whole.  Frye writes:

It works really well. As far as I can tell, ScholarSift is kind of like Turnitin in reverse. It compares the text of a law review article to a huge database of law review articles and tells you which ones are similar. Unsurprisingly, it turns out that machine learning is really good at identifying relevant scholarship. And ScholarSift seems to do a better job at identifying relevant scholarship than pricey legacy platforms like Westlaw and Lexis.

 

One of the many cool things about ScholarSift is its potential to make legal scholarship more equitable. In legal scholarship, as everywhere, fame begets fame. All too often, fame means the usual suspects get all the attention, and it’s a struggle for marginalized scholars to get the attention they deserve. Unlike other kinds of machine learning programs, which seem almost designed to reinforce unfortunate prejudices, ScholarSift seems to do the opposite, highlighting authors who might otherwise be overlooked. That’s important and valuable. I think Anderson and Wenzel are on to something, and I agree that ScholarSift could improve citation practices in legal scholarship.

I'm a bit less optimistic than Frye about ScholarSift, largely because I cannot find any information on how it works, it is unclear what database of documents ScholarSift pulls from, and I am concerned that to the extent that it relies on a database of legal scholarship, the hierarchical problems that Frye identifies in his article may still be imported into results. 

Regarding the lack of information, ScholarSift's website contains virtually no information about how the system operates. I have not been able to locate additional written information on ScholarSift anywhere else--although my search for such information was admittedly a cursory one. I located, and listened to, this Ipse Dixit podcast interview of Rob Anderson who describes how ScholarSift works. But, as is the case with the website, the information is presented in largely conclusory terms--describing how the system will "look at" the text and citations of an article uploaded to it and analyze "relationships" between other articles to sift through a database of "a few hundred thousand articles" and list results in an order of what is most "closely-related." It remains unclear how "relevance" or "closely-related" determinations are made, although it appears that this is done through an analysis of the text, including commonly used words, phrases, and maybe combinations and/or proximities of words or phrases to one another.

The makeup of ScholarSift's database of articles and sources is also unclear. The contents of the database, how determinations are made on what to include, and the age of what is included are all mysterious. On the podcast, Anderson notes that scholars using the platform can upload their own drafts or articles to ensure that they are part of the database, but I expect this would only account for a small portion of what makes up the database. Without more information on the database, its contents, and how its contents are selected, it is impossible to conclude that ScholarSift can conduct exhaustive searches of potentially relevant material.

Finally, Frye and Anderson note that ScholarSift may help break down hierarchy problems in legal academia, where big names from prestigious institutions tend to be overcited, and where the body of scholarship consists largely of articles written by white, male authors. If the processes for locating "relevant" article truly focus on an article's text, perhaps the platform will have some impact. But I have my doubts. 

First, the platform is meant to analyze articles and their citations in locating relevant results (although this will supposedly change as the system develops). If citations are included as inputs, though, they will influence the searches, and authors' biases in selecting their own sources will likely be reflected in the results. 

Second, related to my concern about the database's contents, ScholarSift presumably draws from a body of scholarship in legal academia, in which white male authors are overrepresented (especially if historical writing is taken into account). Even if determinations of relevance are based on the text of submissions and articles in the database, this will still result in results skewing towards white males if they make up the bulk of what is included in the database. 

Third, Anderson notes that there will be features permitting searchers to filter results in various ways, including by high citation rates. This suggests that the system at least includes information on articles' citation rates, and this information may influence what results are deemed "relevant"--which may in turn continue to perpetuate hierarchies of overciting authors from the most prestigious institutions. This last concern is, admittedly, speculative, but without information on how the algorithm works, this concern shouldn't be dismissed. Additionally, as the program develops (and, especially, if it is used to generate a profit), there may be pressure to prioritize results that account for the "prestige" of an author or publications--which could defeat the very purpose of the platform.

If ScholarSift is truly text-focused (as Frye describes, "Turnitin in reverse"), it may have a positive impact on legal research and lead to increased diversity in citations. Hopefully that will be the case. And Anderson notes that ScholarSift may end up being an alternate mean for submitting articles to law reviews--an outcome I would be happy to see. I think it is still early days and that it is too soon to be overly optimistic. But ScholarSift may be worth watching and including as one of many tools used by authors of legal scholarship. 

Michigan Court of Appeal Issues Strong Decision Restricting Government Drone Use

I learned from Greg McNeal's Twitter thread (and related blog post) of an interesting opinion issued by the Michigan Court of Appeals last week in which the court found that the use of a drone by a township to take aerial photographs of private property violated the property owners' reasonable expectations of privacy.

The case is Long Lake Township v. Maxon. A link to the opinion is here, a link to the dissent is here.

Summary of the Case

The township filed a civil action against the property owners, arguing that their property constituted an "illegal salvage or junk yard" in violation of town ordinances. The township relied on aerial photographs of the property taken in 2010, 2016, 2017, and 2018 that purportedly showed an increase in junk stored on the property. The township had used a drone to take the photos.

The property owners moved to suppress the photos, arguing that they had been obtained through an unlawful search in violation of the Fourth Amendment. In response, the township argued that the drone operator it had hired to take the photos had done so in a manner consistent with FAA regulations--noting that they had maintained a consistent line of site with the drone and that it was lower than 400 feet at all times. (A link to the FAA regulations that the township appears to be referencing is here, an FAA summary of the rules is here).

The court recognized that Michigan has a law that restricts individuals from using drones to capture photos, video, or audio recordings of other people in a manner that would invade others' "reasonable expectation of privacy."

The court first noted that in Kyllo v. United States, the United States Supreme Court had found that the use of a thermal imaging device on a home violated the Fourth Amendment. In Kyllo, the Court concluded:

Where, as here, the Government uses a device that is not in general public use, to explore details of the home that would previously have been unknowable without physical intrusion, the surveillance is a "search" and is presumptively unreasonable without a warrant.

The Michigan Court of Appeals took pains to note that Kyllo's holding was not premised on whether the technology used was widely available--noting that other language in Kyllo stated that homeowners should not be "at the mercy of advancing technology" that might be able to see through walls.

The court then considered the cases, California v. Ciraolo (no Fourth Amendment violation where police conducted aerial observation of a defendant's yard from 1,000 feet up) and Florida v. Riley (aerial observation of greenhouse from 400 feet up did not constitute a Fourth Amendment violation). The court emphasized Justice O'Connor's concurrence in Riley, where she recognized that the plurality of the Court relied on FAA regulations governing how high the helicopter could fly, and concluded that compliance with FAA regulations alone does not determine compliance with the Fourth Amendment. Instead she found that "considerable public use of airspace at 400 feet and above" meant that Riley did not have a reasonable expectation that the curtilage of his property was protected from naked-eye aerial observation from 400 feet.

The majority of the Michigan Court of Appeals concluded that the use of the drone by the township was closer to the thermal imaging device used in Kyllo than the aircraft overflights in Ciraolo and Riley. The court noted that FAA regulations were relevant in this case because they demonstrated that drones were "qualitatively different from airplanes and helicopters," as they are smaller, more maneuverable, stealthy, and fly at lower heights. 

The court did not reach the determination of whether a drone overflight was trespassory, finding it unnecessary to do so. Instead, the court found that the determination that the drone use violated reasonable expectations of privacy was enough to establish a Fourth Amendment violation. The court also refused to craft a precise altitude test, instead concluding that "persons have a reasonable expectation of privacy in their property against drone surveillance, and therefore a governmental entity seeking to conduct drone surveillance must obtain a warrant or satisfy a traditional exception to the warrant requirement."

In a dissenting opinion, Judge Hood took issue with the majority's blanket distinguishing of Ciraolo and Riley. Recognizing that drones are smaller, Judge Hood noted there was no evidence that the photos captured were different than those that could be taken from a plane or helicopter at that altitude. Judge Hood further noted that the drone was flying in airspace in which the public could use drones, and emphasized that Kyllo involved technology that was not in general public use--not the case with drones which are generally available for people to use.

Analysis

Years ago, I wrote an article surveying state laws restricting law enforcement drone use. One of my main conclusions was that state-level litigation governing how law enforcement agencies could use drones and the evidence obtained from these drones, was necessary. Absent specifically-drafted legislating that put meaningful restrictions in place, the government would likely engage in broad drone use in light of Ciraolo and Riley. Such broad use could be restricted by laws, or by courts who could craft exceptions to the existing Fourth Amendment case law.

I concluded that leaving it in the hands of courts to carve out restrictions for government drone use (either through passing no laws or passing laws that simply reaffirm Fourth Amendment protections) was not the best approach, and that both privacy advocates and law enforcement advocates should support legislation. Legislation would permit evidence-based restrictions on particular types of drone use and leave law enforcement open to use drones in a manner that balanced potential privacy intrusions with the utility of new technology. These restrictions could be amended as technology develops. Courts, however, would not be in a position issue such carefully crafted restrictions, and would likely be faced with the options of broadly permitting government drone use, or stretching Fourth Amendment case law to craft restrictions that could not be amended or addressed by state legislatures.

This case is an example of the latter outcome. As far as I can tell, Michigan law does not restrict its law enforcement agencies' use of drones. Instead, the most the court cited was a law restricting private drone use--a law couched in broad language based on the Fourth Amendment concept of "reasonable expectations of privacy." As I argue in my article, laws like this provide no guidance for courts beyond what is already set forth in the case law, and are therefore have little to no practical impact. 

In the face of this dearth of meaningful restrictions, the court decided to craft a broad restriction of its own--a prohibition on all warrantless drone surveillance of private property. Law enforcement agencies in Michigan must now obtain a warrant before using a drone in this manner, or use a helicopter for such surveillance. This case is based on the Fourth Amendment, so the state legislature cannot write any laws to restrict the scope of the opinion. The only thing the government can do to try to undo this outcome is to appeal to the state supreme court.

Privacy advocates may cite this case as an example of why laws restricting government drone use are unnecessary. That would be a mistake. The dissenting opinion illustrates how courts could just as easily reach the opposite conclusion and permit government drone use. FAA regulations (which have developed since I wrote my article) now permit fairly flexible drone use--including altitude restrictions of below 400 feet--which may permit much closer imaging of private property. I suspect that other courts in similar situations may rely on Ciraolo and Riley to permit similar drone use.

In civil cases, a strong sign that you've reached a fair settlement is that both sides are unhappy with the outcome. The same is the case with drone restrictions. Privacy advocates would likely take a strong view of Fourth Amendment protections and would consider a legislative restriction approach to be a needless compromise. Law enforcement advocates would not want their ability to use new technology to be curtailed by laws or regulations. But using legislation to curtail specific restrictions on government drone use is the only way to balance the nuances of new technology with the blunt instrument of Fourth Amendment decision-making. Long Lake Township illustrates how failing to take this approach can come down against law enforcement interests, but its dissent should be just as much of a warning to those concerned with law enforcement overreach.