Lawyers: Objecting to Predictive Coding Is Futile

While technology-assisted review isn't for every case, judges are becoming savvy about its advantages.

, Law Technology News

   |4 Comments

Judge receiving paper

Predictive coding is a process where lawyers use technology to help identify documents within a collection that are relevant or privileged. The technology acts as a force-multiplier where the judgment of a senior attorney can be extrapolated to a larger document set using predictive coding technology. If done correctly, predictive coding can do a better job than eyes-on manual review. Like eyes-on manual review, predictive coding can do a poor job if the human judgment driving the predictive analytics is flawed. 

The process itself isn’t that far off of an email spam filter. About 70 percent of email traffic in the world falls into the spam bucket, i.e., unsolicited commercial emails. Your spam filter learns how to automatically identify these garbage emails through human training. You have likely participated in the rough equivalent of predictive coding in training your spam filter—think of the “this is spam” button you may have clicked in the past.

Most spam filters do a decent job of sorting the wheat from the chaff, although occasionally one or two spam emails make it through the filter. The technology itself is not revolutionary or groundbreaking, but its application in the legal context was new in the not-too-distant past.

Predictive coding does not work well in every case. Graphics, audio and video do not jive well with the technologies I am familiar with (caveat: predictive coding analytics are getting more sophisticated as time progresses and not all tools are created equal). Plus, given the effort it takes to get it right, human eyes may be cheaper in small-document cases. If you have a thousand or so documents in your universe, it is still easiest to just look at them all. But predictive coding is a new default approach in the range of standard options for handling cases with high volumes of text-searchable documents.

Much has been written about the defensibility of the predictive coding process. When predictive coding began to see more widespread use a few years back, litigants occasionally sparred over the use of the technology in lieu of straight eyes-on-every-document review. As Law Technology News wrote in 2011, counsel were “left to wonder if predictive coding meets the standard of ‘reasonableness.’ [I]t may be just a matter of time before such methodology is as routinely discussed as contract attorneys and keywords are now.” Then, the first judicial opinion endorsing the use of predictive coding came in Da Silva Moore v. Publicis Groupe, 287 F.R.D. 102 (S.D.N.Y. 2012).  Da Silva Moore was a lengthy and hotly contested case where the court dug deeply into the inner workings of the predictive coding process.

Some say that evolving judicial attitudes toward predictive coding suggest it may be time to retire the defensibility question of the predictive coding process. A recent, and concise, opinion from the Business Litigation Superior Court in Boston indicates the question of defensibility has become a footnote point, literally.

Since Da Silva Moore we have seen a number of concise opinions or excerpts of state court judges accepting the concept of predictive coding as the norm. A Virginia state court endorsed the use of predictive coding, over strenuous objection, in a partially handwritten order in Global Aerospace Inc. v. Landow Aviation, L.P., 2012 Va. Cir. Lexis 50 (Va. Cir. Ct., April 23, 2012).

In the Southern and Northern Districts of New York, Judge Lewis Kaplan and Magistrate Judge Randolph Treece have both cited the availability of predictive coding as part of their analysis in rejecting undue-burden objections to discovery requests. Judge Denise Cote noted in a recent mega-litigation, Federal Housing Finance Agency v. JP Morgan Chase & Co. Inc., that “predictive coding should be given careful consideration in a case like this, and I am absolutely happy to endorse the use of predictive coding and to require that it be used as part of the discovery tools available to the parties.”

The evolving attitude seems to be that predictive coding is presumptively reasonable—a presumption that the human eye and brain (perhaps undeservedly) currently enjoy. The majority of our judiciary has proved aware, at least conceptually, of predictive coding and its potential application in litigation.

Associate Justice Thomas Billings’ decision from the Business Litigation Superior Court in Boston is another concise state court opinion addressing the use of predictive coding, in the vein of Global Aerospace. The decision is short, coming in at barely two double-spaced pages. It notes, in the only footnote, that “The Court has already approved, in principle, the use of predictive coding to accomplish this task.” This is likely the first instance predictive coding has been allowed in Boston’s business litigation section, but it is more significant in the implicit acceptance of the process drawn from its brevity.

What's being said

  • not available

    While courts are becoming more accepting of predictive coding as a concept, its use on a given matter is still subject to legal, procedural, technical, and strategic hurdles. As repeated surveys have demonstrated, predictive coding is still rarely used for "predictive coding". The uncertainty and risk associated with the preseent immaturity of predictive coding methodologies have kept this technology largely limited to filtering and QC.

    The potential challenges to the use of predictive coding on a given case are both lengthy and substantive. As someone who has worked to encourage the adoption of advanced decision analytics and optimization in other industries, I‘ve found that acknowledging technical and procedural issues and addressing them directly through methodology is the best approach to accelerate its acceptance. To imply that predictive coding is "mainstream" or applicable to cases involving as few as 10,000 documents leads to unsustainable expectations and industry backlash.

  • not available

    What exactly are judges becoming "savvy" about? There is no mention of judges sitting with programmers or developers. They are just becoming more accepting. I recently sat in a discussion where a judge described how decisions had been made regarding predictive coding thresholds in one of his cases soley using F1, then one of the statisticians that developed F1 told him they had used it incorrectly. Does that sound like "savvy"?

    http://blog.cluster-text.com/2013/05/08/predictive-coding-performance-and-the-silly-f1-score/

  • not available

    But it is also fervent to be “absolutely happy to endorse…predictive coding” and to “require it be used” when it is only described as “a process.”

  • not available

    Yes, it is futile to discuss objecting to Predictive Coding when it is only described as “a process.”

Comments are not moderated. To report offensive comments, click here.

Preparing comment abuse report for Article# 1202644426053

Thank you!

This article's comments will be reviewed.