#CaseoftheWeek Episode 90: Compelling Defendants to Use Additional Search Terms After Use of TAR

The Case of the Week’s episode 90 is an analysis of the latest installment in the price fixing class action matter, In re Diisocyanates Antitrust Litig., MDL No. 2862 (W.D. Pa. 2022). The decision is from October 19, 2022 by Special Master Francis IV, James C. (Ret.). We’ll be reviewing whether plaintiffs can compel defendants to use additional search terms following production of documents using TAR.

Keep reading or watch the video to understand the eDiscovery issues.

Introduction

Good morning and welcome to Episode 90 of our Case of the Week series, published in partnership with ACEDS. My name is Kelly Twigger, I am the founder and CEO of eDiscovery Assistant, as well as principal at ESI Attorneys. Thank you so much for joining us today.

As always, I’d like to keep you informed of events going on that may be worth your time. I mentioned to you previously that the University of Florida [Levin College of Law] eDiscovery Conference has moved from March to February as a result of the change in the timing of Legalweek. The registration for 2023 conference is now live.

I will be moderating the case law panel at that event and I’m excited to bring that information to you. As always, that conference will have limited in-person availability, but will be free to attend virtually. Last year, we had about 10,000 attendees, and so get your registration in, so we can put on a great show for you.

All right, let’s dive into our decision from the In re Diisocyanates Antitrust Litigation. This one comes to us from Special Master, a former retired judge, James Francis. This decision is dated October 21st of 2022.

Background

All right, let’s dive into the facts of the case to drill down into what Judge Francis talks about here. I’m going to recommend this decision to you to read if you’re involved in TAR at all or in large scale class action cases where you’ve got huge volumes of documents and one party that bears the bulk of the responsibility in providing discovery. The decisions in this case are extremely instructive.

This is an antitrust price fixing class action for two chemicals used in the manufacture of polyurethane. In order to provide responsive information, the defendants used search terms to identify documents and then TAR to review them, they then produced responsive information.

Now, you’ll recall from a couple of weeks ago we had a case on the Case Of The Week series in which we discussed whether search terms and TAR could both be used in the same litigation. Here, that’s not the issue — this is the process that the parties agreed to. You also recall from that discussion that I vehemently disagreed with the Court’s determination that the two did not go hand in hand.

In this particular case, we’ve got a long history of disputes between the parties on search terms that would be used to determine the initial set of documents reviewed used via TAR, as well as the use of TAR. Following multiple decisions in this case, the special master recommended—and the Court approved—that the defendants could choose the search terms and the plaintiffs would have the ability to challenge them following production, if in fact, those productions were deemed deficient.

In those previous decisions, the Court had ordered that once the defendants believed that their search was complete and they had reviewed all the documents following the TAR process, they would do the following: provide the plaintiffs the base number of all the relevant documents obtained from the last two batches searched, identify which of those batches those documents were found in, and identify the number of relevant but privileged documents withheld with respect to each of those two batches, as well as the recall rate and all calculations used to derive that rate.

Essentially, that is the validation process that the Court provided for the defendants to give to the plaintiffs to determine whether or not what they had done in coming up with search terms to create the corpus of the documents and then utilizing the TAR process to review them was sufficient and reasonable under their obligations in discovery. If the plaintiffs agreed following that validation process, then the defendants were done. If not, the parties were required to meet and confer, and submit any disputes to the Special Master.

Well, following that determination, the parties again filed motions, and the Special Master again upheld the District Court’s ruling that the defendants could choose search terms. But Judge Francis did note that the plaintiffs would “have an opportunity to challenge both the search terms selected by the defendants and the point at which defendants chose to halt the TAR process based on the validation information that the defendants had committed to providing.” That second round of motions really just reinforced what the Court and the Special Master had already determined the parties should be abiding by.

Following those two sets of motions, the defendants completed their productions and provided validation letters to the plaintiffs that provided the following: the bates numbers of the responsive documents identified in the penultimate and final batches of the TAR documents that were reviewed; the results of an elusion test, including the number of responsive documents identified and the bates numbers of those documents; and the estimated recall for each defendant’s search.

In addition, the defendants provided the results of the elusion test in which they sampled 2,400 documents drawn from the null set, and the unreviewed TAR population. Those were documents that were included in the TAR review set but were not subject to human review because the TAR algorithm did not give them a high enough classification score. Following that sampling, the defendants identified by bates number those documents from the null set and from the unreviewed TAR population that they had identified as responsive when conducting the elusion test.

Essentially, they went through their entire process, then they looked at the last two batches of documents, as well as a sampling of the null set, an unreviewed TAR population, and provided that information to plaintiffs in order for plaintiffs to determine whether they’ve met their discovery obligations.

Based on that information, as well as the recall rates for the entire review, those validation letters from the defendants stated that they believed it was appropriate to end the TAR review. Of course, because we’re on a motion here, the plaintiffs disagree. In disagreeing with them, they provided specific objections to each of the defendants.

Because they didn’t get anything further that satisfied plaintiff’s objections, they filed this motion to compel challenging two aspects of the defendants’ production, the first being that the search terms that the defendants used were too limited and that they left thousands of documents out of the review process altogether. And second, that the percentage of responsive documents for three of the defendants was too high to stop the review. Those responsiveness rates were at 19%, 18%, and 15%.

On the search term issue, the plaintiffs identified three separate groups of documents that were key to the consideration of whether the search terms were appropriate. Group 1 was search terms that the defendants refused to run at all; group 2 included search terms where the defendants ran a narrower set of terms than those that were proposed; and group 3 included terms to identify meetings and communications between the defendants that are particularly relevant in the antitrust price fixing case. With regard to Group 3, the plaintiffs offered to forego those if the defendants would produce their full calendars. Some of the defendants agreed to do that so that group is not as much an issue on this motion.

With regard to the percentage of responsive documents, the plaintiffs also identified documents that were in the last two TAR batches and the unreviewed batch that they believed were relevant information that reflected communications between competitors, customer complaints, and documents about pricing, swaps, and supply — all of which would be relevant to antitrust claims.

The defendants made four arguments in response to plaintiffs motion. First, that the Court had already decided that the defendants could use the search terms that they had developed. Second, that they had demonstrated the reasonableness of their searches both in terms of quantity and quality by providing recall rates of 73%-89%. Third, that the decision of each defendant to halt its TAR review was supported by the statistics, and that since the percentage of responsive documents located in the last two batches reviewed ranged from 1.3%-19.58%. And fourth, that the any further requirements to produce documents were disproportionate in cost and burden as the defendants have disbanded their review teams when they completed their production.

Analysis

That’s where we stand on our motion to compel, what is the Court’s analysis? Well, as always, Judge Francis, as the Special Master, starts with the applicable legal standards. He notes that

It is common ground among the parties that a producing party must take reasonable steps to identify and produce relevant documents, and that perfection is not required. But ‘reasonable’ is an elastic term, and the parties disagree about its application to the defendant’s search and production in this case.

This is perhaps the most key part of this decision from Judge Francis and what I want you to pay attention to as a takeaway: Judge Francis notes here that reasonableness depends in part on the substantive law of the case, because that law informs the types of documents that are relevant, important, and in turn, dictates the search methodologies that may be appropriate. That is your key takeaway from today. You want to make sure that what you are focusing on is the process for this case is what’s important, considering the evidence that needs to be determined.

Now, as the Judge notes, there are two categories of relevant evidence here on an antitrust price fixing class action. They’re very key because they essentially require opposite considerations in analyzing whether or not the defendants have met their burden in discovery.

The first one is smoking gun evidence. In an antitrust price fixing class action, that smoking gun evidence would be that the defendants had entered into any kind of agreement in violation of the antitrust laws. For example, a recorded phone call in which two competitors agreed to fix prices at a certain level could be direct evidence of that conspiracy and the type of smoking gun that we should be looking for in discovery.

Direct evidence could also consist of an explicit admission. A statement from one of the cited cases that the Court refers to, in which one of the defendants plant managers stated that, “We have an understanding within the industry not to undercut each other’s prices.” That’s the kind of smoking gun evidence that we might be looking for in an antitrust case, and that needs to frame the discussion of what’s appropriate here in terms of process with search terms and TAR.

However, the Court notes that smoking guns are rarely found in antitrust conspiracy cases. As a result, it’s going to be highly unlikely that they would be found in any set of review documents that are going to be collected by the defendants.

The Court goes on to talk about the balancing act that is required in this type of situation:

It is critical not to exclude the possibility of locating such evidence by, for example, utilizing search terms that do not include words that might reflect a conspiratorial communication. The challenge is to do so without making the term so broad that they encompass vast numbers of irrelevant documents. At the same time, broad validation statistics, such as recall, standing alone are of limited utility and ascertaining whether a party has done a reasonable job of searching for such rare documents.

That’s what we’re talking about with regard to these smoking guns. We have to do the broad reach that we’re required to do in discovery, but we need to make sure that we’re getting to the specificity of terminology that the parties might use to be able to identify that smoking gun.

The second type of evidence that the Court notes is key in antitrust cases is circumstantial evidence and the potential inferences that can be drawn from it in order to prove a conspiracy to fix prices. Now, the challenge with circumstantial evidence is, how do you cast a wide enough net to get the evidence that you need to create a critical mass of evidence necessary for an antitrust plaintiff to meet its burden?

That’s essentially a completely different notion than what we talked about with the smoking gun. With the smoking gun, you’re looking for key words, key phrases that you can identify to help you find that. In more of the circumstantial evidence, you’re looking for a wide enough net to be able to gather all of that information so that it can be compiled to build a case for the plaintiffs.

Those are really competing strategies — one, where you want to find something specific, and two, where you want to cast a wide net — both of which have to be balanced regarding the party’s obligations to provide reasonable discovery and the limits of proportionality.

Now, the Court notes that the adequacy of a search has to be assessed in terms of both quantity, i.e. how the search scores with respect to metrics such as recall, and quality, which refers to the importance to the litigation of any documents that might have been left behind and what the relevance of those might be.

The Court notes that the documents that are excluded by defendant’s search terms and the responsive documents that are identified at the tail-end of the TAR review are comparable in their significance. They’re relevant to issues that are important in the litigation, but also are similar to documents that have already been produced.

The Court notes that pertinent quantitative measures show that while the application of search terms has generally been reasonable, the termination of TAR has not with respect to all of the defendants.

The Court then jumps to the discussion of search terms because the plaintiff’s argument initially, is that the search terms that the defendants used were too narrow. The Court starts — by the Court, I should say the Special Master — starts by rejecting the defendants’ argument that the Court has ruled that the defendants get to choose the search terms, and notes that the Court’s second order (that was adopted following a recommendation by the Special Master) was explicit that the plaintiffs will be permitted to challenge search term inadequacies if they believe the search is unreasonable, and that that time is now when the defendants have essentially stated that they’ve met their discovery obligations.

Essentially, the Court looks at all three of the plaintiffs’ arguments with regard to search terms and rejects them. Then he makes a recommendation that the plaintiffs’ motion to have the defendants adopt the disputed search terms be denied.

I recommend highly that you read Judge Francis’ analysis here, specifically, his analysis of the recall statistic. He notes that the parties agreed that the defendants could choose search terms and narrow the documents to be included in TAR, and that those terms have not been shown to be unreasonable. He does go through a significant analysis in reaching that conclusion, and it’s worth a read. So take a read there.

Turning then to TAR, and whether or not the party stopped at an appropriate place in their TAR process, Judge Francis finds that the three defendants that the plaintiffs challenged stopped their review process too early. Looking at the responsiveness rates that the court considers here, three of the defendants—WCA, Huntsman, and Covestro — stopped their review process where the last two batches had responsiveness rates of 15%, 18% and 19% responsiveness.

The Special Master found that, combined with the potential issue on the search terms of documents being excluded by them, it was “particularly important not to stop the review of the remaining documents prematurely.” In essence, the Court says, hey, we talked about earlier that you should have a 10% responsiveness review rate, and I’m going to hold you to that.

The Special Master then goes to the defendant’s arguments that starting a review back up again is disproportional in cost and burden to the defendants. The Court looks at the proportionality considerations of the cost of reassembling the review teams for each defendant and finds that the six factors under Rule 26(b)(1) favor requiring all three defendants to continue their TAR review.

Specifically, Judge Francis points to the amount and controversy of tens of hundreds of millions of dollars, and that the defendants have the possession of the bulk of the evidence here, so they’re in control of what documents are produced. The Special Master goes on to recommend that the defendants continue to review to the point where the last two batches contain 10% responsiveness rates for documents.

One note to remember here is that two days after this decision was issued, Judge Francis was alerted that one of those three defendants that he recommended continuing the review for, had, in fact, provided responsiveness rates below 10%, and that the number that he had assumed was incorrect. Essentially, what was three defendants becomes now two defendants that have to continue their review of documents using TAR down to where the last two batches have a responsiveness rate below 10%.

Takeaways

What are our takeaways from this case? Well, as I mentioned a little bit at the outset, if you need a decision to articulate the value of a Special Master, this is the one that will do that for you. Judge Francis engages here in what I consider to be a very thoughtful and detailed analysis of each of the arguments presented from the perspective of a former judge who clearly understands the ins and outs of TAR and the complexity of a large-scale antitrust matter and balances them very effectively.

I highly recommend that anyone embarking on using TAR in a significant class action review the series of decisions in this case — In re Diisocyanates. All of them are linked on the left-hand side in eDiscovery Assistant. If you use the public links to get to those, you can review those effectively.

It’s going to be critical to be able to support your decisions at each aspect of the TAR process and to ensure that you’re providing the court, or in this case, the Special Master with sufficient detail to allow him or her to do the analysis that is required. Here, the parties provided a lot of contextual documents that were key to Judge Francis’ analysis as to whether or not the defendant search terms were reasonable.

You’ll recall from our factual discussion that many times, the documents that were put forward by the plaintiffs were either near duplicates, or had very similar information to documents that had already been produced, which led Judge Francis to determine that there was no deficiency in the reasonableness of the search terms by the defendants.

We’re still lacking sufficient numbers here for the defendants to make a solid case that the costs associated with starting its review teams back up was disproportional to the needs of the case. We’ve talked multiple times on the Case Of The Week about the need to provide detailed cost estimates to a court. Even if they had, though, I’m not really sure that that would have changed the outcome of this particular case given the size and value of this case. The complexity of the proportionality analysis done by Judge Francis was pretty clear that those costs, even if they were in the range of a couple hundred thousand dollars to complete the review, was not going to be sufficient to shift the burden over to the plaintiffs.

Our most important takeaway from today’s case is this: the rules and processes that govern ediscovery have to be considered on a case-by-case basis. The way in which Judge Francis analyzes the processes here relative to the evidence needed for this type of antitrust case is what’s key. Use that framework when setting up your motions to a court. Put the discovery that you’re seeking in the context of what you need to prove.

Start with your jury instructions and work backwards. In cases with no special master, or one who is not as knowledgeable as Judge Francis is here, you have to teach the Court. Tell the Court why you need what you need to make your case, and why the processes that have been employed are not allowing you to get to what you need. Similarly, if you’re defending those decisions, put them in the context of the evidence that’s required for the other side and what you’ve done to be able to provide that evidence.

Okay, that’s our Case of the Week for this week. Thanks so much for joining me. We’ll be back again next week with another decision from our eDiscovery Assistant database. If you’re interested in doing a free trial of our case law and resource database, sign up and get started.

Thanks so much. Have a great rest of the week. And I’ll see you on our next episode.

Watch Next Episode Live

#CaseoftheWeek streams live on the ACEDS YouTube, Facebook, Twitter, and LinkedIn company pages. Subscribe to the ACEDS social channels and turn on notifications to be notified when we go live.

Catch the Replay

You’ll be able to read episode summaries and watch the recorded show right here on the eDiscovery Assistant blog or access the recorded versions on our Vimeo channel.