#CaseoftheWeek Episode 44: Relevancy Review of Search Terms

In episode 44, Doug Austin of eDiscovery Today discusses how the Court addresses a dispute regarding whether conducting relevancy review of search results is a violation of the 𝐎’𝐃𝐨𝐧𝐧𝐞𝐥𝐥/𝐒𝐚𝐥𝐯𝐚𝐭𝐨𝐫𝐢 𝐈𝐧𝐜. 𝐯. 𝐌𝐢𝐜𝐫𝐨𝐬𝐨𝐟𝐭 𝐂𝐨𝐫𝐩., 𝐍𝐨. 𝐂𝟐𝟎-𝟖𝟖𝟐-𝐌𝐋𝐏 (𝐖.𝐃. 𝐖𝐚𝐬𝐡. 𝐎𝐜𝐭. 𝟏, 𝟐𝟎𝟐𝟏) presided over by US Magistrate Judge Michelle L. Peterson. and provides readers with three key takeaways:

1. Read the Rules, Shankopatumus
2. Have a well written and clear ESI Protocol
3. Don’t get wild with wildcards

Read on and watch the episode for the full analysis.

Good morning and welcome to our Case of the Week for October 26, 2021. I’m Doug Austin, editor of eDiscovery Today. Thanks for joining me.

I’m filling in for Kelly Twigger again this week as she takes some personal time. Thanks again to Kelly and eDiscovery Assistant for allowing me to speak about another interesting case this week.

Through eDiscovery Assistant’s partnership with ACEDS, we choose a recent decision in ediscovery each week that highlights key issues for litigators and those involved in the ediscovery process, and we talk about the practical implications of that decision for you, your practice and your clients.

We’ve got a really instructive case this week with some good lessons learned about knowing the rules and due diligence. I’ll briefly discuss the lesson learned from one of the cases cited in this one, which is a Texas case I covered back in 2016. So you’ll get a bonus case takeaway.

First I should note that a link to the decision is in the comment section of whatever platform you’re viewing us in—whether that’s LinkedIn, YouTube, Twitter, or Facebook. There’s also a link to eDiscovery Today’s coverage of this case as well.

As Kelly kindly mentions every week, you can also view a link to the 2020 Case Law Year in Review report that eDdiscovery Assistant and eDiscovery Today jointly published earlier this year. We’re getting even closer to the end of the year, it’s worth noting that next year’s report will be even bigger, as both eDiscovery Today and Case of the Week have been covering cases all year long. Look for that early next year.

We also invite you to check out the recently updated website at ediscoveryassistant.com. Pop over and sign up for the blog to receive our weekly case Law newsletter and regular blog posts about all things ediscovery.

The case is a dispute over royalties with Microsoft associated with the musical theme and soundtrack for the popular video game Halo. The theme song and soundtrack was developed by Martin O’Donnell and Michael Salvatori, and the ruling notes that the musical theme is reminiscent of a Gregorian chant, which very much is what it sounds like and that it became iconic and well known, which indeed it has.

O’Donnell and Salvatori’s company is the plaintiff in the case and is referred to by Judge Peterson throughout as ODS.

One of the things notable about this case is that the parties agreed to an ESI protocol, which was entered as an ESI order in November of last year. In it, the parties agreed to use search terms to locate ESI likely to contain responsive and discoverable information. The parties had to request Court intervention twice to get its help in determining appropriate search terms under the ESI order, and with the Court’s help, they were able to finally agree on search terms for Microsoft to use, satisfying its obligations under the ESI order in May.

Microsoft proceeded to run the agreed searches and reviewed the documents for privilege as well as responsiveness to ODS’ discovery requests with an instruction for Microsoft’s counsel to attorneys reviewing the documents to err on the site of inclusion. Microsoft produced documents and four rolling productions to ODS between May and August of this year and identified several documents that weren’t relevant to a claim or defense in the case, like employees joining the team or leaving the team, technical issues, buying a boat and so forth. Seems reasonable, right?

Only ODS didn’t think so. They objected to Microsoft’s production because they believe Microsoft wasn’t allowed to conduct a relevance review of the documents that hit on the agreed search terms or withhold those documents from its production. ODS filed the motion to compel Microsoft to produce all non privileged documents hitting on the search terms regardless of whether they were relevant to a claim or defense in the case.

Judge Peterson begins her analysis by referencing Federal Rules 26(b)(1), which says:

Parties may obtain discovery regarding any non privileged matter that is relevant to the party’s claim or defense proportional to the needs of the case. Considering the importance of the issues at stake in the action, the amount and controversy, the party’s relative access to relevant information, the party’s resources, the importance of the discovery in resolving the issues, and whether the burden or expense of the proposed discovery outweighs its likely benefit.

She also notes this has been the relevant standards since the 2015 amendments to Rule 26(b)(1), which went into effect on December 1, 2015. This has been the relevant standard for nearly six more years. She also has an end note to that, which I’ll get into in a moment.

Regardless, she says that ODS has taken the position that the Rule 26(b)(1) relevance standard doesn’t apply to document productions that are the result of the execution of search terms under the Court’s ESI order, and the producing party isn’t allowed to withhold documents based on relevance. In essence, ODS is saying that the ESI order supersedes Rule 26(b)(1), and that requires Microsoft to produce everything retrieved by the searches that isn’t privileged, even if it’s not relevant to the case.

Judge Peterson notes that there’s very little case law on the issue, but she goes through a handful of cases, including a couple that the plaintiff cited in its argument to support its motion and right off the bat, she pokes obvious holes in the case citations by the plaintiff. For starters, Judge Peterson notes that the plaintiff incorrectly attributed a statement from the case, Ball v. Manalto, Inc. That statement was “courts generally do not allow litigants to both restrict their searches through search terms and manually review the results for responsiveness.” ODS used that case in its argument.

The problem is that the judge in the case made that statement but wasn’t making that determination. Instead, the judge was merely quoting the plaintiff’s position in that case.

By the way, the plaintiff in that case was named Craig Ball, but he isn’t the Craig Ball that many of us know as an ediscovery expert in the legal community, but a different guy. Small world.

Anyway, Judge Peterson also notes that other cases cited by ODS directly contradicted ODS’ argument. One of those is FlowRider Surf, Ltd v. Pacific Surf Designs, Inc. where the Court found that the plaintiff didn’t waive its right to conduct a relevance review by agreeing to run search terms agreed to in an ESI order.

ODS didn’t pick cases that really helped its position here.

Judge Peterson also looks at two other cases where the issue was addressed. The first one is BancPass, Inc. v. Highway Toll Admin., LLC, where the parties had discussions over email about search terms. The plaintiff argued that the agreement within the email exchange was to produce all non privileged documents that hit on certain search terms while the defendant wanted to conduct a review for responsiveness after it retrieved a large number of documents, many of which were not responsive to the request. You’ll see at least one reason why, when I circle back to this case in our takeaways as it’s the bonus case. Regardless, the Court in that case ruled that the party’s agreement over email was a way to simplify and limit the scope of production, but it didn’t obligate the parties to produce non responsive documents.

The other case that Judge Peterson references is Palmer v. Cognizant Tech. Sols. Corp, where the Court addressed a similar motion to produce all documents that hit on a particular search term, ruling that production of all search results is not the standard under Rule 26, which provides that discovery may be obtained, as you’ve already heard, regarding any non privileged matter that is relevant to any party’s claim or defense and proportional to the needs of the case. In that case, the judge also stated “the Court will not compel defendants to produce any document simply because it contains a search term, whether or not it is responsive to the discovery request or by extension, whether or not it is relevant proportional to the needs of the action.”

With those cases cited, Judge Peterson ruled that a party’s agreement to run search terms doesn’t waive its right to review the resulting documents for relevance so long as the review can be done in a reasonably timely manner. Noting that Microsoft had the ability to hire contract reviewers who review documents for a living to conduct a document by document review of the production to determine relevance and was therefore able to conduct the review within a reasonable time frame, she denied the motion to compel.

Remember that end note I referenced regarding Rule 26(b)(1) in the 2015 Rules amendments? When discussing that, Judge Peterson inserted an unknown reference which stated:

The Court once again finds it necessary to remind counsel for ODS that the standard regarding the production of relevant evidence in Rule 26(b)(1) is no longer reasonably calculated to lead to the discovery of admissible evidence. The Court reasoned at an earlier hearing in this matter that ODS’s counsel’s failure to understand that Rule 26 had been amended and is no longer the standard that needs to be reasonably calculated to lead to the admissibility of evidence might be one of the problems that we’re having here. Yet, despite being on notice of a significant rule change to the Federal Rules of Civil Procedure, ODS’s counsel nonetheless continues to cite the old standard.

That’s a pretty telling endnote to this case, and it’s a great segue into our takeaways.

The first takeaway is know the rules, or, as my friend and colleague Tom O’Connor likes to say, referencing the old E-Trade commercial, “read the Rules, shankapotomus”. The 2015 Rules changes have been in place for nearly six years, yet I’ve covered many cases where old rules references were used.

One of the most common is the use of that phrase “reasonably calculated to lead to the discovery of admissible evidence”, which has been replaced by six proportionality factors in the new Rule 26(b)(1), which I mentioned earlier when I cited that rule. That’s the standard now—not reasonably calculated to lead to the discovery of the admissible evidence.

Attorneys out there should know this by now. Yet many still reference this as the standard, despite the Rules changes. It’s more than one or two here or there. In fact, out of curiosity, I decided to search for that phrase yesterday in eDiscovery Assistant for cases after December 1, 2015, and I found 698 case rulings which have that phrase in them—698.

While I’m sure some of those references may not be the party attempting to argue using the old standard and may be a mere reference to the phrase, plenty of them are still arguing using that old standard. That includes boilerplate objections, another 2015 rules changes, and another issue we’re still seeing a lot where parties respond with boilerplate objections like vague, improper, overly broad, unduly burdensome, and not reasonably calculated to lead to the discovery of admissible evidence.

Rule 34(b)(2) now requires that objections should state with specificity the grounds for objecting to the request and also whether any responsive materials are being withheld on the basis of that objection. Yet we’re still seeing plenty of boilerplate objections and cases which judges have been routinely waving now. As Tom would say, “read the rules, Shankapotamus.”

The next takeaway is with regard to ESI orders and protocols. Much to my surprise, not everyone is a fan of ESI protocols. I’ve heard a couple of people lately indicate a lack of enthusiasm for them, including Judge Xavier Rodriguez, District Judge of the Western District of Texas, who stated that in a webinar I did with him last week when we were discussing the Nichols v. Noom case from earlier this year.

However, he did acknowledge that he’s also seen a lot of disputes associated with ESI protocols that he’s had to address in his court, so that could be part of the reason why. It may not be the ESI protocol vehicle itself, but it may be how parties are creating and agreeing to them that may be the issue. There are best practices to ensuring or helping ensure that an ESI protocol addresses the most common discovery disputes that many legal professionals may not be following.

Regardless, I’m a big fan of ESI protocols, and I do think they help avoid more disputes than they cause by far. Especially if you know what you’re doing when you negotiate them.

Kelly has been writing an excellent series on ESI protocols on the eDiscovery Assistant blog. Nine blog posts so far on everything from the when, how and why in planning and creating them to the key components of one to the top ten situations you can avoid with an ESI protocol to things like manner and form of production and so forth. Part Eight was about how to handle search term negotiations, and the key takeaway from that one was this how you negotiate the process for deciding on search terms in your ESI protocol requires careful consideration and an understanding of the overall approach to discovery for a case.

I think that also goes with the process associated with the search term results as well, and getting on the same page with the opposing counsel before productions start. In fact, Craig Ball—the ediscovery one not the plaintiff in the earlier case that I referenced—also discussed this case in his excellent. Ball in Your Court blog yesterday, revisiting a blog post and practice tip he published over eight years ago before any of these case rulings happened.

His practice tip was “if you enter into an agreement with the other side to use keywords and queries for search, be clear about expectations with respect to the disposition of items hit by queries,” which is great advice.

Remember, Microsoft submitted four rolling productions over four months. While this dispute was going on, it’s much easier to address these issues up front than it is after production start. That’s another takeaway along with the takeaway to read Kelly’s ESI protocol blog series. It’s excellent.

My final takeaway is about one of the reference cases, BancPass, Inc. v. Highway Toll Admin., LLC, which is the case I covered in my blogging a few years ago. That was a case that was ruled on by Andrew Austin (no relation, at least not that I know of) who’s a Magistrate judge in Texas.

Part of the dispute in that case had to do with the plaintiff’s smartphone app. One of the things notable about the case is that the ruling included the search terms agreed to by the parties. Two of those terms were smartphone within ten of “a-p-p” with the wildcard character and “phone” with the wildcard character within ten of “a-p-p” with the wildcard character.

Clearly, it looks like the parties chose to add a wild card to app to search for app, apps, application, or applications because this is the case involving a smartphone app. Makes sense, right?

Not when you realize how many words there are in the English language that start with “a-p-p”. There are actually 306 words in the English language that start with “a-p-p”. Apparently, it appears the application of the wild card was appealing when it wasn’t approximate or even appropriate, it was appalling. See what I did there?

Clearly, neither party tested these searches beforehand or would have been apparent that they didn’t merit approval. Okay, I’ll stop now.

The point is, it’s very important to test searches before you even propose them or accept searches proposed to you much less agree to them. This example is obvious, but search terms are highly unpredictable, and terms that may be proportional within one data set may be highly disproportional in another data set you just never know so you need to test your searches before you agree to them. Certainly no wonder they wound up with a lot of non responsive documents for the searches in this case given how broad those terms were.

By the way, in case you’re wondering how I know, there are 306 words in the English language that begin with “a-p-p”. There’s a site called MoreWords.Com where you can look up a lot of things related to words. It’s a site designed for Scrabble and Words with Friends players, but it gives you the ability to look for words that start with or end with a character stream like “a-p-p”. That makes it a great resource for testing potential wildcard searches by seeing the words that could be retrieved.

Sometimes you’ll realize if you use that that your search is retrieving a lot more than you expected. Some other times you can actually find it’s retrieving less than you expected because I’ve seen the common mistake of including a word and forgetting to drop the “e”, which adds “i-n-g”, and that’s one of the variations of the word. It’s a great site for being able to test wildcard searches. In fact, in my very first day of daily blogging, back in 2010, I wrote a blog post called “Don’t Get Wild with Wildcards”, where I discussed an experience I had with a client who had already agreed on search terms before I began working with them. One of those terms was “m-i-n” with a wildcard because they were looking for terms related to mines or mining. But there are actually 269 words that begin with “m-i-n”. That one search in this particular case retrieved 300,000 files; it was clearly over broad.

Don’t get wild with wildcards and be sure to test searches with your dataset before agreeing to them.

We’ll be back next week with another episode of Case of the Week from eDiscovery Assistant. If you’re an ACEDS member and interested in using eDiscovery Assistant, there’s a discount available to current ACEDS members and a free trial for folks taking the ACEDS. If you’re interested in either of those, send an email to ACEDS@ediscoveryassistant.com and one of the team will be in touch.

If you’re interested in doing a free trial of eDiscovery Assistant case law and resource database, jump to www.eDiscoveryAssistant.com and sign up to get started.

Thanks to Kelly Twigger for once again letting me keep the seat warm for her. And thanks to eDiscovery Assistant and ACEDS as well.

Have a great week and stay safe and healthy out there.

Watch Next Episode Live

#CaseoftheWeek streams live on the ACEDS YouTube, Facebook, Twitter, and LinkedIn company pages. Subscribe to the ACEDS social channels to be notified when we go live.

Catch the Replay

You’ll be able to read episode summaries and watch the recorded show right here on the eDiscovery Assistant blog or access the recorded versions from the ACEDS social media channels.