Green lights for AI
James Millen, Technical Director, Driver Trett APAC looks at Technology-Assisted Review with eDiscovery – a scientifically proven and court-endorsed AI tool.
Artificial Intelligence (AI) has been identified as Construction technology’s “next frontier” in a recent report by McKinsey & Co on the construction industry’s digital revolution. However, McKinsey & Co also reported that construction and professional services are two of the most reluctant industry sectors to implement AI.
In the context of high value and high-profile construction claims and disputes, reluctance to pioneer new technology is understandable. There is inherent risk in any decision to invest resources into a claim or dispute. A suggestion to employ new or developing technology is often met with two challenges:
- is it reliable? and
- what have the Courts said about it?
AI in eDiscovery: Technology-Assisted Review
In the previous edition of The Digest, Garth McComb wrote about Driver Trett’s use of eDiscovery software. It is a powerful tool for rapidly analysing the vast quantities of documents and data associated with a construction claim or dispute. Our eDiscovery capability implements a form of AI technology known as Technology-Assisted Review (TAR).
Contrary to the doomsayers’ cliché that AI is coming to take all our jobs, TAR relies on a collaboration between AI algorithms and a human professional expert. Broadly a TAR-enabled eDiscovery system prioritises documents that it believes are relevant to a search criteria, and returns a small sample of results. These results are then scored by the human professional expert for relevance. The AI technology in TAR then takes the scoring feedback and progressively ‘learns’ how to produce a more relevant set of results over a few iterations of the review:rank:repeat process. Internet search engines are an everyday example of TAR.
Reliability of TAR
A scientific analysis of TAR in eDiscovery by Grossman & Cormack was published in 2011 under the bold title ‘TAR in eDiscovery Can Be More Effective and More Efficient Than Manual Review’. Grossman & Cormack experimented with five separate and different document review exercises.
Firstly, they found that the professional expert only needed to review 1.9% of the documents in order to ‘teach’ the TAR to return optimal search results. In other words, that’s over 50-times fewer documents to be reviewed than in a full manual search. 50-times less documents = 50-times less time = 50-times less cost, and that’s not even considering the human effect of task-fatigue in a full manual review.
Secondly, their overall results were that “by all measures” the average efficiency and effectiveness of the five technology-assisted reviews were better than the five comparison manual reviews.
The Grossman & Cormack study has been put before the courts in several cases concerning TAR and has been persuasive.
What the Courts have said about TAR
TAR has in fact come before the courts in several jurisdictions. This focused initially on the fundamental issue of whether the courts would approve the use of the technology at all. Subsequently the question progressed to whether the courts would insist on TAR over traditional manual review.
The US - ‘Bring Your Geek to Court Day’
The US courts were the first to approve the use of TAR.
In 2012, US Magistrate Judge Peck ordered the first recorded approval of a litigant’s request to use TAR in a court proceeding, in the case of Da Silva Moore v Publicis Groupe. “Counsel no longer have to worry about being the ‘first’ or ‘guinea pig’ for judicial acceptance of computer-assisted review” he assured. His Order even noted the importance and helpfulness of what Judge Peck described as “bring your geek to court day” - being “able to explain complicated eDiscovery concepts in ways that make it easily understandable to judges who may not be tech-savvy”.
In Gordon v Kaleida Health it was the judge, not the litigants, that suggested the use of a type of TAR called predictive coding. Impatient with the parties’ year-long attempts to agree on how to achieve a cost-effective review of some 200,000-300,000 emails, the Magistrate Judge ordered the parties to try predictive coding.
By 2015, in the case of Rio Tinto v Vale S.A., it was back to Judge Peck to declare that US case law had developed to the point that the courts’ endorsement of using TAR for document review was “black letter law”.
Ireland - Common Law endorsement
The first endorsement of TAR by a common law court came in the 2015 Irish High Court case Irish Bank Resolution Corporation Ltd v Quinn.
The issue before the court was IBRC’s calculation that it would take 10 lawyers nine months to manually review the 680,809 potentially relevant documents, at an estimated cost of €2m. IBRC proposed using TAR to minimise the cost and time, but the opposing party argued against the suggestion.
Quinn argued that TAR could not be relied on to capture all relevant documents, and that there would be no savings in cost and time in real terms due to the human involvement in the AI ‘learning’ process.
The strength of the respective arguments gave weight to the ruling, because Mr Justice Fullam was required to analyse the evidence in detail.
The Court found in favour of TAR, persuaded largely by the Grossman & Cormack study’s finding that TAR requires human review of only 1.9% of documents. In addressing Quinn’s argument of whether TAR was sufficiently accurate, Mr Justice Fullam took the balanced view that “If one were to assume that TAR will only be equally as effective, but no more effective, than a manual review, the fact remains that using TAR will still allow for a more expeditious and economical discovery process [67]”.
The UK - TAR endorsed when contested
In 2016, the UK courts were given a gentle introduction to TAR, because by the time Pyrrho Investments v MWB Property reached the court, both parties had already agreed to use it. They sought approval from the court, and an appropriate order was made.
However, the latter case of Brown v BCA is now considered the UK courts’ landmark decision because, like IBRC in Ireland, the use of TAR was contested between the parties.
BCA held approximately 500,000 potentially relevant documents. It provided evidence that TAR would be 50%-60% cheaper than conducting traditional keyword searches. Brown opposed, arguing that the technology may not be as effective as the more expensive traditional keyword searching.
As with IBRC in Ireland, the court ordered the use of TAR. The Registrar was persuaded by the evidence of likely cost savings, and the corresponding lack of evidence that TAR might be less effective at identifying relevant documents than keyword searching.
Australia – TAR in a Construction Claim
Also in 2016, an Australian court endorsed the use of TAR in a matter specifically concerning a construction project.
McConnell Dowell v Santam & Ors concerned a claim that arose from the design and construction of a natural gas pipeline. Approximately 1.4 million potentially relevant documents were identified, which the court estimated would take a junior solicitor in excess of 10 years to review, even if just one minute was spent on each.
The court appointed a Special Referee to report on a more efficient method for reviewing the documents and endorsed the recommendation to use TAR.
Where next for TAR?
Cases concerning the approval or endorsement of TAR appear to have dried up since 2016, perhaps indicating that the critical mass of precedent has been reached for TAR to be considered a judicially accepted AI technology. In recognition, in late 2016 the Federal Court of Australia updated its Practice Note on ‘Technology and the Court’ Electronic Discovery to expressly encourage “using advanced analytics technologies (or other electronic discovery solutions) to assist in understanding key documents and minimising the document review process”.
The only apparent remaining difficulty the courts have expressed with TAR is, ironically, the level of human ability to properly use the tool. In Triumph Controls v Primus before the UK’s Technology and Construction Court, for example, the parties were ordered to abandon TAR and switch back to a manual document review. In this case, the court held the view that the parties had failed to use the tool in a transparent and verifiable way.
However, given the number of cases evidencing the courts’ endorsement of TAR, across several jurisdictions, it now seems a matter of ‘when’ rather than ‘if’ common use of the technology will cascade down through dispute resolution and into claims practice. A likely by-product of BIM is that the quantity of data and documents generated by large construction projects is likely to continue to increase. In parallel, AI technology is rapidly evolving, which will make TAR more powerful and more economically accessible. As an indicator of the developing technology Grossman & Cormack have continued their research into the application of TAR, publishing further papers in 2014 and 2017 that continued to demonstrate TAR’s accuracy and reliability.
Finally, and given that the time-saving efficiency of TAR is widely accepted, a point made by Simon Waller in his book Analogosaurus - Avoiding Extinction in a World of Digital Business is particularly relevant when considering the use of TAR. Waller explains that using technology such as TAR to save time not only saves cost, but consequently allows professionals to spend that saved time doing valuable, creative, problem-solving work whilst the technology does the mundane. The quicker we can problem-solve, the quicker our clients benefit from cost and time savings.