Services

Our core focus is the intersection of new technologies and techniques that explore, support and advance understanding of legal reasoning, the delivery of legal services and access to justice and legal information.

Services

RAIN NLP BENCHMARK

With the increasing availability of textual data and improved model capabilities, Natural Language Processing (NLP) is gaining wider adoption in industry. However, the field is mainly guided by research-motivated benchmarks which by their nature can fail to adequately measure real-world utility of NLP applications.

At Mishcon de Reya we believe that, in addition to existing benchmarks, application-focused benchmarks will help guide us towards vastly improved Natural Language Understanding (NLU) for industrial applications.

To facilitate greater focus on NLP in industrial settings, we introduce the Real-World Applied Industrial NLP (RAIN) benchmark – a collection of NLP tasks and corresponding datasets with broad practical application.

We formulate several NLP language tasks, and make available new datasets for each, totalling over 150,000 annotations, and provide evaluation of baseline and task-specific models. At the time of release (Q4 2021), we observe a headroom gap to human performance on the overall score of 19.4%.

We will maintain the benchmark and update datasets and include new tasks on an ongoing basis.

Explore
People
We are a researchers, lawyers, engineers, entrepreneurs and technologists who are working on research and publications across of wide variety of different subjects and technical areas across different companies, institutions and universities.
View our people
Projects
We are interested in several broad areas where technologies are helping explore, support and advance understanding of legal reasoning, the delivery of legal services and access to justice and legal information.
View our projects
Publications
Our latest publications and articles.
View our publications
Collaborators
We are always looking for collaborators to work with on the intersection of new technologies and techniques that are concerned with supporting legal reasoning, the delivery of legal services and access to law.
View our collaborators
Contact us