Building the corpus: 134,000 Employment Tribunal judgments

Every part of this project depends on a complete copy of the Employment Tribunal’s published judgments, held locally in a form a computer can search and analyse. This article describes how I built that collection, what it turned out to contain, and where its limits lie. All figures describe the register as it stood on 30 September 2026.

At a glance

Source	Employment Tribunal decisions register, GOV.UK (February 2017 onwards)
Licence	Open Government Licence v3.0
Judgments collected	134,361, from 165,205 PDF files
Substantive judgments	~98,500 (the remainder are very short orders)
Text volume	~290 million words
Storage	~43 GB, most of it the original PDFs
Hardware	One desktop PC: AMD Ryzen 9 5950X, AMD Radeon RX 7900 XTX (24 GB), 64 GB RAM, 2 TB Samsung 980 Pro NVMe SSD

Why the whole register

An early prototype used 22 hand-selected unfair dismissal judgments. It answered straightforward questions well, but failed the first practical test I set it: how do tribunals approach a dismissal for “some other substantial reason” where the underlying problem was a personality clash between colleagues? The early prototype did include dismissals for some other substantial reason, but none where that reason was a personality clash between colleagues. The solution to this was to expand the number of judgments in the database, and that led me to explore downloading the whole register. The register is published under the Open Government Licence, which permits copying and reuse, including analysis, provided the source is acknowledged.

Collecting the judgments

Using an AI coding assistant, I created a tool that downloads every decision on the register and converts it to plain text. It works in two stages. It first works through the register’s search results, fifty decisions at a time, and records the address of each decision. It then visits each address, retrieves the PDF of the judgment and extracts the text.

Compiling the list of decisions took under two hours. Downloading them took around forty hours of running time, spread over about six days, during which the tool was stopped and restarted several times. It saves its progress regularly, checks whether it already holds a judgment before downloading it, and can be stopped and restarted without repeating any work. Running it again later picks up newly published decisions. It also paces its requests, identifies itself to the site, and backs off if the site indicates that it is busy.

Indexing them

With a local copy of the register in place, I used the same coding assistant to build a second tool to process the files. It reads every downloaded judgment and records structured information about it, one row per judgment:

• case number, claim year and parties;
• the protected characteristics the judgment mentions;
• the form of discrimination considered, such as direct discrimination, harassment, victimisation or a failure to make reasonable adjustments;
• other claims, such as unfair dismissal, whistleblowing or holiday pay;
• the likely outcome, and whether it is a remedy judgment;
• whether it mentions injury to feelings or the Vento bands;
• its length, and whether its text is usable.

The index is produced by pattern-matching rather than by an AI model, so it is fast. It is also crude: it records what a judgment mentions, not what it decides. A judgment that considers and rejects a race discrimination claim is still recorded as mentioning race. That is adequate for its purpose, which is to identify, from the 134,000 judgments, those likely to be relevant to a particular question.

Making them searchable

For an AI model to search the judgments by meaning rather than by keyword, each judgment first has to be divided into shorter passages. A long judgment may deal with many issues, and treating it as a single unit would blur them together; dividing it allows a search to find the part that is actually relevant. Each passage is then converted into a numerical representation, known as an embedding. Passages about similar things end up close together even where they use different words, so that a question about a “personality clash” can find a judgment that refers to a “breakdown in working relationships”.

The first search index used a free application called AnythingLLM, which retrieves the passages most relevant to a question and passes them to a language model running on the same computer. It split the judgments into around 600,000 passages and embedded them with a model called Nomic.

That approach worked reasonably well on straightforward questions, but proved unreliable on more specific questions. Asked about health and safety dismissals under section 100(1)(e) of the Employment Rights Act 1996, it replied that the collection contained no cases dealing with that provision.

That answer was, in part, the product of a safeguard. The model had been instructed to answer only from the judgments put before it, and if it could not find a relevant match to say so, rather than fill the gap from its own training, which is where invented authorities tend to come from. The difficulty was that the model saw only a snapshot of the collection, not the collection as a whole. None of the passages it received concerned section 100(1)(e), and it reported that as an absence from the entire register. At the time, a simple text search of the same collection found 144 judgments citing the provision.

To address this, I built a second index. It divides each judgment more finely, at its numbered paragraphs, into around 2.4 million passages, embeds them with a larger and more recent model, and is used alongside a conventional keyword search. Its final build took about nine hours.

The safeguard has also changed. Whether cases exist is no longer a question for the model. It is answered from a separate index of every statutory reference in the collection. Asked the same question now, the tool reports, from the index rather than the model:

145 judgment(s) cite ERA 1996 s.100(1)(e) (107 naming the act beside the section, 38 where the act was inferred).

Where the model’s draft had strayed, the answer says so:

[Removed by the program: a statement about which cases do or don’t exist. Only the database facts below can say that.]

It then lists the judgments that best match the question, with the outcome of each claim as recorded in the tribunal’s order, and passages in which tribunals set out the elements of the test.

Put to the full collection, the question that defeated the prototype now produces a useful answer. Of the first ten judgments returned for a dismissal for some other substantial reason following a personality clash, five deal with exactly that point, each by reference to the Court of Appeal’s decision in Perkin v St George’s Healthcare NHS Trust [2005] EWCA Civ 1174, the leading authority.

What the collection contains

Most judgments are short

The first surprise was how short most judgments are. Among the judgments with substantive text, the median length is about 1,700 characters, or roughly 280 words: well under a page. More than half are under 2,000 characters. Much of the register consists of dismissals on withdrawal, consent judgments, strike-outs and short remedy orders. Fully reasoned judgments form a long tail: about one in ten exceeds 50,000 characters, or around 8,000 words.

This shapes everything that follows. A search that treats every document equally will return a large number of short orders containing no reasoning at all.

When the claims were presented

The register began in 2017, but many of the decisions published since then concern earlier claims. Taking the year from each case number (the /2019 in 1234567/2019):

Judgments in the corpus by the year the claim was presented, taken from the case number. 4,384 judgments with no readable year are excluded. Figures for 2025–26 are low because most of those claims are still in progress.

Claims presented in 2018 and 2019 make up the largest share, with a marked fall from 2020. The fall coincides with the pandemic, but the register alone cannot say whether it reflects fewer claims being presented, slower listing, or both.

The result is a collection that can be searched by keyword and by meaning, and filtered through the structured index. Most of what follows in this series is built on it.