SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Yes, individuals' names are included. The names of parties, lawyers, judges and other individuals may be directly identified. In the UK, court judgements are not anonymised and all information is currently publicly available. Parties should expect to be named in judgements because courts uphold the principle of open justice, promote the rule of law and ensure public confidence in the legal system. However, the court will anonymise a party if the non-disclosure is necessary to secure the proper administration of justice and to protect the interests of that party. For example, asylum seekers, victims of violent offences and children's names are, in most circumstances, anonymised. In addition, courts will not make cases available for publication if important aspects such as safety or overriding interests stand in the way of publication—as a result, such cases will not be contained in our dataset.
Yes, court judgements will directly include sensitive information where it is relevant to the proceedings. In some cases, it will be indirectly identifiable. All such sensitive information is already publicly available in the published court judgements, which means that this dataset does not add new information. Given the impossible task of removing all personal or sensitive information from this dataset, as it would reduce the integrity of the research corpus, we follow GDPR exemptions on the processing of sensitive categories. We have implemented robust safeguards and considerations relevant to processing data of this sensitive nature. Details on these safeguards are described in Section 3 of the paper, which focuses on legal and ethical considerations.
The data was contributed to the research project by CourtCorrect Ltd represented by Ludwig Bull, one of the co-authors. Felix Steffek, a further co-author, has received the data under a licence agreement with CourtCorrect Ltd. Under the license agreement, CourtCorrect Ltd has no rights to interfere with the dataset making it robustly available to the research project.
The corpus was given to the research project in various formats including plain text, Microsoft Word files and PDF files. All files were then converted to a simple standardised XML-format. PDF files were OCRed using Tesseract version (v4.1.1).
We have not intentionally sampled from a larger set. As already described above, the starting point was a raw dataset containing cases from various courts. Courts do not systematically report all of their cases in electronic form, which means that not all cases from every court are published online. As a consequence, this dataset does not necessarily contain all judgements made by the relevant courts. We further excluded all courts that were not within our target jurisdictions of England, Wales and the United Kingdom. Taken together, the dataset is therefore unlikely to contain every case from all the selected courts. However, we have not intentionally omitted any specific case from the dataset.
The data was given to the co-authors of this research project by Felix Steffek, one of the co-authors, having received it from CourtCorrect Ltd. The material was then processed by the authors and stored in XML format. For the annotation of case outcomes, legal experts at the University of Cambridge were hired and paid through the University of Cambridge Temporary Employment Service according to standard compensations for research assistants.
The court cases included in the corpus are cases from 1595 to 2020 and were obtained by the research project in 2021.
Two different ethical reviews of this work have been conducted by two different institutions.
The first of these was conducted at the University of Cambridge. Since the research project is allocated to the Centre of Business Research at the University of Cambridge, the ethical review process of the Centre of Business Research was applicable. The review process consisted of supplying the Research Ethics Committee with the relevant information about this research based on a detailed form to be completed. The information submitted included the involved researchers, the purpose of the research, the research methods, the impact on those affected by the research, the experience of the research team, indemnity insurance, how and for how long data will be stored and the data protection rules to be followed. The Research Ethics Committee discussed these arrangements in a meeting, including the GDPR and other legal and regulatory requirements relating to this project. The discussions of the Research Ethic Committee are confidential and the authors of this paper were not part of the discussions of the Committee.
The Research Ethics Committee decided on the ethical review on 4 April 2022, granting ethics approval. The Committee emphasised that it was satisfied that the relevant requirements were fully met.
In addition, a separate ethical review has been conducted in Sweden by the Swedish Ethical Review Authority. This was done by the Swedish Ethical Review Authority according to Act (2003:460) of Sweden concerning the ethical review of research involving humans. The Swedish Ethical Review Authority is a government agency independent of Uppsala University and assesses whether the research application is in line with Swedish statutes. For this review, we submitted an ethical application containing a general research plan, a CV for the responsible researcher and a standardised form with questions on, for example, research questions and aims, methods used, ethical considerations and a time plan. These documents were then considered by the Swedish Ethical Review Authority.
The ethical review was assigned the number 2022-02063-01 and was approved in May 2022.
We obtained the data from CourtCorrect Ltd, represented by Ludwig Bull, one of the co-authors. Under a licence agreement, CourtCorrect Ltd transferred the data to Felix Steffek, a further co-author, who made the data available to the research project. The licence agreement allows to use and republish the dataset. CourtCorrect Ltd has confirmed that:
1. All cases in the dataset are already accessible for the public.
2. The dataset contains cases from at least three public sources.
3. The dataset has been legally obtained.
4. CourtCorrect Ltd has no power to restrict the use of the dataset by the research project in any way now or in the future.
Our work creates a new corpus by converting the content obtained from CourtCorrect Ltd into a standardised form, adding tags and metadata, adjusting the content to fix errors and remove parts that are excluded, and providing code that enables the corpus' content to be accessed via an API.
No. It would be very difficult to do this in an ethically sound way. We are legally exempt from notification requirements. See details above, which are given in the answer on personally identifiable information, as well as in the main paper.
Did the individuals in question consent to the collection and use of their data?
If so, please describe (or show with screenshots or other information) how consent was requested and provided, and provide a link or other access point to, or otherwise reproduce, the exact language to which the individuals consented.
Not relevant: see above.
Yes. As part of the ethical clearance and considerations, the impact on data subjects has been considered: see Section 3 in the main paper. In brief, the dataset contains publicly available information that does not raise material risks to individuals' rights, freedoms or interests. In any event, our safeguards also act to protect against any risks in further use of the corpus. We implement several mitigation strategies, such as the distribution limitations discussed in the paper, including that we will comply with any request from any data subject that information be removed from the corpus.
Some standard cleaning, such as removing incorrect non-UTF8 characters, correcting obvious OCR errors and identification of incorrect data has been conducted. We have also split the case into header and body text.
The original raw dataset has been stored temporarily but will not be made public because it includes content from courts that are not within our scope and content that is proprietary. Since the case references have been included in the corpus, it is possible to obtain the original cases if needed. We want to keep the corpus representation in the XML format as close as possible to the original documents. Hence, we minimise preprocessing of the material and only correct obvious errors in the digital representation to better represent the originals, for instance obvious OCR errors.
Unfortunately, we are unable to make this available because parts of it are proprietary.
Two small experiments have been conducted to demonstrate example use cases. First, a Latent Dirichlet Allocation topic model analysis has been done on the corpus. In addition, baselines for case outcome extraction and prediction tasks have been conducted.
Yes. The repository for the corpus can be found here: HTTPS://WWW.CST.CAM.AC.UK/RESEARCH/SRG/PROJECTS/LAW.
Some examples might include pre-training language models, fine-tuning, information extraction and summarisation, historical analysis of law, legal applications, etc. Due to its scale, we believe there is a huge potential with this material.
It is difficult to anticipate all future uses of the dataset that might be harmful. Unfair uses of this dataset are possible. We require everyone who uses the dataset to use it in legal and ethical ways only. In addition, please see the answer directly below.
The dataset can only be used for research purposes and cannot be used for any other purposes such as commercial purposes. Also, we will hold users of the dataset to all relevant legal restrictions including the GDPR. For example, the data cannot be used for automated decision-making processes, including profiling, which produce legal effects or similarly significantly affect a person (GDPR art 22). Research outputs, regardless of their format, must never identify natural persons, legal persons or similar entities.
Will the dataset be distributed to third parties outside of the entity (e.g., company, institution, organization) on behalf
of which the dataset was created? If so, please provide a description.
Yes. The data will be made public for non-commercial research use.
The dataset will be made publicly available for research purposes exclusively. Access to the dataset will be granted to applicants who meet the following criteria: (1) the data will be used solely for academic research, (2) users must agree to the terms and conditions, (3) users must provide an ethical clearance form signed by their department or similar, ensuring compliance with ethical procedures when utilising the dataset. These requirements are in place to minimise the potential for misuse of the dataset while maintaining an approach in the spirit of open access. Once these criteria are fulfilled, applicants will be directed to the data download page, where they can access the complete CLC dataset as a zip file. The DOI for the dataset is: 10.17863/CAM.100221.
The dataset will be released upon the release of the accompanying paper describing the material together with ethical and legal issues.
The dataset will be distributed under an individual licence containing the terms ensuring legal and ethical compliance. The terms can be seen on the repository website HTTPS://WWW.CST.CAM.AC.UK/RESEARCH/SRG/PROJECTS/LAW.
There are no particular IP-based or other restrictions imposed on the data by third parties. Crown copyright is acknowledged as regards the court decisions.
There are no export controls or similar regulatory restrictions related to this dataset
Who will be supporting / hosting / maintaining the dataset?
The corpus will be continuously maintained by the authors at the University of Cambridge.
The manager of the dataset can be contacted at clc@law.cam.ac.uk.
Rather than adopting an erratum for a fixed dataset, we will release multiple versions of the dataset which correct any discovered errors, with updates listed in a changelog.
Yes. The corpus will be updated and improved regularly by the research group. We will keep adding more features and cases to the dataset as they become available. We will release new versions of the corpus and will present a changelog on the corpus landing page.
If the dataset relates to people, are there applicable limits on the retention of the data associated with the instances (e.g., were individuals in question told that their data would be retained for a fixed period of time and then deleted)? If so,
please describe these limits and explain how they will be enforced.
This dataset does relate to people, with no limitation on availability. Individuals can request to be removed from the corpus by contacting us using information provided on the landing page, and we will comply if they do.
The corpus is versioned and controlled using GIT. Hence, different versions will be stored, although not necessarily publicly. Data will be versioned using semantic versioning where a major version will be changed if the format or structure of data changes will not be back-compatible. Minor versions will be changed if new features are added and patch versions will be changed if errors are corrected in the corpus.
If others want to extend, augment, build on, or contribute to the dataset, is there a mechanism for them to do so? If so, please provide a description. Will these contributions be validated/verified? If so, please describe how. If not, why not? Is there a process for communicating/distributing these contributions to other users? If so, please provide a description.
We will allow suggestions and contributions to the dataset. These changes will be quality-controlled by random sampling, and if contributions improve the corpus, they will be merged using GIT for traceability. Information on how to contribute suggestions is available on the corpus's landing page.