Title: A Multi-Task Benchmark for Korean Legal Language Understanding and Judgement Prediction
Benchmark: LBOX OPEN
Summary: This work introduces LBOX OPEN, the first large-scale benchmark of Korean legal AI datasets, comprising a legal corpus of 147k precedents and multiple tasks including classification, legal judgment prediction, and summarization. It also presents LCUBE, the first Korean legal language model trained on this corpus.
Phenomenon type: Specific Application (A single use case, where the benchmark is likely to be examples of that use case)
Target phenomenon: legal language understanding and legal judgment prediction in the Korean context​
Phenomenon defined by authors: Yes
Phenomenon definition: we release LBOX OPEN, the first large-scale Korean legal AI benchmark that consists
of six datasets: (1) a large-scale legal precedent corpus (PRECEDENT CORPUS), (2) two classification tasks (CASE NAME, STATUTE), (3) two legal judgement prediction tasks (LJP-CRIMINAL, LJPCIVIL), and (4) one summarization task (SUMMARIZATION).
Definition scope: Subset
Definition integrity: Composite phenomenon
Definition integrity detail: Yes
Phenomenon contested: Contested
Task definition:  legal text classification (predicting case names and statutes from factual case descriptions), legal judgment prediction (estimating punishment ranges or claim acceptance levels from case facts), and summarization (generating summaries of legal rulings and reasoning sections). 
Task item definition: input text and the corresponding label  or output in case of summarisation
Task source: Real task examples (e.g. GitHub issues), Author-crafted task examples (e.g. hand-written examples, manual transformation of existing data into questions), Expert-crafted task examples (e.g. hand-written examples)
Task ecology: Partial real task (e.g. answering medical questions collected from real people), Constructed task (e.g. predicting medical diagnoses from clinicians' notes)
Task ecology detail: it is based on the real document, but not used in the real cases, and tasks as summarisation and classification are the constructed ones
Dataset size: 14.1k
Dataset size notes: 147k precedents (259M tokens)
Dataset metadata available: No
Sampling method: Convenience sample (creators found a set of tasks that was readily accessible), Targeted items (creators defined a task space and chose tasks within it strategically), Specific criteria (items were taken from a larger set based on specified rules)
Split reviewed: Test, Train, Validation
Response format: Short free response (e.g. single word or number), Free response (e.g. summary paragraph, executable code)
Metric definition: Exact Match (accuracy, F1, precision, recall), n-gram (BLEU, ROUGE, chrF)
Metric aggregation: Simple Mean
Metric statistics: simple mean and std
Metric subscores: No
Metric access: Outputs alone
Few-shot evaluation: No
Authorship: Mix (multiple authors from industry and academia)
Benchmark availability: Yes
Task face validity: Yes
Metric face validity: Yes
Results interpreted by authors: Yes
Results compared: No
Results comparison explanation: No comparisons made
Results realism: No
Human baseline: No
Authors discuss validity: Yes
Author validity detail: They discuss how the tasks are grounded in real-world legal processes, and mention that legal judgment prediction tasks remain especially challenging
Taxonomy: Law / 
New taxonomy: Specific Application / Law