Title: Struc-Bench: Are Large Language Models Good at Generating Complex Structured Tabular Data?
Benchmark: Struc-Bench
Summary: The paper introduces a new benchmark to assess LLMs’ proficiency in structuring tables and introduces a novel fine-tuning method, cognizant of data structures, to bolster their performance.
Phenomenon type: Specific Application (A single use case, where the benchmark is likely to be examples of that use case)
Target phenomenon: Generating structured tabular data 
Phenomenon defined by authors: Yes
Phenomenon definition: LLMs are tasked with generating complex struc- tured tables, a process that involves understanding both the content and the specific format require- ments, such as LaTeX syntax. This task extends beyond simple text generation as it demands preci- sion not just in content creation but also in adhering to a detailed and precise structural format.
Definition scope: Comprehensive
Definition integrity: Single cohesive phenomenon
Definition integrity detail: No
Phenomenon contested: Widely-agreed
Task definition: The task is generating structured tabular data.
Task item definition:  text tables, HTML tables, and LaTeX tables and their description
Task source: Modified from another benchmark (e.g. translation into another language)
Task ecology: Constructed task (e.g. predicting medical diagnoses from clinicians' notes)
Dataset size: ~16k
Dataset size notes: Train: 14.1k, Test:1700
Dataset metadata available: No
Sampling method: Random sample (creators defined a task space and sampled from it)
Split reviewed: Test, Train
Response format: Structured response (e.g. valid JSON, API call alone)
Metric definition: P-Score (Prompting Score) and H-Score (Heuristical Score)
Metric aggregation: Simple Mean
Metric statistics: simple mean
Metric subscores: No
Metric access: Outputs alone
Authorship: Academia
Benchmark availability: Yes
Task face validity: Yes
Metric face validity: Yes
Results interpreted by authors: Yes
Results compared: No
Results comparison explanation: No comparisons made
Results realism: No
Human baseline: No
Authors discuss validity: No
Taxonomy: Code Generation / 
New taxonomy: General Application / Data Analysis