Title: Exploring Large Language Models for Classical Philology
Summary: They define two probing tasks to investigate the knowledge acquired by models pre-trained on Classical texts. The experiments provide the first benchmarking analysis of existing models of Ancient Greek. 
Phenomenon type: General Capability (A broadly useful ability, which could be relevant to multiple applications)
Phenomenon defined by authors: No
Phenomenon definition: The tasks are supposed to assess semantic and world knowledge in LLMs.
Definition scope: Comprehensive
Phenomenon contested: Not defined
Task definition: Measuring semantic and world knowledge in LLMs
Task item definition: A sentence
Task source: Author-crafted task examples (e.g. hand-written examples, manual transformation of existing data into questions)
Task ecology: Constructed task (e.g. predicting medical diagnoses from clinicians' notes)
Dataset size: ~550
Dataset metadata available: No
Sampling method: Unknown
Split reviewed: Test, Train
Response format: Multiple choice
Metric definition: Exact Match (accuracy, F1, precision, recall)
Metric subscores: No
Metric access: Outputs alone
Authorship: Academia
Benchmark availability: Link is provided but the data is not there
Metric face validity: Yes
Results interpreted by authors: Yes
Results compared: No
Results realism: No
Human baseline: No
Authors discuss validity: No
Taxonomy: Multilinguality / 
New taxonomy: General Capability / Multilinguality