Title: BEACON: Benchmark for Comprehensive RNA Tasks and Language Models
Benchmark: BEACON
Summary: This paper presents BEACON, the first comprehensive benchmark for evaluating RNA language models across 13 tasks related to RNA structure, function, and engineering. It analyzes various models and components, highlighting the benefits of single nucleotide tokenization and ALiBi positional encoding, and introduces BEACON-B, a strong, resource-efficient baseline model.
Phenomenon type: Specific Application (A single use case, where the benchmark is likely to be examples of that use case)
Target phenomenon: 
RNA understanding — the ability of models to perform comprehensive RNA-related tasks
Phenomenon defined by authors: Yes
Phenomenon definition: the ability of models to perform comprehensive RNA-related tasks such as

Understanding RNA structure (e.g., secondary structure, contact map, distance map)
Predicting RNA function (e.g., splice sites, isoform usage, non-coding RNA function, modifications)
Supporting RNA engineering (e.g., predicting vaccine degradation, programmable RNA switches, CRISPR targeting)
Definition scope: Subset
Definition integrity: Composite phenomenon
Definition integrity detail: Yes
Phenomenon contested: Contested
Task definition: Models are evaluated on 13 RNA-related tasks that span structural analysis (e.g., secondary structure, contact maps), functional prediction (e.g., splice sites, RNA modifications), and engineering applications (e.g., CRISPR targeting, vaccine stability). Each task involves either classification or regression at the nucleotide or sequence level, with specific evaluation metrics for the biological context​.
Task item definition: one RNA sequence, typically composed of nucleotide characters (e.g., A, U, C, G), along with a corresponding label or set of labels depending on the task 
Task source: Real task examples (e.g. GitHub issues), Modified from another benchmark (e.g. translation into another language)
Task ecology: Partial real task (e.g. answering medical questions collected from real people), Constructed task (e.g. predicting medical diagnoses from clinicians' notes)
Dataset size: total: 96,283
Dataset size notes: total: 793,047 and 77,836
Dataset metadata available: No
Sampling method: Convenience sample (creators found a set of tasks that was readily accessible), Targeted items (creators defined a task space and chose tasks within it strategically), Specific criteria (items were taken from a larger set based on specified rules)
Split reviewed: Test, Train, Validation
Response format: Short free response (e.g. single word or number), Structured response (e.g. valid JSON, API call alone)
Metric definition: Exact Match (accuracy, F1, precision, recall), Top L Precision, Top-k ACC, R^2, AUC, MCRMSE, Spearmann core
Metric aggregation: Simple Mean
Metric statistics: Mean and std
Metric subscores: No
Metric access: Outputs alone
Authorship: Academia
Benchmark availability: Yes
Task face validity: Yes
Metric face validity: Yes
Results interpreted by authors: Yes
Results compared: No
Results comparison explanation: No comparisons made
Results realism: No
Human baseline: No
Authors discuss validity: No
Taxonomy: Biology / 
New taxonomy: Specific Application / Physical Sciences