BioReason: Incentivizing Multimodal Biological Reasoning within a DNA-LLM Model
Authors: Adibvafa Fallahpour
Andrew Magnuson
Purav Gupta
Shihao Ma
Jack Naimer
Arnav Shah
Haonan Duan
Omar Ibrahim
Hani Goodarzi
Chris J. Maddison
BO WANG
Venue: NeurIPS 2025 poster ()
Keywords: Biological Reasoning
DNA Foundation Models
Large Language Models (LLMs)
Reinforcement Learning
Multimodal
Primary area: machine_learning_for_sciences
TL;DR: BioReason introduces a novel DNA-LLM architecture where the LLM directly processes genomic information, achieving superior, interpretable multi-step biological reasoning and accelerating mechanistic discovery.

Abstract: Unlocking deep and interpretable biological reasoning from complex genomic data remains a major AI challenge limiting scientific progress. While current DNA foundation models excel at representing sequences, they struggle with multi-step reasoning and lack transparent, biologically meaningful explanations. BioReason addresses this by tightly integrating a DNA foundation model with a large language model (LLM), enabling the LLM to directly interpret and reason over genomic information. Through supervised fine-tuning and reinforcement learning, BioReason learns to produce logical, biologically coherent deductions. It achieves major performance gains, boosting KEGG-based disease pathway prediction accuracy from 86% to 98% and improving variant effect prediction by an average of 15% over strong baselines. BioReason can reason over unseen biological entities and explain its decisions step by step, offering a transformative framework for interpretable, mechanistic AI in biology. All data, code, and checkpoints are available at [https://github.com/bowang-lab/BioReason](https://github.com/bowang-lab/BioReason).

BIOREASON: Incentivizing Multimodal Biological Reasoning within a DNA-LLM Model
Adibvafa Fallahpour
Andrew Magnuson 1,2
adibvafa. fallaapur 0mahhouurmmmillom utooontto.cc
andrew andrew.magnusen www.mmmnmwnnnii...mmmmmmmmmmm utoronto.ca
Purav Gupta 122
Shihao Ma
wurav.gupta guptaamailll...oo utoronto.cco
shihao.. wwwmaillm.........m..mmm.mmm..aa utoronto.ca
Jack Naimer
Arnay Shah 1223 arnav shah@mail www.uutooooo...
jack. naimer@mail wtw.unto..commmm
Haonan Duan
Omar Ibrahim² omar ibrahin22uhn ca
haonan duan@mail utoronto.ca
Hani Goodarzit4. hani. gondrzoodaziiuustf. edu
Chris J. Maddison 1,227
cmaddis@cs. toronto.
Bo Wang 1,2,3
bovangerectorisstitttt....e ai
University of Toronto
Vector Institute
University Health Network (UHN)
Are Institute 'Cohere
University of California, San Francisco
'Google DecpMind
Abstract
Unlocking deep, interpretable biological reasoning from complex genomic data is major Al challenge hindering scientific discovery. Current DNA foundation models, despite strong sequence representation, struggle with multi-step reasoning and lack inherent transparent, biologically intuitive explanations. We introduce BIOREA- SON, pioneering architecture that, for the first time, deeply integrates DNA foun- dation model with a large language model (LLM). This novel connection enables the L.M directly process and reason with genomic information fundamental input, fostering new form of multimodal biological understanding. BIOREASON 's sophisticated multi-step reasoning developed through supervised fine-tuning and targeted reinforcement learning, guiding the system to generate logical, biologi- cally coherent deductions. Across biological reasoning benchmarks, BIOREASON significantly improves performance, raising accuracy on KEGG-based disease pathway prediction from 86% 98% and delivering an average 15% gain over strong single-modalit baselines in variant effect prediction tasks. BIOREASON reasons over unseen biological entities and articulates decision-making through interpretable, step-by-step biological traces, offering a transformative approach for Al in biology that enables deeper mechanistic insights and accelerates testable hypothesis generation from genomic data. Data, code, and checkpoints are publicly available at https: // //t///iittt titu.b..... com/ coo/bowwngg-aa 1ace-laa/PRa wtwlllaaa//aessseeeeeeiass Bioaeeso...
"Equal contribution.
Equal advising.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Introduction
Biological data, spanning genomics, transcriptomics, biomedical literature, and more, is expanding at an unprecedented rate, creating immense opportunities for scientific discovery. This data explosion has catalyzed the development of foundation models (FMs), deep networks trained on vast datasets that enable wide arra of downstream tasks. In genomics, DNA foundation models [6, 10, 31, 45, 13] have demonstrated remarkable capabilities by learning dense sequence representations that drive splice site identification, variant effect prediction, and regulatory element characterization.
Despite these advances, a critical challenge with foundation models still persists: effectively translat- ing these learned representations into mechanistic insights and falsifiable hypotheses. Current DNA foundation models, while powerful in their representational capacity, typically function as "black boxes" that lack the inherent ability to generate transparent, biologically intuitive explanations [4, 26]. These limitations are prominent in complex biological problems requiring mechanistic understanding. such as gene pathway analysis, phenotype prediction, and disease mechanism elucidation [9].
Large language models (LLMs) [32, 2. 12, 34] have rapidly advanced in reasoning capabilities, problem-solving, and knowledge depth. Through sophisticated training methods including rein- forcement learning and supervised fine-tuning, these models demonstrate increasingly sophisti- cated multi-step reasoning across domains from mathematical [14, 28, 30, 18]. However, LLMs alone lack the specialized architecture to effectively process raw genomic sequences and often fail capture nuanced biological patterns in genetic data.
-olving to logical deduction
This disconnect between powerful sequence representations of DNA foundation models and sophis- ticated reasoning capabilities of LLMs creates significant barrier developing Al systems that provide deep mechanistic insights comparable to biology domain experts. To bridge this gap, we present BIOREASON: novel architecture that fundamentally integrates a DNA foundation model with an LLM, enabling a new paradigm of multimodal biological understanding and reasoning.
BIOREASON is distinguished by its ability to create unique flow of information between genomic and natural language. This architecture enables the system to process raw DNA sequences while leveraging the reasoning capabilities of modern LLMs generate biologically coherent explanations and predictions. Through training methodology combining supervised nhe-tuming and reinforcement learning, BIOREASON develops the capacity for sophisticated multi-step reasoning over genomic data; a capability that neither DNA foundation models nor LLMs can achieve independently.
Contributions. Our key contributions include:
Novel multimodal architecture. The first successful integration large DNA foundation model with an LLM, establishing new methodology for Al-driven biological studies.
Advanced reasoning. A systematic training approach combining supervised fine-tuning and reinforcement learning that incentivizes multi-step biological reasoning.
New biological reasoning benchmarks. Development and curation of novel benchmarks for evaluating biological reasoning capabilities, including annotated reasoning dataset for gene pathway and disease prediction dataset from KEGG [20].
Empirical performance improvements. Demonstration that BIOREASON outperforms both DNA foundation models and LLMs with average performance gains of 15%+ baseline.
Interpretable reasoning traces. mechanism for generating step-by-ste by--stp biological reasoning traces that provide interpretable predictions, enhancing scientific insight and hypothesis generation
2 Background & Related Work
2.1 DNA Foundation Models
Recent years have witnessed the emergence of DNA foundation models [6, 10, 43, 31] that have significantly accelerated discovery throughout the biological sciences. These models extract mean- ingful representations directly from nucleotide sequences by pre-traine pre-training vast genomic datasets.

---

Moreover, comprehensive benchmarking studies [15] have demonstrated their proficiency across various genomies tasks in both zero-shot and fine-tuned settings.
Evo2 [6], in particular, represents a significant advancement as one of the largest genomic foundation models date, enabling extremely long-range context windows and predictions. Its ability generate complete bacterial and yeast genomes underscores the potential of these models to capture complex genomic patterns [22]. However, a critical limitation persists: these foundation models operate as "black boxes," lacking the interpretability necessary to explain how they derive conclusions from their embeddings. This opacity hampers the advancement of biological knowledge by obscuring the mechanistic insights that could otherwise be derived from model predictions.
2.2 Large Language Models for Biological Reasoning
LLMs have demonstrated remarkable capabilities understanding and generating human- text, with substantial success in interpreting and reasoning over complex biomedical data, Recent reviews [43] highlight their success across diverse domains, from clinical applications involving patient notes to biological research contexts. The development of specialized models pre-trained on biomedical literature [25], has further enhanced their domain -spncfc performance.
Senomics-focused LLMs such as GeneGPT [19], agentic models such as TxGemma [40], and scientific reasoning models such as rbiol [18] represent initial attempts to integrate language models with genomic databases. ChatNT [35] takes a step further by integrating DNA foundation model representations with language models multimodal framework. However, no previous work has trained such systems for complex biological reasoning. We present the first multimodal framework designed and trained to perform biological reasoning by combining textual knowledge from L.MMs with nucleotide-level representations from DNA foundation models.
2.3 Genomics Benchmarks
DNA foundation models are typically evaluated on established benchmarks encompassing diverse prediction tasks, including regulatory element identification, variant effect prediction, transcription factor binding site prediction, and splice site classification. Comprehensive benchmarking frameworks like BEND [29] provide standardized evaluation protocols that enable meaningful comparisons between models across these supervised tasks.
While these benchmarks effectively measure performance on specific downstream applications, they inadequately evaluate a model's capacity for higher-order reasoning or hypothesis generation, capabilities essential for advancing scientific understanding. This represents critical conceptual gap between current evaluation metrics and the sophisticated reasoning abilities desired from next- generation foundation models. The field requires benchmarks that challenge models perform multi-step logical reasoning and predict potential biological mechanisms. This need motivated our curation of the KEGG pathway database [20] to create multi-step reasoning, variant effect prediction dataset that specifically evaluates a model's capacity for mechanistic biological reasoning.
BioReason Model
We introduce BIOREASON, a multimodal framework designed to unlock deep, interpretable biological reasoning by synergistically integrating genomic and language data. BIOREASON operates two primary input streams: (i) one or more genomic sequences, denoted SDNA: and (ii) textual queries, QTEXT- These queries are processed by an LLM-specific tokenizer, (-),((), into a sequence of M tokens (Wi...., WM) from the LLM's vocabulary VLLM Current methods often fall short in this domain: LLMs treat SDNA as simple strings, thereby missing rich genomic features, while DNA Foundation Models (fDNa) capture these features but primarily yield task-specific discriminative out- puts (e.g., classification or regression scores) rather than interpretable natural language. BIOREASON bridges this gap by deriving contextualized DNA embeddings from the SpNa input(s) and integrating them with the tokenized QTEXT to form unified multimodal input sequence, XIIM, for its core LLM. This direct integration enables the generation of explanatory text, YouT = (V1.... 9K), grounded in genomic nuances. The output YouT presents biological reasoning and the final response. Figure depicts the overall architecture.
3

---

Stacked
Embeddings
|| |000
Qwen3-4B
Qwvnn Tokenizer
Qwend Embedding Matrix
end
dire, starr pxiddraaend De Chrowosoms Number: 12 'e betwork befinicion
Learnable Linear Projection
dim Qwend
ethinis
Step 1: The C-G susstitution at position 40810433
Evo2
BioResson Template
chromesome 12 occurs in the USRK2 gene, which encoces leudine eich repeat kinow 2 bene mulli donaii proten in with boch ki and CTPase activities.
Number:
Network Dehrition of the Pathway: LSGEG CVCS em APAF1 CASP9 CASP3
Stap 2:
Fualy resurs in gain-of- effect. enhanding LRRKZ's activity.
Evo2-10
Lio0ch inucine rich repest kinawe 2 CVCS cytochromee somati: APAF1; apoptotic pezéidase activating factor 1 CASPP: cospase 9 CASP3: cospose
Genes the Pathway:
fruncated response)
with is characteristic moter sympproms. </tbink
CCTCCAGGETCCO
Given context, the biological allele, speciccally what disease contribute
CCTECAGOCTOCOCCTIN
Parkinson's disease
DNA Sequenocs
Question
Response (with reasoning)
Figure 1: BIOREASON Architecture. Schematic representation of our novel multimodal framework that integrates a DNA foundation model with a Large Language Model.
3.1 DNA Foundation Model (SDNA) Encoder
JONa transforms each input SDNA sequence into contextualized embeddings. We utilize established DNA foundation models such as StripedHyena2 (e.g.. Evo2) [6, 33J, or the Nucleotide Transformer (NT), [11]. the SDNA. Each SDNA is first processed by its respective DNA-specific tokenizer, (-) which segments it into a sequence of DNA tokens, = (di...., .z,): each token d, can represent
more nucleotides. If an input SpNa sequence, after tokenization by ToNa, exceeds defined context length (c.g., 2048 DNA tokens), it is truncated. The chosen fDNA architecture then maps each token sequence D to sequence of sighndimensional per-token embeddings EDNA (e1, ......,,.,,) E R dau, These duma dimeensiia embeddings capture context- -eeenxtdeependet genomic features. The weights of the SDNA are kept frozen during BIOREASON'S training and inference.
3.2 Large Language Model (fLL) Backbone
The fULM is the primary reasoning engine and text generator. We employ Qwen3 [41, 42], an autoregressive Transformer-base based L.M,, initialized with its original pre-trained weights. This model receives the multimodal input sequence XILM and is trained to predict the next token yi in the sequence Your, conditioned on the preceding tokens Yei and XLLM- Mathematically, we optimize the parameters OLIM of the fLIM by maximizing the log-likelihood of the observed sequences:
Σ log P(yllvei XLLMS L
The fLLM utilizes special tokens to structure conversational interactions and reasoning within its textual output Your- These include tokens defining user and assistant roles (e.g.. structuring reasoning steps (e.g., <think>
< <lim_start| wwwser/asssstaa ... ...<//hink>), alongside standard padding tokens (< (eenoftext|>>>
3.3 Multimodal Genomic Integration
Genomic information, DNA embeddings from fUNA. is integrated into the fuLM'S 's input by stacking these with embeddings the user's ' query QIEXT and special tokens such as idna_starts and <dna_end>. Key this integration is the preparation of the DNA embedding block, EpNAA formed from one or more input DNA sequences. For each sequence SpNA, its fDNA- generated embedding sequence EDNA, E TRL x (where I is its tokenized length) is projected by learnable linear layer, Proj: R - Rd to yield ED of dimension dum. The resulting E/NA block is obtained by stacking all E, sequences along the sequence dimension.

---

Concurrently, the user's tokenized query QTEXT = (wi...., war) is mapped to its embedding sequence E_.. (E(w))... ((ww)) by the fILM 's input embedding layer, E(-). Similarly, the special tokens tina_star>> and <dna_end end> are embedded via E.(.) to produce é<dna_start> and @<dna_and> ands- These components are then stacked along the sequence dimension to form the multimodal
input XLLM for JLLM-
XITM (C<dna_start>: EDN Cedna_end end> EQua)
All constituent embedding vectors within XLLM receive positional information via Rotary Position Embedding (RoPE) [39], applied according to their final sequence positions. This strategy enables fILM fineggriined attention over both genomic and textual components within a unified modality.
3.4
Group Relative Policy Optimization (GRPO)
To further enhance BIOREASON 's reasoning performance beyond supervised fine- tne-ttning, employ Group Relative Policy Optimization (GRPO) [36, 12), reinforcement learning strategy tailored for refining reasoning generation in language models. GRPO leverages reward signals within groups of sampled outputs, eliminating the need for an explicit value estimator. We implement Dr. GRPO [27], an unbiased variant of GRPO that improves token efficiency while maintaining performance.
For the full formalism, including the composite reward design, advantage normalization, and the clipped surrogate objective with KL regularization, please refer to Appendix A.4.
Datasets
To develop multimodal DNA-LLM model with reasoning capabilities, curated three datasets: one novel dataset specifically designed to incentivize reasoning and two adapted from established benchmarks. The adapted datasets are derived from Clin Var [24] and OMIM [1], which are widely used for variant effect prediction tasks. Our novel dataset is based on KEGG Network Variants data [20] and enhanced with cross-linked metadata from several public variant repositories including Clin [24], OMIM [1], dbSNP [37], and COSMIC [38]. This novel dataset relies on the high-quality manual annotations and descriptions from the curators of KEGG, for gene pathway descriptions and downstream phenotypic effects, like disease.
4.1
KEGG-Derived Biological Reasoning Dataset
4.1.1 Dataset Integration and Statistics
We present a high-quality biological reasoning dataset derived from the Kyoto Encyclopedia of Genes and Genomes (KEGG) pathway database [20), consisting of 1,449 entries that elucidate the mechanistic connections between genetic variants and disease phenotypes. As seen in Figure 2, the dataset construction involved a rigorous multi-stage process that integrates structured pathway information with variant data to enable step-by-step reasoning across molecular networks.
For primary data integration, extracted pathway network data from KEGG [20]. focusing disease-an disease-associatee associated molecular interactions. Pathway data was augmented with variant information from clinical databases (Clin Var, dbSNP, OMIM, COSM) [24, 37, 1, 38] through semi-automated mapping protocol [17, 21] that preserved relational integrity between genomic loci and functional elements within pathways. Each molecular network represented using a standardized symbolic notation (e.g.. "GENE1+GRE +GENE2 - GENE3 -I GENE4") that encapsulates interaction types including activation, inhibition, complex formation, and transcriptional regulation.
To support variant interpretation, we included paired reference and variant sequences with precise alignment coordinates. These sequences have average length of approximately 4,000 base pairs, with most variants differing by only 1-3 nucleotides from their reference sequences.
4.1.2 Reasoning Path Construction and Curation
distinctive feature of this dataset is inclusion of explicit causal reasoning paths connecting genetic variants to discase phenotypes via defined molecular mechanisms; these paths were constructed
5

---

KEGG Varten Networkes Disava
Querton-fessee Recosiing Tros
Mutriten anthuaedd LSRKC intrinalc apapletic pattway
Network: ND1047
Varaat HEURTO COOCTICTTE
Stp DVG scstttution pes the 40200403..
Adteny
USKK? > CVCS -n APM CAMP Chard
LFSE22 kusin dich repect Miuia
adee und sus abeirs Uiitd
Navo
APRF3 sontirle anptiiie actiol lecter
Chudk 377
(S332 CXC en APN CASPY CASPO
cand nocor to Pandromms divcce...
(890) leave ral lase lond voled (CUPa capler
the soc tock has secloraraiiin douuteend eatar cumpuart. 4I' and.
cedarloon in inpristed grews
USVn assocaaed
pataggreed These pathopmnc metisler cnaoc clwans...
abic sacticall ohht dhowe dan this comfututt
298 Variant Networks
1449 QA Pains, representing 173 Variant Networks and 37 Unique Discnsees
Taats
muteel
Samatrd
370
290
L23
X73
1,197
31213
Searryy Doarde
Calri Naer
Collagy
Milliia Oslaro
20200
0000
6700 T56-2
306-3
Figure 2: BIOREASON Dataset Curation and Composition. A. Representative Variant Network element from the 298 networks utilized in study, illustrating the relationship between genomic variants and their corresponding disease annotation that serves as ground truth for generating mechanistic reasoning traces. B. Exemplar of structured question-answer pair with an accompanying multi-step reasoning trace demonstrating the expected logical progression from genomic variant phenotypic outcome. C. Pipeline for data acquisition, integration, and curation across the three BIOREASON tasks. D. Distribution of train/test splits across the three curated datasets. 10% of train dataset was used for validation. E. Distribution of disease categories represented within the datasets, highlighting the diversity of variants and diseases represented in the datasets.
of KEGG
using the Claude 3.7 Sonnet model [2] and grounded with contextual disease information from the KEGG disease database [20]. For training and evaluation, the dataset is structured into standardized question-answer pairs: questions (illustrated in Figure 2B) incorporate variant details, network definitions, and gene descriptions, while answers provide concise mechanism- associations. The accompanying reasoning paths (mean length: 303.8 words) elaborate these mechanistic variant- to-phenotype links with precise molecular information.
4.2
Variant Effect Prediction of Coding Sequences
This dataset originated from the GPN-MSA [5] study. Affected gene names and disease phenotypes were extracted from Clin Var [24] XML records (via NCBI's Entrez Direct tool [21]). while benign variants were sourced from gnomAD v3.1.2 [8] (requiring allele number 2 25,000 and minor allele frequency (MAF) 5%). The data split by chromosome (Chr 1-7, 9-22, X, Y for training; Chr 8 for testing). For training augmentation, GPT-4o [32] generated 50 semantically equivalent question variations per sample. prompting for pathogenic/benign classification and conditional disease phenotype prediction using chromosome and gene context; mutations linked to multiple discases treated as distinct samples for comprehensive phenotype coverage.
4.3 Variant Effect Prediction of Coding Non-SNVs
Coding non-SNVs were sourced from the ClinVar [24] database (2024-02-28 release). We filtered variants to retain only coding non-SNVs within the nuclear genome, affecting <64 base pairs, of certain significance, and with a review status of at least two stars, matched to GRCh38.p14 transcripts. After extracting affected gene names and disease phenotypes where available, a custom algorithm partitioned the dataset to ensure balanced disease representation in train/test splits. Finally, to augment training data, GPT-4o [32] generated 50 semantically equivalent question variations for each entry, using gene and chromosome number as context, prompting for pathogenic/henign classification and, if pathogenic, the associated discase phenotype.

---

Experiments
5.1 Datasets
BIOREASON's performance evaluated on three datasets (detailed in Section 4):
KEGG-Derived Biological Reasoning Dataset. This dataset (1,449 variants, 37 unique diseases) evaluates multi-step mechanistic reasoning. Input: paired reference and variant DNA sequences (SpNa), and textual query (QtEXT) with pathway/gene context. Task: predict the mutation's effect and resulting disease by seguuence-t--sequeeee generation of YoUT containing step-by-step reasoning between <think> (special tokens) and the final disease.
Variant Effect Prediction of Coding Sequences (VEP-Coding). Comprising 50,083 core variant entries, this dataset tests classifying coding variants. Input: paired reference and variant DNA sequences (SDNA), and textual query (QrEXT) providing gene and chromosome context. Task: sequence generation to predict if a variant is benign, pathogenic with its associated disease. Split: Chromosomes (Chr) 1-7, 22, X, Y for train/validation; Chr for testing.
Variant Effect Prediction of Coding Non-SNVs (VEP-Non-SNV), Containing 36,088 core non- SNV entries, this dataset addresses non-SNV alterations (e-g., indels <64 hp). Input: paired reference and variant DNA sequences (SDNA), and an augmented textual query (QTEXT) providing gene and chromosome context. Task: sequence generation predict if a non-SNV is benign, or pathogenic with its associated disease(s). We used stratified train/test splits to ensure balanced disease representation.
5.2 Models and Baselines
To benchmark BIOREASON's performance, we evaluated it against several baseline models, catego- rized as DNA foundation models (fDNa) and Large Language Models (fLLM).
For fDNA baselines, we utilized pre-trained Evo2-1B IB [6] and Nucleotide Transformer (NT-500M) [11] models. For downstream predictions, fDNA models were adapted with an attention head where a single learnable query vector attends the sequence token embeddings produce final sequence representation. For (fLLM) baselines, we fine-tuned pre-trained Qwen3 models of two sizes: Qwen3- 1.7B and Qwen3-4B [41, 42, 34). These models were trained to receive text queries and DNA sequences treated as plain text strings and generate text with reasoning steps and final predictions.
The proposed BIOREASON models, were evaluated in several SDNa and fLIM combinations. Specif- ically, we tested Ev and NT-500M as JUNA encoders, each paired with Qwen3-1.7BB and Qwen3-4B as fLLM backbones. The primary training methodology for all BIOREASON configura- tions was Supervised Fine-Tune Tuning (SFT). Reinforcement Learning (RL) fine-tuning using the GRPO algorithm was subsequently applied to select DNA-LLM models.
5.3 Experimental Setup
Our experimental setup varied by model archiectrr--BroRRAAAN,, LLM-only, or fuNy-only- and task. BIOREASON and LLM-only models underwent Supervised Fine-Tuning (SFT), with LLM parameters efficiently updated via Low-Rank Adaptation (LoRA) [16]. For fiNNA-only baselines, core DNA model weights were frozen; only a task-specific attention head and classifier were trained.
SFT objectives for these models differed: for the KEGG Dataset Task, models generated reasoning steps between <think> tokens and final disease prediction. For VEP Datasets Tasks, they aimed for pathogenic/benign classification and conditional discase prediction for pathogenic variants. During SFT, specialized attention mask restricted loss computation exclusively to the response between <think> tokens and final answer tokens, excluding those from the input query or DNA embeddings. Select BIOREASON models were further optimized with GRPO. Details for LoRA configurations, all SFT and RL hyperparameters, and GRPO reward functions are provided in Appendix A.
Performance evaluation metrics were task-specific. The KEGG Dataset Task utilized Accuracy, Macro Fl Fccoc,,, Macro Precision, and Macro Recall as a multi-class disease prediction assessment, considering potential class imbalances. For VEP Datasets Tasks, Accuracy and F1 F1-score measured the binary pathogenic/benign classification. All LLLM and DNA-LLLM generations were deterministic with decoding temperature of O. We leveraged vLLM for fast inference. [23]
7

---

Table 1: Performance comparison of fDNA-only, LLMMonly, and DNA-LLM (BIOREASON) models
on 290 test datapoints of the KEGG-derived biological reasoning task.
Model
Accuracy
F1-Score
Precision
Recall
[DNA] NT 500M
86.55
69.76
73.23
66.61
[DNA] Evo2 1B
88.28
72.43
75.23
69.83
[LLM] Qwen3
85.17
65.71
71.39
64.19
[LLM] Qwen3 4B
90.00
79.66
88.24
75.08
[DNA-LMM] NT + Qwen3 - 1B
89.31
81.46
88.24
77.30
[DNA-LLM] NT Qwen3 1B (+GRPO)
91.72
75.06
79.41
72.89
(DNA-LLM] NT + Qwen3 4B
95.86
86.25
88.2
84.95
[DNA-LLM] NT Qwen3 4B (+GRPO)
98.28
90.15
91.18
89.62
[DNA-LMM] Evo2 + Qwen3 - 1B
90.42
75.62
77.42
73.91
[DNA-LLM] Evo2 + Qwen3 44B
95.17
86.14
91.18
83.33
[DNA-LLM] Evo2 + Qwen3 44 (+GRPO)
98.28
93.05
94.12
92.48
Table 2: Performance comparison of foNA-only. LLM-only, and DNA-LLM (BIOREASON) models on Variant Effect Prediction (VEP) benchmarks (VE-Coding with 1.23K and VE-Non-SNV with 873 test datapoints), evaluating pathogenic/henign classification.
Variant Effect Coding
Variant Effeet Non-SNV
Model
Accuracy
F1-Score
Accuracy
F1-Score
[DNAJ NT 500M
60.91
45.20
67.93
65.97
[DNA] Evo2 - 1B
70.07
49.19
76.17
66.51
[LLM] Qwen3 1B
46.55
34,82
70.67
76.21
[LLM] Qwen3 44B
48.99
39.58
61.86
67.60
[DNA-LLM] NT Qwen3 1B
55,58
54.50
72.82
76.93
[DNA-LLM] NT Qwen3 4B
60.94
55.66
65.59
73.00
[NA-LLMJ Evo2 + Qwen3 1B
72.83
68.90
88.20
89,91
[DNA-LMM] Evo2 + Qwen3 4B
80.21
80.00
83.85
85.02
Beyond variant-eff tasks, we also experimented with BIOREASON using the supervised DNA foundation model Enformer [3] for chromatin accessibility prediction, see Appendix B.
5.4 Quantitative Results
BIOREASON's DNA-LLM hybrids deliver consistent, substantial gains over single-modality baselines on the KEGG-derived reasoning benchmark (Table 1). The Evo2+Qwen3-4P model with GRPO reaches 98. 38.28% % accuracy and 93.05% F1, outperforming the standalone Qwen3-4B (90.00%/79.66) and Evo2 DNA-only (88. (8.22%%/7..4%% models. Notably, scaling from Qwen3- IB Qwen3-4B substantially amplifies both base performance and GRPO effectiveness: while the IB backbone shows mixed results with GRPO (aceuracy improves from 89.31 % to 91.72%, but actually declines from 81. 11.46% to 75.06%) the 4B backbone demonstrates dramatic and consistent improvement with GRPO, jumping from 95.17% to 98. 28.28% accuracy and from 14% to 93.05% F1. This pattern holds acTOss DNA foundation models; NT+Qwen3-4B with GRPO also reaches 98.28% accuracy with 90.15% FI, confirming that larger LLM backbones provide a significantly more effective substrate for reinforcement learning refinement while simultaneously elevating overall hybrid performance.
On the VEP benchmarks (Table 2), the DNA-LLM hybrids maintain their advantage across variant effect prediction tasks. Evo2+Qwen3-4B achieves 80.21% % accuracy and 80.00% F1 in coding variant classification, far surpassing DNA-only (70.07%/49. 19%) and LLM-only (48.99% 088.99%/39.mm 55%% baselines. For non-SNV classification, Exo2+Qwen3-1 leads with 88 38.20% accuracy and 89.91% 91% F1, surpassing DNA-only (76. 166.17%/66.55 %) and I.L.-only (70.67% %). Class-wise breakdowns for the KEGG benchmark reported in Appendix C.

---

Question
Ground Truth (KEGG)
BioReason's 's Output
Question
Siep The vore- NEGC 800 repiesents
<thikk
sunsitution position 4915969 on chromotome 17.
Step & exration lees altert prste intructture
-Jnn startfuusrr
ocoumiig in the PFNJ gene which encoties profilind
KEGG Data Point
Chromosooe Number
soti monomers or republe actis pelytistatiin
Network Defiviion of the Pathway.
functionally by Important domairs
This redicodde
potor affocting
Stea Thi specitc nusation contrtuee cytssiatio peoteostaa puttwwi....
(ndinnnnneennnn ffhit Aciid/eeennios)
Step The pragpnssive of mator recuter noufts
The C-G meutaiion in PRNZ gone Skel disepts aiaffli
foretinn Impaiing
rep arth dymams The
ACTII actin bets ACTG3) sctin 1 |P NL: prefii
liak clinial feature ALS.
trigingg outosl ars disturetin afrechs motor reurnnason fiiiing
The C-G mutation the PFN gene... cytorieretal defects distunction depentration, uttinotell farrilaa
devdogmmnt familaal aamootoophi Interal ackenes <hhhiic
PFN1 allele specifcall what cheose does this conbrioute
to?
<Jin cndjp-jjer starthasdittee
Amyotmothic Lateral Scleresis (ALS)
Ansawe sweyctrophic bitee edteood (AL ML- wodd-
Figure 3: Case Study of BIOREASON's Output
5.5
Case Study
To illustrate BIOREASON'S 's reasoning capabilities, consider its analysis of PFN1 allele on chromo- some 17 within the pathway Actin (monomeric) // PFN1* // Actin (filamentous). BIORE- ASON correctly predicted Amyotrophic Lateral Sclerosis (ALS) as the resultant disease. Significantly, the model generated plausible 10-step mechanistic rationale, initiating by identifying specific C>G substitution in the PFNI gene. Its reasoning then connected this variant to profilin- dysfunction, impaired actin dynamics critical for cytoskeletal integrity, subsequent disruption of axonal transport in motor neurons, and finally, the motor neuron degeneration characteristic of ALS. This exam- ple highlights BIOREASON'S 's ability to not only make accurate predictions but also to articulate a step-by- btepp biologically coherent pathway from a genomic variant to complex disease phenotype.
6 Discussion
BIOREASON successfully integrates DNA foundation models with large language models, enabling direct LLM reasoning on genomic data. This overcomes key limitations of opaque DNA models and the inability of LLMs to natively process DNA sequences, resulting in enhanced multi-step biological reasoning and superior predictive performance over single-modality approaches.
A core strength of BIOREASON is its interpretable reasoning. By processing contextualized DNA embeddings within the LLM, cultivated through supervised fine-tuning, the system provides not only accurate predictions but also articulates its decision-making via step-hy-step mechanistic explanations formatted with '<think>' tokens. This transparency is crucial, allowing researchers to scrutinize the model's 's logic and translate computational outputs into testable scientific hypotheses.
The broader impact of this work lies in its potential to accelerate biological discovery. BIOREASON offers a robust tool for gaining deeper, mechanistic insights from genomic data, aiding in understand- ing complex disease pathways and the formulation of novel research questions. The development and application of benchmarks focused on multi-step reasoning, as utilized in this study, will further propel the advancement of Al systems capable of sophisticated biological understanding.
Limitations. Despite its strengths, BIOREASON has several limitations. Reliance on curated datasets such as KEGG introduces potential biases and limits coverage of lessccharacterizze regions. The computational cost of encoding long DNA sequences and applying reinforcement learning (GRPO) raises training and inference time, reducing scalability to whole-genome real-time settings. DNA sequences were truncated to 2048 tokens due to hardware limits, potentially omitting distal context. Finally, lack of robust uncertainty quantification limits reliability in high-stake decisions.
Future Work. Future work will focus on expanding BIOREASON's scope and applicability. Key directions include incorporating orthologous sequences to enhance data diversity and model gen- eralizability, and adapting the core framework to other biological modalities such as RNA and protein sequences, thereby addressing broader range of research questions. Additionally, BIOREA- SON's improved variant effect prediction capabilities can be leveraged for impactful applications in genome-wide association studies (GWAS) and clinical mutation interpretation.

---

7 Conclusion
BIOREASON advances computational biology by seamlessly integrating high-capacity DNA sequence encoders with the flexible reasoning of large language models, yielding unified framework that excels both mechanistic pathway inference and variant pathogenicity prediction. Across LEGG-derived reasoning tasks and VEP benchmarks, our DNA-LLM hybrids consistently outperform models restricted to a single modality while generating transparent, stepwise explanations that facilitate expert validation. This tight multimodal fusion, further refined through reinforcement learning, not only boosts accuracy but also opens new avenues for interpretable genomic analysis. Future efforts will focus scaling model size and data, designing leaner architectures, and leveraging modalities such as protein and RNA to broaden BIOREASON 's utility medicine and biological discovery.
Acknowledgments
We would like thank Parsa Idehpour for his foundational contributions the design and implemen- tation of the multimodal GRPO infrastructure, which was essential for large-scale experimentation. We are deeply grateful to Guillaume Filion for his thoughtful guidance the selection of biologically meaningful benchmarks. We also thank Ronald Xie for his valuable insights into model architecture. We appreciate Arman Sayed-Ahmadi for stimulating discussions that helped shape the project's future research directions. Finally, we extend our gratitude to Clem Delangue for his support and encouragement in making our models publicly accessible through the Hugging Face platform.
10

---

References
[1] J. Amberger, C. A. Bocchini, A. F. Scott, and A. Hamosh. Mckusick's online mendelian inheritance in man (omim@). Nucleic Acids Research, 37:D793, 2008. ISSN 03051048. doi: 10. 1093/NAR/KN6655. URL https: httpp:///mc...e.eeee ncbii nlm.nih. htt.://ww.......a.aaaaaaaaaaaaaa
[2] Anthropic. Claude 3.7 sonnet, February 2025. URL
claude-3- 3---------- sonnee.. Accessed: 2025-05-13.
[3] Z. Avsec, V. Agarwall, D. Visentin, J. R. Ledsam, A.
K. R. Taylor, Y. Assael, J. Jumper, P. Kohli, and D. R. Kelley. Effective gene expression prediction from sequence by integrating long-range range interactions. Nature Methods, 18(10):116-1203, Oct. 2021. ISSN 1548- 7105. doi: 102.0880441322-021-112222222221. 1-1592-021-01255
URL https: /twp:::/www.... http:///wwwww.....ee/eeee nature. com/ articles/
[4] G. Benegas, C. Ye, C. Albors, J. C. Li, and Y. S. Song. Genomic language models: Opportunities and challenges. ArXiv, page arXiv:2407.114352 2024. ISSN 2331- 8422. URL https://pmc. ncbi nlm. nih. pubmedcentral. wwaettrra..........mm
mih, waah..gv/arrtee wtt.g/atiiteeeeeeerrrrrrr htt.::.a.aa...dad1117755555
[5] G. Benegas, C. Albors, A. J. Aw, C. Ye, and Y. S. Song. A dna language model based on multispecies alignment predicts the effects of genome-wide variants. Nature Biotechnology, pages 1-6, 1 2025. ISSN 15461696. doi: 10.
101.:/4444401111SSSSSUBBEEAAAAA 14, MACHINE-LLEARNING URL https:///w/w nature.
:5552200,,611KKKR=GGEEEEE WAW.E//WDDDDGTTTTTSSSCCCCIEEEAAE htt/aaaarieeess11s//.aae 020-00011-221--- 02511-W.
[6] G. Brixi, M. G. Durrant, J. Ku, M. Poli, G. Brockman, D. Chang, G. A. Gonzalez, S. H. King, D. B. Li, A. T. Merchant, M. Naghipourfar, E. Nguyen, C. Ricei-Tam, D. W. Romero, Sun, A. Taghibakshi, A. Vorontsov, B. Yang. M. Deng, L. Gorton, N. Nguyen, N. K. Wang, E. Adams, S. Baccus, S. Dillmann, Ermon, D. Guo, R. Ilango. K. Janik, A. X. Lu, R. Mehta, M. R. Mofrad, M. Y. Ng. J. Pannu, C. Ré, J. C. Schmok, J. S. John, J. Sullivan, K. Zhu, G. Zynda, Costa, T. Hernandez- Boussard, E. Ho, M.-Y. Liu, T. McGrath, K. Powell, D. P. Burke, H. Goodarzi, P. D. Hsu, and B. L. Hie. Genome modeling and design across all domains of life with evo bioRxiv, 2025. doi: 10. 10112025.02.. 18. 6388818.. URL https:/ //ww:..//wwww/e http://wwwwwww.o.o//////tt/tt.wt biorxii........................b org/ hetgg/ooneen//eer/e 22AAA///222//222/1222222 /20////22222/22////////22// 02. 18.18.1..... 638918.
D. Balsam, P. Collison,
[7] W. Brown. Granular format rewards for eliciting mathematical reasoning ca- pabilities in small language models. https://gist... wtw.ggittt......iiiiiiiee cww/wo/o/////// www.wwllcwbb////cccccccccccccccc GitHub Gist.
[8] S. Chen, L. C. Francioli, J. K. Goodrich, R. L. Collins, M. Kanai, Q. Wang, J. Alföldi, N. A. Watts, Vittal, L.. D. Gauthier, Poterba, M. W. Wilson, Tarasova, Phu, R. Grant, M. Yohannes, Z. Koenig. Y. Farjoun, E. Banks, S. Donnelly, S. Gabriel, N. Gupta, S, Ferriera, Ruano-Rubio, M. Covarrubias, C. Llanwarne N. Petrillo, G. Wade, T. Jeandet, R. Munshi, K. Tibbetts, M. Abreu, C. A. A. Salinas, T. Ahmad, C. M. Albert, D. Ardissino, I. M. Armean, E. G. Atkinson, G. Atzmon, J. Barnard, S. M. Baxter, L. Beaugerie, E. J. Benjamin, D. Benjamin, M. Boehnke, L. L. Bonnycastle, E. P. Bottinger, D. W. Bowden, M. J. Bown, H. Brand, S. Brant, T. Brookings, S. Bryant, S. E. Calvo, H. Campos, J. C. Chambers, C. Chan, K. R. Chao, S. Chapman, D. Chasman, R. Chisholm, J. Cho, R. Chowdhury, M. K. Chung, W. K. Chung, K. Cibulskis, B. Cohen, K. M. Connolly, A. Correa, B. B. Cummings, D. Dabelea, J. Danesh, D. Darbar, P. Darnowsky, J. Denny, R. Duggirala, J. Dupuis, P. T. Ellinor, R. Elosua, J. Emery, F. England, J. Erdmann, Esko, E. Evangelista, D. Fatkin, J. Florez, A. Franke, J. Fu, M. Färkkila, K. Garimella. J. Gentry, G. Getz, D. C. Glahn, B. Glaser, S. J. Glatt, D. Goldstein, C. Gonzalez, L. Groop, Gudmundsson,
C. Tolonen, S. Novod, L. Bergelson, D. Roazen,
Haessly, C. Haiman, I. Hall, C. L. Hanis, M. Harms, M. Hiltunen, M. M. Holi, C. M. Hultman, C. Jalas, M. Kallela, D. Kaplan, J. Kaprio, S. Kathiresan, E. E. Kenny, B. J. Kim, J. Kim, D. King, Kirov, J. Kooner, S. Koskinen, H. M. Krumholz, Kugathasan, S. H. Kwak, M. Laakso, N. Lake, T. Langsford, K. M. Laricchia, T. Lehtimäki, M. Lek, E. Lipscomb, R. J. Loos, W. Lu, S. A. Lubitz, T. Luna, R. C. Ma, M. Marcus, J. Marrugat, K. M. Mattila, S. McCarroll, M. I. McCarthy, J. L. McCauley, D. McGovern, R. McPherson, J. Meigs, O. Melander, A. Metspalu, D. Meyers, E. Minikel, D. Mitchell, V. K. Mootha,
A. Naheed, S. Nazarian, P. M. Nilsson, M. C. O' O'Dovoa,,, Y. Okada, D. Ongur, L. Orozco,
11

---

M. J. Owen, C. Palmer, N. D. Palmer, A. Palotie, K. S. Park, C. Pato. A. E. Pulver, D. Rader, N. Rahman, A. Reiner, A. M. Remes, D. Rhodes, Rich, J. D. Rioux. S. Ripattii D. M. Roden, J. I. Rotter, N. Sahakian, D. Saleheen, V. Salomaa, A. Saltzman, J. Samani, K. E. Samocha, A. Sanchis-Juan, Scharf, M. Schleicher, H. Schunkert, S. Schönherr, F. G. Seaby, S. H. Shah, M. Shand, T. Sharpe, M. B. Shoemaker, T. Shyong, E. K. Silverman, M. Singer- Berk, P. Sklar, J. T. Smith, J. G. Smith, H. Soininen, H. Sokol, R. G. Son, J. Soto, T. Spector, C. Stevens, N. O. Stitziel, P. F. Sullivan, J. Suvisaari, E. S. Tai, K. D. Taylor, Y. Y. Teo, M. Tsuang, T. Tuomi, D. Turner, T. Tusae-Luna, E. Vartiainen, M. Vawter, L. Wang, A. Wang, J. S. Ware, H. Watkins, R. K. Weersma, B. Weisburd, M. Wessman, N. Whiffin, J. G. Wilson, R. J. Xavier, A. O' D'DnnelllLurree
M. Solomonson, C. Seed, A. R. Martin, M. E. Talkowski, H. L. Rehm, M. J. Daly. G. Tiao, B. M. Neale, D. G. MacArthur, and K. J. Karczewski. A genomic mutational constraint map using variation in 76,156 human genomes. Nature 2023 625:7993, 625:92-100, 12 2023. ISSN 1476-4687. doi: 10. https://wwy http:://wwwwawww..eee/meemme nature.... com/ /tt/aarttccesseeeee/eee/eeeeseet 200p///24411100000005--55--115-0
URL
[9] M. E. Consens, B. Li, A. R. Poetsch, and S. Gilbert. Genomic language models could transform medicine but not yet. npj Digital Medicine, 8:1-4, 12 2025. ISSN 23986352. doi: N0.1038/$41746-025 011077660025-1000-454500 4.03:44.UBIMEMTA-55555 692, W09,70OOKKRD-HEATTHHCC.. 020 01603-4
HEALTH+POLICY. URL https: / //ww://wwwwwc nature. http//ariceesss/1/1/
[10] H. Dalla-Toree Torre, L. Gonzalez, J. Mendoza- N. L. Carranza, A. H. Grzywaczewski, F. Oteri, C. Dallago, E. Trop, B. P. de Almeida, H. Sirelkhatim, G. Richard, M. Skwark, K. Beguir, M. Lopez, and TT. Pierrot. Nucleotide transformer: building and evaluating ro- bust foundation models for human genomics. Nature Methods, 22:287-297, 2 2024. ISSN 15487105. doi: 10. 1008003/411555 1110:00240-22222.SUB/EEE-- com/aarticles//41155222222 024 02--4455-2222---22------------2
14,1111,,,,. 1305, 20,,,,2,,1,,,,,,,,,,,,,,,,,,,,,, nature.
URL
[11] H. Dalla-Torre, I. Gonzalez, J. Mendoza-Revilla, N. Lopez Carranza, A. H. Grzywaczewski, F. Oteri, C. Dallago, E. Trop. B. P. De Almeida, H. Sirelkhatim, G. Richard, M. Skwark, K. Beguir, M. Lopez, and T. Pierrot. Nucleotide transformer: building and evaluating robust foundation models for human genomics. Nature Methods, 22(2):287-297, Feb. 2025. ISSN 1548-7091, 1548-7105 doi: 10.1038/41592-004-02222 htmm/articleess44152222 0252332..
URL https: //ww://www//// nature.
[12] Dee,Seck-AL, D. Guo. D. Yang. H. Zhang, Song, R. Zhang, R. Xu, Q. Zhu, S. Ma. Wang. X. Bi, X. Zhang, X. Yu, Y. Wu, Z. F. Wu, Z. Gou, Z. Shao, Z. Li, Z. Gao. A. Liu, B. Xue, B. Wang, B. Wu, B. Feng, C. Lu, C. Zhao, C. Deng, C. Zhang, C. Ruan, D. Dai, D. Chen, D. Ji. E. Li, F. Lin, F. Dai, F. Luo, G. Hao, G. Chen, G. Li, H. Zhang, H. Bao, H. Xu, H. Wang. H. Ding, H. Xin, H. Gao, H. Qu, H. Li, Guo, J. Li, Wang, Chen, J. Yuan, Qiu, J. Li, Cai, J. Ni, J. Liang. J. Chen, K. Dong, K. Hu, K. Gao, K. Guan, K. Huang, K. Yu, L. Wang, L. Zhang, L. Zhao, L. Wang, L. Zhang, L. Xu, L. Xia, M. Zhang, M. Zhang, M. Tang, M. Li,
M. Wang, M. Li, N. Tian, P. Huang, P. Zhang, Q. Wang, Q. Chen, Q. Du, R. Ge. R. Zhang, R. Pan, R. Wang, R. Chen, R. L. Jin, R. Chen, S. Lu, S. Zhou, S. Chen, S. Ye, S. Wang, S. Yu,
S. Zhou, S. Pan, S. S. Li, S. Zhou. S. Wu. S. Ye, T. Yun. T. Pei. T. Sun. T. Wang, W. Zeng, W. Zhao, W. Liu, W. Liang, W. Gao, W. Yu, W. Zhang, W. L. Xiao, W. An, X. Liu, X. Wang, X. Chen, X. Nie, X. Cheng, X. Liu, X. Xie, X. Liu, X. Yang, X. Li, X. Su, X. Lin, X. Q. Li. X. Jin, X. Shen, X. Chen, X. Sun, X. Wang, X. Song, X. Zhou, X. Wang. X. Shan, Y. K. Li, Y. Q. Wang, Y. X. Wei, Y. Zhang, Y. XU, Y. Li, Y. Zhao, Y. Sun, Y. Wang, Y. Yu, Y. Zhang, Y. Shi, Y. Xiong, Y. He, Y. Piao, Y. Wang, Y. Tan, Y. Ma. Y. Liu, Y. Guo, Y. Ou, Y. Wang, Y. Gong,
Y. Zou, Y. He, Y. Xiong, Y. Luo, Y. You, Y. Liu, Y. Zhou, Y. X. Zhu, Y. Xu, Y. Huang, Y. Li,
Y. Y.Zheng, Y. Zhu, Y. Ma, Y. Tang, Y. Zha, Y. Yan, Z. Z. Ren, Z. Ren, Z. Sha, Z. Fu, Z. Xu, Z. Xie, Z. Zhangg Z. Hao, Z. Ma, Z. Yan, Z. Wu, Z. Gu, 7. Zhu, Z. Liu, 7. Li, Z. Xie, 7. Song, Z. Pan, Z. Huang, 7. Xu, 7. Zhang, and 7. Zhang. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. 2025. URL htthtt/ss///attt /ttp///axxvvvv// ieciiv.orgg/eee org/pat httpp/ff//500 22001.2944
[13] A. Fallahpour, V. Gureghian, G. Filion, A. B. Lindner, and A. Pandi. Codontransformer: A multispecies codon optimizer using context-aware neural networks. Nature Communications, 16(1), Apr 2025. doi: 10. 10.1038/s4 00188/41466--00
[14) A. Fallahpour, J. Ma, A. Munim, H. Lyu, and B. Wang. Medrax: Medical reasoning agent for chest x-ray, 2025. URL /tatps://arxv.... org/ abs////s/..../ 02673.
12

---

[15] H. Feng, L. Wu, B. Zhao, C. Huff, J. Zhang, J. Wu, L. Lin, P. Wei, C. Wu, P. W. pwei, and A. Professor. Benchmarking dna foundation models for genomic sequence classification running title: Dna foundation models benchmarking. doi: 10.1101/2024.0K 00000/2022.08116666888833 16.608288. URI.
https://doo hstps://aoi.orr/10.. 1101/2024 08. 16. 608288.
[16] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen. Lora: Low-rank adaptation of large language models, 2021. URL htttps///rxiv..ie www.xii.orggbbb///666 abs/ 09685.
[17] E. Huckvale and H. N. Moseley. kegg pull: a software package for the restful ac- cess and pulling from the kyoto encyclopedia of gene and genomes. BMC Bioinformal-
ics, 24:1-17, 12 2023. ISSN 14712105.
11116666118899222222200-00117710 doi: 10.1186/S12859-000
12. URL https: http::/s//mmbiiinoooooooooeeeeee htt:://mmmbooiooootiiccsss hwwp:iceeaiiiimmmmeaemaaaaa.. oot.a/aaa.cooo/arrrraaee/e//.... $12859-028 05208-2208220 h httttp://ccrettivcoooooo hwtpaa/s...roooooooooeeeeeeeeeee http/pubiccdommi/eer////.1
[18] A.-.. Istrate, F. Milletari, F. Castrotorres, J. M. Tomczak, M. Torkar, D. Li, and T. Kar- aletsos. rbiol training scientific reasoning Ilms with biological world models soft veri- fiers. bioRxiv, 2025. doi: 10. 101/2000 5..8.18... 18.670981 URL htttps:/wwet biorxiv. org/ content/early2/00 0000002/////2///22 28/21/202 .0.. 18. 670981
[19] Q. Jin, Y. Yang. Q Chen, and Z. Lu. Genegpt: augmenting large language models with domain tools for improved access to biomedical information. Bioinformatics, 40, 2 2024. ISSN 13674811. doi: (0.009/HOOFFORMATTCSS/TTEEET URL https: https://dx...t doi org/10. 1093/bioinf wwmaticss/taettt/
[20] M. Kanehisa, M. Furumichi, Sato, Y. Matsuura, and M. Ishiguro-Watanatee Kegg: biological systems database as a model of the real world. Nucleic Acids Research, 53:D672-D677, 2025. ISSN 0305-1048. doi: D0.1093/AARGKAA9009 URL https: //dx doi www.ooo./....00. nar nar/gkae909
[21] J. Kans. Entrez direct: E-utilities on the unix command line entrez programming utilities help ncbi bookshelf, 4 2013. URL https: //www//wwwww////////////// ncbi nlm. nih. wtw.gg////ooo/BBBB100000
[22] S. H. King, C. L. Driscoll, D. B. Li, D. Guo, A. Merchant, G. Brixi, M. E. Wilkinson, and B. L. Hie. Generative design of novel bacteriophages with genome language mod- els. bioRxiv, 2025. doi: content/ haarly/222509////
110/22225..9.1..65511 URL https://wwet biorxiv. org/ 09. 12. 675911.
[23] W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, and I. Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.
[24] M. Landrum, J. M. Lee, G. Riley, W. Jang, W. S. Rubinstein, D. M. Church, and R. Ma- glott. Clinvar: Public archive of relationships among sequence variation and human phenotype. Nucleic Acids Research, 42, 1 2014. ISSN 03051048. doi: 10. (0.1093/NARGKKT111.. URL https://pubmi: ncbi nlm. nih.
www..o./.......... gww.go///2220 gov/24234437/.
[25] J. Lee, W. Yoon, S. Kim. D. Kim, S. Kim, C. H. So, and J. Kang. Biobert: pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36: 1234-1240, 2 2020. ISSN 1367-4803. doi: 10. https://dkx
URL
doi httiiddddi...ogg/////// http:///....////oooooooota.a..a.
[26] Q. Z. Hu, Y. Wang, L. Li, Y. Fan, I. King, G. Jia, S. Wang, L. Song, and Y. Li. Progress and opportunities of foundation models in bioinformatics. Briefings in Bioinformatics, 25:548, 2024, ISSN 14774054. doi: 10. (0.1093/BIBBBBAES44 URL https: http//xx.dooo doi. bib/bbae548.
[27] Z. Liu, C. Chen, W. L. P. Qi, T. Pang, C. Du, W. S. Lee, and M. Lin. Understanding ri-zero-like training: A critical perspective. In Conference on Language Modeling (COLM), 2025.
[28] Z. Ma, J. Zhang, Z. Liu, J. Zhang, J. Tan, M. Shu, J. C. Niebles, S. Heinecke, H. Wang, C. Xiong, Krishna, and Savarese, Latte: Learning think with vision specialists, 2025.
URL https: //tp///aaivvv//aa//ea/eeeeeawwva org/ ab1/2412...2222 05479.
13

---

[29] F. I. Marin. F. Teufel, M. Horlacher, D. Madsen, D. Pultz, O. Winther, and W. Boomsma. Bend: Benchmarking dna language models on biologically meaningful tasks. 12th International Conference on Learning Representations, ICLR 2024, 11 2023. URL wwwtthtp////aaaeeeee org/
[30] S. M. Narayanan, J. D. Braza, R.-.. Griffiths, A. Bou, G. Wellawatte, M. C. Ramos, I. Mitch- ener, S. G. Rodriques, and A. D. White. Training a scientific reasoning model for chemistry, 2025. URL https:///axas:ti org/ wrrg/bb//2006... 17238.
[31] E. Nguyen, M. Poli, M. Faizi, A. W. Thomas, C. B. Sykes, M. Wornow, A. Patel, C. Rabideau, S. Massaroli, Y. Bengio, S. Ermon, S. A. Baccus, and C. Ré. Hyenadna: Long-range genomic sequence modeling at single nucleotide resolution. ArXiv, 2023. ISSN 2331-8422. URL https://arxiv org/ org/pdf/////poooooo..gg/ogg /2006 15794.
[32] OpenAI, :, A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Beutel, A. Borzunov, A. Carney, A. Chow, A. Kirillov, A. Nichol, A. Paino, A. Renzin, A. T. Passos, A. Kirillov, A. Christakis, A. Conneau, A. Kamali, A. Jabri, A. Moyer, A. Tam, A. Crookes, A. Tootoochian, A. Tootoonchian, A. Kumar, A. Vallone, A. Karpathy, A. Braunstein, A. Cann, A. Codispoti, A. Galu, A. Kondrich, A. Tulloch, A. Mishchenko, A. Back, A. Jiang, A. Pelisse, A. Woodford, A. Gosalia, A. Dhar, A. Pantuliano, A. Nayak, A. Oliver, B. Zoph, B. Ghorbani, B. Leimberger, B. Rossen, B. Sokolowsky, B. Wang. B. Zweig, B. Hoover, B. Samic, B. McGrew, B. Spero, B. Giertler, B. Cheng, B. Lightcap. B. Walkin, B. Quinn, B. Guarraci, B. Hsu, B. Kellogg, B. Eastman, C. Lugaresi, C. Wainwright, C. Bassin, C. Hudson, C. Chu, C. Nelson, C. Li, C. J. Shern, C. Conger, C. Barette, C. Voss, C. Ding, C. Lu, C. Zhang, C. Beaumont, C. Hal- lacy, C. Koch, C. Gibson, C. Kim, C. Choi, C. McLeavey, C. Hesse, C. Fischer, C. Winter, C. Czarnecki, C. Jarvis, C. Wei, C. Koumouzelis, D. Sherburn, D. Kappler, D. Levin, D. Levy, D. Carr, D. Farhi, D. Mely, D. Robinson, D. Sasaki, D. Jin, D. Valladares, D. Tsipras, D. Li, D. P. Nguyen, D. Findlay, E. Oiwoh, E. Wong. E. Asdar, E. Proehl, E. Yang. E. Antonow, E. Kramer, E. Peterson, E. Sigler, E. Wallace, E. Brevdo, E. Mays, F. Khorasani, F. P. Such, F. Raso, F. Zhang, F. von Lohmann. F. Sulit, G. Goh, G. Oden. G. Salmon, G. Starace, G. Brockman, H. Salman, H. Bao. H. Hu, H. Wong, H. Wang. H. Schmidt, H. Whitney, H. Jun. H. Kirchner, H. P. de Oliveira Pinto, H. Ren, H. Chang, H. W. Chung, I. Kivlichan, I. O'Coneell, I. O' O'Connlll,
A. Welihinda, A. Hayes, A. Radford, A. Mądry, A.
I. Osband, I. Silber, Sohl, Okuyucu, I. Lan, I. Kostrikov, Sutskever, I. Kanitscheider,
I. Guirajani, J. Coxon, J. Menick, J. Pachocki, J. Aung. J. Betker, J. Crooks, J. Lennon, J. Kiros, J. Leike, J. Park, J. Kwon, J. Phang. J. Teplitz. Wei, J. Wolfe, Chen, J. Harris, Varavva, J. G. Lee, J. Shieh, J. Lin, J. Yu, J. Weng, J. Tang. J. Yu, J. Jang, J. Q. Candela, J. Beutler, J. Landers, J. Parish, J. Heidecke, J. Schulman, J. Lachman, J. McKay, J. Uesato, J. Ward, W. Kim, J. Huizinga, J. Sitkin, J. Kraaijeveld, J. Gross, J. Kaplan, J. Snyder, J. Achiam, J. Jiao, J. Lee, J. Zhuang, J. Harriman, K. Fricke, K. Hayashi, K. Singhal, K. Shi, K. Karthik, K. Wood, K. Rimbach, K. Hsu, K. Nguyen, K. Gu- Gu-Lemberg, K. Button, K. Liu, K. Howe, K. Muthukumar,
K. Luther, L. Ahmad, L. Kai, L. Itow, L. Workman, L. Pathak, L. Chen, L. Jing. L. Guy, L. Fedus, L. Zhou, L. Mamitsuka, L. Weng. L. McCallum, L. Held, L. Ouyang. L. Feuvrier, L. Zhang, L.. Kondraciuk, L. Kaiser, L. Hewitt, L. Metz, L. Doshi, M. Aflak, M. Simens, M. Boyd, M. Thompson, M. Dukhan, M. Chen, M. Gray, M. Hudnall, M. Zhang, M. Aljubeh, M. Litwin, M. Zeng, M. Johnson, M. Shetty, M. Gupta, M. Shah, M. Yatbaz, M. J. Yang, M. Zhong, M. Glaese, M. Chen, M. Janner, M. Lampe, M. Petrov, M. Wu, M. Wang, M. Fradin, M. Pokrass, M. Castro, M. O. T. de Castro, M. Pavlov, M. Brundage, M. Wang, M. Khan, M. Murati, M. Bavarian, M. Lin, M. Yesildal, N. Soto, N. Gimelshein, N. Cone, N. Staudacher, N. Summers, N. LaFontaine, N. Chowdhury, N. Ryder, N. Stathas, N. Turley, N. Tezak, N. Felix, N. Kudige, N. Keskar, N. Deutsch, N. Bundick, N. Puckett, O. Nachum, O. Okelola. O. Boiko, O. Murk, O. Jaffe, O. Watkins, O. Godement, O. Campbell-Moore, P. Chao. P. McMillan, Belov, P. Su, P. Bak, P. Bakkum, P. Deng. Dolan, P. Hoeschele, P. Welinder, P. Tillet, P. Pronin, P. Tillet, P. Dhariwal, Q. Yuan, R. Dias, R. Lim, R. Arora, R. Troll, R. Lin, R. G. Lopes, R. Puri, R. Miyara, R. Leike, R. Gaubert, R. Zamani, Wang, R. Donnelly, R. Honsby, R. Smith, R. Sahai, R. Ramchandani, R. Huet, R. Carmichael, R. Zellers, R. Chen, R. Chen, R. Nigmatullin, R. Cheu, S. Jain, S. Altman, S. Schoenholz, S. Toizer, S. Miserendino, S. Agar- wal, S. Culver, S. Ethersmith, S. Gray, S. Grove, S. Metzger, S. Hermani, S. Jain, S. Zhao S. Wu, S. Jomoto, S. Wu, Shuaiqi, Xia, S. Phene, Papay, S. Narayanan, S. Coffey, S. Lee, S. Hall, S. Balaji, T. Broda, T. Stramer, T. Xu, T. Gogineni, T. Christianson, T. Sanders,
14

---

T. Patwardhan, T. Cunninghman, T. Degry, T. Dimson, T. Raoux, T. Shadwell, T. Zheng, T. Underwood, T. Markov, T. Sherbakov, T. Rubin, T. Stasi, T. Kaftan, T. Heywood, T. Peter- son, T. Walters, T. Floundou, V. Qi, V. Moeller, V. Monaco, V. Kuo, V. Fomenko, W. Chang, W. Zheng. W. Zhou, W. Manassra, W. Sheu, W. Zaremba, Y. Patil, Y. Qian, Kim, Y. Cheng, Y. Zhang. Y. He, Y. Zhang, Y. Jin, Y. Dai, and Y. Malkov. Gpt-4o system card. 10 2024. URL https://arxee
wwwpg//p/...... /pdf///10.... 21276.
[33] M. Poli, J. Wang. S. Massaroli, J. Quesnelle, R. Carlow, E. Nguyen, and A. Thomas. Striped- Hyena: Moving Beyond Transformers with Hybrid Signal Processing Models, 12 2023. URL https:// //i thub. com http//toethhecooouuie wwwpeeeersriiiiieeeeeeeeee teripehyenaa..
[34] Qwen, :, A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng. B. Yu, C. Li, D. Liu, F. Huang, H. Wei, H. Lin, J. Yang. J. Tu, J. Zhang. J. Yang. J. Yang, Zhou, J. Lin, K. Dang, K. Lu, K. Bao, K. Yang, L. Yu, M. Li, M. Xue, P. Zhang, Q. Zhu, R. Men, R. Lin, T. Li, T. Tang, T. Xia, X. Ren,
X. Ren, Y. Fan, Y. Su, Y. Zhang. Y. Wan, Y. Liu, Z. Cui, Z. Zhang, and Z. Qiu. Qwen2.5
technical report. 12 2024. URL https: /twpvv/aaxvvvv//a///////a....i.a org/pdg. 221..1111..
[35] G. Richard, B. P. Almeida, H. Torre-Torre, C. Blum, I.. Hexemer, P. Pandey, S. Laurent, M. Lopez, A. Laterre, M. Lang, U. Sahin, K. Beguir, and T. Pierrot. Chatnt: A multimodal conversational agent for dna, rna and protein tasks. bioRxiv, 2024. doi: 10.1101/2024.04330 591835. URL https:///we /www. biorxiv org/ 591835.
[36] Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. K. Li, Y. Wu, and D. Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. URL https: //arxiv...iveee// org/ harg//bbe abs/2400 03300.
[37] S. T. Sherry, M. H. Ward, M. Kholodov, J. Baker, L. Phan, E. M. Smigielski, and K. Sirotkin. dbsnp: the nebi database of genetic variation. Nucleic Acids Research, 29:308-311, 2001. ISSN 03000001440 doi: 10. N0.193//AR/29.1..000 URL https: //dx. doi.oog/.1. wtg/10..//////o.. 10.3//000..//////////// 2.08.308
[38] 7. Sondka, N. B. Dhir, D. Carvalho-Silva, S. Jupe, Madhumita, K. McLaren, M. Starkey, Ward, J. Wilding, M. Ahmed, J. Argasinska, D. Beare, M. S. Chawla, S. Duke, I. Fasanella, A. G. N.oogi S. Haller, B. Hetenyi, L. Hodges, A. Holmes, R. Lyne, T. Maurel, S. Nair, H. Pedro, A. Sangrador- Vegas, H. Schuilenburg, Z. Sheard, S. Y. Yong, and J. Teague. Cosmic: a curated database of somatic variants and clinical data for cancer. Nucleic Acids Research, DADD210--D221,, 1 2024. ISSN 0305-1048. doi: 10.1093/NA//KKA9986 URL https: //dx. /oix.doi... wwg/oo////1/ 10. 1093/000////// htt///a//gkkaaa gkad866.
[39] J. Su, Lu,, S. Pan, A. Murtadha, B. Wen, and Y. Liu, Roformer: Enhanced transformer with rotary position embedding. 2023. URL https: //arxiv....iv/// org/ abs abs/2104. 09864.
[40] E. Wang, S. Schmidgall, P. F. Jaeger, F. Zhang, R. Pilgrim, Y. Matias, J. Barral, D. Fleet, and Azizi. Txgemma: Efficient and agentic llms for therapeutics.
[41] A. Yang. B. Yang, B. Hui, B. Zheng, B. Yu, C. Zhou, C. Li, C. Li, D. Liu, F. Huang, G. Dong, H. Wei, H. Lin, J. Tang. J. Wang. J. Yang, J. Tu, J. Zhang, J. Ma, J. Xu, J. Zhou, J. Bai, J. Hc, J. Lin, K. Dang. K. Lu, K. Chen, K. Yang. M. Li. M. Xue, N. Ni, P. Zhang, P. Wang, R. Peng, R. Men, R. Gao, R. Lin, S. Wang. S. Bai, S. Tan, T. Zhu, T. Li, T. Liu, W. Ge, X. Deng, X. Zhou, X. Ren, X. Zhang, X. Wei, X. Ren, Y. Fan, Y. Yao, Y. Zhang, Y. Wan, Y. Chu, Y. Liu, Z. Cui, Z. Zhang, and Z. Fan. Qwen2 technical report. arXiv preprint ar arXiv:2407. 10671, 2024.
[42] A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang. H. Wei, H. Lin,
J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Lin, K. Dang, K. Lu, K. Bao, K. Yang, L. Yu, M. Li, M. Xue, Zhang, Q. Zhu, R. Men, Lin, T. Li, T. Xia, X. Ren, X. Ren, Y. Fan, Y. Su, Y. Zhang, Y. Wan, Y. Liu, Z. Cui, Z. Zhang, and Z. Qiu. Qwen2.5 technical report. arXiv preprint arXiv:2412.15113 2024.
[43] Q. Zhang, K. Ding, T. Lyv, X. Wang, Q. Yin, Y. Zhang, J. Yu, Y. Wang, X. Li, Z. Xiang, K. Feng. X. Zhuang, Z. Wang, M. Qin, M. Zhang, Zhang, Cui, T. Huang, P. Yan, R. Xu, H. Chen, X. Li, X. Fan, H. Xing, and H. Chen. Scientific large language models: A survey on biological and chemical domains. Survey on Biological and Chemical Domains, 1:90, 1 2024. doi: 10.1 1...1/1//nnnnn http://mmmmo....mm.mmmmmm URL https: / aaxivv. org/ arg/pdf/ 2241/.20... 14656v2.
15

---

[44] J. Zhou and O. G. Troyanskaya. Predicting effects of noncoding variants with deep learn- ing based sequence model. Nature Methods, 12(10))931-934, Oct. 2015. ISSN 1548-7091, 1548-7105. doi: 10. 1038/mmeth.3547. URL htwp:::/t/s//ett http::://wwwwwwww.ee///e/meeeeee nature. com/articles ceaechhlss/mmehh..ee.. nmeth. 3547.
[45] Z. Zhou, Ji, W. Li, P. Dutta, R. V. Davuluri, and H. Liu. Dnabert-2: Efficient foundation model and benchmark for multi-species genome. 12th International Conference on Learning Representations, ICLR 2024, 6 2023. URL https: sstps://arxiv. http///xxvvv.or/.//ee wwwriivorgg/a wwt.t/tgttt///t/.............ttt /206 15006.
16

---

A Training Details
A.1 Hyper-Parameters
All experiments share the following base settings unless otherwise noted. Please look at our GitHub for all other details and training scripts.
Optimizer & regularization (SFT).
Optimizer: AdamW
Learning rate: 10-5
Weight decay: 1 10 10-2
Gradient accumulation: 8 steps
Random seed:
Devices:
LoRA adapters (SFT).
Rank: 32, Alpha: 64, Dropout: 0.05
Optimizer & regularization (GRPO).
Optimizer: AdamW
Learning rate: x 10-5
Weight decay: 10-2
Gradient accumulation: 4 steps
Learning Rate Scheduler: Cosine, 0.03 warmup ratio
Random seed: 23
LoRA adapters (GRPO).
Rank: 16, Alpha: 32, Dropout: 0.00
GRPO Parameters.
Number of generations: 8
Per device batch size: 8
Steps: 1000 (7 epochs)
Devices:
Temperature (4B parameters): 0.7
Temperature (1.7B parameters):
Top p: 0.95
Top k: 20
Beta: 0.0
Epsilon: 0.2
DeepSpeed & hardware.
Strategy: deepspeed_stage_2
CPUs per task:
RAM per node: 128-256 GB
Data-loader workers: 4
17

---

Task-specific settings:
KEGG pathway reasoning:
Batch size: I
Epochs: 5
Max legnth DNA: 2048
Max text length: 1024 (for LLM only increases to 8192 to fit the raw DNA sequences)
Variant effect prediction (coding & non-SNV):
Batch size: 2
Epochs: 3
Max legnth DNA: 2048
Max text length: 1024 (for LLM only increases to 8192 fit the raw DNA sequences)
A.2
Computational Resources
We conducted experiments using multiple GPU clusters equipped with NVIDIA A100 and H100 GPUs. A100 systems were equipped with Intel Xeon Silver CPUs, featuring 16-24 CPU cores, 24-32 threads, and 188-251 GB RAM. We used A100 GPUs for reinforcement learning, while other experiments were performed on single HI00 GPUs with Slurm-based orchestration and Deepspeed.
A.3 Reward Details
We use deterministic composite reward function adapted from [7, 12] to guide reinforcement learning with GRPO, emphasizing correctness and strict adherence to the reasoning format. Each completion is parsed using XML--aware extractor that isolates the final answer following the last </think> tag.
For each output i, the total reward r; is defined as the sum of the following components:
Correctness Reward. Rewards +2.0 if the extracted final answer matches or contains the ground- truth answer (case-insensitive substring match), and 0.0 otherwise.
Conciseness Reward. Rewards +0.5 if the extracted final answer contains ten or fewer words, encouraging brevity final responses and preventing loopholes around the correctness reward.
Format Reward. Rewards +0.5 the completion strictly follows the required reasoning structure of single <think> block followed by newline and final answer, with properly closed tags.
Each reward component computed independently and summed per sample, yielding a total reward in [0, 2.5]. Rewards are non-differentiable and used solely within GRPO to compute group-normalized advantages. No learned critic value function is used.
A.4 GRPO Details
Formally, given an input query XLLM, GRPO samples set of G outputs {o1,..., .G. from the current policy Now- Each candidate output comprises reasoning trace and a final response. Outputs are evaluated using composite nomain-specifee reward function r(g.o:), incorporating the rewards from Appendix A.3.
Dr. GRPO normalizes these rewards into advantage using the average and standard deviation:
A = r; mean
The policy parameters 0 are then optimized by maximizing the clipped surrogate objective:
(O) = B[XLIM ~ P(Q), { ~ Ta. (O|XLLM))
G
(o(o: XILM) A,, clip
To(o, To(o;[X[IM)) (IIM))
1 e, 1114) A. BBDKL(TTell ( Tew (o, XLIM)
min
Tou (o; XLLM)
1=1
with hyperparameters and 8.
18

---

A.5 GRPO Learning Curve
Figure illustrates the reward progression during GRPO training across three model configurations. Several key patterns emerge from the training dynamics:
Rapid initial learning. All models exhibit steep reward increases in the first 100-200 steps, demon- strating that GRPO efficiently guides the policy toward correct reasoning patterns early training.
Model size effects. The 4B parameter models reach stable high reward values significantly faster than the 1.7B model, with the larger models stabilizing around step 400 compared to step 800 for the smaller model. The 4B models also demonstrate lower variance in the later training stages, suggesting that increased model capacity provides more robust optimization under GRPO.
Near-optimal performance. All models eventually reach rewards approaching the theoretical maximum of 2.5, indicating successful acquisition of both correct answer generation and adherence to the required reasoning format. The plateau behavior after stabilization suggests stable policy optimization without catastrophic forgetting.
Architecture-agnostic learning, Both NT' (Nucleotide Transformer) and Evo2 DNA encoders paired with Qwen3-4B backbones follow nearly identical learning curves, indicating that the GRPO training procedure generalizes effectively across different DNA foundation model architectures.
Figure Mean reward progression during GRPO training across different BioReason model configu- rations over 1000 training steps. Shaded regions represent per-step reward variance across the batch of generations per query.
19

---

B Generalization to Supervised DNA Models
To demonstrate that BIOREASON's architecture generalizes beyond unsupervised DNA foundation models, we conducted an additional experiment integrating a supervised DNA model, Enformer [3], for chromatin accessibility prediction. This experiment addresses two key questions:
(I) Can BIOREASON leverage supervised DNA encoders trained on specific genomic tasks?
(2) Does the framework extend prediction tasks beyond trriset-to--iseaee reasoning?
While our primary experiments utilized unsupervised DNA foundation models that learn general- purpose representations from diverse genomic data, supervised models like Enformer offer comple- mentary strengths. Enformer is explicitly trained to predict genomic annotations including chromatin accessibility, outperforming DeepSEA DNase-seq variant effect prediction [3]. By integrating En- former into the BIOREASON framework, we evaluate whether our architecture can extract additional value from task task-specific pretrained representations.
B.1 Experimental Setup
We utilized public dataset derived from the DeepSEA paper [44] containing 60,000 data points for long-range genomics. The human genome segmented into 200bp bins, with each bin representing a classification instance. The task was formulated as multi-label binary classification to predict chromatin accessibility states ("open" vs "closed" for transcription factor binding) across 20 different DNase-seq tracks encompassing diverse cell types.
We chose to frame this classification task because DNase-seq data naturally represents binary chromatin accessibility states. While Enformer was originally trained for regression on epigenomic coverage prediction, our autoregressive architecture is optimized for discrete prediction tasks. En- former's demonstrated competency with DNase-seq data, as shown in its original evaluation, suggests it retains relevant knowledge for this formulation.
B.2 Model Configurations
We evaluated the following models:
Enformer (DNA-only): Standalone Enformer with frozen weights and a trained attention head for classification.
Qwen3-1.7B and Qwen3-4B (LLM-only): Qwen3 models fine-tuned on the task with DNA sequences treated text.
(irormeer-Qee3-1.777 and Enformer- -wenn-44- (DNA-LLM): BIOREASON hybrids integrat- ing Enformer embeddings with Qwen3 backbones.
All models were trained on a subset of the data and evaluated on held-out test set following the experimental setup.
B.3 Results
Table 3 presents comprehensive results across multiple evaluation metrics. The BIOREASON hybrids significantly outperform both standalone DNA and LLM baselines. The Enformer- -wernerr-Qwen3-4BB model achieves a Macro F1-score of 33.70%, nearly doubling the performance of standalone Enformer (17.18%). This substantial improvement demonstrates that integrating sequence embeddings with an L.M via the BIORFASON architecture provides significant performance gains even when using supervised DNA encoders. The Enformer--wen3-1.78 configuration similarly outperforms both component models in isolation.
These results provide two critical insights into BIOREASON's design. First, the significant perfor- mance improvement with Enformer validates that the synergy between DNA sequence embeddings and LLM reasoning is general architectural principle, not an artifact of using unsupervised models. This confirms the fundamental value of combining specialized sequence encoders with flexible LLM reasoning engines, regardless of the encoder's 's training paradigm.
Second, these results illuminate our rationale prioritizing unsupervised DNA foundation models in the main experiments. While supervised models like Enformer demonstrate strong performance when
20

---

Table 3: Performance comparison of JDNA-only, LLM-only, and DNA-LLM (BIOREASON) models
on chromatin accessibility prediction across 20 DNase-seq tracks (all metrics in %). "M" and "W" denote Macro and Weighted averages, respectively.
Model
FI-M
FI-W
Prec-M
Pree-W
Ree-M
Ree-W
[DNA] Enformer
17.18
15.86
15.61
14.85
31.07
29.92
[LLM] Qwen3 1.7BB
13.01
12.94
20.46
20.11
9.89
9.87
[LLM] Qwen3 4B
18.96
18.32
24.47
23.62
16.49
16.06
[DNA-LLM] Enformer Qwen3 1.7B
25.89
24.39
27.02
25.55
33.97
32.45
[DNA-LLM] Enformer Qwen3 4B
33.70
33.08
34.01
33.49
40.02
39.29
integrated into BIOREASON, they are highly specialized for their training objectives, in Enformer's case, predicting predefined set of epigenetic marks. The standalone Enformer's 's modest performance (17.18% F1) on this task suggests that its specialized embeddings, while powerful, may not be as transferable novel downstream tasks compared to the rich, general-purpose representations learned by unsupervised models from vast, diverse genomic data.
For BIOREASON's central goal of enabling broad, multi-step biological reasoning across diverse queries and tasks, unsupervised models provide more robust foundation. Nevertheless, this ex- periment demonstrates that BIOREASON's architecture is flexible enough to accommodate both supervised and unsupervised DNA encoders, allowing users to select the most appropriate foundation model for their specific application domain.
C Per-Disease Performance on the KEGG-Derived Reasoning Benchmark
This section reports class-wise results for the KEGG-derived biological reasoning benchmark, pro- viding a detailed view of BioReason's performance across individual disease categories. Each class corresponds distinct disease entity, with metries averaged across multiple variant instances. The results demonstrate consistently strong performance across diverse diseases, indicating that BioReason generalizes well beyond high-frequency classes and maintains stable reasoning quality across varied mechanistic contexts.
Table Per-disease performance on the KEGG-ddrived reasoning benchmark for NT Qwen3-4B and Evo2 Qwen3-4B (all metrics in %)).
NT + Qwen3-4B
Evo2 + Qwen3-4B
Disease
Freq.
Acc.
Prec.
Rec.
F1
Aec.
Prec.
Rec.
F1
Parkinson's disease
47
99.7
100.0
98.9
100.0
100.0
100.0
100.0
Alzheimer's disease
40
99.7
100.0
97.5
98.7
100.0
100.0
100.0
100.0
Spinocerebellar ataxia
36
100.0
100.0
100.0
100.0
99.3
97.2
97.2
97.2
Amyotrophic lateral sclerosis
35
99.7
100.0
97.1
98.6
100.0
100.0
100.0
100.0
Melanoma
17
99.7
100.0
94.1
97.0
99.7
100.0
94.1
97.0
Prion discasc
100.0
100.0
100.0
100.0
100.0
100.0
100.0
100.0
Colorectal cancer
12
99.3
85.7
100.0
92.3
99.7
92.3
100.0
96.0
Huntington's 's disease
10
100.0
100.0
100.0
100.0
100.0
100.0
100.0
100.0
Gaucher disease
7
100.0
100.0
100.0
100.0
100.0
100.0
100.0
100.0
Acute myeloid leukemia
100.0
100.0
100.0
100.0
100.0
100.0
100.0
100.0
21

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our abstract and introduction accurately reflect the contributions and scope of the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: See 6.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
22

---

Justification:
This paper does not include any theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Appendix A lists the necessary information in addition to our content pages.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
23

---

Answer: [Yes]
Justification: We provide full, anonymized source code and data alongside instructions for reproducing the main experimental results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/nips. wwtpccppppppppeeeeeeeeeccecceeee httggguddds/oeeeee waweeSubmissionolle smiiisssiiilcc for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Appendix A lists the necessary information in addition to our content pages. Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core of the paper to a level detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Comprehensive statistical analyses with multiple runs were limited by the significant computational resources required for these large-scale models. To ensure fair comparison and reproducibility from our single experimental runs, all text generations from BioReason and LLM-only models were performed deterministically by setting the decoding temperature to O, as detailed in Section 5.3. This approach provides stable point estimates for performance evaluation, with further investigation of inter-run variability deferred to future work.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
24

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Appendix A lists the necessary information in addition to content pages. Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https:
Answer: [Yes]
Justification:
The research conducted this paper conforms to the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. if there special consid- eration due to laws or regulations in their jurisdiction).
Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper highlights BioReason's 's substantial positive societal impact, pri- marily through its potential accelerate biological discovery and deepen the mechanistic understanding of diseases by generating testable hypotheses. Concurently, Section 6 (Limi- tations) addresses crucial considerations for responsible development, such as mitigating potential biases arising from curated training datasets and emphasizing the critical need for robust validation of the model's interpretable outputs to ensure their cautious and effective application in scientific research.
25

---

Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considcrations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning conrectly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This research does not release nor produce any data models that have high risk for misuse.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The creators and original owners of assets used in the paper have been appropriately cited.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
26

---

The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We release a codebase and our datasets alongside proper documentation and experiment scripts.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This research did not utilize human subjects or crowdsourcing.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This research did not utilize human subjects crowdsourcing.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
27

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: LLMs were not an important, original, or non-standard component of the core methods of this paper.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https: htops://neurips.. for what should or should not be described.
28


Reviews:
[{"review_id": "ajraA1Axa4", "reviewer": "Reviewer_X1Q7", "summary": "The paper introduces BIOREASON, a groundbreaking multimodal AI framework that integrates a DNA foundation model with a large language model (LLM) to enable deep, interpretable biological reasoning. Unlike traditional DNA models that act as black boxes, BIOREASON allows the LLM to directly process genomic sequences and generate step-by-step, biologically coherent explanations.\n\nKey highlights include:\n\n1. A novel architecture that fuses genomic and textual data for multi-step reasoning.\n\n2. Training with supervised fine-tuning and reinforcement learning to enhance logical deduction.\n\n3. Superior performance on tasks like disease pathway prediction and variant effect analysis, with up to 15% improvement over single-modality baselines.\n\n4. Transparent reasoning traces that explain predictions in a biologically meaningful way.\n\nBIOREASON represents a major step forward in AI-driven biology, offering both accuracy and interpretability for complex genomic analysis.", "questions": "refer to Strengths And Weaknesses", "limitations": "refer to Strengths And Weaknesses", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. Multimodal Innovation: BIOREASON is the first architecture to deeply integrate a DNA foundation model with a large language model (LLM), enabling direct reasoning over genomic sequences.\n\n2. Interpretable Reasoning: Unlike traditional “black-box” models, BIOREASON generates step-by-step, biologically coherent explanations using special tokens like <think>, enhancing transparency and scientific trust.\n\n3. Strong Empirical Performance: Achieves up to 97.24% accuracy and 86.30% F1-score on KEGG-based reasoning tasks, outperforming both DNA-only and LLM-only baselines by significant margins.\n\n4. Benchmark Contributions: Introduces new biological reasoning benchmarks, including a curated KEGG-derived dataset with mechanistic variant-to-disease pathways.\n\n5. Reinforcement Learning Enhancement: Uses Group Relative Policy Optimization (GRPO) to further refine reasoning quality beyond supervised fine-tuning.\n\n6. Case Study Validation: Demonstrates real-world utility by accurately predicting disease outcomes (e.g., ALS) with detailed mechanistic justifications.\n\n7. Generalization Across Tasks: Performs well on both mechanistic reasoning and variant effect prediction, showing versatility across biological domains.\n\nWeaknesses\n1. Dataset Bias: Heavy reliance on curated datasets like KEGG may introduce biases and limit generalizability to under-characterized genomic regions.\n\n2. Computational Overhead: Integrating long DNA sequences and applying reinforcement learning increases training and inference time, which may hinder scalability.\n\n3. Lack of Uncertainty Quantification: The model does not currently provide confidence estimates, which is a limitation for clinical or high-stakes applications.\n\n4. Implementation Complexity: The architecture and training pipeline (e.g., multimodal input fusion, GRPO) are technically complex and may be difficult to reproduce or deploy without significant resources.\n\n5. Limited Modal Diversity: Currently focused on DNA; future extensions to RNA, proteins, or multi-omics data are needed to broaden its biological utility.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thorough evaluation and insightful feedback on our work. We appreciate the recognition of our contributions in multimodal innovation, interpretable reasoning, and benchmark development. Below, we address the identified weaknesses.\n\n\n\n**Dataset Bias and Generalizability.**\nWe selected the KEGG dataset, despite its bias towards well-studied pathways, because its expert-curated nature was essential for our primary goal of training a model to perform deep, multi-step reasoning on genomic sequence. This required a foundation of ground-truth causal chains linking variants to disease phenotypes. KEGG provided the ideal, high-quality data necessary to rigorously benchmark this novel capability, which is a core contribution of our work.\n\nTo prove the framework generalizes beyond these curated examples, we trained and evaluated it on two large-scale Variant Effect Prediction (VEP) benchmarks. These tasks challenge the model with over 86,000 coding variants sourced from broader databases like ClinVar and gnomAD, covering diverse mutation types including SNVs and non-SNVs. Many of these variants are under-characterized compared to the KEGG examples. BioReason’s high accuracy across these benchmarks confirms its ability to generalize, effectively classifying a wide array of variants without relying on the detailed pathway information used during its reasoning training.\n\n\n\n**Computational Overhead.**\nWe have made several deliberate design choices to ensure our framework remains efficient. During training, the large DNA foundation model is kept frozen, and we employ Low-Rank Adaptation (LoRA) for parameter-efficient fine-tuning of the language model, which significantly reduces the training burden. Additionally, we truncate input DNA sequences to a fixed context length during training to save memory. For inference, our publicly available framework is designed for compatibility with optimized inference servers like vLLM, which dramatically reduces latency and enhances scalability for practical applications through efficient batching and memory management. This also drives down the computational cost of GRPO stage, which is mostly bottlenecked by compute required for generations.\n\n\n\n**Lack of Uncertainty Quantification.**\nWe thank the reviewer for highlighting this critical issue. The lack of robust uncertainty quantification is indeed a significant limitation and a well-recognized, field-wide challenge for LLMs. Current LLMs are fundamentally optimized to predict the next most probable sequence of tokens, which often results in confident-sounding, definitive outputs, even when the underlying information is ambiguous. Because BioReason leverages an LLM as its core reasoning engine, it naturally inherits this limitation. However, we designed BioReason's core mechanism of generating interpretable reasoning traces as a practical step towards addressing this trust deficit. While the model does not output a formal confidence score, it produces transparent, step-by-step biological explanations that allow domain experts to scrutinize the model's logic, evaluate its deductions, and ultimately build confidence in its conclusions in a way that is impossible with conventional black-box models.\n\n\n\n**Implementation Complexity and Reproducibility.**\nTo ensure our results are fully reproducible, we will make all data, training code, and model checkpoints publicly available. For deployment, we are preparing our models for release on standard platforms like HuggingFace and have designed the framework to be compatible with high-throughput inference engines like vLLM, ensuring fast and efficient operation. Importantly, our approach is not limited to large-scale systems; we have demonstrated that the BioReason architecture provides significant performance gains even when using smaller 1.7B parameter models, making the benefits of our framework accessible across different computational budgets.\n\n\n\n**Limited Modal Diversity and Scope.**\nWe thank the reviewer for this elegant future work suggestion. Our primary goal with this work was to establish BioReason as a foundational proof-of-concept. We focused on the deep integration of a DNA foundation model with a large language model to prove that our architecture could successfully unlock interpretable, multi-step reasoning from fundamental genomic sequences. This focused approach allowed us to rigorously validate this novel architectural paradigm. Adapting the core framework to other modalities, such as RNA and protein sequences, is a key priority. This extension is a central part of our upcoming BioReason2 model and in line with our vision of training multimodal biological LLMs that can reason across DNA, RNA, and Protein."}, {"review_id": "1eESnpIUJE", "reviewer": "Reviewer_TdiE", "summary": "This paper introduces BioReason, a DNA-LLM model that can reason about the effects of sequence mutations on downstream pathways and diseases. It merges two types of language models: genomic language models (such as EVO2 and NT) and large language models (such as Qwen3). Each types of these models are good for their own designed tasks. The gLMs are great at understanding DNA and its syntaxes and LLM are great for natural language. There is a disconnect between these two family of models, and BioReason tries to bridge the gap and make a native model that gets input from both DNA sequences and text query. The authors have benchmarked it on KEGG-derived biological reasoning datasets and variant effect prediction tasks and shown that it outperforms any other model (gLM and LLM) individually. This clearly shows a potential of merging these two types of models in order to boost biological reasoning.", "questions": "- Why have you only used unsupervised gLMs? I wonder what would be the performance of a supervised model like Borzoi and alphaGenome rather than EVO2.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n- Merging gLMs and LLMs is an innovative and interesting idea to enhance reasoning.\n\n- The benchmarks are convincing and clearly show the advantages of such hybrid models. \n\n- The multi-step reasoning associating variants to downstream pathways and diseases is very helpful and has great use cases.\n\nWeaknesses:\n\n- The only gLMs used here is EVO and NT which are unsupervised models. It would be great to also show the performance of supervised models such as Enformer, Borzoi, and alphaGenome.\n\n- Other than sequence to disease reasoning, it would be great to show how BioReason can handle sequence to gene expression and chromatin accessibility as well.", "was_revised": true, "final_justification": "The authors have addressed my concerns in the rebuttal.", "rebuttal": "We thank the reviewer for their positive assessment and insightful feedback. We are encouraged that they found our approach innovative and the results convincing. In response to the constructive weaknesses identified, we have conducted a new experiment that addresses both suggestions: evaluating our framework with a supervised DNA foundation model on a new task beyond sequence-to-disease reasoning.\n\n\n---\n\n\n### **New Experiment: Chromatin Accessibility Prediction**\n\nWe evaluated the BioReason framework on a chromatin accessibility prediction task using the supervised DNA model, Enformer.\n\n* **Model Choice**: We specifically chose Enformer as it is a powerful, state-of-the-art supervised model explicitly trained to predict genomic annotations like chromatin accessibility, directly aligning with the reviewer's suggestions.\n\n* **Data and Task**: We utilized a public dataset for long-range genomics containing 60,000 data points. We trained all models on a subset of this data and evaluated their performance on a held-out test set. The task was formulated as a multi-label binary classification problem to predict chromatin accessibility across 20 different tracks.\n\n\n\n### **Results**\n\nOur results demonstrate that the BioReason framework's core advantages hold true. The hybrid Enformer-Qwen models significantly outperformed both the standalone Enformer model and the Qwen LLMs.\n\n\n| Model | F1 Macro | F1 Weighted | Precision Macro | Precision Weighted | Recall Macro | Recall Weighted |\n| :--- | :--- | :--- | :--- | :--- | :--- | :--- |\n| Enformer | 0.1718 | 0.1586 | 0.1561 | 0.1485 | 0.3107 | 0.2992 |\n| Qwen3-1.7B | 0.1301 | 0.1294 | 0.2046 | 0.2011 | 0.0989 | 0.0987 |\n| Qwen3-4B | 0.1896 | 0.1832 | 0.2447 | 0.2362 | 0.1649 | 0.1606 |\n| Enformer-Qwen3-1.7B | 0.2589 | 0.2439 | 0.2702 | 0.2555 | 0.3397 | 0.3245 |\n| **Enformer-Qwen3-4B**| **0.3370**| **0.3308**| **0.3401**| **0.3349**| **0.4002**| **0.3929**|\n\n\nThe Enformer-Qwen3-4B model doubled the F1 Macro score of the standalone Enformer model. This clearly shows that integrating sequence embeddings with an LLM via the BioReason architecture provides a substantial performance boost, even when using a supervised DNA encoder.\n\n\n\n---\n### **Discussion**\n\nThis experiment provides two key insights:\n\n1.  The BioReason framework is robust and generalizable. The significant performance lift demonstrates that the synergy between a DNA model and an LLM is a general principle, not an artifact of using unsupervised DNA encoders like Evo2 or Nucleotide Transformer (NT). This confirms the fundamental value of combining a specialized sequence encoder with the flexible reasoning engine of an LLM, regardless of the encoder’s training paradigm.\n\n2.  Our initial choice of unsupervised DNA foundation models in the manuscript was deliberate. While supervised models like Enformer are powerful, they are highly specialized for the tasks they were trained on, such as predicting a predefined set of epigenetic marks. In contrast, unsupervised models like Evo2 and NT learn more general-purpose representations of genomic syntax and semantics by being trained on vast, diverse genomic data without being constrained to a specific set of labels.\n\nIn our new experiment, the standalone Enformer model's performance was modest, suggesting its specialized embeddings, while potent, may not be as rich or transferable for novel downstream tasks. While our framework still extracts significant value, the overall performance ceiling is ultimately influenced by the quality and generalizability of the initial DNA embeddings. For our paper's central goal of creating a system for broad, multi-step biological reasoning, we believe the rich, flexible embeddings from unsupervised models provide a more robust foundation, better suited for the diverse and complex queries we aim for BioReason to handle.\n\n\nWe thank the reviewer again for their valuable suggestions, which have helped us strengthen our work and further validate the robustness of the BioReason framework."}, {"review_id": "JeN4tCFnyx", "reviewer": "Reviewer_ZvqQ", "summary": "This paper presents BioReason, a model architecture that integrates a DNA foundation model with a LLM to improve prediction accuracy and enable natural language reasoning for each prediction. The training process combines supervised fine-tuning with reinforcement learning to encourage multi-step reasoning. As part of this work, the authors also developed and curated new benchmark datasets for biological reasoning. BioReason is evaluated using two different DNA foundation models and two LLMs across these datasets and shows marked accuracy improvements as well as useful step-by-step reasoning.", "questions": "1. Line 131: What are the effects of truncation?\n2. Suggestion: The notation is clear with a detailed read, but including labels for major components (eg $\\textbf{E}^\\prime_{\\text{DNA},k}$ vs $\\textbf{E}^\\prime_{\\text{DNA}}$ in Figure 1 could help ease the comprehension, especially for the steps described in 3.3. “Stacking” along which dimension? \n3. In section 4.2, what was the reasoning for the chosen data split by chromosome? \n4. What was the performance like across different disease representations? \n5. How do models of similar size (ie $\\mathcal{f}\\_{LargeLLM} \\approx \\text{BioReason}(\\mathcal{f}_{SmallLLM}$ , $\\mathcal{f}\\_{DNA})$ compare? Could improved performance just be a factor of a larger joint model? \n6. Suggestion: Reference Figure 3 in section 5.5, perhaps at the end of the first sentence.", "limitations": "1. Since GPT was used in the dataset creation process, there is a non-negligible risk of errors or hallucinations in the generated data. This introduces potential noise that could affect model training or evaluation, and it would be valuable to include an error analysis or quality check to quantify this risk.\n2. A user study could further explore the interpretability claims—specifically, by evaluating whether domain experts find BioReason’s outputs more trustworthy, understandable, or useful compared to a typical classifier. If experts tend to agree with BioReason’s explanations, it would also be important to examine how they respond when the model is wrong—do the explanations still seem convincing, and could they mislead users?", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths\n- Clarity: This is a very well-written paper—clear, well-organized, and easy to follow. The motivations are explicitly laid out, arguments are well supported, and the structure flows naturally, almost telling a story. The writing is polished, concise, and approachable. Thank you for a very enjoyable read.\n- Significance/Originality/Quality: The introduction of curated datasets and benchmarking for DNA LLM reasoning represents a valuable contribution to the field.\n- Quality: The inclusion of stratification to ensure balanced disease representation is a thoughtful design choice.\n- Quality/Significance: The modular design, in which $\\mathcal{f}\\_{\\text{LLM}}$ and $\\mathcal{f}\\_{\\text{DNA}}$ can adopt different backbones allows for flexibility in model size and the potential for performance improvements as new architectures emerge.\n- Quality: The pairing of two SOTA $\\mathcal{f}\\_{\\text{LLM}}$ and two SOTA $\\mathcal{f}\\_{\\text{DNA}}$ models adds to the rigor to the evaluation, with results presented clearly using standard evaluation metrics.\n\nWeakness:\n- The term interpretability is overloaded in the field, so clearly defining it at the outset would help set expectations and clarify the nature of the claims. For instance, the approach does not offer inherent interpretability. Moreover, the LLM-generated explanations may not accurately reflect the true reasoning behind the model’s predictions—LLMs are known to produce plausible-sounding justifications for incorrect outputs and nonsensical rationales for correct ones. Without clearer framing at the outset, the scope and limitations of interpretability in this work only become apparent after a close and thorough reading of the paper.", "was_revised": true, "final_justification": "This paper introduces BioReason, a modular framework that integrates DNA foundation models with LLMs for biological reasoning. The contributions are significant: curated benchmark datasets, evaluation across multiple backbones, and convincing performance towards explainable multimodal model. The paper is exceptionally well-written, clear, and accessible. My main concern was around the framing of interpretability. The authors clarified this in the rebuttal, explicitly defining their scope and limitations. They also provided thorough responses to all methodological questions and strengthened the case that performance gains stem from high-quality model performance rather than model size alone.\n\nOverall, this is a technically strong, clear, and impactful paper. I maintain my Strong Accept (6) recommendation.", "rebuttal": "We thank the reviewer for their thorough review, detailed feedback, and especially the thoughtful suggestions! Below, we address the identified weaknesses.\n\n**Interpretability.** Thank you for raising this important point and suggestion. We will revise our writing in the form of a new subsection of the paper, touching on the scope and limitations of our framing of interpretability.\n\n\n\n**Questions.**\n1. We appreciate this question about the effects of truncation. We used a 2048 nucleotide context window centered on each variant, following established practices from prior Nucleotide Transformer and Evo2 studies. As the context window was chosen after data curation, we truncated our sequences during batching to fit sequences to this 2048 nucleotide window.\n\n   We acknowledge that truncation may result in the loss of distal regulatory elements or long-range genomic context that could be relevant for variant interpretation. However, we have not systematically evaluated how increasing the context window would affect performance, as this was constrained by our computational resources and model limitations. This represents an important direction for future work - understanding the trade-offs between computational efficiency and the inclusion of longer-range genomic context.\n\n   We will clarify the truncation methodology in section 3.1 of the final manuscript and acknowledge this limitation in our discussion.\n\n2. We thank the reviewer for pointing out this space for improved communication. We will update Figure 1 with labels matching the steps from 3.3, as well as specify that we concatenate the embeddings along the token dimension.\n\n3. We utilized the established chromosome-based split strategy from the InstaDeepAI/genomics-long-range benchmark, which is a processed version of the data from the GPN-MSA paper. This splitting approach tests model generalizability to an entirely new genomic context and represents a rigorous evaluation strategy. We maintained this split in our dataset to ensure direct comparability with existing benchmarks and to provide a fair assessment of BioReason’s generalization capabilities across chromosomal regions.\n\n4. Please see Tables 1-2 below for results.\n\n**Performance Across Disease Representations.**\n### Table 1: NT + Qwen3 - 4B\n\n| # | Disease | Frequency | Accuracy | Precision | Recall | F1-Score |\n|---|---|---|---|---|---|---|\n| 0 | Parkinson’s disease | 47 | 0.997 | 0.979 | 1.000 | 0.989 |\n| 1 | Alzheimer’s disease | 40 | 0.997 | 1.000 | 0.975 | 0.987 |\n| 2 | Spinocerebellar ataxia | 36 | 1.000 | 1.000 | 1.000 | 1.000 |\n| 3 | Amyotrophic lateral sclerosis | 35 | 0.997 | 1.000 | 0.971 | 0.986 |\n| 4 | Melanoma | 17 | 0.997 | 1.000 | 0.941 | 0.970 |\n| 5 | Prion disease | 15 | 1.000 | 1.000 | 1.000 | 1.000 |\n| 6 | Colorectal cancer | 12 | 0.993 | 0.857 | 1.000 | 0.923 |\n| 7 | Huntington’s disease | 10 | 1.000 | 1.000 | 1.000 | 1.000 |\n| 8 | Gaucher disease | 7 | 1.000 | 1.000 | 1.000 | 1.000 |\n| 9 | Acute myeloid leukemia | 7 | 1.000 | 1.000 | 1.000 | 1.000 |\n\n### Table 2: Evo2 + Qwen3 - 4B\n\n| # | Disease | Frequency | Accuracy | Precision | Recall | F1-Score |\n|---|---|---|---|---|---|---|\n| 0 | Parkinson’s disease | 47 | 1.000 | 1.000 | 1.000 | 1.000 |\n| 1 | Alzheimer’s disease | 40 | 1.000 | 1.000 | 1.000 | 1.000 |\n| 2 | Spinocerebellar ataxia | 36 | 0.993 | 0.972 | 0.972 | 0.972 |\n| 3 | Amyotrophic lateral sclerosis | 35 | 1.000 | 1.000 | 1.000 | 1.000 |\n| 4 | Melanoma | 17 | 0.997 | 1.000 | 0.941 | 0.970 |\n| 5 | Prion disease | 15 | 1.000 | 1.000 | 1.000 | 1.000 |\n| 6 | Colorectal cancer | 12 | 0.997 | 0.923 | 1.000 | 0.960 |\n| 7 | Huntington’s disease | 10 | 1.000 | 1.000 | 1.000 | 1.000 |\n| 8 | Gaucher disease | 7 | 1.000 | 1.000 | 1.000 | 1.000 |\n| 9 | Acute myeloid leukemia | 7 | 1.000 | 1.000 | 1.000 | 1.000 |\n\n\n5. We thank the reviewer for this insightful question. Table 2 of the paper provides strong evidence that our performance gains are not simply due to increased model size, but due to a higher quality DNA embedding being passed into the LLM.\n\n   A particularly insightful comparison is between Qwen3-4B and NT + Qwen3-1B. The BioReason model (NT + Qwen3-1B) has fewer total parameters than the standalone Qwen3-4B, yet performs significantly better on variant effect prediction tasks. We believe this is due to the quality of the cross-modal embeddings from the DNA model, which is then integrated as part of the LLM’s prediction context. The LLM can then leverage these representations to achieve superior performance on biological tasks, even with fewer parameters. This suggests that BioReason's improvements result from the combination of domain-specific DNA representations and LLM reasoning capabilities.\n\n6. We thank the reviewer for this suggestion to tie the discussion directly to the figure, and will make this change in the final manuscript.\n\n\n\n\n\n\n\n**Dataset Creation Hallucination Risk.**\nWe acknowledge that this is an important concern. To mitigate this risk, we curated ground truth data on mutation effects and downstream phenotypic impacts from the KEGG database, which contains expert-curated summaries with citations on mechanistic effects. This served as our gold standard, and we used Claude solely to reformat these verified paragraphs into chain-of-thought reasoning traces rather than generating novel content.\n\nTo assess quality, we conducted manual validation on a random sample of generated reasoning traces to verify biological accuracy and consistency. While quantifying exact confidence scores remains an open issue for LLM-generated content, our approach of providing comprehensive context and restricting the model's role to reformatting ground truth data mitigates hallucination risk.\n\nThis methodology, combined with our validation sampling, provides confidence that our reasoning traces are biologically accurate and constitute reliable training data.\n\n\n\n**User Study.**\nWe thank the reviewer for this excellent suggestion regarding user studies to evaluate our interpretability claims. We wholeheartedly agree that a comprehensive evaluation with domain experts would provide invaluable insights into BioReason's real-world utility and guide improvements for practical use with biologists studying unknown mutations.\n\nSuch a study would require extensive coordination with domain experts, such as clinical geneticists. We believe this evaluation is important, but would be better suited as dedicated follow-up work that can properly address the complexities of interpretability in genomics."}]

Author rebuttal:

Meta-review: his paper presents BioReason, a DNA-LLM model for reasoning about the effects of sequence mutations on downstream pathways and diseases. The model integrates genomic language models, like EVO2 and NT, with large language models, like Qwen3. Evaluation was conducted on biological reasoning datasets derived from KEGG and on variant effect prediction tasks. Results indicate that the combined model performs better than either genomic or large language models used individually.
Decision: Accept (poster)