CellDuality: Unlocking Biological Reasoning in LLMs with Self-Supervised RLVR
Authors: Yuhang Chen
Zhen Tan
Ruichen Zhang
Mufan Qiu
Tianlong Chen
Venue: ICLR 2026 Poster (2026)
Keywords: Reinforcement Learning
Biological Reasoning
Foundation Models
Single-Cell Biology
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: We introduce a self-supervised RL framework that trains LLMs for biological reasoning by rewarding the consistency of a forward-inverse task loop.

Abstract: \begin{abstract}
Developing generalist large language models (LLMs) capable of complex biological reasoning is a central challenge in computational biology. While existing LLMs excel at predictive tasks like cell type annotation and logically-constrained problems, enabling open-ended and mechanistic reasoning remains a challenge. A promising direction is Reinforcement Learning from Verifiable Rewards (RLVR), which has been shown to significantly enhance complex reasoning in general domains like mathematics and code synthesis. However, its application in biology is hindered, as most biological outcomes are non-verifiable. For example, verifying a generated gene sequence is usually infeasible. In this paper, we introduce CellDuality, a self-supervised framework that enables LLM agents for robust reasoning in single-cell biology. Our framework is built on the principle of complementary task duality, a self-verification process that leverages a bidirectional reasoning loop. First, the model performs a forward reasoning task by predicting a biological outcome (e.g., a cell's response to a drug). Then, in a complementary inverse task, it must reason backward from its own prediction to reconstruct the initial conditions (e.g., the original drug perturbation). The fidelity of this reconstruction serves as an intrinsic reward signal, creating a feedback loop that enforces logical and biological consistency. We use these intrinsic rewards to align the base LLM via reinforcement learning, without requiring ground-truth verification labels. We demonstrate that CellDuality achieves state-of-the-art performance and provides coherent biological explanations across a diverse suite of single-cell reasoning tasks. Critically, on the challenging out-of-distribution perturbation prediction benchmark, our self-supervised approach significantly outperforms the standard fine-tuning baseline and narrows the performance gap to a supervised RLVR baseline. Our work showcases a new path toward scalable training of biological foundation models.

Published as a conference paper at ICLR 2026
CellDuality: UNLOCKING BIOLOGICAL REASONING IN LLMS WITH SELF-SUPERVISED RLVR
Yuhang Chen' Zhen Tan Ruichen Zhang' Mufan Qiu- Tianlong Chen
University of North Carolina at Chapel Hill 2 Arizona State University {yuhang, ruichen, mufan, tianlong] (cs unc. edu 2tan360asu. edu
ABSTRACT
Developing generalist large language models (LLMs) capable of complex bio- logical reasoning is a central challenge computational biology. While existing LLMs excel at predictive tasks like cell type annotation and logically-constraine problems, enabling open-ended and mechanistic reasoning remains a challenge, promising direction is Reinforcement Learning from Verifiable Rewards (RLVR). which has been shown to significantly enhance complex reasoning in general do- mains like mathematics and code synthesis. However, its application in biology is hindered, as most biological outcomes are non-verifiable. For example, verify- ing a generated gene sequence is usually infeasible. In this paper, we introduce CellDuality, a self-supervised framework that enables LLM agents for robust reasoning in single-cell biology. Our framework is built on the principle of com- plementary task duality, a self-verification process that leverages a bidirectional reasoning loop. First, the model performs forward reasoning task by predicting biological outcome (e.g., a cell's response to a drug). Then, in a complemen- tary inverse task, it must reason backward from its own prediction to reconstruct the initial conditions (e.g., the original drug perturbation). The fidelity of this reconstruction serves as an intrinsic reward signal, creating a feedback loop that enforces logical and biological consistency. We use these intrinsic rewards align the base LLM via reinforcement learning, without requiring ground-trufe verifi- cation labels. We demonstrate that CellDuality achieves oftte-of-theaar per- formance and provides coherent biological explanations across a diverse suite of single-cell reasoning tasks. Critically, the challenging out-of-distribution per- turbation prediction benchmark, our self-supervised approach significantly out- performs the standard fine-tuning bascline and narrows the performance gap a supervised RLVR baseline. Our work showcases new path toward scalable training of biological foundation models.
INTRODUCTION
Developing generalist large language models (LLMs) capable of biological reasoning is central goal of computational biology (Fang et al., 2025b; Istrate et al., 2025; Lotfollahi et al., 2019). This reasoning ability involves inferring complex, mechanistic principles from cellular data (Fang al., 2025b; Matsumoto et al., 2025). This capability is paramount in single-cell biology, where under- standing causal chains, such as how cell responds to drug, is key therapeutic discovery (Fang et al., 2025a). However, achieving robust biological reasoning is fundamentally challenging due to the stochastic nature of cellular systems and the intricate, nigh-dimensional dependencies between biological entities. This complexity creates significant hurdle for current methods, especially the foundation models (Hao et al., 2024; Cui et al., 2024), which we categorize into three limitations.
First, most models are optimized for prediction, not mechanistic reasoning. Architectures like scGPT (Cui et al., 2024) and C2S-Scale (Rizvi et al., 2025) excel at learning correlational patterns for tasks like cell type annotation but not explicitly trained to generate the coherent, explana- tory steps that capture underlying biological pathways. Second, existing reasoning- models often operate in logically-constraiee
paradigms. For instance, Cell-ol (Fang al., 2025b) models a deductive puzzle-solving process rather than the open- open-ended, hypothesis driven inquiry of scien- tifie exploration. Finally, there exists trade-off between depth and generality. Specialized models

---

Published as a conference paper at ICLR 2026
Single-cell Expression Profiles
Unifed Task Framework
Sample Reasoning
Genes
High
Traces
Coll 1
Type Annotation
Conditional
Gencration
Cell 2
'g2
84
Cell Type Label
81
8g
84
...
...
Sur
8m Low
Cell Type Label
"g2
Descending Ordering
Unified GaT Dotuset
Drug Seusitivity Chasification
Perturbation Respouse Genersttio
Training
Cell Sentence
82
84 -. B11
"B2
Bq -.. M. B11
Stage Cold Start
Cell
82
84
Unified CoT
LLama
Sensitive Resistant
Cell
83
8g
Ground Truth-based Reward
Our Reward X
Stage RL
Large scale
Input:
X
Ingut
X
Input:
Tack
Gen
Uniabeled Data
GRPO
LLama
Output:
Gen Y
GTY
Output:
Gen
Output:
Gen
Exect
Sentance
Reward
B
Gen B
Reward
Match
Sinslarity
An overview of the CellDuality framework. Single-cell expression profiles are first converted into ranked "Cell Sentences," which are inputs to our task formulation covering four rea- soning tasks. A high-quality CoT dataset is generated using a teacher model and Reject Sampling. This dataset is used for Stage SFT cold start of an LLaMA model. The model is then further aligned in Stage 2 via self-supervised RL (GRPO) on large-scale unlabeled data. The core innova- tion is our duality-based reward mechanism, which replaces the need for external Ground Truth by rewarding the consistency between a Primal Task and its complementary Dual Task.
Figure 1:
achieve deep reasoning in single task, while versatile, multi-takk agents like InstructCell (Fang et al., 2025a) currently lack the same level of mechanistic insight. Therefore, developing a frame- work that takes step toward deep reasoning across diverse tasks remains an open challenge. Such framework must achieve generality over two core biological themes: cell identity and cell dynamics.
promising direction Reinforcement Learning from Verifiable Rewards (RLVR), a paradigm that has successfully enhanced I.LMs' general reasoning ability, such as mathematics and code synthe- sis (Shao al., 2024; Rafailov et al., 2023; Lee al., 2023). However, its application in biology is severely limited because most biological reasoning tasks are inherently non-verifiable. For exam- ple, a specific gene sequence output of conditional cell generation has no single correct version for given cell type, making simple verification infeasible. This data-dependency fundamentally con- strains the training of more ambitious, unified models on the open-ended, cause-and -nd-effeeeettttteeeeeeeee-e--e-- scenarios that would foster biological understanding.
To address this challenge, we introduce CellDuality, an agent for open-ended biological reason- ing. It operates within structured framework of four core tasks designed span the fundamental biological themes of cell identity and cell dynamics (details Sec. 3. Crucially, CellDuality is trained via supersupe paradigm inspired by DuPO (She al., 2025), built on the principle of Complementary Task Duality. This framework leverages a bidirectional reasoning loop to generate its own supervisory signals: first, the model performs forward reasoning task (e.g., predicting cell's response to a drug); then, in complementary inverse task, given generated results and known input conditions, it will reason backward reconstruct the unknown input conditions. The reward is then determined by directly comparing the reconstructed input with the original. This consistency score becomes the intrinsic reward signal, compelling the model to produce forward predictions that accurate and logically reversible, without needing ground-truth labels for the predictions themselves.
We implement this principle in two-stage training paradigm. An initial Supervised Fine-Tuning (SFT) stage on a small, curated set of examples, containing both forward and inverse reasoning traces. This stage serves cold-start the model, teaching it the language and format of biological reasoning. This is followed by a large-scale, self-supervised Reinforcement Learning (RL) stage, where the model is aligned using these intrinsic rewards on vast unlabeled data. This stage refines
2

---

Published as a conference paper at ICLR 2026
the model's ability to produce outputs that not only stylistically correct but also biologically and logically coherent.
Empirical evaluations demonstrate that CellDuality, despite being trained without any ground- truth verification during its RL phase, substantially outperforms its SFT-only counterpart. Critically, on the challenging OOD perturbation benchmark, our self-supervised approach closes 35-56% of the performance gap to a fully-supervised RLVR model that was trained with ground-truthh rewards. This showcases the sample efficiency and generalization potential of our framework. Our main contributions are:
We propose structured framework that organizes complex biological inquiry into four core rea- soning tasks spanning both cell identity and cell dynamics. This provides promising step toward developing and evaluating agents with broader capabilities in single-cell biology.
We introduce the principle of Complementary Task Duality, new mechanism for generating annotation- free rewards. This framework incentivizes LLMs to lear the intrinsic mechanistic consistency of biological processes by rewarding the fidelity of a bidirectional reasoning loop, eliminating the need for ground-truth labels during the RI. phase.
We show empirically that the generalist model significantly outperforms standard SFT baselines and narrows the performance gap fully-supervised oracle model the challenging out-of- distribution perturbation prediction benchmark.
2 RELATED WORK
Foundation Models in Single-Cell Biology. Foundation models are revolutionizing single-cell bi- ology by learning representations from massive transcriptomic data (Cui al., 2024; Theodoris et al., 2023). The field has rapidly progressed from models focused on predictive tasks, such as scGPT (Cui et al., 2024) for annotation and C2S-Scale (Rizvi al., 2025) for multi-task general- ity, to those attempting explicit reasoning. However, these reasoming aware models often operate in narrow paradigms; for instance, Cell-ol (Fang et al., 2025b) frames reasoning as a logically- constrained puzzle, while agentic frameworks like ESCARGOT (Matsumoto et al., 2025) rely on external knowledge graphs. A key challenge remains developing a single, generalist agent that can perform open-ended, mechanistic reasoning directly from cellular data. Our work addresses this gap, aiming for the generality of models like InstructCell (Fang et al., 2025a) but with a training objective that explicitly fosters deep, intrinsic reasoning.
Reinforcement Learning from Verifiable Rewards Reinforcement learning is increasingly used to refine LLMs beyond standard SFT, with paradigms evolving to reduce reliance on external super- vision (Ouyang et al., 2022; Shao et al., 2024). A particularly scalable paradigm is Reinforcement Learning from Verifiable Rewards (RIVR), which replaces subjective feedback (Bai al., 2022) with objective, ground-truth-base rewards from deterministic verifiers (Shao et al., 2024). How- ever, the prerequisite of verifiable output severely limits RLVR's application in biology, where outcomes are inherently stochastic. Recent work has sought to address this by generating self- supervised rewards through task duality. For instance, the DuPO (She et al., 2025) framework introduced generalized duality for non invertiibe tasks, such as mathematical reasoning, by recon- structing input components to create a reward signal. Inspired by DuPO, our work adapts this duality principle to the unique challenges of biology (detailed in Sec. 3.2). We introduce Complementary Task Duality to generate intrinsic, self-verifiable rewards from the internal consistency of cellular processes, thus extending the RLVR paradigm to the non-verinee verifiable biological domain.
METHODOLOGY
This section delineates the methodology for training single-cell biological reasoning model. An overview of our entire framework is presented in Figure We first define the concepts and the task formulation. We then introduce the generalized framework for self-supervision. Finally, detail our training pipeline: an initial cold start stage, followed by a self-supervised Reinforcement Learning stage that uses duality principle enhance the model for deeper reasoning.
3

---

Published as a conference paper at ICLR 2026
Table 1: Task Formulation for Single-Cell Reasoning.
Theme
Classification Tasks
Generative Tasks
Cell Type Annotation
Conditional Cell Generation
Cell Identity
(laput: Cell. Output: Label)
(Input: Label, Output: Cell)
Drug Sensitivity Prediction
Perturbation Response Generation (Input: Cell + Drug, Output: New Cell)
Cell Dynamies
(Input: Cell + Drug, Output: Label)
3.1
PRELIMINARIES AND TASK FORMULATION
We denote the LLM policy as Te, parameterized by 8. Let V be the global vocabulary of all con- sidered gene names. Our work addresses structured set of four core single-cell reasoning tasks, organized into a 2x2 matrix spanning two fundamental biological themes: Cell Identity and Cell Dynamics. The core data structures for these tasks are defined as follows:
Cell Representation: A cell {g9, 1. -K.] is represented of its top K expressed genes, where each gene 9, E V.
descending order sequence
Perturbation:
perturbation p is a structured tuple describing an intervention. e.g., p = (operation, target), where operation (knockdown, overexpression) and target E V.
Cell Type and Sensitivity Labels: A cell type is categorical label from predefined set T. Similarly, a drug sensitivity label, s, is a categorical label from a set
All inputs to the LLM are constructed as textual prompts x that combine these components. The model's output is textual response y, generated autoregressively according to the policy y ~ (-x)..). A response may include reasoning trace z and a final answer a, i.e., y = az
3.2 THE PRINCIPLE OF COMPLEMENTARY TASK DUALITY
A primary obstacle applying Reinforcement Learning (RL) to the four tasks defined above is the absence of scalable reward source. single-cell biology, obtaining ground-truth signals from experiments is prohibitively expensive and slow. Our work is motivated by central question: Can we generate a reliable, intrinsic reward signal directly from the structure of these biological problems themselves, thus enabling RL without external supervision?
To achieve this, inspired by DuPO (She et al., 2025), we introduce a self-supervised reward genera- tion framework. Adapting the duality principle from well-structured domains (e.g.. mathematies) to biology is non-trivial: biological outputs are inherently stochastic and high-dimensional, requiring the design of domain-specific task formulations, such conditional gene inpainting (Sec. 3.4), to produce stable reward signals. The core idea is to reframe a single biological question into pair of mutually-verifyiin tasks, a primal task and complementary dual task. This creates an internal logic loop that the model must satisfy, providing natural source for an RL reward.
Definition 3.1 (Complementary Task Duality). Let the input space X' of primal task T, be decom- posed into disjoint subspaces: X (known components) and Xr (unknown components), such that X' = X₂ U X,. The primal task Tp is mapping from Tp X + Y. Its complementary dual task Tea a mapping that leverages the primal output y and the known component X3. to reconstruct unknown component x,:
Ted : (y, xx) + X..
Pair (Tp: Tea) forms a generalized dual pair if satisfies the complementary consistency principle:
Vx X, y
(x(x): y=(xx))
d(xu, Tealy, xx)) $ c,
where :(-.-) Xe, X Nr, is domain-specific distance metric, and 2 0 is a tolerance threshold.
The power of this framework lies in its ability to transform an unsupervised problem into a self- verifying one. The consistency principle defined above provides the mechanism to generate rewards: the fidelity of the dual task's 's reconstruction, d(x.,. x.), serves as a direct, intrinsic measure of the logical and biological coherence of the primal task's output y. This approach elegantly sidesteps the challenges classical dual learning (irreversibility and asymmetry) by leveraging the known component X contextual anchor, ensuring the dual task is well-posed.
4

---

Published as a conference paper at ICLR 2026
3.3 TRAINING STAGE 1: SUPERVISED
FOR CAPABILITY COLD-START
Before RL Training, we first initialize the base LLM with foundational biological knowledge and reasoning patterns through Supervised Fine- Line-nuning (SFT). This essential cold-start phase ensures the model can effectively engage with the complex, self-supervised tasks in the subsequent align- ment stage. The process involves two key steps: generating a high-quality Shain-of-Thhughh dataset, and then using it to train the model.
3.3.1 THAIN-OF-THOUGHT REASONING DATASET GENERATION
We construct comprehensive SFT dataset, SFT, by leveraging powerful teacher models (e.g,, GPT-4o, Gemini 2.5 Pro) to generate Chain-of-Thought (CoT) reasoning traces. A critical aspect of our approach is that DsrT must equip our model with capabilities for both the primal (forward) reasoning and the complementary dual (inverse) reasoning required in our RL stage. Therefore, generate and curate distinct data subsets for each direction.
Primal Task SFT Data. For each of our four core tasks, generate primal task data. Given an input prompt X,, we prompt a teacher model Teacher to generate N candidate responses {ys,k = kZ a. { We then apply task-specific filtering to select high-quality instances for our primal SFT set, SFT
For Classification Tasks (Annotation Sensitivity): We use strict Rejection Sampling proto- col. A candidate yı,k is accepted if its final answer a exactly matches the boound-truth label aç- We define an indicator for correctness 6,, I(a., = =a)). The accepted set for prompt x; is (yr,elG,, = -}. This ensures all training examples are factually correct.
For Generative Tasks (Cell & Response Generation): As no single, unique ground-truth quence exists for these tasks, a simple exact match infeasible. Instead, we adopt a Rank-Aware Filtering protocol. For each prompt X; with corresponding ground-truth cell sequence aj, the teacher model generates a candidate response a.). The candidate is accepted into DsFT only if the generated cell sequence a; demonstrates high fidelity to the ground truth in terms of both gene overlap and expression ranking. We quantify this using our proposed Rank-Wal Weighted Jaccard Similarity metric (Eq. 3). A candidate is accepted only its similarity score exceeds predefined threshold.
Dual Task SFT Data. To explicitly teach the model the inverse reasoning required for duality framework, we construct a corresponding dual task SFT set, D For each instance in our curated primal set, formulate its complementary dual problem.
For primal instance (x = (Xk, yx). we construct a dual prompt Xdual = (y", ;x)) The ground-truth answer for this dual task is the original unknown component, Ydual X,-
For example, for perturbation response instance where X Cpres X, = P. and y" Cposs the dual SFT sample would be: prompt (cpua Cpre) paired with the ground-truth answer p.
The teacher model is then prompted to generate CoT reasoning for these dual problems. The final SFT This hybrid strategy ensures the model is proficient
SFT dataset is the union Dsrt = Derinal D in both forward and inverse reasoning before entering the RL stage.
3.3.2 SUPERVISED FINE-TUNING OBJECTIVE
The model is then trained on DSFt by minimizing the standard negative log-likelihood loss CSFT(0) over the complete reasoning trajectories:
yi Σ
CSFT (0) = - (x.... Dave
log mo (vi,, x,, vi.es)
(I)
The resulting model, TSFT, possesses the baseline capabilities required for the subsequent self- supervised alignment.
5

---

Published as a conference paper at ICLR 2026
3.4 STAGE 2: SELF-SUPERVISED QUALITY-GUIDER REINFORCEMENT LEARNING
This stage constitutes the core of our self-supervised methodology. We refine the capabilities of the SFF-initialized model, TTF.. by aligning it with the principle of complementary consistency. This is achieved through Reinforcement Learning (RL) framework that operates on a large, unlabeled dataset DRL and is guided by intrinsic rewards, eliminating the need for any ground-truth data.
3.4.1 SELF-SUPERVISED REWARD GENERATION
The cornerstone of our alignment stage is the generation of intrinsic rewards derived from the com- plementary duality principle. For any prompt x (X,,X, x,) and model-generated primal output y, compute a reward by executing the complementary dual task and measuring its reconstruction fidelity. We employ two types of rewards, categorical and sequence-based, tailored to the nature of core tasks.
Categorical Rewards from Inverse Task Consistency. For generative tasks such as Perturbation Response Generation and Conditional Cell Generation, the duality provides clean, categorical reward signal. In both cases, the primal task generates a high-dimensional cell sequence (Cpusi or c), and the complementary dual task attempts reconstruct categorical input label (the sensitivity or the cell type t). The reward is a binary signal based on the exact reconstruction of this label:
r(ylx) 1(x, Xx);
where x, is the original categorical input (c.g., and x, is its reconstruction (c.g.. t). This re- ward directly measures the logical consistency of the generated output: biologically plausible cell sequence should unambiguously encode the conditions that generated it.
Continuous Rewards from Conditional Inpainting. For classification tasks such as Cell Type Annotation and Drug Sensitivity Prediction, where the primal output is low-information label, we design a reward based on a conditional gene inpainting objective. Here, the input cell sequence artificially decomposed into an observed part Cabs and hidden part Caid, which serves as the unknown component X.. The dual task is to reconstruct čnaa conditioned on both the observed genes Cobs and the model's predicted primal label (i or 8). The reward a continuous score reflecting the quality of this reconstruction: r((lc) RWJS Chia). The meik-Weighted Jaccard Similarity (RWJS) extends the standard Jaceard index by weighting each gene by its reciprocal rank, wly, c) = 1/rank(g, c), that
genes contribute more. For ground-truth sequence c* and
generated sequence Cgen. it is defined as:
Eyestns w(g.e") (ulg..p)
RWIS(C", -Cgen) EyES- w(g, c*) +*) Eggssnn w(g. Cgen)
2
where S = Set(c") and Sgen = Set(Cgen). RWIS ranges from 0 (no overlap) to (identical), providing a biologically meaningful measure that prioritizes high-expression genes. This reward incentivizes the model to base its classification a deep understanding of the cell's underlying gene signature, as a correct label should provide the necessary context for accurate gene inpainting.
3.4.2 POLICY OPTIMIZATION WITH GRPO
We optimize the policy ne to maximize the expected self-supervised reward [(0) Ex-Da. Ex-DD,,rry[x)) [r[y[x)]. We employ Group Relative Policy Optimization (GRPO), a memory-efficient and stable critic-free RL algorithm. The optimization follows iterative, online process: for each prompt, the current policy generates a group of G candidate responses, each of which is then assigned a self-supervised reward based its dual-task performance. This group of responses and rewards is then used to update the policy as follows.
Advantage Estimation. For each prompt, after generating group of G responses and their cor- responding rewards {r we compute the advantage for each candidate. This is achieved by normalizing the rewards relative to the group's 's performance, which serves as an empirical bascline, thus obviating the need for separate value function:
Fk mcan (fr;)
A std((r,) +

---

Published as a conference paper at ICLR 2026
Table 2: Performance comparison on the Cell Type Annotation task. Baselines are trained on each dataset individually. We report Accuracy (Acc.) and Macro Fl-score (F1) for all five benchmarks, differentiating between ID and OOD evaluation for our model.
In Distribution (ID)
Evaluation
Out-of-Distribution (OOD) Evaluation
Model
He-Li220-Liver
Segerstolpo-201
Xin-2016
Ma-2020
Raslidac-Ponce-D0
Aec. (%6
(1())
Aec. (%)
(%)
Acv. Aev.(%)
(%)
Arc: (%)
(1-))
Acc: (%)
F1 (1)))
SBERT
85.28
94,08
99.52
99.64
99,25
95.79
$2.92
51.73
86.67
79,60
SCIPT
94.88
01:75
98.09
97.87
99. 10
95. 40
82.54
79.40
91.43
87.01
Geneformer
95.0n
92.57
99.52
99.49
99.70
99.39
85.79
$4.89
88.50
83.81
CellZSemience
94.88
94.42
99.52
99.64
99.35
98.77
82.40
$1.05
80.59
76.82
InstretCCll-inn instrcc
90.06
95.24
100,00
100,00
99.30
988.9
85.59
54.50
91.10
88,69
CeliDunlity (SFT-only)
94.33.0.11
94.07.0.18
96
08.76
98.73
99:45 99.45_.0112
99.01
90,22
74.95
72.87.0.28
57.24
DusiDolltty
96.34
95.41
00.51
09.78
90.52
90:08
82.05
81.76
88:45
78, 12.11
Objective Function. The policy is updated by maximizing the GRPO objective, which includes clipped surrogate objective to stabilize training and a KL penalty to prevent large deviations from a reference policy Net (typically the initial SFT model TsFT):
(5) where (f) = To (g( |x) Te... (yolz) is the probability ratio, A, is the advantage at token & (in our case, Aş is applied to all tokens of response k), e, the clipping ratio, and / is the KL coeffi- cient. This iterative, multi-task training process progressively refines the model's ability to generate biologically coherent and logically consistent responses.
(8)
min (0) A,, clip(p: Ec, cc) (Tel
EXPERIMENT
4.1 EXPERIMENTAL SETUP
Tasks and Datasets. Our evaluation centered around the four core single-cell reasoning tasks introduced in framework. To ensure fair and direct comparison with state-of-the-art models, adopt the exact datasets and train/test splits used in seminal works, including 2025) and InstructCell (Fang et al., 2025a). Our training strategy involves fine-tuning single base model on a designated primary training set for each task theme, and then evaluating its performance on both in-distribution (ID) and diiitri-iitiitti (OOD) test sets.
(Rizvi et al.,
Cell Identity Tasks (Annotation & Generation):
Training Dataset: To build a robust multi-task model for cell identity, we construct a mixed training dataset by combining the training splits of four diverse public benchmarks: He- 2020-Liverr (He et al., 2020), Segerstolpe-2016 (Segerstolpe et al., 2016), Xin-2016 (Xin et al., 2016), and Human Immune Tissue Dataset (Domínguez Conde et al., 2022). This mixed dataset serves the sole source of supervision for our model on all identity tasks.
ID Test Set: For the cell type annotation task, datasets included in our training mix (He-2020-Liver, Segerstolpe-2011 Xin-2016). For cell generation,
the held-out test splits of the three
the held-out test splits of Human Immune Datasets.
OOD Test Set: We use datasets entirely unseen during training: Ma-2020 (Ma al., 2020) and Bastidas-Ponce-2011 (Bastidas-Ponce et al., 2019).
Cell Dynamics Tasks (Sensitivity Prediction & Response Generation):
Training Dataset: We construct comprehensive mixed training dataset by combining three distinct perturbation benchmarks: the L1000 dataset (Subramanian et al., 2017), which covers two human drug response datasets, GSE149383 (Lung) (Aissa et al., 2021) and GSE117872 (Oral Cavity) (Sharma et al., 2018). This diverse dataset, containing exam- ples for both response generation and sensitivity classification, serves as the sole source of supervision for our model on all dynamics-related tasks.
ID Test Sets: The held-out test splits of the two datasets explicitly included for the classifi-
cation task: GSE149383 and GSE117872.
-OD Test Sets: We use two benchmarks entirely unseen during training. For cross-species classification,
the complete GSE1 10894 (Mouse Bone Marrow) dataset (Bell et al., 2019). For generative causal reasoning, we use the OOD splits of the sci-Plex3 Human Perturbation dataset (Srivatsan et al., 2020).
7

---

Published as a conference paper at ICLR 2026
Table Performance comparison on the Drug Sensitivity Classification task. Baselines are trained on each dataset individually. We report Accuracy (Acc.) and Macro F1-score (F1).
In- Di-Disitibutin (ID)
Evaluation
Out-of-Distribution (OOD) Evaluation
Model
68E:149333 (Human Lung)
GSE117872 (Human Oral)
GSE110894 (Mouse Bone)
Acc. (%)
F1 (%)
Acc. (%)
F1 (%)
Acc. (%)
F1 (%)
ScBERT
99.56
99,56
95.42
96.01
95.80
95,79
scGPT
97.79
97.79
82.44
84.76
95.80
95.79
Geneformer
98.23
98.23
94,66
95.27
93.01
92.91
Cell2Sentence
93.36
93.36
90.84
90.72
95.10
95.08
InstractCell-nstruett
97.35
97.34
100.00
100.00
97.20
97.19
CellDuality (SFT-only)
98.91
98.89
96.78_0.22
97.12.
96.45--0.18
96.42
CellDuality
99.12
29.10-00144
29.10
97:23-00:0
97.23
-0.20
97.58
96.
96.12.-0:21
96.08.40.2
Table 4: Performance Perturbation Response Generation (sci-Plex3 benchmark). Our model was trained on a separate perturbation dataset, while baselines were trained on the in-distribution splits of sci-Plex3, Lower scores are better for distribution-bast based metrics.
Model
Supervision Type
scFID (J)
MMD (1)
Wasserstein (L)
seGen
Supervised
0.95
1.05
0.98
CellOT
Supervised
0.88
1.03
0.95
scGPT
Supervised
0.29
0.42
0.54
C2S-Scale IB (SFT)
Supervised
0.02
0,01
C2S-Scale (GRPO
Ground- Truth RL
0.02
0.01
0.21
CellDuality (SFT-only)
Supervised
0.045
0.028-0.002
0.267
CellDuality
Self Sule-Sueeriieed
0.038 10.002
0.019
0.000
0.001
0.245
10.011
Evaluation Metrics.
For Classification Tasks (Cell Type Annotation, Drug Sensitivity): We report Accuracy as the primary metric. We also include the Macro FI-score to account for class imbalance.
For Generative Tasks (Conditional Generation, Perturbation Response): For Perturbation Re- sponse Generation, follow C2S-Scale (Rizvi et al., 2025) and report distribution- metrics (scFII and MMD) calculated in pre-trained embedding space (scGPT (Cui et al., 2024)) to as- sess the quality and realism of generated cell populations. For Conditional Cell Generation, follow Cell2Sentence (Le Vine et al., 2024) and report NN Accuracy. The k-NN classifier is evaluated with multiple neighbor values (k {3, 5, 10, 25))).
(GW) Distance and k-
Baseline Models. We benehmark CellDuality against a comprehensive set of state-of-the-are models, with all performance metrics cited directly from the original publications for fair com- parison. For classification tasks (Annotation and Sensitivity), we compare against domain-specifi foundation models such as scGPT (Cui et al., 2024) and Geneformer (Theodoris al., 2023), as well as LLM-based agents like InstructCell (Fang et al., 2025a). For generative tasks (Cell and Response Generation), baselines include specialized generative models like seGen (Lotfollahi al., 2019) and seDiffusion (Luo et al., 2024), and the powerful LLM-based framework C2S-Scale (Rizvi al., 2025).
Implementation Details. Our CellDuality model is based on the Llama-3.2-33 3.2-3B architecture. The SFT stage is conducted for 3 epochs with learning rate of le 5. The subsequent self supervisdd RL alignment is performed using GRPO with a group size of G = 8, a train batch size of 512, a mini-batch size of 32, and is run for 200 optimization steps. All experiments are conducted 8x A6000 GPUs, All our scores are shown as mean + std over runs.
4.2
MAIN RESULTS
Across all four reasoning tasks, our self-supervised framework, Cell- Sall--Duaity,, demonstrates highly competitive performance against wide range of state-of-the-art baselines. As detailed in Tables through 5, our multi-task model, trained on mixed datasets, consistently matches or sur- passes specialist models that were trained on individual benchmarks. This is particularly evident the classification tasks (Cell Type Annotation and Drug Sensitivity), where CellDuality shows robust generalization dut-of-distribution and cross-species datasets.
8

---

Published as a conference paper at ICLR 2026
Table Performance on Conditional Cell Generation on the Human Immune dataset (In-
Distribution). Baseline results are cited from Cell2Sentence (Le et al., 2024). k-NN Accuracy reported for multiple values of
k-NN Accuracy (%)
Model
GW Distance (L)
k=3
k=5
k=10
k=25
sc
24.36 +000062
24.00
+0.0064
24.25 1.00034
23.48 40.0032
302.13 00,9338
seGen
10.0110
23.30.00.099
20.005
23.35 23.35
10.0041
315.95 112441
seDiffusion
23.35 10.0125
22.88
10.0111
23.68
10.0067
23.06 10.0049
72.02 1.0.9337
scGPT
18.38 10.0086
17.88
10.0169
18.11
10.0199
18.82
10.0071
2989.81 1.4.9229
Cell2Sentence- 160M
25.88 10.0061
25.65.40.0060
27.46.00.007
27.15 17.0000
54.30 100.410
CellDuality (SFT-)nly)
24.92 L0.0058
24.71 10.0055
25.83
10.0062
25.49
10.0059
63.87 10.0421
CellDuality
26.34
+-000566
10.003
26.21
+-00060
25.98
+0,0057
61.45 1-0.400
1.0
1.0
0.5
0.5
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.0
0.2
0.4
0.6
0.8
1.0
(a) Categorical Reward (Generative Tasks)
(h) Continuous Reward (Classification Tasks)
Figure 2: Training dynamies of self-supervised rewards during the RL alignment stage. The plots show the moving average of (a) the categorical accuracy-base -based reward for generative tasks and (b) the continuous
reward for classification tasks.
The most significant impact of supe-supe approach is observed in the generative tasks quiring deep mechanistic reasoning. For Perturbation Response Generation, the duality-guided RL stage provides substantial performance boost over the already strong SFT bascline. Critically, self-supervised model successfully narrows the performance gap to a fully-supervised oracle that re- quires ground-truth labels for alignment, proving the efficacy of our annotation-free strategy. While our model also demonstrates strong performance on Conditional Cell Generation by outperforming most classical and deep learning-basc generative models, its primary strength lies in its ability to learn the intrinsic, mechanistic consistency of biological processes, showcasing new path toward scalable and robust scientific reasoning agents. The central value our ceff-supervised approach lies in eliminating the dependence on expensive, often unavailable ground-truth labels during the RL alignment phase. Our framework achieves competitive performance without any such supervi- sion, providing practical and scalable path to strong reasoning capabilities in data-scarce biological domains.
4.3 ABLATION STUDY
Self-Supervised vs. Ground-Truth Supervised RL To rigorously quantify the efficacy of our self- supervised alignment strategy, we conduct head-to-head comparison against standard supervised RL approach. We evaluate three key models on their respective in-distribution test sets: (1) the SFT-only baseline, (2) supervised RL oracle trained with ground-trule truuh rewards, and (3) our self- supervised CellDualite model. As shown in Table 6, our self-supervised RL approach consistently and significantly boosts performance over the SFT-only baseline across all tasks. Critically, our annotation-free method substantially narrows the performance gap to the fully-supervised oracle, and even surpasses the oracle's Macro F1-score on the He-2020-Livee annotation task, suggesting learns more robust decision boundaries.
CONCLUSION
We introduced CellDuality, an agent that learns complex biological reasoning through novel self-supervised framework. Our contribution, the principle of complementary task duality, enables reinforcement learning alignment on non-verifiblle single-cell tasks by generating intrinsic rewards from bidirectional reasoning loop. Trained via our sample-efficient, two-stage paradigm,

---

Published as a conference paper at ICLR 2026
Table 6: Core ablation study comparing Self-Supervised RL against a Ground-Truv Supervised oracle. All models are initialized from the same SFT checkpoint and evaluated on their respective in-distribution (ID) test sets.
He-2020-Liver
GSE149333 (Lung)
sci-Plex3
Method Configuration
Acc. (1)
F1 (1)
Acc. (1)
F1 (1)
ScFID (1)
MMD (L)
2aama-3.2-3B-Inttre
22.45 1.23
52.82-1.45
29.67
61.34 1.12
SFT-only
95.83 +0.21
=0.67
98.91 +0.15
98.89 0016
0.045 0.0053
0.028 00.002
RI with Ground-Trunt
97.21 #0.16
94.85+0.14
99.34 +0.12
99.31 -0.13
0.025 ++0.001
0.012
Ours (Self-Supervised RL.
96.34 +0.19
95.41 +0.16
99.12 +0.13
99.10 -0.14
0.038 +0.002
0.019 +0.001
CellDuality achieves Mtate-fftthe-ar performance across four distinct reasoning tasks, providing coherent biological explanations. Critically, our self-supervised approach demonstrates its efficacy by narrowing the performance gap to a supervised RLVR baseline. This work presents promising step toward scalable foundation models biology, offering new paradigm that learns to reason from the intrinsic logical structure of scientific problems, rather than from external labels.
LIMITATIONS
Our framework is validated four representative transcriptomic tasks and has not yet been tended to other omics modalities such as ATAC-seq or proteomics. The cell sentence representation discards absolute expression magnitudes, trading encoding precision for compatibility with off-the- shelf LLMs. While our duality-based reward enforces logical consistency, it does not guarantee biological correctness. Mutually consistent yet biologically implausible predictions remain possi- ble, and direct interpretability evaluation at the pathway level is left for future work. The RL gains are most pronounced for generative tasks and comparatively modest for classification tasks where SFT alone already achieves strong performance. Finally, our experiments use a 3B ---arrmmeter model, which may limit reasoning depth. Exploring the scaling behavior of our approach is an important future direction.
ACKNOWLEDGEMENTS
This research was partially funded by the National Institutes of Health (NIH) under award 1R01EB03710101. The views and conclusions contained in this document are those the au- thors and should not be interpreted representing the official policies, either expressed implied, of the NIH. This research was also partially supported by the Amazon Research Award.
THE USE OF LARGE LANGUAGE MODELS
We utilized Google's Gemini Pro 2.5 as a writing assistant in the preparation of this manuscript. Its function was strictly limited language refinement tasks, such as enhancing clarity, correcting grammar, and rephrasing sentences to improve readability within an academic context. All scientific content, including the core ideas, experimental design, and interpretation of results, was generated exclusively by the human authors.
REFERENCES
Alexandre F Aissa, Abul BMMK Islam, Majd M Ariss, Cammille C Go, Alexandra E Rader, Ryan D Conrardy, Alexa M Gajda, Carlota Rubio-Perez, Klara Valyi-Nagy, Mary Pasquinelli, et al. Single-cell transcriptional changes associated with drug tolerance and response to combi- nation therapies in cancer. Nature communications, 12(1):1628, 2021.
Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional ai: Harm- lessness from ai feedback. arXiv preprint arXiv:22 2022.
10

---

Published as a conference paper at ICLR 2026
Aimée Bastidas- Sophie Tritschler. Leander Dony, Katharina Scheibner, Marta Tarquis- Medina, Ciro Salinno, Silvia Schirge, Ingo Burtscher, Anika Böttcher, Fabian Theis, et al. Comprchensive single cell mmn profiling reveals a detailed roadmap for pancreatic endocrino- genesis. Development, 146 (ed112)ddv113388 173449, 2019.
Charles C Bell, Katie A Fennell, Yih-Chih Chan, Florian Rambow, Miriam M Yeung, Dane Vas- siliadis, Luis Lara, Paul Yeh, Luciano Martelotto, Aljosja Rogiers, et al. Targeting enhancer switching overcomes non-genetic drug resistance in acute myeloid leukaemia. Nature communi- cations, 10(1):2723 2019.
Haotian Cui, Chloe Wang, Hassaan Maan, Kuan Pang, Fengning Luo, Nan Duan, and Bo Wang. sogpt: toward building a foundation model for single-cell multi-omics using generative ai. Nature Methods, pp. 1-11, 2024.
Domínguez Conde, Chao Xu, Louic B Jarvis, Daniel Rainbow, Sara Wells, Tamir Gomes, SK Howlett, O Suchanek, K Polanski, HW King, et al. Cross-tissue immune cell analysis reveals tissue-specific features in humans. Science, 37666594)eaa15117,, 2022.
Yin Fang, Xinle Deng, Kangwei Liu, Ningyu Zhang, Jingyang Qian, Penghui Yang, Xiaohui Fan, and Huajun Chen. multi-modal ai copilot for single-cell analysis with instruction following. arXiv preprint arXiv:2501.08166 2025a.
Yin Fang, Qiao Jin, Guangzhi Xiong, Bowen Jin, Xianrui Zhong, Siru Ouyang, Aidong Zhang, Jiawei Han, and Zhiyong Lu. Cell-ol: Training Ilms to solve single-cell reasoning puzzles with reinforcement learning. arXiv preprint arXiv:2506.029) 2025b.
Minsheng Hao, Jing Gong, Xin Zeng, Chiming Liu, Yucheng Guo, Xingyi Cheng, Taifeng Wang, Jianzhu Ma, Xuegong Zhang, and Le Song. Large-scale foundation model on single-cell tran- scriptomics. Nature Methods, pp. 1-11, 2024.
Shuai He, Lin-He Wang, Yang Liu, Yi-Qi Li, Hai-Tian Chen, Jing-Hong Xu, Wan Peng, Guo- Wang Lin, Pan-Pan Wei, Bo Li, et al. Single-cell transcriptome profiling of an adult human cell atlas of 15 major organs. Genome biology, 21:1-34, 2020.
Ana-Maria Istrate, Fausto Milletari, Fabrizio Castrotorres, Jakub M Tomczak, Michaela Torkar, Donghui Li, and Theofanis Karaletsos. rbiol-training scientific reasoning llms with biological world models as soft verifiers. bioRxiv, pp. 2025 08, 2025.
Harrison Lee, Samrat Phatale, Hassan Mansoor, Kellie Ren Lu, Thomas Mesnard, Johan Ferret, Colton Bishop, Ethan Hall, Victor Carbune, and Abhinav Rastogi. Rlaif: Scaling reinforcement learning from human feedback with ai feedback. 2023.
Daniel LeVine, Syed Asad Rizvi, Sacha Lévy, Nazreen Pallikkavaliyaveetii, David Zhang, Xingyu Chen, Sina Ghadermarzi, Ruiming Wu, Zihe Zheng, Ivan Vrkie, Anna Zhong, Daphne Raskin, Insu Han, Antonio Henrique de Oliveira Fonseca, Josue Ortega Caro, Amin Karbasi, Rahul Mad- hav Dhodapkar. and David van Dijk. Cell2sentence: Teaching large language models the language of biology. In ICML. OpenReview.net, 2024.
Mohammad Lotfollahi, F Alexander Wolf, and Fabian J Theis. segen predicts single-cell perturba- tion responses. Nature methods, 16(8):715-721, 2019.
Erpai Luo, Minsheng Hao, Lei Wei, and Xuegong Zhang. sediffusion: conditional generation of high-quality single-cell data using diffusion model. Bioinformatics, 40(9):btae518, 2024.
Sai Ma, Bing Zhang, Lindsay M LaFave, Andrew S Earl, Zachary Chiang, Yan Hu, Jiarui Ding, Alison Brack, Vinay K Kartha, Tristan Tay, et al. Chromatin potential identified by shared single- cell profiling of ma and chromatin. Cell, 183(4): 103-1116. 2020.
Nicholas Matsumoto, Hyunjun Choi, Jay Moran, Miguel Hernandez, Mythreye Venkatesan, Xi Li, Jui-Hsuan Chang, Paul Wang, and Jason H Moore. Escargot: an ai agent leveraging large language models, dynamic graph thoughts, and biomedical knowledge graphs for enhanced reasoning. Bioinformatics, 41(2):btaf031, 2025.
11

---

Published as a conference paper at ICLR 2026
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. In NeurlPS, 2022.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning. Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 01133788-37744 2023.
Syed Asad Rizvi, Daniel Levine, Aakash Patel, Shiyang Zhang, Eric Wang, Sizhuang He, David Zhang, Cerise Tang, Zhuoyang Lyu, Rayyan Darji, et al. Scaling large language models for next- generation single-cell analysis. bioRxiv, pp. 2025-04, 2025.
Asa Segerstolpe, Athanasia Palasantza, Pernilla Eliasson, Eva-Marie Andersson, Anne-Christine Andréasson, Xiaoyan Sun, Simone Picelli, Alan Sabirsh, Maryam Clausen, Magnus K Bjursell, et al. Single-cell transcriptome profiling of human pancreatic islets in health and type 2 diabetes. Cell metabolism, 24(4):593-607, 2016.
Zhihong Shao, Peiyi Wang. Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang. Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024.
Ankur Sharma, Elaine Yiqun Cao, Vibhor Kumar, Xiaoqian Zhang, Hui Sun Leong, Angeline Mei Lin Wong, Neeraja Ramakrishnan, Muhammad Hakimullah, Hui Min Vivian Teo, Fui Teen Chong, et al. Longitudinal single-cell ma sequencing of patient-derived primary cells reveals drug-induced infidelity in stem cell hierarchy. Nature communications, 9(1):4931, 2018
Shuaijie She, Yu Bao, Yu Lu, Lu Xu, Tao Li, Wenhao Zhu, Shujian Huang. Shanbo Cheng, Lu Lu, and Yuxuan Wang. Dupo: Enabling reliable Ilm self-verification via dual preference optimization. arXiv preprint arXiv:2508. /4460, 2025.
Sanjay R Srivatsan, José McFaline-Figueree Vijay Ramani, Lauren Saunders, Junyue Cao, Jonathan Packer, Hannah A Pliner, Dana L Jackson, Riza M Daza, Lena Christiansen, et al. Mas- sively multiplex chemical transcriptomics at single-cell resolution. Science, 367(6473):45557 2020.
Aravind Subramanian, Rajiv Narayan, Steven M Corsello, David D Peck, Ted E Natoli, Xiaodong Lu, Joshua Gould, John F Davis, Andrew A Tubelli, Jacob K Asiedu, et al. A next generation connectivity map: L1000 platform and the first 1,000,000 profiles. Cell. 171(6):143-14452 2017.
Christina Theodoris, Ling Xiao, Anant Chopra, Mark Chaffin, Zeina R Al Sayed, Matthew Hill, Helene Mantineo, Elizabeth M Brydon, Zexian Zeng, X Shirley Liu, et al. Transfer learning enables predictions network biology. Nature, 618(7965):616-666 624, 2023.
Yurong Xin, Jinrang Kim, Haruka Okamoto, Min Ni, Yi Wei, Christina Adler, Andrew J Murphy, George D Yancopoulos, Calvin Lin, and Jesper Gromada. Rna sequencing of single human islet cells reveals type 2 diabetes genes. Cell metabolism, 24(4):608-615, 2016.
12


Reviews:
[{"review_id": "JSwOcyJNOK", "reviewer": "Reviewer_Kuj3", "summary": "Summary:\nThis paper introduces CellDuality, a self-superivsed framework to enhance the biological reasoning ability of LLM. It tries to address the challenge that it’s hard to build reward functions for most biological outcomes as it is non-verifiable. The framework uses complementary task duality. The method has achieved SOTA performance without needing ground-truth verification labels.", "questions": "See Weaknesses", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Pros:\n- The paper introduces CellDuality, a new self-supervised method to enhanve biological reasoning ability of LLMs\n- The method has achieved SOTA performance compared with baseline methods.", "weaknesses": "Cons:\n- The “cell sentences” representation used in the method is a simplified representation discarding the actual expression value. There should be better way to encode single-cell transcripomics data.\n- The claim of unified framework may be a bit too broad, as it only includes 4 downstream tasks with limited settings. \n- The proposed method is not consistently outperform baselines methods plus the additional reinforcement learning only bring very marginal benefits.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hvlx2jGeWo", "reviewer": "Reviewer_vKnC", "summary": "The paper introduces CellDuality, a self-supervised reinforcement learning framework for training large language models to perform biological reasoning in single-cell analysis. The method leverages Complementary Task Duality, where forward (primal) and backward (dual) reasoning tasks generate intrinsic rewards that enable alignment without ground-truth supervision. Extensive benchmarking on classification and generative tasks is also present.", "questions": "I would ask the authors to address the weaknesses listed above:\n1) Please include biologically grounded evaluations, such as verifying whether the model’s reasoning outputs (for example, perturbation responses) are consistent with known pathways or regulatory relationships.\n2) Provide an analysis of computational cost and scalability, including training time and hardware requirements for the RL alignment stage.\n3) Please discuss how the framework could be extended to multimodal datasets (for instance, integrating ATAC-seq or proteomic data) to further validate the generality of the approach.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1) The paper presents a technically sound and well-motivated framework that adapts reinforcement learning with verifiable rewards to inherently non-verifiable biological problems.\n2) The principle of complementary task duality is conceptually elegant, providing a self-consistent mechanism for intrinsic reward generation without labeled data.\n3) The experimental results are extensive, covering multiple benchmark datasets, and the ablation studies convincingly isolate the contribution of the self-supervised RL stage.", "weaknesses": "1) The study lacks an explicit evaluation of biological interpretability: while metrics demonstrate numerical improvements, there is no assessment of whether reasoning outputs capture biologically meaningful mechanisms.\n2) The framework’s scalability and computational cost are not detailed, especially regarding the RL alignment phase, which could be a practical limitation.\n3) The proposed approach is validated only on transcriptomic data; extending it to multimodal biological datasets would strengthen its generality claims.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rN9MI0ciSd", "reviewer": "Reviewer_nVH5", "summary": "This paper proposes CellDuality, a self-supervised RL-framework that enables large language models (LLMs) to perform open-ended biological reasoning in single-cell analysis. The key idea is to make sure each biological reasoning task such as drug response prediction is paired with an inverse task such as  recovering perturbation from response, forming a bidirectional reasoning loop. The consistency between the forward and inverse predictions is used as an intrinsic reward signal without labeling or annotation, enabling RL alignment without ground-truth verification labels. Experiments across four representative single-cell reasoning tasks demonstrate that CellDuality achieves competitive or slightly better performance compared to SFT baselines and narrows the gap to fully label-supervised RLVR models.", "questions": "- How many samples or optimization steps are passed for a complete RL stage? Could the authors annotate such information in Figure 2 (a)(b)? \n- Regarding the differences between the proposed self-consistency reward and ground-truth reward, what if the model predict the wrong label (X <> Y) during rollout sampling? Should the model learn useful knowledge or will this cause the model being enforced to predict the wrong label? \n- What is the base model and performance for CellDuality in Table 2-5? Line 377 indicates this is LLama-3.2-3B. Could the authors add the performance of base model before SFT for better comparison in these tables? \n- Could the authors attach the exact text prompts for constructing the CellDuality in the appendix, as a common practice, to enhance the reproducibility?\n- Could you also show some failure modes when the model exhibits degenerated consistency (eg., mutually consistent but biologically incorrect predictions).", "limitations": "", "rating": 2, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The proposed self-supervised RL training paradigm is training-efficient and agnostic to the choice of policy optimization algorithm, showing that consistent rewards can provide alignment signals without external supervision.\n- The Complementary Task Duality concept is clearly formulated and grounded in prior work (eg., DuPO), providing a natural extension to domains where canonical verifiable rewards are not available or expensive to acquire.", "weaknesses": "- The self-consistency reward effectively measures semantic or reconstruction fidelity rather than true biological correctness. While this provides a stable and well-defined signal, it may not inject new knowledge or enable deeper reasoning as canonical verifiers do in math, code, or symbolic domains. In this sense, the biological reasoning learned may remain shallow.\n- Across multiple evaluation cases (Tab 2-6), CellDuality shows seemingly marginal gains over its SFT baseline and does not consistently outperform strong supervised or specialized models. This raises doubts about the real effectiveness of the proposed self-supervised alignment compared to simpler fine-tuning approaches and how is applied to other relevant domains in biology.\n- The methodological novelty appears limited relative to DuPO (She et al., 2025), with CellDuality serving primarily as an instantiation in the single-cell domain rather than a substantial conceptual advancement.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)