CP-Agent: Context‑Aware Multimodal Reasoning for Cellular Morphological Profiling under Chemical Perturbations
Authors: Yuxin Zhang
Yiyao Li
Ping Shu Ho
Simon See
Zhenqin Wu
Kevin Kin-Man Tsia
Venue: ICLR 2026 Poster (2026)
Keywords: Cell painting microscopy; Biomedicine; Multimodal reasoning; LLM; Agent
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: We present CP-Agent, a multimodal agent that interprets drug-induced cell's morphological changes  by context-aware alignment.

Abstract: Cell Painting combines multiplexed fluorescent staining, high‑content imaging, and quantitative analysis to generate high-dimensional phenotypic readouts to support diverse downstream tasks such as mechanism-of-action (MoA) inference, toxicity prediction, and construction of drug–disease atlases. However, existing workflows are slow, costly and difficult to interpret. Approaches for drug screening modeling predominantly focus on molecular representation learning, while neglecting actual experimental context (e.g., cell line, dosing schedule, etc.), limiting generalization and MoA resolution. We introduce CP-Agent, an agentic multimodal large language model (MLLM) capable of generating mechanism-relevant, human-interpretable rationales for cell morphological changes under drug perturbations. At its core, CP-Agent leverages a context-aware alignment module, CP-CLIP, that jointly embeds high-content images and experimental metadata to enable robust treatment and MoA discrimination (achieving a maximum F1-score of 0.896). By integrating CP-CLIP outputs with agentic tool usage and reasoning, CP‑Agent compiles rationales into a structured report to guide experimental design and hypothesis refinement. These capabilities highlight CP-Agent’s potential to accelerate drug discovery by enabling more interpretable, scalable, and context-aware phenotypic screening---streamlining iterative cycles of hypothesis generation in drug discovery.

Published as a conference paper ICLR 2026
CP-AGENT: CONTEXT-AWARE MULTIMODAL REASON- ING FOR CELLULAR MORPHOLOGICAL PROFILING UNDER CHEMICAL PERTURBATIONS
Yuxin Zhang",, Yiyao Liz", Ping Shu Ho', Simon See", Zhenqin Wu?t, Kevin Tsia
Department of Electrical and Computer Engineering, The University of Hong Kong
School of Computing and Data Science, The University of Hong Kong
SSchool of Biomedical Engineering, The University of Hong Kong
*Nvidia AI Technology Center
Advanced Biomedical Instrumentation Centre
ABSTRACT
Cell Painting combines multiplexed fluorescent staining, high-content imaging, and quan- titative analysis to generate high-dimensional phenotypic readouts to support diverse downstream tasks such as mechanism-of-acciee (MoA) inference, toxicity prediction, and construction of drug-disease atlases. However, existing workflows are slow, costly and difficult to interpret. Approaches for drug screening modeling predominantly focus on molecular representation learning, while neglecting actual experimental context (e.g., cell line, dosing schedule, etc.), limiting generalization and MoA resolution. We introduce CP-Agent, an agentic multimodal large language model (MLLM) capable of generating mechanism-relevant, humaa--nterpretabll rationales for cell morphological changes under drug perturbations, At its core, CP-Agent leverages a context-aware alignment module, CP-CLIP, that jointly embeds high-content images and experimental metadata to enable robust treatment and MoA discrimination (achieving maximum Fl-score of 0.896). By integrating CP-CLIP outputs with agentic tool usage and reasoning, CP-Agent compiles rationales into structured report to guide experimental design and hypothesis refinement. These capabilities highlight CP-Agent's 's potential to accelerate drug discovery by enabling more interpretable, scalable, and context-aware phenotypic crreennng-streamlinnn iter- ative cycles of hypothesis generation in drug discovery.
1 INTRODUCTION
High-content imaging with Cell Painting has become workhorse for scalable phenotypic drug discov- ery. This technique, integrating advanced microscopy, multiplexed fluorescent staining and quantitative image analysis, allows us to establish high-dimensional morphological cell profiles that capture rich mul- tiscale cellular responses to chemical perturbations. These profiles have been proven valuable in support- ing mechanism-of-accion (MoA) inference (Tian al., 2023), toxicity prediction (Ewald et al.) (002)), hit triage (Vincent et al. 2020), and drug repurposing (Fredin Haslum et al.] 2024). while also enabling the construction of reference atlases and improved target deconvolution (Moffat et al. 2017).
In Cell Painting workflows, cells are perturbed under diverse conditions and the experimental context is not a nuisance to control but signal to model. For instance, dose and time define trajectories; cellular back- ground modulates pathway readouts (Appendix B.2). The resulting profiles guide follow-up experiments and can advance phenotype-driven drug discovery. However, Cell trinting-based drug discovery remains
'Equal contribution. CCoresponding authors. Project page: http:///gtaaaccommeeeii/aaag/aCa

---

Published as a conference paper ICLR 2026
limited by several challenges: (i) complex intermediate dependencies: Morphological responses are highly context-dependent. For example, concentration-depee deppedeent profiles show low correlations across dose levels (Pearson r 0.21-0.26) (Trpptssi et al., 2022). and MoA prediction is sensitive cell line context (Seal et al., 2024). Ignoring these structures conflates biology with acquisition artifacts and wastes the valuable metadata; (ii) convergent morphologies: Compounds with distinct mechanisms may induce morphological readouts convergence, reducing MoA resolution, thereby complicating the extraction of standardized, inter- pretable descriptors, (iii) Lack of semantic grounding: Representing image embeddings as unstructured feature vectors restricts their capacity for semantic reasoning and downstream biological inference.
Recently, various Al methods have been introduced to Cell Painting datasets, such as generative ap- proaches to synthesize images under perturbations (Navidi et al. 2024 Cross-Zamirski al. 2023 Palma et al. 2025), multimodal frameworks integrating chemical and genetic annotations with cell painting im- ages (Sanchez-Fernandez et al., (2023) (Fradkin al. 2024 Lu et al. 2025). For example, CLOOME firstly introduced a CLIP-style model to align Cell Painting images with molecular structures. MolPhenix and Cell- CLIP further extend this direction by leveraging strong unimodal foundation models to align the molecule. However, many existing models offer visual embeddings as black-box features, which lack semantic inter- pretability. Morcover, experimental context is often under-used: metadata is appended via late fusion or treated as unstructured text, yielding less informative representations and hindering iterative, closed-loop experimental design. Meanwhile, emerging multimodal large language models (MLLMs) offer reasoning capabilities and have been applied in diverse biological domains, such as genomics, biomedical imaging,
and omics data analysis (Zhang et al.] 2024a] Lin et al. 2025) [ii et al. [2024b Hu et al., 2024b Zhang et al. 2024b). Yet their applications in drug screening remain underexplored.
In this work, we introduce CP-Agent, a context-aware, agentic MLLM framework for Cell Painting drug perturbation screening. At its core is CP-CLIP, contrastive alignment module that jointly embeds Cell Painting images and structured experimental context, including drug compounds and other essential exper- imental conditions, enhancing the biological relevance of cell morphology. The model is pretrained on 1.9 million image-context pairs, with a customized token injection strategy that embeds key fields for better alignment. Comprehensive evaluations across curated classification tasks show that CP-CLIP outperforms general-purpose baselines. Built on this perception layer, CP-Agent integrates tool-augmented reasoning and task-adapted MLLMs grounded in phenotype descriptors and MoA ontologies to generate structured, interpretable outputs. Together, this agentic system supports scalable and interoperable phenotypic analysis, enabling cross-study generalization and providing actionable insights for assay prioritization and iteration, thereby accelerating hypothesis generation and improving decision-making in phenotypic drug discovery.
2
METHOD
2.1 DATASET
We employed three open-access Cell Painting datasets, consisting of approximately 1.9 million pairs: BBBC021 (Caie et al., 2010), CPJUMP1 (Chandrasekaran et al.; 2024), and RxRx3 (Fay et al. (023). encompassing diverse image and its associated experimental context (e.g., cell lines, experimental treatment conditions) We cu- rated compounds to ensure traceable MoA labels across datasets. For each collection, we matched SMILES representations of the perturbing chemical compounds to ChEMBL, retrieved their targets and MoAs, and retained only compounds with publicly resolvable MoA names. A summary of the curated setting is provided in Table T More details about dataset backgrounds are provided in Appendix C
cinpound-induce phenotypes. Each image-context pair comprises a microscopy
The training set comprises 1,846,436 image-text pairs, while the validation contains 9,395 pairs. For zero-shot evaluation, we curated a held-out set of compounds spanning all three datasets, selected to assess generalization unseen perturbations.
2

---

Published as a conference paper ICLR 2026
Table I: Summary of datasets used in this study
Dataset
Cell line
Channel
Compound
Concentration
Time
Image Pair
BBBC021
MCF-7 (p53 WT)
3
34
Variable 8-point half-log
24 h
144,411
CPJUMPI
U2OS, A549
62
5.0 μM
24 h, 48
562,687
RXRX3
HUVEC
6
380
Fixed 8-point half-log
~0 h
1,265,984
2.2
MOLECULAR DRUG ENCODING
Several established approaches map compound perturbations to vector representations, enabling alignment with image embeddings and facilitating multimodal learning (Winter et al., 2019 [W et al.] 2025). For instance, SMILES-basee (e.g.. ChemBERTa) and graph-based models learn molecular embeddings from structure, often using RDKit for preprocessing. Alternatively, one can compute continuous molecular de- scriptor embeddings (c.g.. physicochemical and topological descriptors), formalized as a parameterized fea- ture extractor: deese P) [fi (a; P1), fe (x; P2) ..... fa (æ; Pa)] E IR where æ is an input molecular representation (e.g., SMILES strings or molecular graphs), and each fi(æ; P.) extracts specific property, forming a d-dimensional real-valued feature vector. contrast, binary fingerprint embeddings that encode the presence/absence of substructures (e.g., Morgan/cireular, MACCS, or path-based fingerprints) (Bento) et al. 2020 Φrp? : .M- + {0, 1) d or Ng. yield binary or count-based encoding over the molecular space M.
2.3 CP-CLIP: REPROCESSING
To harmonize Cell Painting images across datasets with varying resolution and signal quality, we defined a channel-wise preprocessing step: P: IR Ho Wo + # x applied independently to each fluorescence chan- nel. This includes Contrast Limited Adaptive Histogram Equalization (CLAHE), random Laplacian sharpen- ing, and gamma correction, yielding enhanced images ī = P(T). Enhanced single-channel images are then cropped into 512 x 512 patches and stacked, yielding input tiles Tp E 12512x12xCC; For each perturbation tile Ep. a corresponding control tile 2;, E 1S12x612x is independently sampled from a matching control set Ω (Ir)). which share all experimental contexts (e.g.. plate, cell line, channel) with Tp. except for the per- turbation compound. That is x ~ 24 (Ω (zp)). The final image branch input is formed by concatenating the grayscale perturbation and control tiles along the channel dimension, & concat (Te, E 1R512x512x2, This paired design encourages the model to learn the contrasts between treated and untreated states.
Molecular descriptors are projected via fixed dimensional mapping faese : X' - IRd, where each feature dimension corresponds to predefined physicochemical topological property (See Appendix D Let v = Jaese (x) E Rdd denote the raw descriptor vector for compound æ E X. To ensure numerical stability and comparability across compounds, dimensions containing undefined values (e.g.. NaNs or Infs) are removed, and z-score normalization is applied independently to each feature dimension 02 #:-44
To account for the compound-speeifie dosing scheme, each molecule is represented by a normalized dosing pair pair|pmax, (C)), where Pmaxx denotes the molecular mass-normalized maximum concentration (in m),mL), and s(C) is the log-scaled dose step index corresponding to a given concentration. Let M E Rş0 denote the molecular weight (in Da or //mo)), and Ciaxx Rş0 the nominal maximum concentration (in jM). So, the molecular maximum mass concentration is given by:
Punax fing/mL M[Da] Cmax [uM]
106
where the denominator a reflects the conversion from μM and Da to mg/mL. While for each titration point C E (C1,.... Cs). pseudo-step index is computed on a log scale to reflect dilution ratios:
sC' logıo (Cmax) lC)
A log = 0.5
(2)
A log
3

---

Published as a conference paper ICLR 2026
Generated Drug Screoning Reports
Perception
Tools
Think Actions
Fealure Selector
Dala Analys
Calling APR
Alggmmnt
Users
Image
Text Bransh
Rum Teat
Outtanal Condition
Imaging Conditon
Drug Conditio
GPT4 Tokenbar
(V
Text Protedddiin
Itpp charees Se.. the saggrr cojactiio
Sbuctored Conditon Tranko
campunt AESLLRAA TakDeo
-Nocedonn (Mim babee whrr alaad Time
cony zan
MAN MTLL and
So di and 1LN
Tranfforer
1900 AIR mor
1100 Fott, Ie FOR
Figure 1: Illustration of the CP-agent (top) and CP-CLIP (bottom). CP-Agent connects perception, memory retrieval, and modular analysis into unified pipeline for generating reports for Cell Painting experiments. CP-CLIP forms the backbone of the CP-Agent's perception module, providing joint embeddings of Cell Painting images and structured experimental context.
where the denominator 0.5 corresponds to the log-fold change between adjacent titration levels in a 2-fold serial dilution protocol. A detailed derivation is provided in Appendix E
For observation time, let E R20 denote time in days. Temporal normalization rescales into the unit interval via: / Tilm with T max
112. The 112-day (16-week) window reflects the FDA's stopping rule, adopted by Watkins al. (2022) in their pharmacoeconomic analysis. These representations ensure that the input space remains consistent across compounds with varying dosing schemes and time-points.
2.4 CP-CLIP: CONTEXT- TOKEN PROJECTION
Our contrastive framework uses a structured text encoder tailored to the metadata obtained from drug screen- ing experiments (Figure [ bottom). Each experiment represented as a prompt-like sequence composed of cell culture, imaging. and drug compound perturbation conditions. So the "raw text" refers to structured experimental metadata such as cell line, culture medium, imaging parameters, compound identity, dosage, time and other cultural information if have. These contextual descriptions are first composed into a natural language-style sentence and tokenized into input IDs using the standard GPT-2. To accommodate structured context and consistent representations of perturbing compounds, we introduced field-specific placeholder tokens (i.e. <CMPD>, <CONC>, <T <TMM>> ) for compound descriptors zempd = &dese (æ; P) E IRd,, normal-
ized concentration Zcone = [Pmax: s sO)] E IR?, and normalized time 2tmee = f R. The special placcholder tokens are directly inserted into the text sequence and registered into the tokenizer's vocabulary. During tok- enization, they are automatically recognized as atomic units and their positions are preserved without being split or altered. Their embeddings are then dynamically computed via field-specific Multilayer Perceptron
4

---

Published as a conference paper ICLR 2026
User
Best matched Context!
Cuture Conditio
Imaging Condition
Drug Conditoo
Keyacrd
Vodled coll....
Extraded Ferzres
Are
CP-CLIP Agent
Towit 44
Mawaha sungle
Mess scDeertii Ne S0re
Exiar
Segmentor
Tools
Inpurarety Unirarayy
YONA
background specits Gell mochan teight changes....
user, promitt Given Painsing Ahich marphslogical features maci auily be afacded
Intornddiate Answwer2
FeadRank Agent
Painting imvagng channel provite
"ramm"
Tuault,
prompt:
You areal tromacizal
Surnayy Theatie
ReportGen Agant
featurs, stutistic, summary: Coopuen pne faaiie riteow time parturiatinn
Loer, peorpt: Compare onpected fior)
pertuibaion conctiin vo observeed and uluun meitennss
Answentt
System prompt:
Yours wiag Lun.e/ demratt
-ed in diforonzes
pteeomios aralys epepiliingg Painting
-cotarup ficenccc intorwal
(018% ddainice cance Jp.
spadiwrts
Figure 2: Automated cell-phenotype assessment pipeline of CP-Agent. Upon user query, CP-CLIP retrieves the relevant experimental context guide cell segmentation and feature extraction. Downstream agents then rank morphological changes and generate interpretable, end-to-end phenotype reports.
(MLP) trunks f. : Rdd D::
fempd (Zempu) IR"
Cempd
Geone
fconc (Zcone) IR'
Ctime fume (Ztime) E
IR
where Sempa; Scone; and ftime are lightweight MIP trunks encoding compound identity, concentration, and time-point used in place of the placeholders. The resulting text input is a hybrid sequence:
x = [CLS, f1, t2,...,
Cempd
Gouns
Cime
(4)
<CMPD>
<CONC>
<TIME>
This hybrid sequence, combining standard subword embeddings 4, E TR. D with structured embeddings e, E RD from field-specific MLPs, is fed into the text Transformer to produce final text representation. Implementation details are in Appendix F By replacing placeholder tokens with learned embeddings, the model fuses continuous metadata with discrete language tokens in shared embedding space. The text en- coder thus captures both experimental signals and linguistic coherence, enabling better semantic alignment.
2.5 CP-AGENT WORKFLOW
CP-Agent adopts modular, eeeoory-uggment architecture that connects perception, tooling, and analysis into a single-par palleppasspps pipeline (Figure top). Given user-provided Cell Painting images, a lightweight memory retriever powered by CP-CLIP fetches the most probable experimental context (i.e., cell line, fluorescence channels, imaging settings, chemical perturbations). Once the experimental context is retrieved, the pipeline proceeds to visual analysis. Rather than relying on vision backbones that produce holistic, biologically opaque embeddings,
extract handcrafted single-cell morphological features. These interpretable repre- sentations are processed by a modular, MLLM-driven agent architecture, where the MLLM serves as a policy layer that dynamically routes tasks to interchangeable tools and integrates their outputs. We frame this sys- tem as "agentic" in the sense of procedural autonomy (Xu et al.) 2025): unlike reinforcement learning-base planners, CP-Agent employs the MLLM cognitive controller within a structured workflow. It relies
5

---

Published as a conference paper at ICLR 2026
on the model's learned reasoning capabilities- than fixed logical scripts-to dynamically prioritize morphological features, interpret statistical distribution shifts, and synthesize mechanism-level hypotheses based on retrieved experimental context.
We instantiate this concept on fluorescence Cell Painting data via a specialized CP-Agent workflow (Fig-
ure 2. which comprises the following steps:
CPContext Agent Given paired Cell Painting images (control vs. perturbation) acquired under matched conditions, the CPContext Agent employs pre-trained CP-CLIP retriever to obtain experimental context from curated knowledge base. Simultancously, harmonizes metadata via contolled-vocabult and channel labeling to generate standardized descriptors. Retrieved context is routed both (A) as a context bundle to FeatRank Agent, ReportGen Agent, and (B) as metadata keywords to the CellFeat Agent.
tagging
ChannelSeg Agent Given Cell Painting images, the ChannelSeg Agent performs nuclei instance segmen- tation on DNA-stained channels and whole-cell segmentation on non-DNA channels (e.g., RNA, Actin, ER, etc.). It outputs channel-specific instance masks, which are passed to the CellFeat Agent.
CellFeat Agent Given Cell Painting images, corresponding masks, and harmonized metadata, the CellFeat Agent extracts per-cell morphological, intensity, texture, granularity, neighborhood, and occupancy fea- tures using configured CellProfiler pipeline (Appendix HH). Output is routed both (A) as extracted feature items to the FeatRank Agent for mechanism-aware selection, and (B) as channel-wise single-cell feature matrices to the StatSynth Agent for statistical evidence synthesis.
FeatRank Agent Given extracted feature items and experimental context, the FeatRank Agent scores and ranks features by their likelihood of being influenced by the perturbation. It generates confidence confidence-eighheed rationales to support prioritization. Output is routed as a prioritized feature list with explanations to the StatSynth Agent.
StatSynth Agent Given the prioritized feature list, full feature matrices, and experiment-level context, the StatSynth Agens computes per-feature statistical evidence between control and perturbation conditions based on the prioritized features. It summarizes distribution shifts, effect sizes, confidence intervals, and statistical significance. Outputs are routed as statistical summaries and interpretations to the Reporten Agent for final report composition.
ReportGen Agent Given statistical summaries, prioritized features, visual exemplars, and experimental context, the ReportGen Agent composes an integrated interpretation of the perturbation's biological im- pact. It identifies key morphological shifts and evaluates their consistency with expected cellular responses to infer plausible mechanisms. The resulting report summarizes these findings, provides follow-up rec- ommendations and visualizations, and is delivered to the users for downstream access.
The agent tool stack integrates both classical and learning- based components. For segmentation, we fine- tuned VISTA-2D He al. for 20 epochs using diverse augmentation strategies to mitigate optics-induced batch effects. The model generates channel-specific masks that enable biologically consistent segmentation across diverse imaging conditions. More details regarding dataset preparation and training of the segmenta- tion model are provided in Appendix]] [] The StatSynth Agent is tasked with reasoning over high-dimensional single-cell morphological data (typically 30-300 cells per image). which impractical for direct LLM ap- plication due to length constraints and noise (Fang et al., (204)). Instead, we curate agentic tools that (i) aggregate summary statistics for key features, and (ii) quantify distribution shifts between control and per- turbed samples. These compact, interpretable summaries support reliable procedures for this step are provided in Appendix L
reasoning. Detailed
3 EXPERIMENT AND RESULTS To assess the effectiveness of CP-Agent. we isolated and evaluated its components before measuring end-to-end reporting quality: (a) CP-CLIP (context-aware retrieval and alignment): we evaluate its accuracy on in-distribution classification (seen-drug) and generalization (unseen-drug matching), ablations against MILM baselines and CLIP variants; (b) Vision embedding structure: we evaluate whether CP-CLIP em- beddings encode chemically grounded, dose- and MoA-dependent morphology; (c) Statistical synthesis and
6

---

Published as a conference paper ICLR 2026
Table 2: Model performance on classification tasks
Model
Cell
Channel
Perturbation Compound
Fändokaller
Rececadetil
AZM 475271
Misoquenti
Trazodone
Oruztnbb
Ruttramidz
Lunincosib
BJRB Methosslte
Rasion Cupsing
0.143
0.10
0.10
0.10
0.10
0.30
0.10
Grde-4
0.448
0.228
0.215
0.174
0.190
0.034
0.0
0.0
0.102
GPT-5
0.377
[.439
0.59
0.168
0.0
0.0
0.353
0.0
0.0
0.0
0.0
0.0
0.074
Claste 4 Soneet
0.400
0.195
0.0
0.00
0.0
0355
0.0
0.9
0.0
0.0
0211
0.
0,027
Cemini
0.526
0.628
0.0
0.0
0145
0.007
CLOOME VIT-Rr16
0.764
0.764
0.653
0.55
CLIP VIT-8/16
10000
0.955
0.776
0.600
0.661
0.500
0.600
0.575
0637
SIALIP B/1
1,000
0.925
0.224
0.411
0.315
0.826
0.291
0.638
0.395
0.272
0.604
0.400
0.514
CP CLIP SIJUIP VIT Br16
(Avuriittrr
1,000
0.034
0.655
0.442
0.483
0.776
0.351
0.850
0.255
0.186
0.660
0.620
0,532
CLIP VIT
0.991
0.899
0.862
0.891
0.894
0.540
0.971
0.887
CP CLP Vi1-
1,000
0.882
0.907
0.869
0.357
0.540
0.384
0.854
0922
0.8%6
CPCCIP V11-1/16
(Aouriptoo,
1,000
01849
0.388
0,800
0.5%6
07346
03843
01M2
0
01819
0915
0.441
0.891
Table 3: Unseen drugs similarity score
Model
Regorafenib
Sacubitril
Bupartisib
Dexamethasone
Nimodipine
AZ258
Nilotnab
MG
Average
CLIP VT-B/116
0:207 c,002
0.2058 = 0.304
0.269 0.04s
0.3601 + 0.000
0.377 a.01
0.528 c.000
00.000
0.340 = 0.or2
0.286
SigLIP VIT-/116
0,038
0.095
0.129
0.146
0.183
0.090
00005
0.143
0.096
CP-CLIP SigLIP-VIT--O VIT-BI1
(dezcriptor)
0.37824.007
0.420 10
0.323 2 102
0. 0.503 o 100
0.516 31662.00 0.05
0.488 2:0.11
0.303 a 0.0000
0.380
0.414
CP-CLIP VIT-B/16
(fugerprind)
0.257
0.222
0.375
0.468
0.461
0:420
0.210
0.420
0.300
CP-CLIP WT-B/16
(dexcriptor)
0.452
0,412 0.0M
0.396 :
00.075 0.005
0. 0.012
0,468 C0.10
0.324 + 0,055
0.448 0.oss
0,432
CP-CLIP ViT-L/16 (descriptor)
0.455 2..118
0.445 2.330
0.408 = 0,053
0.530
0.523 0.00
0.448
U. 1.206- +:
0.448 2
0.444
reporting: whether compact summaries enable robust comparisons between control and perturbation in the generated report. Finally, we assessed the effectiveness of full CP-Agent reports via expert review.
3.1 MODEL VARIANTS AND MLLM BASELINES
To contextualize the performance of our proposed model, we compared it against several leading MLLMs. Specifically, we included Grok-4 (xAT, 2025), GPT-5 (OpenAl 2025). Claude-4-Sonnet (Anthropic 2025).
and Gemini-2.5-Pro (Google DeepMind, 2025), which have demonstrated strong performance across range of general-purpos multimodal benchmarks. Following recent benchmarking protocols for MLLMS in biomedical and healthcare settings (Lozano et al.] 2024] Burgess et al. (025). we adopt a zero-shot, two-stage prompting pipeline: first, the models were prompted to curate background knowledge relevant to Cell Painting experiments; then, they were asked to answer multiple-choice questions about experimental conditions given the curated background knowledge, paired control and perturbation images, and masked textual prompts. To further narrow the adaptation gap and make the comparison more conservative, we additionally evaluated few-shot variant in which the MLLMs were provided with a small visual memory bank (two labeled exemplars per class) before answering the same tasks. Detailed prompt templates and the corresponding zero-shot and few-shot results are reported in the Appendix M
Alongside these MLLMs, we benchmarked multiple variants of our contrastive learning framework, CP- CLIP, which extends the CLIP architecture by integrating structured experimental context into training. As a baseline, we used the original CLIP model based on the ViT-B/16 vision backbone, retrained on natural language text aligned with Cell Painting images. All CP-CLIP variants enhance this setup by injecting serialized numerical metadata, as detailed in Section 2.4 We evaluated CP-CLIP variants that differ in compound encoding and loss function (See Appendix G including: (i) a descriptor-based model used continuous molecular descriptors, and (ii) a fingerprint-basee based model used binary fingerprints. We also tested a SigLIP variant that uses sigmoid-based pairwise contrastive objective (Zhai et al. 2023). To assess the impact of vision model capacity on performance, we tested a CP-CLIP variant with VIT-L/16 vision backbone.
7

---

Published as conference paper ICLR 2026
(a)
(b)
Compound-based
Danes
MOA-based
ab
ecethiree
Cleose Martte Iraoe lat
deoadin
arbumynn
ectent--zoeetngg
demaage
dccetanol
micrsibbuee stab protein degredation prosein syntheds
-40
mg n2:132
derracakine
Envodtaton
oc541
Anisomycin
Bryostatin
Floxuridine
(c)
Oraine
1.1 12 13 14 15 10
12 14
1-7 1.8
13 14
14 147
1-1 1-2 5-3
1-5 145 1-7 148
Concentatioon
Concentraiion
Concentation
Concenizatio
Figure 3: CP-CLIP captures pharmacologically meaningful morphology. UMAP projections of CP-CLIP image embeddings, colored by (a) compound identity and (b) mechanism of action (MoA). The clear clus- tering indicates that the learned representation encodes biologically relevant morphology. (c) Concentration- dependent morphological changes captured using image embeddings extracted from samples treated with varying compound doses.
3.2 TASK SEEN-DRUG CLASSIFICATION
To benchmark distribution performance, we designed classification tasks across three categories: cell line, fluorescence channel and compound. The classification is performed retrieval-based inference by ranking cosine similarity scores between image embeddings and set of candidate textual prompts, following the standard CLIP paradigm. Those contextual metadata includes both textual and numerical variables, which are encoded jointly as a natural language sequence, enabling prompt-based querying without the needs for task-specific heads. For example, for compound classification, 10 compounds were randomly sampled to form a balanced 10-class setting. Table 2 summarizes the results. Among all general- purpose MLLMs, Gemini-2.55 2.r-Pro achieved the best performance the cell line and channel prediction tasks (F1: 0.526 and 0.628). However, on compound classification, performance dropped sharply: All models fell below random baseline, except for Grok-4, which slightly exceeded it. Confusion matrices (Appendix MM.3) revealed near- zero Fl scores, indicating systematic failure in identifying perturbing chemical compounds and limited generalization of current MLLMs. In contrast, CP-CLIP consistently outperformed both the baseline CLIP and all MLLMM across tasks. Descriptor-basee based models slightly outperformed fingerprint-base ones on compound classification (F1: 0.891 vs. 0.887), indicating that continuous encodings provide richer chemical contexts. Scaling the vision encoder from ViT-B/16 to ViT-L/16 yielded no significant gain (FI: 0.896 vs. 0.891), indicating that lightweight backbone suffices when paired with strong chemical priors. Taken together, these MLLMs results also constitute a "no-CPContext" baseline, reinforcing the conclusion that without explicit perturbation-awe aware grounding, current MLLMs fail to extract meaningful biological signals from Cell Painting image. This emphasizes the essential role of CP-CLIP as the perception in CP-agent.
8

---

Published as a conference paper at ICLR 2026
3.3 TASK II: UNSUEN-DRUG MATCHING
To evaluate generalization, we performed zero-shot prompt-image matching on held-out compounds by computing cosine similarity between image and prompt embeddings (Table 3). The baseline CLIP model (ViT-B/16) yielded low alignment on unseen drugs (avg. similarity = 0.286), while CP-CLIP (descrip- tor, ViT-B/16) achieved 0.432, 14.6% absolute increase. Descriptor-based models also outperformed fingerprint- ones (0.432 vs. 0.360), indicating that continuous encodings capture more relevant chemi- cal contexts. Scaling the vision encoder from ViT-B/16 to ViT-L/16 further improved performance 0.444, suggesting enhanced robustmess to morphological variation. To provide a comparative reference, we also evaluated similarity seen drugs (Appendix DD. Notably, performance on unseen drugs remained close, indicating strong generalization. Specifically, descriptor- based ViT-B/16 and ViT-L/16 models achieved 0.549/0.432 and 0.561/0.444 on seen/unseen drugs, suggesting that CP-CLIP captures mechanism-rrelevant biology, rather than memorizing labels. This zero-shot capability supports practical applications such as MoA hypothesis generation, hit prioritization, and generalization to novel perturbation contexts.
3.4 VISION EMBEDDING ANALYSES
F- shows UMAP projections of embeddings from CP-CLIP ViT-B/16 (descriptor). The UMAP pro- jection reveals clustering by MoA, indicating the learned representation encodes pharmacologically mean- ingful morphology beyond compound identity. Figure shows pnnucentratioo-rllate patterns for four drugs selected from the BBBC021 I and RxRx3 datasets. CP-CLIP embeddings exhibited clear dose-response tra- jectories, reflecting concentration-ddeendder morphological change. particular, the sharp dose-responses observed for Anisomycin and Bryostatin are consistent with previous reports [ranston et al. )1982); Mar- shall et al. (2002). In contrast, drugs with minimal morphological impacts show flatter trends across dosage. More examples and a detailed explanation of this schematic are provided in the Appendix K
3.5 CP-AGENT REPORTS
We present three case studies from different datasets to demonstrate CP-Agent generated reports (Figure (i)Example (BBBC021, MCF7 + Taxol): Taxol induces clear cytoskeletal phenotype by stabilizing mi- crotubules and arresting mitosis (Kiwanuka et al.) 2022). CP-Agent detected the localized changes in tubulin texture and correctly linked them to microtubule stabilization and mitotic arrest, demonstrating its ability to recognize canonical, visually prominent phenotypes. (ii) Example 2 (CPJUMP, A549 Sorbinil): Sorbinil is an aldose reductase inhibitor that produces a subtle and uncertain phenotype (Zietek et al., (005).. CP-Agent detected modest shifts (e.g., smoother RNA texture, reduced granularity), and suggested potential stress granule suppression. Meanwhile, it also flagged ambiguity and suggested further validation, illustrating its ability to reason under uncertainty. (iii) Example 3 (RxRx3, HUVEC + BGT226): BGT226 is PI3K/m'TO inhibitor, leading to a multi-comparlmeni phenotype affecting organelles, cell shape, and density (Kampa- Schittenhelm al. 2013). By integrating mitochondrial texture, cell area, and density changes, CP-Agent inferred PI3K/TTO inhibition, showcasing its capacity to synthesize complex morphological cues into mechanistic insights. Together, these cases show that CP-Agent adapts diverse biological contexts, rang- ing from clear to ambiguous phenotypes, and generates biologically grounded summaries. Additional ex- amples and reasoning details are provided in Appendix O
We conducted an expert survey to assess whether LLM-based CP-Agent produces accurate and well- reasoned screening reports. Four LLMs (mentioned in Section 3.1) each generated reports for ten con- trol-perturbation image pairs. Experts (N = 11), ranging from PhD students to professors in pharmacology or related fields, rated 40 reports (10 pairs x models) on a 1-7 scale across ten criteria from Waqas et al. (2025), covering language guality and reasoning quality. Full criteria definitions and examples provided in Appendix PP As shown in Figure [6] most metrics received high scores across models. CP-Agent pow- ered by GPT-5 showed the strongest overall reasoning performance, followed closely by Gemini-2.5-Pro.. To further assess the interpretability and consistency of the CP-Agent framework, we conducted a system- atic evaluation of the two LLW-re modules: FeatRank Agent and ReportGen Agent. As shown in
9

---

Published as a conference paper at ICLR 2026
Control
Perturbed
Generated Brief Reports
Nrt Scid
Sars Jarie 21
Mechaniem linked cummary
The spceen In marut vid Butt vhelhe by cherred mashaaop of wit the fycofl wer sterzca mager
BBBC021
stdellan. iroon alle virrube conantis di suparaatio in risii amatt thes coul prisert Higher Lduinn tadur white codennelcoorraes tlavene enerrpp and abrong
ahfir 2.00
Mibbrrt 1.00.2
Experinental Connott
Call line NDFT
-urare medi-nn APNI 1640, 10% Fed. uigtt
Trexiin 20X abjection Compoand Tunst 1o4 utt 201)
Sarpie con peruite snall 10) and searra fass lagge
and and Falme wir dose enpersediinei ritele daveghan the riiluiin
Mechaniem linked summary
repected sokaer
HiVt
erae water reoholegy argne hepoffee
CPJUMP
ixpored
mey highly lhs alulty
abongly
area 1966 pravearty Chen pecooee
Most Matched Experimental Content: -Cll lis A549
whith dovil are comore cabiiie
fexture
medium
tmoglng RMA chorme, 2006 otpectivv
the thet seste
Mochamam acdinn Alboue canue
995
Machaniem linked summary
R61770 tonee
ant undei
uin
PalONTOR inbtli vedeo far Key dgg Ecassy eeborad eo spadd
Roibd
and rick
-Sell HJVEC
ale caldd and souiiing ui oystindion
-Gilire rentim: EGM2 reetium
thr wail
-Sompaant: BC122022 phi, 201)
oftereet vodis
fore
-Wchhnion action PKTK clses inmibror
the recterae
Figure 4: Summary reports generated from CP-Agent. The examples show CP-Agent's 's ability to recognize clear (Taxol), subtle (Sorbinil), and complex (BGT226) morphological responses, linking them to plausible biological mechanisms.
Table [5] the selected morphological features remained highly consistent across runs, indicating robust and stable feature prioritization. Table also revealed a stable corpus-level consistency of the report generated from ReportGen Agent.
4 DISCUSSION AND CONCLUSION
We present CP-Agent, a context-aware multimodal reasoning framework for interpretable analysis of Cell Painting drug responses. Its core, CP-CLIP, aligns imaging data with experimental context, enhanced by nu- merically grounded token injection. This yields strong generalization and outperforms baselines on multiple classification tasks. CP-Agent separates and coordinates perception, retrieval, analysis, and reporting into specialized agents (i.e,,CPContex: ChannelSeg, CellFeat, FeatRank, StatSynth, ReportGen). This enables an evidence-first workflow where CP-Agent converts high-dimensional morphological features, together with the experimental context, into compact, calibrated summaries that an MI.LM synthesizes into inter- pretable narratives. Hence, CP-Agent allows end-to-end biological interpretability. Users can trace predicted mechanisms back to corresponding morphological features-frow images to masks, features, statistics, and final explanations. Unlike histology tasks, where many agent-based pipelines can perform well without train- ing by using well-designed chain-of- thought with off-the-shelf MLLMs, our results show that zero-shot prompting for Cell Painting datasets consistently underperforms, and biologically grounded supervision is essential for meaningful reasoning. CP-Agent also generalizes to various imaging modalities such as quan- titative phase imaging (QPI), digital holographic microscopy, and brightfield time-lapse imaging (Lo et al.) 2024| Siu et al. 2023 Zhang et al. 2023 Lee et al. 2025) and integrates flexibly with tools like ilastik, Fiji, and Icy. Overall, it establishes new paradigm for combining MLLMs with mechanistically grounded anal- ysis, offering a foundation for next-generation AI systems in phenotypic drug discovery. Looking forward, the modular agentic architecture of CP-Agent could flexibly be extended for experimental planning (e.g.. dose strategy refinement), multi-omics fusion, well causal priors for counterfactual reasoning.
10

---

Published as a conference paper ICLR 2026
ETHICAL STATEMENT
This work does not involve human subjects, animal experiments, or personally identifiable data. All experi- ments are conducted on publicly available Cell Painting datasets.
ACKNOWLEDGEMENTS
The work is supported by Advanced Biomedical Instrumentation Center, the Research Grants Council (grant no. 17125121, 14125924, 17128225, CFS2021-7S06) the Innovation and Technology Commission of the Hong Kong Special Administrative Region of China (grant no. ITS/318/22FP, ITS/408/23FP).
REFERENCES
Anthropic. Claude 4 sonnet. Anthropic Model Card, 2025. Available at: htttps//www.atthrrrs/ com [Accessed: September 2025].
Reza Averly, Frazier N Baker, Ian A Watson, and Xia Ning. Liddia: Language- based intelligent drug discovery agent. arXiv preprint UrXv:22502.1395, 2025.
Nitin Sai Beesabathuni, Neil Alvin B Adia, Eshan Thilakaratne, Ritika Gangaraju, and Priya S Shah, Image- based temporal profiling of autophagy relattd phenotypes. Autophagy reports, 4(1):2484835, 2025.
A Patrícia Bento, Anne Hersey, Eloy Félix, Greg Landrum, Anna Gaulton, Francis Atkinson, Louisa J Bellis, Marleen De Veij. and Andrew R Leach. An open source chemical structure curation pipeline using rdkit. Journal of Cheminformaties, 12(1):51, 2020.
Mark-Anthony Bray, Shantanu Singh, Han Han, Chadwick T Davis, Blake Borgeson, Cathy Hartland, Maria Kost-Alimova, Sigrun M Gustafsdottir, Christopher C Gibson, and Anne E Carpenter. Cell painting, a high-content amage-based assay for morphological profiling using multiplexed fluorescent dyes. Nature protocols, (9):1777-17744 2016.
James Burgess, Jeffrey J Nirschl, Laura Bravo-Sánchez, Alejandro Lozano, Sanket Rajan Gupte, Jesus Galaz-Montoys Yuhui Zhang, Yuchang Su, Disha Bhowmik, Zachary Coman, et al. Microvqa: A mul- timodal reasoning benchmark for microscopy-baatt based scientific research. Proceedings of the Computer Vision and Puttern Recognition Conference, pp. 19552-19564, 2025.
Peter D Caie, Rebecca E Walls, Alexandra Ingleston-Orme, Sandeep Daya, Tom Houslay, Rob Eagle, Mark Roberts, and Neil O Carragher. High-content phenotypic profiling of drug response signatures across distinct cancer cells. Molecular cancer therapeutics, 96)) 2010.
Srinivas Niranj Chandrasekaran, Beth A Cimini, Amy Goodale, Lisa Miller, Maria Kost-Alimova, Nasim Jamali, John G Doench, Briana Fritchman, Adam Skepner, Michelle Melanson, et al. Three million im- ages and morphological profiles of cells treated with matched chemical and genetic perturbations. Nature 2024.
Methods, 21(
Jonathan Choy, Yanqing Kan, Steve Cifelli, Josephine Johnson, Michelle Chen, Lin-Lin Shiao, Haihong Zhou, Stephen Previs, Ying Lei, Richard Johnstone, et al. High-throughput screening to identify small molecules that selectively inhibit apoll protein level in podocytes. SLAS DISCOVERY: Advancing the Science of Drug Discovery, 26(9):1225-1237 2021.
WI Cranston, RF Hellon, and Y Townsend. Further observations on the suppression of fever in rabbits by intracerebral action of anisomycin. The Journal of Physiology, 322(():441 445, 1982.
Jan Oscar Cross-Zamirsk Praveen Anand, Guy Williams, Elizabeth Mouchet. Yinhai Wang, and Carola- Bibiane Schönlieb. Class-guided image-to-image diffusion: Cell painting from brightfield images with class labels. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3800- 3809, 2023.
11

---

Published as a conference paper at ICLR 2026
Jessica D Ewald, Katherine L Titterton, Alex Bäuerle, Alex Beatson, Daniil A Boiko, Ángel Cabrera, Jaime Cheah, Beth A Cimini, Bram Gorissen, Thouis Jones, et al. Cell painting for cytotoxicity and mode-of-action analysis in primary human hepatocytes. bioRxiv, 2025.
Xi Fang, Weijie Xu, Fiona Anting Tan, Jiani Zhang, Ziqing Hu, Yanjun Qi, Scott Nickleach, Diego So- colinsky, Srinivasan Sengamedu, and Christos Faloutsos. Large language models (Ims) on tabular data: Prediction, generation, and understanding- survey. arXiv preprint arXiv:2402.1793 17944, 2024.
Marta Fay, Oren Kraus, Mason Victors, Lakshmanan Arumugam, Kamal Vuggumudi, John Urbanik, Kyle Hansen, Safiye Celik, Nico Cernek, Ganesh Jagannathan, et al. RxXx3: Phenomics map of biology. Biorxiv, pp. 2023-02, 2023.
Philip Fradkin, Puria Azadi Moghadam, Karush Suri, Frederik Wenkel, Ali Bashashati, Maciej Sypetkowski, and Dominique Beaini. How molecules impact cells: Unlocking contrastive phenomolecular retrieval. Advances in Neural Information Processing Systems, 11111666-111001. 2024.
Johan Fredin Haslum, Charles-Hugues Lardeau, Johan Karlsson, Riku Turkki, Karl-Johan Leuchowius,
Kevin Smith, and Erik Müllers. Cell painting-based bioactivity prediction boosts high-throughput screen- ing hit-rates and compound diversity, Nuture Communications, 15(1):3470, 2024.
Google DeepMind. Gemini 2.5 pro. Google DeepMind, 2025. Available at:
google (Accessed: September 2025].
Linda Harkness, Xiaoli Chen, Marinann Gillard, Peter Paul Gray, and Anthony Mitchell Davies. Media composition modulates human embryonic stem cell morphology and may influence preferential lineage differentiation potential. PLoS One. 14(3)xe011378,, 2019.
He, Guo, Y Tang, A Myronenko, V Nath, Xu, et al. Vista3d: a unified segmentation foundation model for 3d medical imaging (2024).
Haigen Hu, Xiaoyuan Wang. Yan Zhang, Qi Chen, and Qiu Guan. A comprehensive survey on contrastive 2024a.
learning. Neurocomputing,
Mingzhe Hu, Joshua Qian, Shaoyan Pan, Yuheng Li, Richard LJ Qiu, and Xiaofeng Yang. Advancing medical imaging with language models: featuring a spotlight on chatgpt. Physics in Medicine & BBioloyy, 69 10T101, 2024b.
David J Huggins, Ashok R Venkitaraman, and David R Spring. Rational methods for the selection of diverse screening compounds. ACS chemical biology, 6(3):208 217, 2011.
Kerstin Maria Kampa-Schittenhelm Michael Charles Heinrich, Figen Akmut, Katharina Henriette Rasp, Barbara Illing, Hartmut Döhner, Konstanze Döhner, and Marcus Matthias Schittenhelm. Cell cycle- dependent activity of the novel dual pi3k-mtorc1/2 inhibitor nvp-bgt226 in acute leukemia. Molecular cancer, 12(1):46, 2013.
Martin Kiwanuka, Ghodeejah Higgins, Silindile Ngcobo, Juliet Nagawa, Dirk M Lang, Muhammad H Za- man, Neil H Davies, and Thomas Franz. Effect of paclitaxel treatment on cellular mechanics and mor- phology of human oesophageal squamous cell carcinoma in 2d and 3d environments. Integrative Biology, 14(6):137-149, 2022.
Thiel Lee, Evelyn HY Cheung, Kelvin CM Lee, Dickson MD Siu, Michelle CK Lo, Edmund Y Lam, Ruchi Goswami, Salvatore Girardo, Kyoohyun Kim, Felix Reichel et al. High-throughput multimodal optofluidic biophysical imaging cytometry. Lab Chip, 2025.
12

---

Published as a conference paper ICLR 2026
Vanille Lejal, David Rouquié, and Olivier Taboureau. Cell morphology and gene expression: tracking changes and complementarity across time and cell lines. Toxicology and Applied Pharmacology. 504: 117530, 2025. ISSN 0041-008X. doi: https//ddd.aogg00000006..1.aaa2.
URL https://
www. www.scceeeenerett.cc com/ ott.a//cc......o.oooooooooo....a
Anqi Lin, Junpu Ye, Chang Qi, Lingxuan Zhu, Weiming Mou, Wenyi Gan, Donggiang Zeng, Bufu Tang, Mingjia Xiao, Guangdi Chu, et al. Bridging artificial intelligence and biological sciences: a compre- hensive review of large language models in bioinformatics. Briefings in Bioinformatics, 26(4):bbaf357, 2025.
Fangyu Liu, Olivier Mailhot, Isabella S Glenn, Seth F Vigneron, Violla Bassim, Xinyu Xu, Karla Fonseca- Valencia, Matthew Smith, Dmytro Radchenko, James S Fraser, et al. The impact of library size and scale of testing on virtual screening. Nature chemical biology, pp. 1-7, 2025a.
Haoyang Liu, Shuyu Chen, Ye Zhang, and Haohan Wang. Genotex: An Ilm agent benchmark for automated gene expression data analysis. arXiv preprint arXiv:2406. 15341, 2024a.
Tianyu Liu, Yijia Xiao, Xiao Luo, Hua Xu, W Jim Zheng, and Hongyu Zhao. Geneverse: A collection of open-source multimodal large language models for genomic and proteomic research. arXiv preprint arXiv:2406. 15534, 2024b.
Wuchao Liu, Han Peng, Wengen Li, Yichao Zhang, Jihong Guan, and Shuigeng Zhou. sci2el: Effec- tively integrating single-cell multi-omics by intra-and inter-omics contrastive learning. arXiv preprint arXiv:2508. 18304, 2025b.
Michelle CK Lo, Dickson MD Siu, Kelvin CM Lee, Justin SJ Wong, Maximus CF Yeung, Michael KY Hsin, James CM Ho, and Kevin K Tsia. Information-distille generative label-free morphological profiling encodes cellular heterogeneity. Advanced science, 1 11(29):300551, 2024.
Alejandro Lozano, Jeffrey Nirschl, James Burgess, Sanket Rajan Gupte, Yuhui Zhang, Alyssa Unell, and Serena Yeung-Levy. ( [mmu)-benntl A vision-language benchmark for microscopy understanding. arXiv preprint arXiv:2407.01 791, 2024.
Ming Y Lu, Bowen Chen, Drew FK Williamson, Richard J Chen, Melissa Zhao, Aaron K Chow, Kenji Ikemura, Ahrong Kim, Dimitra Pouli, Ankush Patel, et multimodal generative copilot for human pathology. Nature, 63446033):4466475 2024.
Mingyu Lu, Ethan Weinberger, Chanwoo Kim, and Su-In Lee. Cellclip-leaming perturbation effects in cell painting via text-guided contrastive learning. arXiv preprint 2rXiv:2506.06290, 2025.
John L Marshall, Neelesh Bangalore, Dorraya El-Ashry, Yair Fuxman, Michael Johnson, Brian Norris, Michael Oberst, Elizabeth Ness, Slawomir Wojtowicz-Praga, Pankaj Bhargava, et al. Phase study of prolonged infusion bryostatin-1 in patients. Cancer biology & therapy, 1(4):409-416, 2002.
Akira Miyajima, Fumiya Nishimura, Daigo Natsuhara, Yuka Kiba, Shunya Okamoto, Moeto Nagai, Tadashi Yamamuro, Masashi Kitamura, and Takayuki Shibata. Parallel dilution microfluidic device for enabling logarithmic concentration generation in molecular diagnostics. Lab on a Chip, 25(13):3222-3253 2025.
John Moffat, Fabien Vincent, Jonathan A Lee, Jörg Eder, and Mareo Prunotto. Opportunities and chal- lenges in phenotypic drug discovery: industry perspective. Nature reviews Drug discovery, 16(8): 531-543, 2017.
Zeinab Navidi, Jun Ma, Esteban A Miglietta, Le Liu, Anne E Carpenter, Beth A Cimini, Benjamin Haibe- Kains, and Bo Wang. Morphodiff: Cellular morphology painting with diffusion models. bioRxiv, 2024.
13

---

Published as a conference paper ICLR 2026
Floriane Odje, David Meijer, Elena Von Coburg, Justin JJ van der Hooft, Sebastian Dunst, Marnix Medema, and Andrea Volkamer. Unleashing the potential of cell painting assays for compound activ- ities and hazards prediction. Frontiers toxicology, 6: 6:1401036, 2024.
OpenAl. Gpt-5. OpenAl Blog, 2025. Available at: https:// com [Accessed: September 2025].
Alessandro Palma, Fabian J Theis, and Mohammad Lotfollahi. Predicting cell morphological responses to perturbations using generative modeling. Nature Communications, 16(1):505, 2025.
Ana Sanchez-Fermander Elisabeth Rumetshofer, Sepp Hochreiter, and Günter Klambauer. Cloome: con- trastive learning unlocks bioimaging databases for queries with chemical structures. Nature Communica- fions, 14(1):3339, 2023.
Srijit Seal, Maria-Anna Trapotsi, Ola Spjuth, Shantanu Singh, Jordi Carreras-Puigverr Nigel Greene, An- dreas Bender, and Anne Carpenter. A decade in a systematic review: The evolution and impact of cell painting. bioRviv, 2024.
Rohit Singh, Samuel Sledzieski, Bryan Bryson, Lenore Cowen, and Bonnie Berger. Contrastive learning in protein language space predicts interactions between drugs and protein targets. Proceedings of the National Academy of Sciences, 120124):c2220778120, 2023.
Dickson MD Siu, Kelvin CM Lee, Bob MF Chung, Justin SJ Wong, Guoan Zheng, and Kevin K Tsia. Optofluidic imaging meets deep learning: from merging emerging. Lab Chip, 23(5):0011-005 1-0033 2023.
Houcheng Su, Weicai Long, and Yanlin Zhang. Biomaster: Multi aggntt system for automated bioinformatics analysis workflow. bioRxiv, pp. 2025 01, 2025.
Guangyan Tian, Philip Harrison, Akshai Sreenivasan, Jordi Carreras- Puigver,, and Ola Spjuth. Combin- ing molecular and cell painting image data for mechanism of action prediction. Artificial Intelligence in the Life Sciences, 3: 1000000 2023.
Maria-Anna Trapotsi, Elizabeth Mouchet, Guy Williams, Tiziana Monteverde, Karolina Juhani, Riku Turkki, Filip Miljkovic, Anton Martinsson, Lewis Mervin, Kenneth R Pryde, et al. Cell morpholog- ical profiling enables high-throughput screening for proteolysis targeting chimera (protac) phenotypic signature. ACS Chemical Biology, 17(7):1733-1744 2022.
Fabien Vincent, Paula M Loria, Andrea D Weston, Claire M Steppan, Regis Doyonnas, Yue-Ming Wang, Kristin L Rockwell, and Marie-Claire Peakman. Hit triage and validation in phenotypic screening: con- siderations and strategies. Cell Chemical Biology, 27(1 133-11:333--44, 2020.
Hanchen Wang, Yichun He, Paula P Coelho, Matthew Bucci, Abbas Nazir, Bob Chen, Linh Trinh, Serena Zhang, Kexin Huang. Vineethkrishna Chandrasekar, et al. Jure leskovec, and aviv regev. Metric mirages in cell embeddings. bioRxiv, pp. 2024-04, 2024.
Asim Waqas, Asma Khan, Zarifa Gahramanli Ozturk, Daryoush Saced-Vates Vafa, Weishen Chen, Jasreman Dhillon, Andrey Bychkov, Marilyn M Bui, Ehsan Ullah, Farah Khalil, et al. Reasoning beyond accu- racy: Expert evaluation of large language models in diagnostic pathology. medRxiv, 2025.
Stephanie Watkins, Joshua C Toliver, Nina Kim, Sarah Whitmire, and W Timothy Garvey. Economic out- comes of antiobesity medication use among adults in the united states: retrospective cohort study. Jour- nal of managed care & specialty pharmacy, 28 28(10):1066- 1066-19999 2022.
14

---

Published as conference paper ICLR 2026
Robin Winter, Floriane Montanari, Frank Noé, and Djork-Amé Clevert. Learning continuous and data- driven molecular descriptors by translating equivalent chemical representations. Chemical science, 10(6): 1692-1701, 2019.
Ting Wu, Peilin Zhan, Wei Chen, Miaoqing Lin, Quanyuan Qiu, Yinan Hu, Jiuhang Song, and Xiaoqing Lin. Chemberta embeddings and ensemble learning for prediction of density and melting point of deep eutectic solvents with hybrid features. Computers & Chemical Engineering. 196: 196:109063. 2025.
XAI. Grok 4. xAI Documentation, 2025. Available at: https://x: ai
Accessed:: September 2025].
Gelei Xu, Xueyang Li, Yixiong Chen, Yuying Duan, Shuqing Wu, Alexander Yu, Ching-Hao Chiu, Juntong Ni, Ningzhi Tang, Toby Jia-Jun Li, et al. A comprehensive survey of agentic ai in healthcare. 2025.
Yuxin Yang, Abby Jerger, Song Feng, Zixu Wang, Christina Brasfield, Margaret S Cheung, Jeremy Zucker, and Qiang Guan. Improved enzyme functional annotation prediction using contrastive learning with structural inference. Communications Biology, 7(1):1690, 2024.
Li Yiyao, Nirvi Vakharia, Weixin Liang, Aaron T Mayer, Ruibang Luo, Alexandro E Trevino, and Zhenqin Wu. Omiesnavigator: an llm-driven multi-agent system for autonomous zero-shot biological analysis in spatial omics. bioRxiv, pp. 2025-07 07, 2025.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 11975- 11986, 2023.
Duzhen Zhang, Yahan Yu, Jiahua Dong, Chenxing Li, Dan Su, Chenhui Chu, and Dong Yu. Mm-lims: Recent advances in multimodal large language models. arXiv preprint arXiv:2401.13601 2024a.
Shanghang Zhang, Gaole Dai, Tiejun Huang, and Jianxu Chen. Multimodal large language models for bioimage analysis. nature methods, 218:113390 1390,,113330000 2024b.
Ziqi Zhang, Kelvin CM Lee, Dickson MD Siu, Michelle CK Lo, Queenie TK Lai, Edmund Y Lam, and Kevin K Tsia. Morphological profiling by high-throughput single-cell biophysical fractometry. Commu- nications biology, 6(1) 6(1):449, 2023.
Matylda A Zietek, Akshar Lohith, Derfel Terciano, Beverley M Rabbitts, Aswad Khadilkar, John B MacMil- lan, and R Scott Lokey. Cell painting in activated cells illuminates phenotypic dark space and uncovers novel drug mechanisms of action. bioRxiv, 2025.
A USE OF LARGE LANGUAGE MODELS (LLMs)
We used large language models (e.g., GPT-4) for non-substantive assistance during manuscript preparation. Specifically, LLMs were used to improve writing clarity, grammar, and phrasing, but not for generating scientific content experimental design. All technical contributions, experiments, and interpretations were conceived and conducted by the authors.
The authors take full responsibility for the content of the manuseript, including any text generatedor pol- ished by the LLM. We have ensured that the [L.M-generated text adheres to ethical guidelinesand does not contribute plagiarism scientific misconduct.
15

---

Published as a conference paper ICLR 2026
B PRELIMINARIES AND BACKGROUND
B.1 HIGH-CONTENT IMAGING
High-content imaging (HCI) leverages automated microscopy and quantitative morphology to profile com- pound effects. Cell Painting stains multiple cellular components and extracts hundreds of single-cell fea- tures, producing high-dimensional representations that enable poss-ppeturbaaion comparisons, including compound clustering, target and pathway inference, and prediction of unannotated mechanisms (Bray al. 2016
Odje et al. 2024).
B.2 MULTIDIMENSTONAL EXPERIMENTAL. DESIGN IN CELL PAINTING ASSAYS
Drug screening with Cell Painting involves diverse experimental factors that strongly shape cell morphol- ogy. (Overview of high-content imaging (HCT) can be referred to Appendix B.I). Key sources of variability
include the cell line (Lejal al. 2025 ) culture medium Harkness et al. (209)) incubation environment,
and drug administration, each capable of inducing substantial morphological shifts. Drug libraries typically contain hundreds of thousands of molecules (Huggins et al. 2011 Liu et al. 2025a), with concentrations sampled using hag--logaii dilution series to capture rese-response characteristics across orders of mag- nitude (Choy al. 202T Miyajima al., 2025). Meanwhile, temporal variables staging further increase complexity, as different observation time points can capture different call phases of treatment response, revealing both immediate and progressive morphological changes (Beesabathuni et al.) [025 Lejal al.)
2025). The interplay of experimental variables defines high-dimensional space which condition combina- tions yield diverse morphological phenotypes.
B.3 MLLM AGENTS FOR BIOINFORMATICS
Large language models (LLMs) demonstrating growing potential across diverse domains of bioinformat- ics, with applications ranging from gene expression analysis (Liu et al. (024a) and drug discovery (Averly (et al. 2025) to pathology image interpretation (Lu et al. 2024), spatial transcriptomics (Wang al. 2024 and gene perturbation studies. Because datasets in these fields are often high- digh-dimensional, recent efforts have increasingly turned multimodal large language models (MLLMs), which integrate visual features from images with prior textual knowledge. Leveraging logical inference strategies such as deduction, induc- tion, abduction, and analogy, MILMMs can support existing pipelines and facilitate novel scientific insights.
More recently, emerging paradigm has focused on deploying MILMs as autonomous or semi- autonomous agents to execute complex bioinformatics workflows (Yiyao et al., 2025| Su et. 2025). Such agents integrate heterogeneous tools and interact through natural language, enabling biological data anal- ysis guided by human instructions. While early studies highlight the promise MLLM-driven agents augmenting traditional pipelines, their scope has largely been limited to direct perception and recognition tasks. They remain insufficient for deeper understanding of complex biological processes and for generating novel hypotheses. Addressing this gap, we introduce CL-CLIP, multi-agent system that extends beyond the visual capacities of current state-of-the-art MIL.Ms to capture subtle pharmacological features, provide interpretable analysis, and facilitate hypothesis generation in pharmacological research.
B.4 CONTRASTIVE LEARNING
Contrastive learning is self- supervisdd paradigm that learns representations by pulling semantically related pairs closer and pushing unrelated pairs apart in shared embedding space (Hu et al.] (022a)). In biol- ogy, contrastive learning has underpinned several applications, such single-cell multi-omies integration (scRNA-seq and SeATAC-seq) (Liu et al.] (0255), protein function prediction for classify enzyme activ- ities (Yang et al. 2024). drug-target interaction prediction through protein-compoume embedding (Singh
16

---

Published as conference paper ICLR 2026
et al. 2023). CLIP exemplifies the dual-encoder contrastive paradigm for multi-modal learning, trains an image encoder and a text encoder so that matched image-text pairs have high cosine similarity while mismatched pairs are pushed apart. By scaling to large, CLIP can produce transferable embeddings that generalize across tasks.
C DATASET BACKGROUNDS
BBBC021 profiles MCF-7 cells treated with 38 reference drugs covering 12 mechanisms of action, imaged across up to eight half-log doses and three channels (DNA, 3-tubblin, actin) (Caie et al. 2010). CPJUMP1 includes 301 small molecules (46 controls) perturbed in U2OS and A549 cells, imaged in five channels (DNA; mitochondria; actin/Golgi/plasma membrane; nucleoli and cytoplasmic RNA; endoplasmic reticu- lum) (Chandrasckaran et al. 2024). RxRx3 assays HUVECs with 1,674 bioactive compounds across eight concentrations and six fluorescence channels to capture dose-response phenotypes (Fay et al. 2023)
D DETAILED RDKIT2D FEATURE OVERVIEW
Table 4: Categorized RDKit2D Descriptors Used in This Study (174 descriptors)
Feature Category
Descriptors
Topological and Complexity De- scriptors Basic Physicochemical Properties
BalabanJ, BertzCT, Chio, ChiOn, ChiOv, Chil, Chiln, Chily, Chi2n, Chi2v, Chi3n, Chi3v, Chi4n, Chi4v, Ipc, Kappal, Kappa2, Kappa3
MolWt, ExactMolWt, HeavyAtomMolWt, MolLogP. MolMR.
LabuteASA, TPSA
Atom and Bond Counts
VumnValencEllectronns Bundotatableeonds NumHAcceptors. NumHDonors, NHOHCount, NOCount,
HeavyAtomoonn
NumHeteroatoms, FractionCSP3
Ring Structure Descriptors
RingCount, NumAromatticings, NumSaturatcdRings, NumAliphati-
cRings, Num.AromaticCarbooyyll.. HumnAromatccHetercyyylle. NumSat- urated(Cabboyylls. NumsSaturatedHHeeeeocyyyy.. NumAliphaticCarboccy
cles, NumaAliphaiicHoeeroyydss
Electrotopological State (EState)
MaxEStateIndex, MinEStateIndex, MaxAbssSStaclndex, MinAbsES-
Descriptors
tateIndex
VSA (Van der Waals Surface Area) Descriptors
EState VSAI-11, VSAI-14, SMR.VSAI-IE SlogP.VSAI-D. ESA.EState1-10
Fingerprint Density Descriptors
FpDensityMorgan,, Morgann,,
FpDensity hanaraaoanngnn.nnDDnnnnnnggaaaa
Fragment- Based Functional Group Descriptors
ALLCOO fr.O..H. fr.AI.H.noTerr. fr_ANN, fr.Ar.COO, fr.Ar.N fr_Ar_NH, fr_Ar_OH, fr_COO, fr_COO2, fr_C.O, fr.C.O,noCOO, HOCCN, fr Imine, fr NH0, fr NHI, fr NH2, fr Ndealkylationi, fr.Ndellkylation2, fr Nhpyrrole, fr. frdeheeeed, laaaaay.. hhalde,, fralllyiccoxidd fr_amide, fr_amidine, fr.aniline, fr.aryl azo, fr benzene, fr. bicyclcli,, fr. diyydropyrrddine, epoxide, ester, fr ether, ft furan, fr halogen. fr fr_imidazoe,, imide, fr ketone, fr ketone. T fr lactone fr_methoxy, fr fr.oorphlinne, fr fr.nitrile, fr.nitro, fr_nitro_ane arom. fr fr.nitro_arow catrrorrrommooooo nonootho,, fr_para_hydree gpara_hydroxylatoon fr_phenol, fr phos ester, fr. piperdine, fr piperzine, fr priamide, fr pyridine, fr sulfide, sulfonand, fr sulfone, thiazole. fr thiophene, unbrch alkane, fr_urea ged
hdrzine.
fr fr_phos_acis _ccid,
Drug-Likeness. Score
17

---

Published as conference paper ICLR 2026
E LOG-DOSE INDEXING FOR SERIAL DILUTION
To represent compound concentrations on consistent and model-friendly scale, we transform raw con- centrations into log-scaled step values. This transformation is based on the assumption that concentrations follow a serial dilution protocol in logarithmic space.
Let Caax E 2o denote the nominal maximum concentration for compound, and let C E R any intermediate concentration point. In standard protocol with logarithmic dilution spacing, each dose is reduced by a fixed factor per step. This can be expressed as:
C C 10 log
h: == 0,0,,,
(5)
where A log 0 is the logarithmic step size (in base 10. For example, Alog 0.5 corresponds to a 3.16-fold dilution between adjacent doses, since 10-0.5 as 0.3162.
To recover the step index s(C)) corresponding to any concentration C, we invert the above relation:
C = maxx 10 ((C) AA log
log10 (C) log10 (Cmax) - s(C) A log
(6)
sCC) log10 (Cmax) 1og (C')
A log
Thus, the log-scaled step transformation is defined as:
(C) log10 (Cmax) log (C)
A log 0.5
(7)
A Alog
This representation maps concentrations to a normalized step index in log space, which is more suitable for modeling, especially in contexts where peneenttratio-responss relationships are approximately log-linear.
F CONTEXT-AWARE TOKEN PROJECTION MODULES
Algorithm 1 CP-CLIP: Context-Aware Token Projection Modules
1: function ENCODEIMAGE(ZiN)
2: fimg < V(Zime)
3: return normalize([fmn)
4: end function
5: function ENCODETEXT(o.. e)
6: X <- "ooeenFmeeddinggnn.
7: if <CONC> in गय then
8: X[<CONC>] + conc_mlp conc_mlp(c)
b C E R², conc.mlp 122 + 1Rd - IRd
end if
<TIME> in Fua then 10: if
X[<TIME>| + time_mil (mm_mlp(t)
b E R time mlp R! - Rddd IRd
12:
end if
13:
<CMPD> in Jia then X||<MMP>>] + compound.mlp(e end
14:
E E Rdow, compound.mlp Rd - Rd Rdd
15:
16:
X - PosEmb()))
17:
fua + T(X)
18:
return (ormalize(fua)
19:
end function
18

---

Published as conference paper ICLR 2026
G TRAINING LOSSES
We train the alignment with a symmetric CLIP-style contrastive objective. Specifically, we employ the InfoNCE loss which encourages matched image-text pairs to have high similarity while contrasting them against all other mismatched pairs in the batch:
N
LInfONCE
1
Σ k=1
(5/hiis : wb) + loE (stti) Mike
(8)
2.V
Here, F; = (f( ..... f. ITT E RNxd and F, = F image and text embeddings. The similarity matrices are computed as S s FFF E RNXN The ground-truth labels yı E (0, 1} indicate the correct matching pair for each sample in the batch. fcu(+,-) denotes the standard cross-entropy between the similarity scores and the target labels.
N are the batch of normalized
our experiments, we additionally compare InfoNCE loss with an alternative loss recently proposed in SigLIP. which simplifies the contrastive objective by directly operate joint embeddings in a shared represen- tation space.
N
Sigh
log
(9)
N
Here, is learnable temperature parameter. To isolate the effect of the loss function from the model architecture, we apply both loss types within our CP-CLIP framework for fair comparison.
H CELLPROFILER PIPELINE
For all DNA channels, we extracted per-cell features using the workflow described in Table This pipeline specifically optimized for nuclear segmentation and feature extraction, using modules that mea- sure grayscale features like shape, texture, and granularity. These features are particularly suitable for DNA stains. For all non-DNA channels (such as Actin, Tubulin, etc.), we applied consistent pipeline template described in 6 This workflow is tailored to cytoplasmic or filamentous structures, which differ in spatial organization and image characteristics compared to nuclei.
Some feature modules differ between the two workflows, particularly in how certain parameters are config- ured. For example, texture features were computed at different spatial scales: for DNA, we used smaller scales (e.g., 3, 5, to capture fine-grainee nuclear texture, while for non-DNA channels, larger scales (c.g., 5, 10, 15) were used to capture broader cytoskeletal patterns. Similarly, granularity features and shape de- scriptors such Zernike moments each channel. These differences pipeline configuration ensure that the measurements are biologically meaningful and adapted to the unique characteristics of each fluorescence channel.
customized reflect the typical size and morphology of structures
19

---

Published as conference paper ICLR 2026
Table 5: CellProfiler pipeline modules and measured features for DNA channel.
Module
Key Settings Notes
Measured Features
Images
Load images; filter by:
isinage, exclude folders with regex
2. Metadata
Extract metadata from filename and folder using regex patterns
Plate, Well, Sitc, ChannelNumbor, Date
3. AamesAndTyppe
Assign names: DNA
Image names: DNA, mask; Object names: nuclei, Nucleus
(grayscale), nuclei mask (objects); match rules: file contains "DNA", file cont ains "nuclei"
4. Groups
Grouping disabled
5. MessurelmaggArrr@ocuuddd
Measure area of nuclei objects
AreaOccupied.mucle:
6. MensureOOhjetLNegghhorss
Measure neighbors of nuclei within 10 pixels
Neighhbors.10px.count.
7. MeasureeOheetteigghbrrs
Measure neighbors nuclei within 50 pixels
Ne Wciighborra.0pxx.coumr Ne Weiigbborr..50p22Perreettttthct
8. MeasureoOheetSizzShape
Measure nuclei; include Zemiike moments and advanced features
Shape: Area. Peraa.errittter. FormFactor, Zeniikkk Zemnik::
Zernike.0.0 to Zernike.9.9
9. Texture
Texture of DNA in nuclei;
Texture features per scale: Contrast, Entropy, Correla on etc.
scales: 3, 5, 7; levels: 256; both image and object
mode:
10. McasureGramularity
Granularity of DNA in nuclei; radius spectrum range =
4
11. ExportToSpreadsshet
Export all features with metadata; output file: DATA. cav with prefix Expt_
All per-object and per-image features above, including per-image
mcan/mncian/std
20

---

Published as conference paper ICLR 2026
Table 6: CellProfiler pipeline modules and measured features for Actin channel.
Module
Key Settings Notes
Mcasured Features
Images
Load images; filter by: i mage, exclude folders with regex
2. Metadata
Extract metadata from filename and folder using regex patterns
Plate, Wcll, Sitc, MaanneiNumber Date
3. Tymess
Assign names: Act (grayscale), cel mask (objects);
Image names: INNA, mask Object names: nuclei, Nucleus
Match rules: file contains "Actin", file contains "cell"
4. Groups
Grouping disabled
5. Veasure/mageArraaccuupe
Measure area of cell objects Measure neighbors of cell within 10 pixels
AreaOccupied.com
6. MessureOhjecNeiighbory
Ne Neighbors 10px Count, Neighbors. Percentiouching
7. MessureOhjecLNiighhoor
Measure neighbors of cell within 50 pixels
Neighbors.50px. Percent:louching
8. MessureOOheetSizzShaue
Measure cell include Zernike moments and advanced features
Shape: Area, Area, Perimeter, Solidity, FormFactor, MaxFeretDiametee: EquivalentDDameree etc. Zemike: Zerni ke_0_0 to Vernike 9
MexsureTexture
Texture Act in cell; scales: 5, 7; levels: 256
Texture features per scale: Contrast, Correlation intion, Entropy, sopEn DifferenceEntzapy. InfoMeass.. InfoMeas2 Granularity 1--4 Act Actin.ineelll 11. ExportToSpreadsheet
10. MeasureGranularity
Granularity of Act in in cell: radius 8, spectrum range = 4
Export all features with metadata; output file: DATA. csv
All per-object and per-image features above, including per-image mean/median/std
I SIMILARITY PERFORMANCE ON SEEN DRUG COMPOUNDS
Table 7: Similarity Performance Seen Drug Compounds
Model
Fladskklterr
Resecadorii
AZM475271
Misogrostol
Trazodone
Orantini
Rullnanide
Jumincosib
BIRB-796
Methoxsules
CLIP WITBB115
0.406
0.417
0.427
0.438
Siyl
0.323 muro
0.320=0m0
0.522
0.222 mone
0.211aame
0.2807 anows
CP CLIP SigL
(deacnpoor
1/16
0,588 0.065
0,530 0357
0.456 so00
0.501 1000
0.465_0.0
0.545 10945
0,452 co0
0.445 s.oe
0.479_.xx
0.625 Lost:
(foperpiit)
VIT-R116
0.0022.0000
0,308 0.336
0.310 s.uu
0.309-0003
0.
0,002 1.940
0.499 0,00
0.315_amm
0.51 cost:
CPACIP VIT-R016
(denviptor
0.510a 0.57
0.505:2007
0 5.504
0.511 m.0
0.107
0.525 2.mm
20
(deavijpt))
0.620 sunza
c.mm
0.503 musss
0.620 FILER
0.508 cvoo
0.100i
0.51320me
0.500-2mm4
21

---

Published as conference paper ICLR 2026
Table 8: Seen drugs similarity averaged score
CP-CLIP SigLIP-VI--BI1 (descriptor)
CP-CLIP VIT-B/16 (fngerpiint)
CP-CLIP VIT-B/16
CP-CLIP VIT-L/16
CLIP VIT-B/16
SigLIP ViT-B/16
(descriptor)
(descriptor)
0.467
0.269
0.496
0.552
0,549
0.561
J VISTA-2D FINE-TUNE
The original VISTA2D model does not consistently achieve accurate segmentation across all fluorescent channels, especially when applied to diverse cell painting datasets. To address this limitation, we fine-tuned the segmentation model using the Cell Painting dataset. Figures below illustrate representative instance segmentation results across different channels and datasets (BBBC021, RxRx1, and CPJUMP, respectively), demonstrating improved mask quality and channel-specific accuracy. Three standard instance segmentation metries are used evaluate the fine-tuned model's instance mask quality on 500 test data, with improvements shown in Table []:
Intersection over Union (IoU) The IoU evaluates the overlap between a predicted instance / and ground truth instance label T:
[P PNT
I) = PPUT
(10)
Where [| is number of pixels in the intersection of P and T.
Aggregated Jaccard Index (AJI): The AJI generalizes IoU to entire image containing multiple in- stances. It is the ratio of the total number of overlapping pixels between matched ground truth and prediction pairs, the total number of pixels in their union plus the pixels in all unmatched predicted instances, and can be formulated as:
Σ
T
AJI((,T)) E T.P + Ejev PP5l
(11)
Where æ(i) the index mapping that assigns predicted instances align with ground truth instances. U is the set of unmatched predicted instances.
Panoptic Quality (PQ): PQ is metric that jointly evaluates segmentation quality and recognition quality in instance segmentation. It reflects both how accurately the matched segments overlap (IoU) and how well all instances are detected (accounting for false positives and false negatives). PQ rewards correct segmenta- tions while penalizing missing spurious predictions. PQ can be formulated as:
[M
PQ(P.T)
LoU(p, t)
(12)
M
P Punmatched
Tunmarched
(M.
Detection Quality (DQ)
Segmentation Quality (SQ)
Where M is the number of ground truth pairs, P Punmatched is unmatched predicted instances (False Positives). Tunmatched unmatched ground truth instances (False Negatives).
Table 9: Instance Mask Evaluation Metrics VISTA-2d
IoU
AJII
PQ
before fine tune
0.272
0.290
0.151
after fine tune
0.824
0.791
0.682
22

---

Published as conference paper ICLR 2026
Input Image
VISTA
Fine-tuned VISTA
Ground Truth
DNA
Mito
CPJUMP
ER
RNA
AGP
Figure 5: Segmentation performance comparison on CP-JUMP dataset across different imaging channels.
23

---

Published as conference paper ICLR 2026
Input Image
VISTA
Fine-tuned VISTA
Ground Truth
DNA
Mito
RNA
RXRX3
Actin
AGP
Figure 6: Segmentation performance comparison on RXRX3 dataset across different imaging channels.
24

---

Published as conference paper ICLR 2026
Input Image
VISTA
Fine-tuned VISTA
Ground Truth
DNA
BBBC021
Actin
Tubulin
Figure 7: Segmentation performance comparison on BBBC021 dataset across different imaging channels.
K DOSE RESPONSE EXAMPLES
To further illustrate the diversity of fose-response behaviors captured by CP-CLIP embeddings, Figure K shows additional examples from two datasets: BBBCO21 and RxRx3 since only the two datasets designed dose scheme based experiments. For each compound, we compute the cosine distance between image em- beddings at different concentration levels, focusing on perturbation effects within individual imaging chan- nels.
The x-axis denotes concentration step pairs relative to the first experimental dose. Because different datasets use either fixed or variable half-log concentration series, we normalize the comparisons by indexing each dose level (e.g., 1 for the lowest concentration, 8 for the highest). A label such as "1-2" indicates the cosine distance between embeddings concentration step and step 2. For example, the lowest concentration 0.0001 μM and half-log step is used, then: step is 0.0001 μM, step 2 is 0.000316 μM, step8 is 0 μM. The cosine distance computed between embeddings 2, and 2j at two different doses and j. where
d, = 2, Zj
(13)
| ll ||z
The y-axis reflects this cosine distance, providing a quantitative measure of morphological difference be- tween concentrations. A rising trend along the x-axis indicates increasing morphological divergence from the baseline as concentration increases, which indicating a hallmark dose-dependent phenotype. Sharp trajectories observed for drugs such Alsterpaullone, Camptothecin, Cisplatin, Emetine, Mitox-
25

---

Published as conference paper ICLR 2026
antrone, Acetophenazine, Buclizine, and Thiothixene, which are also consistent with their known mecha- nisms. In contrast, compounds such Eszopiclone and Methsuximide produce more stable embeddings across doses, suggesting limited morphological response. These visualizations provide additional support for the claim that CP-CLIP embeddings can sensitively capture dose-dependent morphological variation across diverse chemical perturbations.
Alsterpaulione
Camptothecin
Cisplatin
3.7
Coaino Dabino
Oodine Disannce
02
0.5
0:
BBBC021
1-1
1-3
13
Commentriiinn stop par
Compentiiiion stap par
Conmenmition step pai
Lactacystin
Emetine
Mitoxantrone
0.7
06
08
0.
5
Cooere
08
0.0
3.
02
0.1 14
1-1
1/2
1-3
1-5
166
1.7
1.1 14
16 1-8
16 1-8 1-1
1-2
1.3
Concentration
Acetophenazine
Buclizine
Thiothixene
Dipance
0.5
0.3
84
04
3.2
82
RXRX3
0.1
011
Concentraton dee par
Concentration step par
Doncentraion step par
Eszopiclone
0.0 Methsuximide
Garenoxacin
Costre Deteroo
69
244
04
04
14
1-2 1-3. 14, 1-7 14
14 52 1-3.
148 1-7
14 14
Figure 8: Dose-response consistency across compounds in BBBCO21 and RxRx3 datasets, measured by cosine distance between CP-CLIP embeddings at different concentration step pairs.
26

---

Published as conference paper ICLR 2026
L STATISTICAL EVIDENCE SYNTHESIZER EQUATIONS
Table 10: Summary of statistical parameters for image
Parameter Name
Expression Variable Description
n.control
la]
Number of cells from the control group
n perturb bb
b: Number of cells from the perturbation group
Table 11: Summary of statistical parameters for each feature metric and their definitions
Parameter Name
Expression
Variable Description
median_contren
median(e)
median: Median of
median.perturb
median 6)
median: Median of
mad.control
median la median (a)
MAD: Median absolute deviation of a
mad.perturb
median (b median(b)) (b)
MAD: Median absolute deviation of b
p10.control
Qa(0.10)
Qa(p): p-th quantile of control group
p25_control
(0.25)
Same as above
p50.control
(a(0.5))
Same above
p75.control
(0.7.))
Same as above
p90.control
Qa(0.90)
Same as above
(v(0.10)
(p): p-th quantile of perturbation group
p25.perturb
Qs(0.25)
Same as above
p50.perturb
(0.5.)))
Same as above
p75.perturb
Q.(0.75)
Same as above
p90.perturb
(0.9.))
Same as above
delta_median an
median (b) median(a)
Difference medians between groups
bootstrap.ei.Jower
Cljow
Lower bound of bootstrap confidence interval Upper bound of bootstrap confidence interval d: Cliff's delta effect size
bootstrap ci upper cliffs delta p.value
Clup
d
p
p: Statistical significance from hypothesis test
The lower and upper bounds of the bootstrap confidence interval, denoted as Claw and Clup. estimate the confidence interval of the median difference between control and perturbed sample using the bootstrap resampling method. Specifically, 1000 rounds of bootstrap sampling are performed. It can be computed as:
Ch Percentile 2.5 ((57))
(14)
Clup Percentile 97.5 ({8)))
(15)
Let o denote the median difference obtained in the i-th round of bootstrap resampling, the collection (6t) represents the set of median differences obtained from N rounds of bootstrap resampling.
Cliff's delta a nonparametric effect size that quantifies the magnitude of difference between two distribu- tions. It is computed as:
[ 4z) (x: <1)]
(16)
a
27

---

Published as a conference paper ICLR 2026
Where I; denotes the i-th sample from the control group, and yj denotes the j-th sample perturbation (or treatment) group. The indicator function I(-) returns 1 if the condition inside the brackets is true, and 0 otherwise. Cliff's delta, which quantifies the degree of difference between the two groups. Its value ranges from -1 to 1, where d = 0 indicates no difference, d = 1 indicates the control group has much bigger value.
The p-value corresponds to the result of two-sided Mann-Whitney U test. It helps assess whether the observed difference could be explained by random variation, under the assumption that the null hypothesis is true. The p-value is computed as:
U
p p= 2- db
(17)
(+h.
Where U is the Mann-Whitney U statistic, and Yha, D6, are the sample sizes of the two groups being compared. The term ((-) denotes the cumulative distribution function (CDF) of the standard normal distribution. The numerator measures the deviation of the observed U value from its expected value under the null hypothesis. This standardization transforms the U statistic into a z-score, which is then used to compute the two-tailed p-value. A small p-value indicates that the observed difference in distributions is unlikely to have occurred by chance.
M MLLMS BASELINE DETAILS
M. 1
METHODS
To evaluate the reasoning capability of current mainstream MILMs on the Cell Painting dataset, we test four APl-accessible models: Grok-4, GPT-5, Claude-4-Sonee, and Gemini-2.5-Pro. The experimental workflow consists two stages. First, each MLLM performs background knowledge curation as a single preliminary task. The curated information is then used as context for zero-shot VQA across three tasks: the cell line task, the channel task, and the perturbation compound task. During background knowledge curation, the decoding parameters set to temperature 0.7 and top-p = 0.95, whereas for VQA they set to temperature I and top-p 1 to ensure response stability. All MLLMs are prompted with the same structured instructions specifying the evaluation criteria. In the VQA stage, the models receive both control and perturbation images together with masked textual descriptions. Their task is to select the correct answer from multiple-choie options that include the ground-truth label and provide both a confidence estimate and concise rationale. An example prompt is shown below.
In addition to the zero-shot setting described above, we further evaluate few-shot variant of the same protocol to make the comparison with CP-Agent more conservative. For each of the three tasks (cell line, channel, and perturbation compound), we construct a small visual memory bank consisting of two labeled exemplar image pairs per class (control + perturbation). These exemplars are selected from the training split and are fixed across all MILLMs to ensure comparability. In the few-shot condition, the VQA prompt augmented with these exemplars: before answering query, the model is shown the memory bank with the corresponding class labels and is instructed to use these examples as visual references when reasoning about the new control-perturbation pair.
The overall prompting structure, background knowledge curation stage, and decoding parameters remain identical to the zero-shot setup. The only difference is the inclusion of the exemplar memory bank in the VQA stage. As reported in Table [2] few-shot prompting yields modest improvements on the cell line and channel tasks, indicating that current MILMs can benefit from limited visual grounding. However, performance on the perturbation compound task remains very low, and the models do not exhibit reliable compound-level discrimination despite changes in the prediction distribution. This suggests that the subtle
28

---

Published as conference paper ICLR 2026
morphological signatures induced by chemical perturbations are difficult for general- purrposep MLLMs acquire from small number of Cell Painting exemplars.
M.2 PROMPTS
Task: Cell Line
Background Information Curation
You knwwindgabble biological research assislani spucializing Cell Painting-baen phenotypic profilig.
Goal:
background
analyse Painting experiments
perturistion images Painting Gallery
Scope:
Candidate lines: (A549, MCF1, HUVEC).
Available imsging chrenels (subect apperr per somple): (DNA, RNA, ER, Mito, Actin, AGP).
Yeur responsibilries:
each candidate
provide concisc
including:
Camonieal morphology (oell sbape/size, ofbreionnsprroding colony patterss, prolisecation tendencies).
Nacclear features (heteregencity, muclooli prominencs) cytoplasmic
floorescence microncopy.
Chrenc)-oncored
Ceil Paiming (what typically observable DNA AAAAANAAAA/iiiiiii Tubalin ER: Mito note chumnel not infommative), Robust coes send to persist across many perturbations vs coes that are sensitive dose, time, confluency, or imaging settings. morphology.
Typical culture conditions (medis, supplements) that may
Cruss-lin line comparisen:
Key discriminative
help distinguish
macenetlly).
A compact "Coee Lino-speciffcc" visual oeservation checklist that standardizes what look for across samplex.
Cenfounders and limitations:
Common tochnical biological confounders (plate fatch Illumination, magnification, confluency/oveerrooohh desictime, availablliny. Fields that may rrivially identify the line (e.g.. specific media namesi; mark these as "identity revealing" variables.
weinht ignore cues required Output requicements:
missing.
Be accurate, concise, and avoid redundancy or speculation, information is uncertain, state "unknown".
Provide parts: structured and (B) machine-reulable JSON black
Task: Cell Line
Task: Cell Line
System Instruction for VQA
User Input Template for VQA
Painting imaging expert deep knswllddge
Predict the mooked cell line.
EXPERIMENT DESCRIPTION:
Yeu receive:
[1) two microscopy images: Image control, Image B
BACKGROUND KNOWLEDGE JSON:
pesturbation sume experiment,
(ackgroudd)
(2) experiment description with one attribute masked (the cell line),
CANDIDATE CRLL LINES:
(3) structured background knowledge ISON describing candidase their canonical morphology, channel-speeifie and discriminative features.
A549, MCF7, U20S, HUVEC
ATTACHED IMAGES:
Image A: Control
Image
Your task:
AUTPUT_ISOX _ SCHEMA:
infer most likely sell lime for the musked attribure by comparing Image and Image
background knogeedge, Be conserative; if evidencu werk or ambiguous, distribute probability mass across candidates rather guessing Fukeesniidnne.
"eell_line prediction"..
"pred": "oooe o[[A549, MCF7, U20S, U20S, HUVEC
E( Boot, "HUVEC": (loat)).
"probs":
Return only ISON response matching the required schemna.
"vonfidence": float, !f equuls probs(pred]
"tatinale "<describe differencee and why they match the predicted line"
costrol
Now answer in ISON only.
29

---

Published as conference paper ICLR 2026
Task: Channel
Background Information Curation
You are knowkdggebbe biological research assistami specializing Paainiing phonotypic profiling.
Goal:
Curate buckground knowloige that helps analyze Cell. Painting experiments with cuntrol and perturaatin images from (Cell Painking Gallery).
Scope:
Candidatte channek: (DNA, RNA, ER, Mito, Actin, AGP)
Available
may appear per sample):
ER, Mito, Actin, AGP).
Your respassibiltiee::
Clocnel dussicrs (for each choune (candidate, cluonsls)):
What the lobels biologically (trocturr(ppocess)) and expected subeelluiar localization.
Canonical appearace fluoreseence images: texturetopology (e suclea-dominnet, nuelcoli visiility, filamentous netwos ks; par perimeller reticulum; punctate tubulcc organciles; membrano/ Golgi pattecns).
Distinctive eses vs. look- look-alikes (how to tell this channel apart from virally similar ones and why confasiin cecurs).
Robust vs. scusitive caes: which patterns persist neroes ceil types/pottuettioos and which chrnge doseltime, confluency, or imaging settings
Qsalityllmtefact considerations: seturaiion, bleed-throoee throngh, nen-mniform illumination, focus blur, recommended processing background nomalization, flat-fildiig, gentle control cohmnennent).
Cross nous-channel comparisom:
A concise table discriminzzive feotures (c.g- "moleea dominant", "illamentous synoskeleton"; "perimuleea reticulum", "mitochondrial panctaae network", "cortical scinn or strees libers", "golgi perinuclear creseen membeine outline") with chaene''s is tynica) strength (0-1).
Common confasion pairs RNA DNA in mucleoli; Actin Tubulin filaments; ER Mito the maclens) and how resolve them.
Confounders & identity leakage:
Technical confounders: batchiplate
illumination non-umi uniformity
mugnification, hit-depih, camera grin.
Biological confounders; confluency/overr growth, cell- cell-cycle stage, apoptosischeensis.
rovealinn metadata to avoid relying on (e.g. file names or emboided channel tags).
Scoring heuristics for downstream
Propose small set booolmiievieeeee checks (ace keys below) and lightweight decision nubric (ifithen rules or weights) to combine them into per-charnel likelihoods.
Include "Uninown'aniiguuuss fallback when evidence insufficient.
Oufpul requireeents:
Be accurate, avoid redundancy speculation; information uncertzin, state "unknown".
Provide two paris: (A) sbort, structured narralive, and (8) muchine-readable ISON block Gollowing the sebeema below.
Task: Cell Line
Task: Cell Line
System Instruction VQA
User Input Template for VQA
You biomedical imaging expert doup knowlolge Painting assays.
TASK:
Prodict the masked image channel.
EXPERIMENT, DESCRIPTION:
You will receive:
(maskod lexi)
131 two microscopy images: Image control, Image H
BACKGROUND KNOWLEDGE JISO;
parturistion of the experiment,
(onkground)
(2) za experiment description with one actribute masked (the image ciannel))
CANDIDTTE CHANNELS:
AGP, Actin, DNA, Mito, RNA,
(3) stractured background knowledgs ISON deseriiing visual fingerpeiats each Cell Painting channel (AGP, Actin, DNA,, BR.
ATTACHED IMAGES:
Image A: Control
Milo, RNA, Tubulin).
Image Pettrrbation
OUTPUT JSON SCHEMA:
task:
infer most likely image channel for the masked attribute by comparing Image A and Image with the background knowlsdge. calihrited: if evidence
"Insk: "image channel prediction",
"pred": "soone of [AGP, Actin, DNA, ER, Mito, RNA, Tubulin]st "DNA" float "ER" float, "Mite: flcat, "RNA": fleat, "Tubalin": fleat). "confidence": float.
conflicting, distribute probubility mass over candidates rather than overconfident guessing. only respanae matching required schema.
"probe": "AGG" flost,
"rationle": "skry intrinsic staining obsurveds*
30

---

Published as a conference paper ICLR 2026
Task: Perturbation Compound
Background Information Curation
You are knowkdggabbe biological research specializing Paainting phonotypic profiling.
Goal:
Curate buckground knowloige that helps analyze Call Painting experiments with cuntrol and periutaatin images from (Cell Painking Gallery).
Scope:
Candidatt compounds: (flidokaliner, racecadotill, arm47527), miiogrostol, trazodone, onntiini, rufinamide, lumirscoxib, 796, methoxsslen).
respasibálliie:
Compound dossters (for cach compound in [acctotiexamide, azm475271, esomepnazol,, flindokalner, letrozoli, misoprostol, nimodipine, orantinib, sacubitri, (sseodone)) }):
Mochanism sction (MoA; write "unknown" funclear) and pomary (with confidence: hightmecinumsloo)
Expected morphological phanotype when comparing control - perturiation, using sheenel-ggnoii language (e call sizespreed, counding/oottracion Elassentous bundled potteens, stress
nuclear biteoottrgeaaiityy micronucki, nucicoli prominenes, cytoplasmic gnaularity, vacnoles, minoti--ateet-likk pantrns, chznges in pogulation donsity).
Robust vs scusltive cues: which polterns petsist across typescondiiion:; which see sensitive dose, time, conffueccy imaging conditions.
Dose time typical effective concencraioo (uM,
allowed) window (hours). Common of--argetsscconde phenotype that may confound interiettaion..
confusions (compounds MoA)
dissmbiguate using image-only cues.
Cross compound compartson:
concise table discriminative (eatures 5. "meclesr dominant", ; "fllamentous cyteskeleten", "perinuclerr reticulum", "mitochenérial punciate network",
"cortical actin band
fibens", "golgi, perinuclear
membrane outline") with each charnel's rypical strength (0 l).
Common confasion pairs (egg, RNA va DNA in meclooli; Actin vs Tuhalin filamenis; ER Mitn near the macleus) and how to resalve them.
lealage:
Technical confoundes: butchiplate, illuminazion non-unilenn non-unifommity. facus bur, magnificction, saturation, bleedd through.
Biological confoundes: oon oefllunsy//verrrrwil scrum %2 cell-gycle stage, spoplocis
Identity revealing fields (drug synonyms, SMILES) should be decumented but flagged 'not be used as shorteurs", prediction image evidence.
Output requirements;
Be accurate, concise, and avoid redundancy speculation, information is uncertain, state "unknown"".
Provide two parts: (A) sbort, siractured nurraiive; and (B) muchine-reaabble ISON block following tbe schema below.
Task: Perturbation Compound
Task: Perturbation Compound
System Instruction VQA
User Input Template VQA
You biomedical imaging expert with knowledge of Cell Puinting assays.
TASK:
Predict the musked image compound.
EXPERIMENT EXPERIMENT.DESCRIPTION DESCRIPPIONN:
You will receive:
(masked tex]]
(1) microscopy images: Image control, Image
BACKGROUND KNOWLEDGE_SSOM
perturiationn of the experiment,
(backgroudd)
(2) un experiment description with one actribute masked (the image channel)
CANDIDATE COMPOUNDS:
aoctohesamide, 42m475271, eoomppsnll,, flinkokaberr,
(3) structured background knowledge in ISON describing candidate compounds (their Musitargess, expected image-ontly mor moophological signstares, and doce-time prices).
misoprostol, nimodipins, orantinil, sacubitri,, trazodone
ATTACHED IMAGES:
Image Control
Image 9: Pecturbation
task:
OUTPUT SCHEMA:
infer the most libely image channel for the masked attribute by companing Image Imrgee Be cnlibrated: if evidence week conflicting, distribute probubility mass over candidates rzther than overconfident guessing. response matiching required
backgrorand knowledge
"compoundd prediction",
"pred": "<ooe cennndndi compounds=*]=""
"probs": ("szoopound Is floss, "..": float),
"conffdence" floss, aquals probi[pred]]
"rationale": "owends; key A-D visual differences and how they support the predicted compounds"
31

---

Published as conference paper ICLR 2026
M.3
DETAILED RESULTS
Figure 9: Confusion matrix on cell line task.
CO-4
GITS
Ob dedeendeerr
Cemar 2250
Figure 10: Confusion matrix on image channel task.
Band
OPER
Figure 11: Confusion matrix perturbation compound task.
M.4 FEW-SHOT RESULTS
Table Few-shot Model performance on classification tasks
Madel
Cell line
Channel
Perturbation Componad
Flindokaleer
Racecaooiil AZM--47271
Misopraato
Trazodose
Oraniiah
Rulnanide Lunincosib
BIRB-796
Methorsals
Macro
Gni-4
0,515 (40.67) 0.250 (+0.032)
0.224
0.184
0.0
0.0
0.300
0.175
0.034
0.000
0.0
0.000
0.101
0.510
0.000
0.000
0.035
(+0600)
Gemini-2.5. 5hhho
0,600 (40.074)
0.730 (40.00))
00
0,000
0.000
0.000
0.000
0.0
0.000
0.160
0.044
0.000
0.023
N CP-AGENT PROMPTS
The prompts guide the CP-Agent through a multi-step reasoning process to interpret morphological effects of perturbations in Cell Painting data. Figure introduces tasks: (1) background curation step, where the agent synthesizes prior biological knowledge about compound's mechanism action (MoA) and pre- diets which CellProfiler feature classes likely be affected in specific imaging channel, and (2)
32

---

Published as conference paper ICLR 2026
feature ranking task, where individual features are prioritized based their relevance to the predicted mor- phological response. Figure 13 guides the CP-Agent to evaluate whether observed morphological changes under perturbation are consistent with the proposed mechanism of action (MoA). Using prior biological knowledge and quantitative feature summaries, the agent assesses each feature's 's directional change, links it to the expected mechanism, and assigns confidence scores. The agent then provides an overall judgment of mechanism plausibility, highlighting supporting or conflicting evidence. All prompts enforce structured JSON outputs to ensure compatibility with automated downstream analysis and promote reproducibility.
Task: Report Generation
Background Information Curation
You hiumedicall research asistunt with chemical hiology and phenotypic profiling.
Given chemical porter bation specifie Painting imaging clonnel, provide mechunisic insight hypothesize expasted morpooogy chonges specific to that ccllular componont
Inpet:
Perturbation condition (comspound
Cell Paintng channel.
pestubbtion cntuubbtoon_onditio, condition
responibillties:
Mechaniem uumary: 1-2 serience description componnd' mechanien action (MaA).
Chonmel-edevant typothoses a list morphology-lve effects espected for the given cellular component (e g. "Tubalin depolym-
criation lowas micmtublle testure" "ER fingmentaton gramilarity increase") with ressoning foessod on the selected channel. types CellProfiler types likely chunge this AnaSSupe")
impacted
lrity
Output farmat:
Retuin ouly ISON in the following lormat:
mechorism summary"
descriptions".
morphology hypotheses": I I'<hypothesis 1>" "chypothesis 2>") "Beely festurn types" ( ""pest"AArrSSiieeee A'AnnStuge", "Testine Tabalin'. 'immulaiit Tubuliis,, Wiigh "Tapiborr"""""asiiiiii "Lacation", "Namber")
Notes:
Facus all hypotheses and feature types on the specific imaging channel. If the compoind known to affect chis structure, be specific. If the effect is indireet or uncertain, say Be biologically grounded hut cencise.
Task: Report Generation
Feature Ranking Template for VQA
feature p_.peddccioo_spy
hiomedical specialist morphological
Follow instructions exactly. Remain grounded in provided context.
Return ISON only, with no extra text.
feature preeieppeeittoon._m
Painting pertorlation experiment, prodict morphological featunes move likely affected.
Inputs:
Backgmand biological knowlodge (hoon piir curation step).
background cunntoon_eeeee
Pertrbation concitoo:: perturtation condition
Candidate CellProfiler feature names the. the only allowed feuture namespace): festure, names, juoon
Your respnsiilltiee
Scloet from the provided festar name: list only.
Prediet which
show mospholoicaal change
perticbutinn.
semsoning
knowa biological mechanism expected morphological effects.
possible, reluze features to biological strucuures (e noclear shape, testure, grandariiy, arto, neighbor count, etc.)
Be conservative: de not overchim. If uncertrin, assign lower confidence..
Outpet format:
only the followwing
'featons ranked" [
"dleatire provided
"attonale": sentence grounded
diffesences contexts",
"onfidence": <foot between 0 and 1>
uummary: deid one-genggrah vummary morphology observed
Figure 12: Prompt templates for background curation and feature ranking.
33

---

Published as a conference paper at ICLR 2026
Task: Report Generation
Feature Mechanism Consistency Template VQA
feature mechanism consiteeny sys:
You are biomodiical imsging and phenomics anslyst spocializing Cell Paincing nesays
primary evidence quanitative feature explicilly provided inputs.
derived from CellProfiler culputs, Visual evidence ma be refermmeed only
Please output ONLY valid ISON object without any explanation, murktown formartingg or excaa teat.
feature mechanism consistency
Thhe perturbactin condition provided below may be incorrect. nona, or advenarial (e g.... fake drug nume).
Prior biological knowledge sbout the perturbsnion a SOFT peior only. You must verify agaiieet quanitustive evidence. If there is mismstch, you
evidence, prior mechanism. conservative: mechanism tume bllias recognized or could be contused legg. take or uncommson, set plansiblity low and avoid strong clums..
Dividence priorit:: (1) quamitative statistcss,
Inputs:
Cuntrol image: (DMSO)
Perturbation image: (pertuted))
Pesturbasion condition: (l cenurration_condtionn condition 1
Beekiground mechisiism and mox mooplol expectation (from prior knowledge): (( background, controon_ }]
Quantiative summary festures (based populition statatics:: (1 summary sammmary_oo_faeeee wtt.oo/oosss__spsooe__
Your responsiilities:
YYou are given two grayscale microssapy images A control (DMSO), - pertaee.
Compare expected from perturbation condition vs observed data from quanitative summary of features and visual evidence.
summtry. evallate whaller change supports, costendiss, insufficiens relative the mechusiso. Provide contidence score (0.0-1 (l) your jadgment
For each feature
You most evaliste *hovery fraturp* proviced, even not statisicallly significant. If eivideene is woak (egg high gevelue, Cleresss O, or smull effect size), sute that explicitly and assign low confidence.
may order footures by significnnz,,
du skip
Summarize the dominant morphology change and explain the quantiative trends supoort er contrdiet expectation. contradiet, explain why
(e.g. incorrect mecinsisst, off-tnrget, low dosettmme, or similar phenoeype to moother class).
Provide concise overall assesoment of whether the observed phenotype aligns with the proposed mechonism, highlighting supporting conflicting besed quarittaivve summary festures.
Output format:
Only neurn ISON. Do not include any sen-ISON test, comments, or miekdown.
rinked': [
"name": "<feeture__nee name from provided list>".
"dinectinn: : "sinreaeleemselmbiiuuo
abserved_eviddneee
sentences citing quattiiative stuts (delta CDCltf's deliaig) and, clear, differences. No claims beyand
provided evidence >""
"mechanisu, link": "eway this feature's change would suponticnttriii adiet the proposed mecitanisss, unclens, state ambiguutt
"supgars_pooposal proposst_mechhiineee mechanism": cnappttscoradiiclnntttttto
'upport confidence"
"eaechanism cassisteney" "cupeetioontadiiiiiestti
(plassibllit_sooo score": between and estimate crodible the pestutbttoo_conciie conditionnn based selected feature same and suporting scores for the propesad mochanism. Lower if cooflicting>>*
"cooppiology_ summary" "'One concise partgraph summarizing the dominatt marphological changes observed in Partubed (B) Control (A). based prananty quanitative teatures.
evidence available and cleatly supgorts trends (e.g.. traganented mutochondria, of structure), you may hriafly mention secondary supports.. "overall meshanien pesessment"
Write 5
order:
Prelade: briefly
al what the imagelcnndtion is hypothesized to show ithe pruposed MoA phenotype guess), and
h) what this aesesment will do nest (evaliatt consistency using quanitattive fexturns and mochunistic ressoning).
Example: "The specimen loputhesized to reflect ppmposed Munighenttppe) Below. I assess whether the observed morphology aligns with treds mechanistcc linkage.
this hypothesis using quntitative
Mechanism veniee.
Evidence summary.
Mechanistic linkage ventence: explicilly link Mos celluher expected morphlogies observed festures.
"Given the proposed MoA [MoA], it known to alter jprocess Al via [action], which should cause [outeome B); this would present as [feutures
(observed: Imeerics).""
"If [MoA] perturs pprccess could plausibly produce (outeome BJ. aligning with [leatures] (observed: [metrics]]; however, this tentative [limitation).
Caveats and altematives have (1-2 sentences): note inconsistencies limitttoos and suggest next checks (e g., dose-response, tite course.
orthogonal markers, replicate expansion).
Do causality beyond provided evidence; keep wording falsifiabli and bounded by
Figure 13: Prompt template for evaluating mechanism- consistency in Cell Painting data.
34

---

Published as a conference paper at ICLR 2026
O ADDITIONAL CASE STUDIES
0.1 ADDITIONAL CASE I: TAXOL IN MCF7
Cell Painting Drug Experiment Brief Taxol MCF7 (Tubulin, 20X; μM, 24
Matched Experimental Contest
2. Visual Overview: Control ve Perturbation Images
Iic: MCF7 thuman breast cancer)
-Culure medium: RPMI 1640, FDS, GlutaMAX, uglmL
imaging: Tubulin channel, objectivo
Control
Perturbed
Compound: Taxol (24 JM, 24h)
MMcchnism action: Microlubule stabiization
3. Anticipated Feature Changes
Mochanism
Expected effects taxol (microtubule stabilizer):
enrichment 24 tavoi hours MCF7 expeected stabilized, thick microoubule bundles with perfnuiear corficll
spindielaster formations.
drangia
heleogannsly while increesiing large-scila leading
Entropy/ contrest large offsets
Angular second moment, comreation, homogenelty
Scale desendent granularty shifts (, scale. 1 midiscallo
4. Key Feature Evidence from Data
AreaShaps_ Soaitttyy
Taxnwo, Varlance Tubutin_F F. 5.0r
Tennure SuvrVarinne, Tubulee_ 2.aNNe_7_66 $1
Dewdloon decrease
Directon: increso
Dieecten: increse
Mesion deoneased by -0.052
inreased
(1177
0,114 -0 (60) large effer (CaP's 5 0 B2X hight
1992)t atrong offect (Cars = 8a7g licant
6274)) soring effect (CBl'ss =-0 600x diy Wooen (q 000510)
kign 10 276-56)
D.DA 3).
Stabiloed, innenslly within colss Suppo Support prodosed mechanien in C Yes 85 oo continene))
spirdess
writillly across relyhtorhoods
amrat and srape imsudartiss recluting sc8d ly wis protusdnssadeerik smudtisss
expacaed with bundled
spindie
atruchaness
pports proposed
confidence))
proposed mecharien: You confdenont
Anecitrses Jeidity
Dffrrencedntraayy..... Tubulo_ 2_52_200 226
Infob/oas Diaccoon decrssse
Drection: incraase
Drocion necrese
Obseved evilenou: Median noreased by -852 ICI [254,
Observed nesi Median intreaped by -0 580 (CI
Obscrwes evterese Medar ducroaaed by -0 040 (CI
shoing
ageltom
airifiart
0 0:000017)
093255)
sighlitart nosee)
Mecherler Ink Thick neght hund neat to defoer
Moct entum ind Bunding and scindds poiss introdues
Meeharism live Lowkr in infomtiionaa mozure
syopleam incease looal contrast consinann with
pronout oed ly afforeooe clevaing offorenze-at
conductoon roflect shrnnger structured holkrggnniiyy
buncleed Suppors proportd macherlmm supporte (17
Supports propooed mechertam: @ supports 1outs
S0 Ripeott propoeed medhinim & support (0.32 contieeoos
contteewc))
corlidence)
5. Mechanism Assessmant and Conclusion
Morphology. summary
Kay Evience (Features Effect Sizes]:
Perturbed cals fower (16 and show strong incresses Tubulin texture variaroe, sum-variance, contrast, and difleence-errtrppy slongvide a marked decresse solidiy These quantisiio trends indicate mor pronounced, bright microtubolle structures and megular cell outlnes, concistent with bundledlsabilizd microtubules and milolic arrest phenokypes. Other texture and shape metrics show weak or non gnificant changes.
Variance, Tabulin 21,1__.60 296 +2877
q indicates Increased intensily heterogenerly due to bunated microdututes
Overall, mechanism likked,
The specimen hypothestzed reflect microtubule stablization loading morphology tubuinn miotic Balmw, whether mechanistic Inkage. The allga hypothesia quantlative phenotype supports microlubule stabiization. propored (mecharism
2. Textore SumVerisooe_ Tubufin7_01_ ___0__56 Median by +4228 a 6. 10e-04 Sugpests incressed global veriability in tubulin texture
micrhubuu stalizerss
after micretubule dynamiss suppression depolymerizalion which cause bundled, bright
3. Texture Contrast Tubuslo 7 01 205
arrest: this would present as higher tubulin varlancerconhaall and reduced call solidiy (cbserved large posiive cortiass@MMeeencc-cm entropy
( 126-04
Meeron by +632
strong salidity decrease).
varlancal-
Reflects sharper contrast from bright bundles vs. backhroundd
Caveats: Sample size in perturbed small (n 16), and several feotures are non-agriitcant offtanget effects or imaging differncess cannor excluded. Fallowup
4. Texture DifferenceEntioyy. Tubuthn 7. 256 Median +0 689
with doce responseetmeeeeeeeeoo course, mitsicc markers (e.g. pl1)) and replicate expension is rocommanded to strongthot the attribution.
35

---

Published as a conference paper at ICLR 2026
0.2 ADDITIONAL CASE 2: VINCRISTINE IN MCF7
Call Painting Drug Expariment Brief - Vincristine in MCF7 (Tubulin, 20X; 24 piM, 24
Matched Experimental Context
Visual Overview: Control vn Portutteiion Imseges
MCF7 thuman breast F68, CluaMMX, agint. C418
Tubulin channel, 20x onjective
Control
Perturbed
Compoundt Vicctetine 24h) Mochanism action: Microuubule dessabilzers
3. Anticipated Festure Changes Based on Machanism
eeeeceed
(micamtubule
cylcakolettd colispae, smesl, morphologizo changes
Relative centrol vincristin treatment expected
LLook microbuutlle stracluee and laslure
+Cylcshubiall collapae vesiting more roundedd smaller calla
Acccumuation of cel's in mhosis (Inreaseed cel number]
These changes are expected lo manilest in Cell Puinting foatures as
+1 Tubulin texture contrast and entropy to Textare Contrast Tubulin, Texure Entropy, Tubulin)
Angiter setord momentt g. Teahone AngidaeS@woniiMemeet Tuburin)) duw to more crnform absining
Erseniicty Faeter (mone cimuse calls)
to misoc rounding) Scall-dkpendertt gramularit
AmnaShapo_
Oijac
(a.g. Cranutaityy Tubufri)
Kay Feature Evidence from Data
220
Namberr Nanterr Choareed
Humber
Chanolanty_
Deaden In2mass
uede Mokuer ressose Hap9 tq 15e-0 te 0.43)
5.i1e, CiI's cuta 2.50)
decrenie
Madan decooee 4i5
borse, dala- -23mg
Dinctoo: iones
Mndtin inwwnedd 40-776
trecvasion Insc bitraii cueas
the
in oeo mini anno, ohtn boxing appelode and
indtows Dapaymattne of tine nicve ivanc and rexmendyy sager aggrgaaies combiler
aggrecster panseyraal sec det anpty opmpoieii
addeessnt odi dush setuction call count
antial belveon adjeard
epports propoeed eacher am oor (chnne))
Sepporss propoced thomiion a Yes cood
Suppore proposwd ao You contidence)
Yeutiee digrrirtooos/Mooen 2. OT 201
Areechape ForssFucter
AreaOhape Excsetriity
Obearved Madiar decrsmedd 20224
secress
Desanee: witeece Median incrassed by - 660 (C) ItC.
Cesanac mvitence Madien dec recespeed by 4:433 it
cotte, Ual's ckta 2:300)
of but 1180222)
vuiraccaly ge loxt
Vecca vim line Complemettry Forefadtr
Esmerboat tedtur sontel celae replaced by
lnc Bundlngg and spinds price prrduue
inlomm houndin dan cytodae colapos
vods.
tekerggmmou parkere lorght uggregatio dark
in guicb with skbellood mecolubeioo.
confernoo)
Sapports proporad macharlem You (oontdanna)
expports
LMM
Mechanism Assessment Conclusion
Marphology_
The dominant marphologioal change primuud danupioo of the tubilnn cyloskaleion Cuanite- tydly, this captured by lighty sintficant increase in Texure, Contrast. Tubulin and Granuann ly 2 Tahain rellrdnng the colspas the flemenious netwcck coens, bright aggiegeas. This cyloakohtal ialure is cona starl with he doservel (lhaugh net staislioslyy signirrant) sends toward mome munced coll shape, inded by an increase in AmasShage, FormFactor Furthermors the treatment induced foant cyjoooskyy evidenced by marked decreasu the Number. Object Number.
Ewdence (Features with Large Effect f. Faxture Contraut, Fubulin, 3. of 258 Median by +439.3
= 2 15e-07
indlaatea ineraase local plaol untanssily vanation due to poulh network colaase and agprepation
Grammdarrtyy Tabuiin
Overall mwwhanism_ Anker
Median by +3.276
specien hypothestedd reteect microtubule destab lzatonn induced vinotetine Below, assesa whither the observed morpholoyy alins with Pias hypothesia using cuantlative feature trens and mechanistc Inkage.
Quuerifiaa
emergeencc
formaiion
The obeerved phanehype sboughy supports the proposed machankem wilion
3.7eeturee Angutträneonnsoomen Twbu-
The most sigwficantt changes are tubuin testure, with a mazeive increase in Texture, Con
3. 01 256
traal Tuterin (dellx medie::
and decrwoo homoppenely ( (Tectur AngalerSweand
Median by -0 000002 i 0.0155
Moment Tubdiin), indiaiing lubulin agueeation.
accompanted a aigilf decruss cell (Number Object Number), suggecting sytobbtityy
Redezts loss homogenoous Romontoos texdure consstent with miconowle attuuttion
proposed
micehubuee
microtubule fomation, mintic smesl the would present less fiwmeetous stuthiee, formation of husoli and cell rounding gronuorty rooservect < 0.02 for texture features).
polymrtizaion
shoul sytoskelesal
4.Number Objject Number
Medion by
with the resenneed incowosss subuin contrel and
inalcatis reduced consistert with mitoto
Whiee the festural and cytotaulaity evidence detnittve, the expected changes in cell shape did not reach ataisiiow aigrifica row, which could be due to insulieent stalstical power population
whorlsire Induced cyfatoxiityy
netergennity
36

---

Published as a conference paper at ICLR 2026
0.3 ADDITIONAL CASE 3: SORBINIL IN A549
Cell Painting Drug Experiment Brief - Sorbinif In A549 (RNA, 20X; 45yM 48h)
1. Most Experimental Context
Vieual Overviewr Control un Perfurtntion mages
Inc:
channel GMEM
Control
Pertubbe
Comparrd Surtarel (68 uld, 45h)
Mechariam adion Abtose Reductase Inhibilor
5. Anticipated Feature Changes Eased on Machaniam
Exsecsed effects sprbini (akoosa reductase under coktrriellaamoee stressl:
expected lo perture RNA organization through redowocamotc spess, loading nuolcolar compaction cytoplasmio RNA puncia formation,
IIN
Relative control,
treatmentis expected
Inorwwseed RNA welure haarrggrrllyy
Farmation siruss grande-ike puncla
Red etricution of RNA to nuces eoll and pertrucleor zones
Thise changes are expected
manillasi Call Painiing features as:
11 RNA texture entropy (2-g. Texture Entropy RNA G 02. 258, Todture Entropy RNA, 02. 2581
-1 ReLA beatume contred g. Testure Cantned FevÀ, (0 250, Teature, Contreel RNA 02 256)
Loca intonslyy variaiiity g.g. Textire DffornnceEniipyy RNA,
Textrre noeettnne RKA,5. 206) 5_00 (56)
g. granubarly (eg.. Cramuleniyy RNA, Grandarity
4
4. Key Feature Evidence from Data
Tonkare detrrryyyy....... 251 Nendoon escinai
Tackan_ Exergy_ 2_58,158
Tastane www.raattrreeeeeeeeeeeccccccoooe 200
Ohserent sedence Vediondeormaoo deceasoddty -1 566 (DI 2.51 4880s lerge offec 0.20s Eut0er
Ortorved ew ionoe Meddan revored +0.200 3o
-2 508 lags effec (S arss o 0,785 e- 5,55 00 Rhah sayaal consicent with mudieoerr compoction
0-400] at unng sffed (Cllta 5 20000
Mechariee wik Heduced karpe ontipgy dig with
Mocria sian likk Hight or ho nogonne by 8008 saganit
conad cadion redso/temed: rram from aldms clss briiitrr.
nuckeoer comoaitttion expects with nudedaa emess Suppe mechoee
condersed reciadar aignal and recuoed use 20uA. Support prope
Suppera oe rethoriirs Yeu is conlidnnctt
contitenc))
Tackare Tactan_DVffrereee Affrereedniiopy...cc....... J.
Rawdin Recinisn
-D4 taz 12gs modima clec th' b DAT3
Ohseered relonca Mocian doom syed 046 781
55454
ADD 5.00 sral--orraaa ofrect (Cifss o Tx
Mlech live inorssow fivonon varance sharper bourcarie or local and fot ia bount with sorse.
Meshorion lik uneer beer
more
Vecia
velee uncotter tedtro or consoldaicc ent
donadiitiineeeeringg during nucleter strees
Suppore propased mechhooki Yhe 13 contdenca)
Supports prop mechasirn: soofdrend))
Supgor propctoo nec vansen teadioceet 30
Gramalaityy.
Obeanval swdeena Modiar poasuaed by -8 140 (Ct
Vicha lnd, Mad benuliouaai
sould
kow HMA por cil which cnly partully malcher expersaions
Support propooud ineeroniion aotifiinee yo
Mechanism Assessment and Condusion
The porturbed condiion ombbls markedl reduced RMA texture enoooyy and increased homegencity (higher irvess difenenox moment) indloiing a more oroweed RNA datribtiion
Morphology
Key Evdence (Features with Large Effect Sizest
Toxture, Entropy RMA_5_02 286 Muckin by -3762
Di woreoeeeeiioto winee and dffwerss-eerrrey mubric suggael ataerpet but more predictatie local grey-lavel relytonships compotible with donoolidatoo of RNA signal into stuctured subructeor regions and reduction of fluee nudiooplan same RNA, Gramulatly messures teard coomward small smal-to-mi scales statiaticlly marginal, implyng
5565--00
Indectte increaed ordering and reduced
incressed cytopiasmic purets
change
ontesiiyy neterogenaty
compaction
Overail mww.anisss....... Inktdd summary
Median -2.656
The spaciman i hypoiasized relaci wkloow reduciass aebiilion aotinl causing redosloometic stress with nudedlarRRA processing sttratoos.
3.555-00
assess whether observed morphology aligns with
Ruducto Nagge-soala RPA Komiogeniaaionn
hypothesia
consittent with nuchoolar consalattion
broacly stant Fuidenos summany Tectur Ertrpyy RNA and pd decrenss strong proposed mechanism.
Mcchan verdit
RNA
3. Tastture_ Awweraddieeeeeeeeeenn R-
7. 1e-7) and Texture InvereO flore rerkmmott increases (2 +0,293 a 1o-6). nndlating Nigher homogonelly Ditrrenodmmreyy decreases and Diftrencevanaance morcasss a 0,005) supporting more ordored yet sharper kooi sinuctun Grandarly metric show and mann fart deowsos contred clargees are the sspactdd drectton but not signiicatt.
NA, E. 02 265 Mechem
1.338-06
Supgeets increased homogenetty of RMA Difheweeeanwnee
02 259
proposed vldose reduciase intibitor perlabs redox balance and nsmate atreso could plausilyy produce nucioclar swess leading compact onionss
Median -5.277-04
RNNultc regions and reduoed difuss (coservec) strong
n 00001
presert
enropy highte homegenety
indicatea sharper local cov-l aWferenness companibe with mocleder for
doorcases entropy
diference
Caveats and attenaiies:: Many
are not signiicant and direct evidence syloplemmt whees--gneriellle. purdle lecking Raplioal axperatonn dowe-neperee. and incuaiin of nudooiammnnaatee matiers would help diaamiggaoe nusiestar consolicaion from general transorptional downsnit or imag nnitemmenttionnn antfacts.
contrsst
Tosturee DwfferonceEitrroyy.oo. 0_0__6002_ Medion by -1.5 946
37

---

Published as a conference paper at ICLR 2026
0.4 ADDITIONAL CASE 4: BGT226 IN HUVEC
Cell Painting Drug Experiment Brief BG7226 (Mito, 10X; 20μM, 20h)
Most Matched Experimental Context
2. Visual Overview: Control Perturbation Images
-Cal Inc: HUMEC
Cuuture medium: EGM2 medium
Imaging: Mito channel, 10X objective
Control
Parturbad
Compound: BCT226 (20 μM, 20h)
Mechanism actinn Pig-kinawe Class I Inhibitor
3. Anticipated Feature Changes Based Mechanism
Expected effects of BGT225 (OR inhib
BGT226 Insalmnnt is expected to impair mitcchontiaal inlegrity through invibiiion of PI3KMWTOR signalinng leacing to mitochondrill fragmentaation reduced mitochondrial moss, cristae disruption.
Relative to
BG 8E1226 trealment expected
of mitochondrill
mese)
Mitcchondrial fragmentation
disruption
Changes in mitochandral al shape (loss elongated, more ciroular)
Allered cristwe siructure and internal texture
Moddited spatial organization of mitcchardial objects
These changes ar espected to manifastt in Call Painting featurss as:
-1 Mitcchordrial area (e (e.g..AresSaageeee AreSShape_ Area)
-1 Mitachondrial compactness and torm factor (e.g.. AreaSShaee Compachness, AreaShape FormFactor)
Focentricity. MijorAsisi angth (e AraSShapa_ Fccentricily AmesShupe Mar.MMaaailllggghll singh)
Granulartt medium
Granulaniyy 3. Mto)
Taxture contrast entropy (o.g. Texture Contrast Mito 256, Textura, Entropy 01
Solidity g. AreSStepp Solidly)
Changes in mitcchonrra al nelghoormood structure (e s. Neighbors NumberOHNeeghorrs 10)
4. Key Feature Evidence from Data
Tecture, 2..0hast_.011.000 Cieccio:: increase
Directon: MobontaisCangth
Neighbors MunbbrOiiiigtooor. decoease
Oheerved en denon Styrkoant hnrwaee from 5o
Obeenved eedence Decresse from 6 4 neighbors -20 Ei 1-2.0 11 cath 8 86, q Be-da)
la -98.4 CI [65.0, 142.1 Cll's -0.57 2 6.8--08)
$2.0 2227 of 110 29-8] car 8 -0.6 (6, q
Mucheriam bak, PINKWIOO emvatton coud donugs
loading
decthrlon lnk Piic TOR inte too induce oell
poliiestio sunvent leading lower densiyy and fower reightsing cols.
hateroyensll contore milscoodtiaa
stros and alwe cytodow aiai orgen oation pakentialy
coining patwemx
causing coll atongation part stroos reeporsse
Supports propored mechoniom O Yoe (I confidento)
Suppore proposed medreniors bonn C Yhe 2ou comtteencs)
procowed medeintom Yeu 10.7 coveidencc)
Saloy
Diwciao: Inchess
Machortam inko Cr amobiton can cause oel cydee small and smess entured cal entrgement scrandent
AraaShape Area
Oiserved thence Sign fexrd tromes hore 2179.5 to
"6 de 02)
dj
Meshariam ins. Plakin TCR inveation darupt cytokidulal organization on and col achesion, kuiding contact more inegular col la hopes.
AreaShipe_ Compactners
Directon: nnreaae
Oboeerve oedemoe:
CaT's from
Se-00r)
eddencel Decrease foo 0 to Of (-6 15 -0 07), Clifs 8 0.50 8 6e-07) Mecharinn likk Coneateet with compatinss charges, dangee
AreaShape, FormFoodor
moppicoogy
making was cinalar
Suppora proposed mechanten: Yes confisence)
gruing.
Supports
10.7
Supports propeced mechaniem ( Yes (07 contidens))
Granuletty 9.80lo
Toxturo Entroy__. J. 1,011.00 _26
AmaShappe Salidbyy
Direction deciensn
Tindton: decrroon
Observed Increase from 7:10 7.74 (d: G1 56. 368 cats 6 -048, q 6 Te-67) Medhaniom Inc. POR inibl on dooust mitochondaal organization finctoo, oeteogeessss disirdened milochondraal Supports propoced tmm O Yes (0.8 confie nce)
Chowrned mucdenne Decrnoos from D.s
0.24, CI: 0. 19) care D 16, q 0.11)
Madteram lnk Po indt ton right neducss
Mect wnlnn Ptaln TOR inte fro can wfled cyttokedete organiaation and cal adheedoon leating loo ankt cal shapee with more profuueoees imegstarties sec Sappprrt proposed
minchenonn granudarty trrogg attered mitoonontraa
Supports praposed machersam Irautkcent (022 cambbereca
AreeShape, Eccastrietty
Obsenved ensdence: Week moruase from 0.75 3.50 +6.02. CF [-0 2/01j CM a -0.16, 0 11)
eccensiiity
elongattion the efnit
conduetin undiumion
Support proposed mechoniem: & IneutSiciet (0
contienoo)
38

---

Published as conference paper ICLR 2026
Painting Drug Experiment
(Mito, 10X; 20μM, 200)
Mechanism Assessment and Conclusion
Morphology, summary
The perturbed cols show consistent penern of atrespreeated morphological changes including signifiantly incraased oall stza (araa 441%), akongation (major axs +35%), and reduree cel densily flower reighbors). Mitccoodria organization appeors disrupted with increased testure contrast and entropy, susgesing fragmented or disorganized mitochnnrral networks, Call shape becomes less compact and cirailer, with redured solddty indicating sytoskeletal disruption and iregular morpholegy typical metabolic stress. Overall, smoother, ordered
Key Evidenco (Features with Large Effect Sizeg!:
Tastura, Contreat Mito_3, 01_256 258
Median by +95.4
increased
T.Teruure Entropy Mito 3_01_256
crstae discuption
heterogenelly
dominamt
nucicolar
Medeer by +0.65
g 9.7e-07
Suggeals incressed millchonddial textura disertir and
Iragmentation
Overall mechanism linked summary
3.AreaShape, Area
specienn hypothesized reflect PI3K infiitton BGT which morphology
Mexian by +895.5
metabellam
pathways
whethar reasuning,
4.9c-07
signs
Roflects shass-induuad
observed phensbipe strondly supports proposed PIKKITOR evidence indldes significantty incressed melechondiall
ArenShupe__ MnjorAaissLength
antopy (n-1e-680
Medtan by
entarged and elongsted cells with imegular shapes. and reduced cell denstty. Given the proposed MoA FI3K class Inhiboon known to afor colliarr metabollsm and survival signaling via disupled P13KAAImTTR paltways. which should cause miochhndraa dyslunction cell cycle arrest, and cytoskeletai disruption; this would present entarged imreular cells with aisergarized mittchondris (observed. +64% textuee contrast. +41% call area, reducad form factor). Two features showed muuffiiientt evidence specificity
8.8i-088
Consistent with call elongation under stress cytosteletal remodalig
5. Meigghbors_ NumbarOONeighooor_.. 10
Median by -20
statistical significance, and
6.86-08
indicales redlvced cel densly, porsibly profferation arest cyfotouicity
sylotookchy.
39

---

Published as a conference paper at ICLR 2026
0.5 ADDITIONAL CASE 5: AZ841 MCF7
Cell Painting Drug Experiment Brief AZ841 in MCF7 (DNA, 20X; 24μM, 24h)
1. Most Matched Experimental Context
Visual Overview Control Perturbation images
-Cel Ins: MCF7
-Cuture medium: RPMI 1640, 10% FaS, 1% GlusaMAX, 900 ugimi G418
imaging: DNA channel, 20x dbjective
Perturbed
Compound A2841/24 yMt 24h)
Mecharism action Aurora Kinase tor
3. Anticipated Feature Changes
on Mechanism
Expected efects of Aurora kinase inhibilan AZ841, 24ph 24h):
Aurom kinsse imfibiion expected cause mitoic arrest, cytokinesia failure, and spoptatic faagmenattion. This leads mixed nuclear phenotype induding small condensed mitaiic eniarged multiucleated cel's, micronuzter nuclcar debris. expected chromalin
Relative
Aurora inhib
MMoliic
Polypiokly mudiincceationn cytokiesss failure
imagular micronuctei
fragmentation
innreeased chromatin heterogenelty and punciate signats
Possible
in
parking
These changes are expected manttest Coll Painting features
-1 Muciaer siza dipersion (e.g. AreaSrppa, Araa AnwsShaee_ FqyivakenDiammdder, AmaShape, MaeFanelDameter)
-1 Shape inagudarity and fragmamiatio (4 g. ArsaShape Eccantricity ArsaS/apa Soris_Skklyyyy AreaStupe FormFacto: ArseShapee Parimeter,
AreaSrape EuerNummerr
-1 Mid- and coane-scale DNA grandarrty (eig Granulanit_2 2_ D_DAA GmmularityS _3 (NNA
*1 DNA texture hoterogenalty and local veriabilty Byy (e.g. Texure Contrast DNA 02 255, Texture Entropy DNA, 6. 92 255, Texture Variance D-
NA. 02. 256, Texture Sumvarancce DMA, 5. 02 250, Texture. DiferenceEntrppy. DNA, 5. 02 256)
-I Tichure smoothinias homaginalty (# Teature Invessddiiioonnsaanennn [DNA 5, 258, Texturo, Corellainnaannnn LONooe__NN____ 5, 256, Textura An-
gular gulaBeccoddommart DNA, 5 02. 255)
-1 Olject count (dus to call loss) (eg Number_ Object Number)
-1 Neight bar contact due to clusterigg or entargad ruciii (e.g. NeighoonsPPeee ParpanToucchnng.. 50)
Kay Feature Evidence Data
Granslerty_2
Granulertt.. 3
AreaShappe Masraantilaattrr
Dieecto:: remoon
Deecton ircress
Observed endence: Medaan incuased frr 200 to 0.81d 520: (2-37, 2.535 very large etledt (Calfsid
Chsenned edderce: Medar incressed from o 20d
Obeerved wnderce:: Macken incressed 38 20
(Clfs
(7.21
8811
= 4 0.76) 1.36-18
-O 01211 2 1o-08
Necharism ink Auron sinese enitttion oan cousse
Mechersam Larganazat divamann cumppingtnes tnirr---
Meshartem Phkyttoabyruuu trrtttttto can ylald larger Suppart propoeed machan amr Ven $2.90 conticenco)
condereedion flageente chromesin and micorrcie
gronularty
granuartyy Supports proposed modhaniee riom: O Yeo 06 confidenos)
Suppors propoeed mechaniion: Yos 13 58 confidenoo)
stuch
ArcoShape Sulisity
Area Fartraattrr
AreaShape Parineder
incinne
Chseeved exconoo Med decressed from c to
clao (A- Cil j-d -b.co)] modembl wflect
Ossersed widdeca: Mecian decrocood hom 911 6.71
5950 1-1,115 -4 beaay modwnete eftec (CBT di
+91 87 CII 7.78 45,21) moderate sfict (Cal'ss
50500, a 3 3.50-05
Sapports proposed
Medhansnn fis. Insuulrr or loulited nu mudikina truckor
asprogates from oyloirecis fallur lovor soiddyy
foggie tarion expocted with divorosone mes dog ogalion
Machiriim ink, Nudlor imogu lafty, indortation, or pericl
Support propoaed mechan sim. Yos 10 54 confideno)
frmm midtin negi nezeaee perineter
Frtenger
mechumeer Yie BG conl dinas)
1m
ArsuShapee_A Area
Areedtapee FovislerrDaase
Teative Variance 2008
medennee Modia naseese 655 1501
Cossrves evidnns: Nedkr incrssee from 34 43 72
Observed evidnnes Macker decreased from 994 322
+d8t: [147 728 mode the whet (CI di 416) (2 nd pn: Ci 12 500: mocerate oftec (Cll's di
(b -221 37: ci 1-335,0 86, -73 styl mocerate oftec: (CWM) oi a 2-420-04
q
1.250-04
Mechorism
nucis with
notennedd chepmosomes witl midic anvorms nusiel can redure local intensity varannos ai thie. scale.
Kecharism
oytt4 medis bell em senbel farger clnor sin.
equinmenn clemeter
Suppors proposed mendinniion: You 13 confidenoo)
Suppo to prosoeed mechanan: O Yes (2:30 conridence)
40

---

Published as a conference paper at ICLR 2026
Cell Painting Drug Experiment Brief - AZ841 in MCF7 (ONA, 20X; 24pM, 24h)
Key Feature Evidence from
Sunioriano, 202
Neignbors PecsentTruchinngc Desecioo:: ant gunut
Dlawclani
Obsonvot or dendnnss Modian deoreasod from 1610 A7 300 68 (b 3 754.8 oil 1080 56, 226.01) moderate
Ditoersed sieence: Madian decreseed from 5e5 358 (A ci (-331 stjt moderate
Cheeeced wddtnnc Medien 100% toth groupe dillt
Medismiu No dhange inormatioo
Moehare lik: Homogeneous divomatin
Mecharsam link Chromain oor opndensation redice controat matric componiti with more unitom asecing Suppors proposee mechartent Yex (0:25 contidenca)
Supoorts propoeed mechoran:: d. rruutccw (0.0
segrevted micronuiet can reduce sum vafarce.
Suppors proposed mediarkom Yes (2.62 comfidence)
5. Mochanism Assassment and Conclusion
Morphology summary
The pertur bedurbed nucht show strong increases in DNA granularity at muliple scales and larger, more ireguurr ruc iear shapes thisher area, eculvalent diameter, partmeter recuced solidty and form factor). Cell counts per fiedd trand lower. DNA techur metrics genarally decmase in variinoalcontrsst si suggasting more compact or discretized chrometin. Together, trese patterss are consistent with micoic emors leading to nuttrudiaalomippyypkeee and chromatin condansation Overal, the cominant change smoother, more ordered RNA texture comsistent with nucleolar compactio consciidaiio.
Eudence (Fenturss Large Fifict Sizaa):
1.Granuarity__2 2_ LDNA:
Median by +3.52, q 8o-18
Strangeat evidence chromatin condenss--
popfmiccrouudel 2.8 2.Granularity DNA:
Overall mechanism Jinked summary
Medon by +3.88, iq 1. 1.3e-13
spesimen hypothesized reflect knase intibiiion
mitaicc disruption. Bolow, hypothesis using quantitativa feature
Further supports increased coarse chromatin structune
whathar tha observed morphokoyy sligns
mecharistic linkage.
J.AresShape__te /uzForetDiammter////// Area/
Mecharism verdict: The quantiative phanotype broadl consistent with aurora Kinase inhibitonn. Evidenee summary. Strong incressss in Gramulrlty 22__ DNA (dellas +0,520 10,52/+ qs 1e-13) and
EquivalentDiameter Parimotor: Indicales nuchar entergement ivaguler
entarged, more ineeular nuslel (Area/ Petteeeeitetee eret up: Saldhy/Fomf actor down: a te-04)
boundaries
dominate. Texture motric iss (Varlance, SumVariance, Contrast) decraas with a. 0.002, while cal number modeslly drope with CI touching O Non-informative features show no significant change
S.Are Solidity/ / FormFactor:
Indicates nuclear farmentation or lebulstion
Mechanistic lnkage sentence: Glven the proposed MoA aurora kinase nhiibtorrs, is known to alter inhibiiion of
5. Texture metrics (Variance, SumVariance, Contrast]:
miltic nutth.ckkatto//pppplaai
onloytkkneess which should mitaiic
would inegular nudel potental moreoude (observed: Granaforit__2 2/3 Area/Euuvaieemn httyy/nnnnDDmmeeePPeeeeeee Caveats attamatives: Same textura changes modest and several features are non-signiliantt and we only have DNA chanael a single timepoint. Follow-up with mullichonnel Cell Painting (tubuiniactii) call cycla protling, and dose-response www.rressstoeeeooooooooo tim--cuus would strongthen mocharisse aitrbution and ditingush from other milolic poisons.
increased DNA granularty larger,
Suggest reduzed internal helerugenetl.
consistent with chromatio compaction
up: Solidty/Form down).
P REASONING EVALUATION CRITERIA
The survey is designed via Google Form, and can be accessed here: httpps://bcc..ooogl/ Www.chhaAArrroNeTITTTEVEVEEEEEET
com/ http:///ww...a.aooooooooooccoaa. wiewwormnusp-heedeer
To facilitate consistent and high-quality responses, we shared the following rubric and example list with par- ticipated experts as initial guidance. This framework outlines key criteria for evaluating Language Qual- ity and Reasoning Quality of model-generated explanations in biological tasks. The rubric emphasizes five core aspects of language quality-includin well five reasoning quality metrics such pattern recognition, stepwise reasoning, biological deduc- tion, hypothesis formation, and mechanistic insight. Each criterion is paired with both positive and negative examples to help clarify expectations and common pitfalls.
accuracy, relevance, coherence, depth, and conciseness,
41

---

Published as conference paper ICLR 2026
P. LANGUAGE QUALITY CRITERIA
Language Quality Criteria
Criteria
Parformance
Excellent
Positive Examples
Negative Examples
"Eg5 Inhibiton leads to monapolar spindle formation, halmmrk of mitoic arrest' Explanation: Comectly links Eg5 inhibition monopolar spindle formation and milolic demonstrating
E/Example 1:
Terminology.
Accuracy
Uses correct biological terms, machaniam and data. Describes processes drug machenisms acourately.
Exampie: "Granulartty the cytoplasm reflects chromatin
machanism descriplions are correct; errors.
consermation during Explanation: Chromatin condensation occurs the nudeus, not the cytoplasm. This shows incorrect terminology.
Exammle 70F11, also known as Eg5,
ecserfall for centrosome separation during miloais."
Explanation: Acourately identifies KIF Eg5 and correctly explains its role centrosome scparatin.
Stays the image-based mechanism, features, and hypothosis tasting. Avaids discussing lated pattways.
Focused on the core problem: avaids unrolated content.
Example: "Texture changes are evaluated in the context of mitotic arrest, not other cell cycle stages."
Example "EGFR inhibiors are commanly used lung cancer therapy and act on tyrosine kinase domains." Explanation: the task about Eg5 infibition, discussing EGFR irreevanti and off-lpiic.
Explerition: Suys on-topic linking texture features specifically mitotic
Example 1: 'Because Eg5 inhibtion blacks
bipotar spindle formation, the observed Increase DNA granulerily is expected."
Coherrece
Explanation: Uses 'because therafore" logically
Logical
cause-elfect observelions interpretations.
mechanism
Example Therefore, Egs irtibition the cause." Explanation: Jumps conclusion without explaining intermediate steps like spindle defects miteric arrest.
sruusured reasoning, natural trarsitios.
"We first observe increased chromadi granularity and reduced call number. Given these findings, hypothesize Eg5 inhibilion the litely mechanism, which sligns dysfunction phanotypes. progression hypothesis biological
Example 2:
Explaration:
Example 1: 'Although increased granulantyy
suggest mitotic amed, could also reflect apoptcels; further staining needed. Explenation: Considers
hypotheses
X granulerily reduced number.
beyend explain 'why": considers
mochanistic reesoring, alteraaive
proposes valiation, showing analytical
Example 2:
"To confimm that increased
Explanation: Only observes phenomena without expleining their significance underfying cause.
aitennative
explanstions, validation proposalss.
granularity results from mitolic arreet, time-lppe imaging could used track time."
mechanisms
limibations.
Explanetion: Suggeels forward-kookin approach, demonstrating deeper reasoning.
Example 1: 'The texture of the chromatin to also increased granularty Explanation: Repettivee phrasing; same idea is stated twice.
Example "Granularily' 1, Entropy consislent with chromatin condensation Eg5 inhiition."
Conciseness
Expresses complete logic using minimal words.
Explanation: Uses symbolic shorthand to summarize findings clearly and effeclively.
Glear and efficient redundancy.
Example 2: "Due to the polantial
inhibilion of Eg5, which during may possibly experience something in mitotic progression.
"Mitotic anest infereed from compaction." Explanation: Cmils unnecessary words yet remains scientificlly complete precise.
@ Example 2:
monopolar spindies
blockage
Explanation: Wordy, vague, and redundant. Can be simpiiied to: "Eg5 inhibition likely caused mitatic arrest."
Figure 14: Language quality criteria for evaluating CP-Agent generated Cell Painting reports.
42

---

Published as a conference paper ICLR 2026
P.2 REASONING QUALITY CRITERIA
Reasoning Quality Criteria
Criteria
Excellent Performance
Poellie Examples
X Negelive Examples
X Example 1: "The colls look
6.
Recognition
identify
morphology consistentwihh mitotic Explanation: Recognizes dense, granular chromatin as a sign arrest.
@ Example "Granular, compact chromatin
mosily the
Explanation:
evident merphdiogical charges..
recognize
key visual
Connects visual features with plausible mechanisms.
X Example 2: "The blurry area in the cytoplasm might be the nucleclus.
difforecess such as cell morphology or staining
Example 2: 'Reduced cell count and round,
patterns and Ink them to biological meaning.
compact sccumulation consistent with mitotic
Explenation: Confuses
Explanation: intogrates multiplo visual cues explain biological state.
cytoplasmic structure organelles, showing poor structural understanding.
& Example 1: "Slep Check DNA granularity Step Consider milolic Step Confirm shift Conclusion: Eg5 inhibition lkeey." flow.
X Example 1: "This dleerly due
7. Algorithmic
Reasoning (Stepwise
Eg5 Explanation: Conclusion without supporting steps or evidence.
Thinking)
Follows clear "obsorvo -- infor - varily" logic chain.
Explanation: Follows disgnosic-style ressoning
Systomatic
X Example 2: "Mayoe it's
reasoning from visual featurees diagnostic
stop by-stop
El Example 2: "Metric: High DNA granularity low heteragonoity Hypothesis: Nuclear compaction Biological
apoplosis, but the chromatin
and also the granularity Explanation:
Consistent with melaphase kely cause: Eg5 inhibitin."
high Eg5 Disorganized
spindles."
conclusion.
Explanation: Builds multistep logic from feature mechanism.
reasoning, lacks structured flow.
@ Example 1: Eg5 is inhibited, bipoler spindle formation chromatin condenses."
X Example 1:' Eg5 is inhibiled, chromatin this." Explanation: Skips requined mecharielic ressoring steps; lacks causality.
cells accumulate mitosis
Deductive Reasoning
Explains observed features using established mechanisms or canonical pathways..
Explanation: Demonsirates clear biological cause- affect chain from inhibition phonotype.
Uses known
biological rules to prodict specific autcomes.
E Example 2: "Apoptoeis leads nuclear
X Example "Because mitoeis complicated, maybe thes's chromatin dense"
liagmentationn increased heteregeneity, This appear imegular,
punctato chromatin staining.
Explanation: Vague unscientific language: lacks specific mechanistic explanation.
Explanation: Applies known apoptosis features to inteppret image cata.
& Example t'Hypethess:: DNA granularity suggests either miloic arrest or apoptosis. Evidenoe: Low Entropy High Contrnat - favors milowis"
9. Induction Hypothesis
Proposes attemative hypatheses, weighs evidence,
Explanation: Proposes attrnatives and uses feetures to evaluate them.
X Example T: must be Eg5 inhibiion."
Testing- - Forms hypotheees from observations and supports refines
Explanation: States conclusion without forming testing hypothesis.
& Example 2: 2'Hypothesis: Granular chromatin
and draws them reasoned evidence.
nitetic
condusiion..
validate:
staining
sccumulation.
Explanation: Suggeets hypothesis and a conoroto method for testing it.
cantrosome separation, leading monopolar spindies, which induee checkpoint-mediideee mitotic arrest." Explanation: Demonstrates full causal chain from drug action to phenotype.
T Example
inhibition
10. Mechanietic Insight Links visual observations to underlying
Traces a causel palh from molecular intervention callular strucuueeflunction image features.
X Example 1:This must be mitotic arrest because the nuclel look dense."
@ Example 2: "Milalic cells lose substrate adhesion reorganization cortical actin and detachment fiam resulting rounded morphology imainng.
Explanation: Observation not linked to molecular or cellular mechariam; reasoning.
molecular celluiar pathways.
Explanation: Explains how sytoskeletal changes translate to visual cell shape.
Figure 15: Reasoning quality criteria for evaluating CP-Agent generated Cell Painting reports.
43

---

Published as conference paper ICLR 2026
Q
EXPERT RATINGS OF CP-AGENT GENERATED REPORTS ACROSS LANGUAGE AND REASONING CRITERIA
Q.1 HUMAN EXPERT ASSESSMENT OF PERTURBATION REPORT QUALITY ACROSS LLMS
Figure 16 summarizes expert evaluations across ten rubric criteria, split into five language quality dimen- sions (Figure I()) and five reasoning quality dimensions (Figure T On average. all four LLMs received high ratings (mostly above 5.0 on a 7-point scale), indicating strong performance in generating biologically grounded screening reports. Among the models, GPT-5 consistently achieved the highest scores across most reasoning metrics metries--inluding pattern recognition, algorithmic reasoning, and mechanistic insight- while also maintaining strong language quality. Gemini-2.5-Par closely followed, particularly excelling rele- vance and coherence. Claude-Sonnet-4 underperformed slightly in mechanistic insight and inductive reason- ing, indicating slightly weaker performance in higher-order biological inference. Grok-4 showed relatively balanced language quality but lagged slightly in depth and coherence compared to oop-perfomming models. The bar chart (Figure 16p) further illustrates per-metric mean scores, reinforcing the finding that reason- ing dimensions pose greater challenge than surface-level language quality, especially in tasks requiring mechanistic interpretation and hypothesis generation.
(a)
A. Language Quality Metrics
(b)
B. Reasoning Quality Metrics
Algortthmic Reasoning
Galde-Sermi54 Gamii255h
Deductive
Accuracy
Pattern Recognition
Inductive
Conciseness
Mechanistic Insights
(c)
LLM Evaluation by Metric
Model Clucee Someet
Acoeeny
Reterante
Cobaerce
Depai
Gevisontt
Putten Recaparion
Alputtienc Rearning
Deluctve
Intectiv
Mechembbic Ancignss
Figure 16: Expert evaluation of LLM-generated screening reports across language and reasoning di- mensions.(a-b) Mean expert ratings (on 7-point scale) for language and reasoning quality, based on ten rubric-based evaluation criteria. (c) Bar chart summarizing per-metric mean scores across the four evaluated models: Claude-Sonnet-4 (blue), Gemini-2.5 2.5-Pro (orange), GPT-5 (green), and Grok-4 (red).
44

---

Published as a conference paper ICLR 2026
Q.2 EXPERT EVALUATION CONSISTENCY ANALYSIS
To assess the reliability and consistency of expert evaluations used in our study, a comprehensive inter-rater agreement analysis is conducted. The results are detailed in Table [3] and summarized in Table 14 We employed several standard metries to quantify agreement among expert annotators, including Kendall's W, standard, deviation, exact agreement percentage, #1 rating agreement, Mean absolute, difference (MAD). Kendall's W values ranged from 0.33 to 0.37 across different models, indicating fair agreement among annotators. At the pairwise level, 73.3% of all rating pairs fell within =1 point on the 0-7 scale, and the mean absolute difference (MAD) was 1.04, suggesting a high degree of rating consistency. Given the limited number of annotators, bootstrap resampling is employed compute 95% confidence intervals for all key reliability metrics, ensuring robust statistical estimation.
Although the expert ratings did not reveal statistically significant differences between models, observed consistent trends across raters-particularty that GPT-5 achieved marginally higher average scores across most evaluation dimensions. However, these differences were relatively small, implying that all evaluated LLMS were able to generate informative, relevant, and biologically meaningful reports when operating under our CP-Agent generation pipeline.
These findings collectively support the internal consistency and reliability of the expert evaluation process. reinforcing confidence in the qualitative assessments reported in the main text.
Table 13: Inter-rater Evaluation for Expert Evaluation
Model
Metric
Kendhll W
Kenball W.
Kenball, W.
SD
SD,
SD,
Esacti
Essst,
Exadi
Neark
Near.
Neau.
MAD
MAD,
MAD,
Accanay
0.308
0.148
0.405
1.01
0.89
1.14
24.1
20.2
29,4
68.7
62.6
74.2
1.36
1.02
3.31
Reieanee
0.3
77.1
0:36
1:36
Colerence
0.87
50.8
Condieness
0.244
0.454
1,05
(198)
22.0
65.0
1,05
1.30
Patee Recognitin
0.312
0.138
0.530
0.89
0.77
1.04
20.9
26.2
33.2
77.1
71.2
63.3
0.99
0.86
1.34
Algerhhic Reasoning
0.308
0.133
0.000
0.88
1030
2X.1
26.9
33.0
75.9
70M9
N1.1
1.00
0.30
1.00
Deluxtive
0
0.200
0.628
0.50
D.3
0.99
20.3
26.4
33.5
74.0
68.4
78.7
1.01
0.91
1.12
Indurtio
06699
0.62
$4.4
66.B
81.5
1.00
0.58
Mechanstic Insights
0.342
0.56
1.03
0.90
60.7
72.7
1.07
1,30
Accuncy
0.370
0.555
0.94
1156
22.2
06.3
0.97
1.38
Relesance
0.409
0.191
0.64
0.54
0.57
1.00
25.0
22.4
23.3
72.3
68.2
76.5
1.08
1.50
1.36
Canemno
01315
0.181
01533
0.64
11.66
11000
78.6
54.9
35.1
70.
74.0
68.5
0.94
0.6%
1.08
Desth
0.300
0.119
0.18
0.85
0.00
0.00
30.4
27:1
33.8
71.0
73.4
50.9
0.95
0.58
1,00
Gemini-2...Pro
Conniencc
Recogaition
0.3
73.0
93.0
0:95
Algorihnnc
0.597
0.79
Dedective
0.62
0,50
1179
68.8
0.56
1.35
Inductiv:
0.365
0.174
0.51
0.83
0.67
0.90
20.6
24.9
35.2
68.7
63.6
74.1
1.66
0.96
1.14
Mecharistic Insights
0.30
0.168
0.495
0.93
115
1.00
25.3
23.3
33.4
71.9
05.2
7567
1,06
0.95
1.19
Aocunayy
0.504
0.96
0:06
1.31
Retesunce
0.69
Dogth
0.307
0.139
0.56
0.52
103
63.8
0.92
1.15
5
Parrer Recegaition
Cuncisence
0.303
0.350
0.30
0.140
0469
0.40
0.97
0.11
0.3
11773
1100
116
25.2
53.7
24.0
75.0
33.0
33.5
71.5
79.6
63.5
74.4
66.7
77.6
0.91
0.94
0.7%
1.00
1.38
Algorthnic Reasoning
0.422
0.365
0:8
0.86
0.71
199
90A
74.5
37.3
76.5
66.7
0.96
0.79
1.15
Delactive
0.2
0.57
65.2
1,00
Me-hanistc Insights
0.98
1.22
Accunacy
0,278
0.120
0.493
0.93
0.1
1.06
27.8
23.7
32.2
23.3
65.4
50.2
1.06
0.91
1.21
Reievurce
0,403
0.214
0.63
0.92
D.96
25.7
22.9
20.1
73.4
68.5
79.0
1,05
0.34
1.15
Cheeemace
0.96
0.11
11.2
93.8
18.9
0.7%
1.05
Desth
0.9
65.0
1.25
Gmk-4
Canciuace
Purem Recogattion
0.349
0.52
050
67.1
030
1.36
Algorihancc Reasoning
0.359
0.236
0.593
0.55
0.69
0.90
33.5
26.2
41.5
76.7
714
68.6
0.92
0.15
1.11
Deductive
01858
0.279
0.700
0.89
1181
119
25.5
24.8
31.5
76.3
71.6
79.8
1.00
0.91
1.31
Industis:
0.463
0,270
0.677
0.58
[.0)
1.05
25.4
22.5
28.
69.0
64.6
73.5
1.15
1.24
1.23
Mechariattc Insighes
0.335
0.149
0.519
0.99
1.11
28.1
16.0
31A
66.6
5k.9
74.7
1.12
0.97
1.37
45

---

Published as conference paper ICLR 2026
Table 14: Summary Inter-rater Evaluations Across LLMs
Model
Kendall_W W
Score_Avg
Score_Std
Near_ Agreement %
HlaudeSSoneet,
0.33
5,45
0,94
72.65
Gemini-2.55Prr
0.36
5.53
0.89
75.07
GPT-5
0.37
5.59
0.91
72.48
Grok-4
0.37
5.51
0.93
73.09
R ROBUSTNESS EVALUATION: FEATRANK AGENT AND REPORTGEN AGENT
The reproducibility of CPAgent's reasoning outputs is primarily influenced by the temperature parameter in the large language models (LLMs) since the perturbation codition is infered from CP-CLIP, which makes the conclusion of the report deterministic already. In our pipeline, following the initial pretrained CLIP model, there are two LLM modules, the "FeatRank Agent', which ranks CellProfiler extracted morphology features, and 'ReportGen Agent', which generates natural language reports based on the ranked features and contextual information. To ensure that the feature ranking step as deterministic possible, we set the temperature of the FeatRank Agent to O. To systematically evaluate the reproducibility, we designed five experiments with varying temperature settings, the ReportGien Agent's temperature was set ranging from 0.0 to 1.0, while keeping the FeatRank Agent temperature fixed at 0.0. In each setting, we repeated the pipeline 30 times on the same input samples and analyzed the consistency of the selected features and generated reports.
SYSTEM PROMPT
You are scientific evaluator specialized in assessing corpora of mechanism assessment reports derived from Cell Painting assays.
Your task to evaluate the internal consistency set of 30 mechanism assessment reports. Each report analyzes how observed morphological features align with hypothesized mechanism of action for specific chemical perturbation.
You are not evaluating the scientific accuracy or biological correctness of any individual report. Your focus is on how mutually consistent the reports are with each other terms of:
Scientific focus and biological reasoning style
Use of technical terminology and feature names
Presentation structure and rhetorical flow
You will be provided with full set reports or representative excerpts. Based on this, you will assign scores for each consistency dimension and briefly justify your evaluation.
Return only the JSON structure specified the user prompt. Do not include any commentary or additional explanation.
USER PROMPT
INPUTS:
A corpus of 30 Cell Painting-base based mechanism assessment reports
46

---

Published as conference paper ICLR 2026
Each report typically includes: Mechanism verdict, feature-based evidence summary, mechanistic linkage explanation and caveats or alternative hypotheses.
TASK: Evaluate the corpus for internal consistency across the following three dimensions. You are not judging scientific accuracy. Focus on mutual alignment in scientific reasoning, terminology, and structure.
1. THEMATIC CONSISTENCY
Definition: Do all reports demonstrate scientific purpose and consistent style of biological reasoning?
What to look for:
Reports clearly assess whether observed phenotypes support a hypothesized MoA
Use of biological logic at cellular or subcellular level
Use of structured reasoning patterns: "If MoA X is true, then we expect phenotype, which we
observe." ""Tii phenotype is consistent with known outcomes of X (c.g., ER stress)"
Avoidance of vague or off-topic content
Consistency depth of biological interpretation
Scoring Guide:
10: Reports are clear, coherent, and mechanistic
8-9: Mostly consistent with minor variation in depth
6-7: Some reports descriptive only
4 5: Inconsistent styles lack MoA focus
1-3: Highly divergent in purpose reasoning
2. TERMINOLOOIICA CONSISTENCY
Definition: Are technical terms, feature names, and mechanistic labels used consistently?
What to look for:
Uniform use of Cell Painting features (e.g., Texture_Entropy)
Consistent naming of biological phenomena
Standardized of MoA terms (e.g., "oxidative stress")
Avoidance of ambiguous or informal language
Quantitative descriptors (e.g., "+0.3 increase") preferred over vague phrases
Scoring Guide:
10: Terminology precise and consistent
8-9: Minor synonym use
6 7: Mixed naming for same terms
4-5; Frequent inconsistencies
1-3: Terms vague informal
47

---

Published as conference paper ICLR 2026
STRUCTURAL CONSISTENCY
Definition: Do reports follow similar structure and rhetorical flow?
What to look for:
Inclusion of key components: Mechanism verdict, evidence summary, mechanistic linkage, caveats or alternatives.
Similar order, depth, and sentence structure
Avoidance of missing or reordered sections
Scoring Guide:
10: Fully consistent structure
8-9: Minor variation in order or depth
6-7: Incomplete or reordered sections
4-5: Frequent structural differences
1-3: Highly inconsistent organization
SCORING INSTRUCTIONS
Assign score between 1-10 for each dimension
Provide concise justification (<50 words)
Return only the JSON in the following format
JSON Output Format
"Thematic Consistency":
"score": <1-10>, "justification": "<brief explanation>"
}
"Terminological Consistency"; {
"score": <1-10>, "justification"" "<brief explanation>
b
"Structural Consistency";
"score": <1-10>,
"justification": "<brief explanation>"
48

---

Published as a conference paper ICLR 2026
CORPUS EVALUATION
Table 13: FeatRank Agent's Repeatability
Temp
Features Number Avg
Features Number std
Top 5 Stable Features
0.0
18.57
2.37
AreaStape Aren, AreaStupe Eccentiiityy Texture Contrss 5 3202 256, Gramolarity 2. Gramdarity
0.0
18:20
2.26
ArasShupeAAen, AreaShupe
AreaShuge_Reeemmniing Fecentricity, Texture_ Contrass 5,002,506, Ginmolarity.. Grumdarity. 3
0.0
18.27
2.31
ArcaShape.Arcl AvaSStupeEceentiicgy Eccenticity, EexiureContraa 5, 256, Gramdanity 2. Gramulaiiy..
0.0
18.87
1.96
AreaShap_AArea. AreaShape Eccentricity ArcaStape Arca, ArcaShape Eccentricity Texture Contrast 5.02 256. Grandlatty 2.
5.02 256. Grandlaity 2, Granularity.] 3
0.0
17.17
2.28
Granularity 3
Table 16: Corpus score comparison across different temperature setting.
Temp1/Temp2
Thematic Consistency
Terminological Consistency Structural Consistency Averaged Corpus Score
0.0/0.0
8.00
7.00
9.00
8.00
0.0/0.1
8.00
7.00
8.00
7.67
0.0/0.2
8.00
7.00
8.00
7.67
0.0/0.5
8.00
7.00
8.00
7.67
0.0/1.0
8.00
7.00
8.00
7.67
S COUNTERFACTUAL PROMPT EXPERIMENTS: DOSAGE AND TIME
To address the concern that the high retrieval performance may be due to metadata correlations (e.g.. com- pound identity being indirectly inferred from time or dosage), rather than genuine multimodal alignment, we conducted controlled ablation experiments to isolate and evaluate the extent to which different textual components contribute to model performance. Specifically, we masked individual fields in the text prompts: compound name + MOA, concentration, inme-and measured the calculating the changes in retrieval accuracy.
As shown in Table masking the compound name and MOA (Experiment 1) results in catastrophic performance drop (c.g., text-to-image R@ drops from 98. to 3.50; MRR drops by nearly 90%), indicating that the model relies heavily compound-speciffe textual information. We mask the compound name and its mechanism of action (MOA) jointly, rather than separately, because these two fields are semantically correlated and often co-informative. This design choice is consistent with our setup in the drug classification experiment (Table [ and ensures a fair and aligned evaluation across experiments. In contrast, masking concentration or time results in only moderate to negligible performance degradation (e.g.. R @ I drops 3.50 when masking compound, and only to 93.00 when masking time). This pattern suggests that while the model encodes and leverages compound identity meaningfully. To further test CP-CLIP robustness to misleading contextual cues, we designed counterfactual prompt experiments in two settings:
Clean Prompt: The target classification field was masked, but all other contextual metadata (c.g.. time, channel) remained correct.
Disturbed Prompt:
The target field was masked, and unrelated metadata fields were shuffled randomly, keeping only the compound identity correct. Since we have demonstrated that compound identity has strong influence on retrieval performance.
49

---

Published as conference paper ICLR 2026
Table 17: Retrieval performance before and after masking different textual components.
Experiment
Metrie Original
Masked
6 Absolute
I Relative (%)
Experiment 1: Mask Compound Name MOA
R@1
98.70
3.50
-95.20
-96,45
R@5 100.00
16.80
-83.20
-83.20
Text-to-Image
R 100.00
29.60
-70.40
-70.40
MRR
0.9935
0.1178
-0.8757
-88.15
R@! 98.70
3.30
-95,40
-96,66
R@s 100.00
16.10
-83.90
-83.90
Image-to- text
R@ 100.00
28.70
-71.30
-71.30
MRR
0.9935
0.1102
-0.8833
-88900
Experiment Mask Concentration Only
R@1 98.70
57.00
41.70
42,25
R@s
100.00
84.00
-66.00
-66000
Text-to-Image
R@ 10
100.00
91.40
-8.60
-8.60
MRR
0.9935
0.6829
-0.3106
-31.26
R
98.70
55.20
-43.50
44.07
R @
100.00
79.20
-00800
-00.80
Image-to-Tee
R@ 10
100.00
87.40
-12.60
-12.60
MRR
0.9935
0.6617
-0.3318
-33.39
Experiment 3: Mask Time Only
R@I
98.70
93.00
-5.70
-5.78
R@S
100.00
99.50
-0.50
-0.50
Text-to-Image
R@10
100.00
100.00
0.00
0.00
MRR
0,9935
0,9590
-0.0345
-3.47
R@1
98.70
94.70
-4.00
-4.05
R@s
100.00
100.00
0.00
0.00
Image-to-Tex, Text
R 10
100.00
100.00
0.00
0.00
MRR
0,9935
0.9726
-0.0209
-2.11
50

---

Published as conference paper ICLR 2026
Table 18: Concentration Classification under Masked and Counterfactual Prompts
Concentration
Firecoxib
Opicapone
Neratinih
Hydrofumethiaaziee
Acetuminophen
Primidone
0.00316uM CLIP (disturb)
0.4866
0.3341
0.2965
0.1634
0.2298
0.2174
0.003 0.00316uM 16uM CP-CLIP (disturb)
0.4663
0.5052
0.4890
0.4887
0.2350
0.3155
0.3866
0.003 16 CLIP
0.7371
0.5203
0.6491
0.5209
0.4780
0.4237
0.4167
0.00316pM CP-CLIP
0.6866
0.6233
0,2819
0.01pM CLIP (disturb)
0.5588
0.2708
0.1757
0.3731
0.2147
0.2161
0.2612
0.01pM CP-CLIP (disturb)
0,3212
0.4409
0.3694
0.4677
0.4906
0.4598
0.2321
0.01pM
0.7887
0.5391
0.6267
0.4383
0,3473
0.01pM CP-CLIP
0.4356
0.6343
0,4913
0.5521
0.5783
0.5561
0.2635
0.0316pM CLIIP (disturh)
0.5714
0,1921
0.3178
0.2890
0.2017
112873
0,1670
0.0316GM CP-CLIP (disturb)
0.4335
0.4714
0.3928
0.7949
0.3725
0,5151
0,4015
0.0316pM CLIP
0.7437
0.4749
0.5839
0.5330
0.5033
0.5387
0,4628
0.0316pM CP-CLIP
0.5754
0,6048
0.6374
0.9205
0.4771
0.6364
0,4773
0.1uM CLIP (disturb)
0.2992
0.2652
0.3369
0.2866
0.2486
0.2232
0.1938
0.1μM CP-CLIP (disturb)
0.4660
0,2935
0.4015
0.3035
0.4051
0,3000
0.4148
0. CLIP
0.4775
0.3554
0,3690
0,5203
0.2819
0.4823
0.2661
0. CP-CLIP
0.6402
0.4459
0.5442
0.4974
0.5014
0,3832
0.4971
0.3 16 CLIP (disturb)
0.3343
0,2301
0.2776
0.2131
0.2505
0.1994
0.2961
0.316 CP-CLIP (disturb)
0.5079
0,3333
0,4027
0,4595
0.2848
0.4962
0,2642
0.316pM CL.IP
0.5401
0.4179
0.6288
0.4591
0.6056
0.4077
0.5475
0.316GM CP-CLIP
0.6272
0.4686
0.5068
0.6438
0.4399
0.5477
0.3217
1.0uM CLIP (disturb)
0.4430
0,2748
0,2505
0.5455
0.1803
0.3082
0,3073
1.0gM CP-CLIP (disturb)
0.6364
0.4411
0.4103
0.4792
0.4459
0,3974
0.3702
1.0UM CLIP
0.6611
0.5773
0.6211
0.8856
0.4909
0.5766
0.4473
1.0UM CP-CLIP
0.8137
0.6469
0.6500
0.6217
0.5133
0.4469
0.5673
3. 3.162uM CLIP (disturb)
0.3219
0,2933
0,2815
0.2119
0.2016
0,2635
0.2114
3.162uM CP-CLIP (disturb)
0.3324
0,2900
0.4479
0.2600
0.1577
0,3991
3.162μM CLIP
0.5528
0.5902
0.5459
0.6442
0.5826
0.5459
0.3255
3.162μM CP-CLIP
0.7599
0.5243
0.4577
0,5812
0,3433
0,2415
0.4454
10.0μM CLIP (disturb)
0.3871
0.4191
0.2997
0.1502
0.1696
0.1813
0,2152
10.0pM CP-CLIP (disturb)
0.6340
0.3924
0.6029
0.5010
0.3401
0.3562
0.3499
10.0μM CLIP
0.7079
0.7125
0.5729
0.6616
0.3300
0.3685
0.5673
10.0μM CP-CLIP
0.7895
0.6469
0.5874
0.5133
0.4469
Table 19: Time Comparison (clean vs disturb prompt): CLIP vs CP-CLIP
Time
Isabepilone
Melhox.saenn
Sulinpymuune
Triamierene
Miconazole
Ceriinih
Actocetamile
CLIP (distarb)
015903
0.6473
0.7212
0.6882
CP-CTIP
06980
0.9566
09101
48% CLIP (clssturt)
0.7386
0.6229
06218
0.079
0.6522
0.6927
07650
48% CP-C1.IP (dlsnmbb
0
0.9902
09338
0.8836
0.9340
0.9375
09387
4o CLIP
0.9950
1,0000
10000
1,0000
1,000
0.9950
10000
48h CP-CLIP
0.9980
1,0000
1 1000
1,0000
1,0000
1.00K0
10000
Table 20: Summary of Concentration Classification Performance: CP-CLIP vs CLIP under clean dis- turbed prompts.
Compound
CLIP
CP-CLIP Disturhed
Clean
Disturhed
A
Clean
firocoxib
0.5637
0.4163
-0.1474
0.6025
0.5038
-0.0987
opicapone
0.4244
0.2831
-0.1413
0.4631
0.3956
-0.0675
cinoxacin
0,3962
0.2875
-0.1087
0,4500
0.4231
-0.0269
neratinib
0.5131
0.2875
-0.2256
0.5269
0.4950
-0.0319
hydroflumethiazzde
0,3669
0,2025
-0.1644
0.3350
0,3650
+0.0300
acetaminophen
0.3531
0.2381
-0.1150
0,3762
0,3844
+0.0082
primidone
0.3550
0.2350
-0.1200
0.3588
0.3556
-0.0032
CP-CLIP 1 over CLIP (Clean)
+7,83% Accuracy
+7.72% F1-Scure
CP-CLIP over CLIP (Disturbed)
+54.05% Accuracy
+50.55% F1-Score
51

---

Published as conference paper ICLR 2026
Table 21: Summary of Time Classification Performance: CP-CLIP vs CLIP under clean vs. disturbed prompts.
Compound
CLIP Disturbed
CP-CLIP
Clean
Clean
Disturbed
ixabepilone
0.9950
0.7550
-0.2400
0.9975
0,9600
+0.0375
methoxsalen
1.0000
0.6425
-0.3575
1.0000
0,9575
-0.0425
sulfinpyraoone
1.0000
0.6350
-0.3650
1.0000
0.9175
+0.0825
triamterene
L.0000
0.7100
-0.2900
1.0000
0.8850
+0.1150
miconazole
1.0000
0.6625
-0.3375
1.0000
0.9325
-0.0675
ceritinib
0.9950
0,6650
-0.3300
1.0000
0,9350
-0.0650
acetohexamide
I 1,0000
0.7650
-0.2350
1.0000
0.9350
-0.0650
CP-CLIP 1 CLIP (Clean)
+0.08% Accuracy
+0.08% FI-Score
CP-CLIP CLIP (Disturbed)
+34.91% Accuracy
+35.23% F1-Score
This setup isolates the model's reliance different types of metadata and tests whether it still make accurate predictions under misleading or noisy context. For the concentration classification task (Table 20) CP-CLIP achieves a +7.72% Fl-score improvement over CLIP under clean prompts, and +50.55% F1- score gain under disturbed prompts. Similarly, for the time classification task (Table CP-CLIP maintains stable performance with only marginal degradation under disturbed prompts, achieving +35.2335 Fi-score improvement over CLIP. The detailed scores for each category are available in Table [8 and Table 19 In contrast, CLIP exhibits substantial performance drops disturbed scenarios, suggesting that more susceptible spurious correlations in the metadata.
This results confirm that CP-CLIP is not learning shortcuts based on metadata correlations but is instead capturing robust multimodal associations between visual morphology and semantic input prompts. The counterfactual experiment and contextual masking experiments effectively disentangles various sources of information, demonstrating that CP-CLIP generalize to various classification tasks even under adversar- ial or misleading contextual conditions.
T RETRIEVAL PERFORMANCE BENCHMARKS
addition to the classification accuracy metries presented in Table Table which are used for bench- marking an-distribution performance and out-of distriibitiiitoonn performance across various conditions (e.g., cell line, channel, and compound), we further report retrieval-based metrics to evaluate the effectiveness of contrastive learning models.
Specifically, we use Recall and Mean Reciprocal Rank (MRR) on held-out validation set assess both text-to-image (T-I) and image-to-text (I-TT) retrieval performance. These metrics provide a complemen- tary perspective on model alignment quality between visual and textual modalities, particularly settings where ranking-basse based retrieval is desirable.
Table summarizes the retrieval performance several contrastive models, including CLIP, SigLIP, and vari- ants of our proposed CP-CLIP, under both text-to-ine -it-to-immae and image-to-text settings (Table 23 To provide more complete picture, we also include the retrieval performance of CLOOME, shown in Table 22 While CLOOME is not general-purpose contrastive model, report its performance molecule- image and image-molecule retrieval tasks for completeness. Notably, CLOOME's retrieval limited to molecular inputs only, and is not applicable for cell-line other biological metadata queries.
52

---

Published as conference paper ICLR 2026
Table 22: Performance CLOOME on Image and Image-Molecule Retrieval Tasks.
Model
Molecule-Image
Image-Molecule
R
R@5
R@ 10
R@20
R@50
MRR
R @
R@5
R@ 10
R@20
R@50
MRR
CLOOME
95.58
99.94
99.94
99.94
99.94
0.9754
59.86
59.98
60.52
65.59
71.89
0.6063
Table 23: Context-to-Image (T-I) and Image-to PoaggettoCConter Context (I-T) retrieval performance using Recall @ K.
Model
Context-Image Image
Image-Contee Conge-Context
R@1
R@
R@
R
R@
CLIP VIT-B/16
66.8
90.80
95.40
99.38
0.7719
58.55
80.67
86.54
91.71
96.85
0.6820
SigL DELLI--YT-BII
43.85
67.38
77.44
85.75
93.19
0.5457
38.65
56.15
63.86
71.56
81.92
0.4700
DELIP-VIT-ER B/16 (D)
25.93
52.21
61.37
71.45
85.38
0,3842
20.71
39.87
48.83
59.65
74.33
0.3015
CP-CLIP VIT 8/16 (fp)
72.97
93.96
97.47
99.10
99.86
0.8213
64.20
86.55
91.99
95.73
98.75
0.7385
CP-CLIP VIT-B/16 (D)
94.69
97.87
99.74
0,8479
68.92
87.77
92.14
95.56
98.55
0.7716
CP-CLIP VIT-L/16 (D)
73.83
92.93
96.44
98.49
99.61
0.8215
64.77
85.02
90.13
93.83
97.52
0.7351
53


Reviews:
[{"review_id": "CycDuwI1gj", "reviewer": "Reviewer_k4QM", "summary": "The paper introduces CP-Agent, an agentic multimodal large language model (MLLM) system designed to analyze Cell Painting assay data to study how chemical perturbations affect cell morphology. The core of CP-Agent is CP-CLIP, pretrained with cell paint images and experimental metadata pairs by contrastive learning. Then CP-Agent leverages the context inferred from CP-CLIP to help generate structured report including experimental design and hypothesis refinement.", "questions": "1. Can you report  image-only, text-only baselines and counterfactual context (swap dose/time ) to test the performance change?\n\n2. Are agent outputs fully deterministic or do responses vary with randomness? How reproducible are the reasoning outputs across runs?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. The paper is original in explicitly treating experimental context as signal and injecting it into the text encoder for semantically image–text alignment with CLIP style training. The pretraining corpus including 1.9M image–context pairs, which is reasonable scale for Cell Painting experiments and well-suited to learning robust, context-aware representations.\n\n2. The system is presented as a clean, step-wise workflow with structured JSON outputs and rich case studies that connect reasoning steps, making the reasoning traceable for practitioners.\n\n3. CP-CLIP achieves robust MoA/treatment discrimination. The framework is portable to other imaging modalities and broader phenotypic screening use cases.", "weaknesses": "1. More ablation study is needed for the proposed method. The ablation study could include: 1) remove each context field (<CMPD>, <CONC>, <TIME>) separately, and then retrain the model, to show how the model performance change 2) remove control images from the image embedding, use only the perturbation tile, to verify the contribution of control embedding \n\n2. Table 2 results with high performance on compound recovery, may be results from the fact that the CP-CLIP is training on the meta data text, and the model can recover compound from the text context. Additional experiment to test performance change: image-only, text-only, and shuffled-context controls (time and dosage).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XHJQTCal4z", "reviewer": "Reviewer_uvt6", "summary": "This paper proposes an agent-based framework comprising two main components: CP-CLIP, a contrastive learning model for chemical perturbation–image alignment, and CP-Agent, a agentic system built atop CP-CLIP.", "questions": "See above", "limitations": "", "rating": 2, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The main novelty of this work lies in the introduction of an agentic framework specifically designed for Cell Painting images. While the underlying components (e.g., CLIP-based contrastive learning and vision encoders) are not new, applying an agent-based paradigm to organize multimodal reasoning, analysis, and interpretation within this biological context is conceptually original. This represents an interesting step toward structured and interpretable automation in cellular image understanding and drug discovery workflows.", "weaknesses": "While incorporating agentic system for Cell Painting is interesting, there are several weakness of the proposed work,\n\n\n**Insufficient survey of related work in cross-modal contrastive learning for Cell Painting**\n* The related work section lacks sufficient discussion of recent multimodal and contrastive learning methods for Cell Painting. While the paper introduces CP-CLIP as a novel contribution, several recent works within this context are not adequately discussed [1, 2, 3]. \n* In addition, the manuscript lacks quantitative comparisons with prior frameworks. Although CP-CLIP is presented as a key innovation, the evaluation primarily contrasts against generic CLIP variants and omits stronger baselines from recent multimodal or contrastive approaches. This makes it difficult to disentangle whether the reported improvements stem from the proposed context-aware token injection mechanism or from other architectural or training differences.\n\n**Method**\n* While interpretability is highlighted as a major advantage of CP-Agent, the paper lacks quantitative evaluation of this aspect (e.g., faithfulness, consistency, or attribution accuracy).\n* The authors pretrained CP-CLIP on roughly 500 distinct compounds, which is small relative to the nearly two million image–context pairs. Since standard CLIP loss assumes each image–text pair is unique, simply aligning the same perturbation would potentially lead to overfitting or biased representations. \n\n**Limited evaluation and non-standard evaluation metrics for cross-modal contrastive learning**\n* The number of chemical perturbations explored in this work is relatively limited. Prior studies in cross-modal contrastive learning for Cell Painting [1,2,3] have leveraged datasets with over 10k compounds, whereas this study appears to utilize a smaller subset (approximately 500 compounds across 1.9M image–context pairs). This limitation constrains the generalizability of CP-CLIP and raises concerns regarding its robustness across broader chemical spaces.\n* The evaluation of CP-CLIP relies primarily on cosine similarity scores--the optimization objective of CLIP loss itself--rather than standard retrieval metrics such as Recall@K. This metric choice makes the reported improvements less convincing, as higher cosine similarity naturally aligns with the training objective.\n* The paper benchmarks CP-Agent primarily against general-purpose MLLMs (e.g., GPT-5, Gemini-2.5-Pro, Claude-4-Sonnet), which are not pretrained on Cell Painting data. A more appropriate baseline would involve fine-tuning a VLM on the same Cell Painting datasets for a fairer comparison.\n* How classification is done for standard CLIP, e.g.,whether a separate classifier is trained or retrieval is done by ranking cosine similarity, is not clearly explained.\n\n**Missing Ablation Studies and Component Analysis**\n * The agentic system (CP-Agent) includes several modules (e.g., CPContext, FeatRank, StatSynth, ReportGen), yet the manuscript provides no ablation to assess their individual contributions. Without such analysis, it remains unclear which components are necessary or most influential for final performance.\n\n**Unclear dataset and evaluation setup**\n* The description of training, validation, and test splits is ambiguous. It is unclear which datasets are used for each stage and how compounds are selected for evaluation. Given the paper’s focus on chemical perturbations, omitting comprehensive datasets such as [4] limits the scope. \n\n**Presentation and clarity issues**\n* Several figures (e.g., Figures 1–4) lack sufficient captions and details. For example, the use of a GPT-2 tokenizer in Figure 1 is not clear, what constitutes “raw text” and how it is tokenized remain unclear. Similarly, later figures provide only high-level summaries without specifying details.\n\n[1] CLOOME: contrastive learning unlocks bioimaging databases for queries with chemical structures Sanchez-Fernandez et al Nature com 2023\n\n[2] How Molecules Impact Cells: Unlocking Contrastive PhenoMolecular Retrieval Fradkin et al, NeurIPS 2024\n\n[3] CellCLIP – Learning Perturbation Effects in Cell Painting via Text-Guided Contrastive Learning Lu et al NeurIPS 2025\n\n[4] Cell Painting, a high-content image-based assay for morphological profiling using multiplexed fluorescent dyes, Bray et al Nature protocal 2016", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "R73yQm8AqK", "reviewer": "Reviewer_3gDr", "summary": "This paper introduces CP-Agent, a multimodal system for analyzing Cell Painting drug perturbation experiments. The core technical idea is inserting learned embeddings for compound, dose, and time as placeholder tokens into the text encoder to align images with experimental context. The system achieves F1=0.896 on compound classification using 1.9M training pairs across three datasets which were normalized and fused in a scalable way. CP-Clip is turned into CP-Agent via subtool definitions, scaffolding and integration with GPT-5 to generate reports.", "questions": "* For the expert evaluation, what is inter-rater reliability? Are differences between models statistically significant? How do the reports compare to human-written reports or simpler template-based baselines? Without these comparisons, the scores lack context.\n* Can you clarify what makes this system \"agentic\" vs being a fixed pipeline?\n* Can you provide image-free baselines using only molecular fingerprints and metadata (no microscopy images) for the classification tasks? This would quantify how much the images actually contribute to performance.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* Clear reframing that experimental context (compound, concentration, time) is signal and should be fused into the text branch via learned token projections.\n* Principled multi-dataset curation with MoA harmonization. This is scalable and onboarding additional datasets should be cheap\n* The dataset integration and cross-normalization seems sensible and principled and onboarding new Cell Painting datasets should be cheap.\n* Clarity: I found the paper reasonably easy to read.", "weaknesses": "* “Agentic” framing: The system is a single-pass orchestrated pipeline with predefined tool calls, there is little evidence of learned action selection or closed-loop planning beyond routing, so the agentic claim feels a little overstated.\n* I found the MLLM comparisons somewhat unfair and uninformative. CP-CLIP receives >1M domain-specific training pairs while GPT-5/Gemini/Claude receive (presumably) zero Cell Painting training and only minimal 2-stage prompting. MLLMs with extensive prompt engineering or few-shot learning would be a better comparison\n* Weak expert evaluation. No inter-rater reliability metrics or comparison to baseline methods (human-written reports, template-based summaries) are provided. This makes the high scores uninterpretable without reference points.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)