Transforming Generic Coder LLMs to Effective Binary Code Embedding Models for Similarity Detection
Authors: Litao Li
Leo Song
Steven Ding
Benjamin C. M. Fung
Philippe Charland
Venue: NeurIPS 2025 poster ()
Keywords: cybersecurity
embedding
binary code
Primary area: deep_learning

Abstract: Cybersecurity and software research have crossed paths with modern deep learning research for a few years. The power of large language models (LLMs) in particular has intrigued us to apply them to understanding binary code. In this paper, we investigate some of the many ways LLMs can be applied to binary code similarity detection, as it is a significantly more difficult task compared to source code similarity detection due to the sparsity of information and less meaningful syntax. It also has great practical implications, such as vulnerability and malware detection. We find that pretrained LLMs are mostly capable of detecting similar binary code, even with a zero-shot setting. Our main contributions and findings are to provide several supervised fine-tuning methods that, when combined, significantly surpass zero-shot LLMs and state-of-the-art binary code similarity detection methods. Specifically, we up-train the model through data augmentation, translation-style causal learning, LLM2Vec, and cumulative GTE loss. With a complete ablation study, we show that our training method can transform a generic language model into a powerful binary similarity expert, and is also robust and general enough for cross-optimization, cross-architecture, and cross-obfuscation detection.

Transforming Generic Coder LLMs to Effective Binary Code Embedding Models for Similarity Detection
Litao Li" Queen's University litao. wiiqueensu..co ca
Leo Song" Queen's University 1eo. songSqueensu.cm ca
Steven H. H. Ding
McGill University steven h. ding@mcgill.....mmmm ca
Benjamin C.M. Fung McGill University ben. fung@mogill..... ca
Philippe Charland
Mission Critical Cyber Security Section, Defence R&D Canada philippe. char Sharlandddddc-rdee ca
Abstract
Cybersecurity and software research have crossed paths with modern deep learning research for few years. The power of large language models (LL.Ms) in particular has intrigued us to apply them to understanding binary code. In this paper, we investigate some the many ways LLMss can be applied to binary code similarity detection, it is significantly more difficult task compared to source code similarity detection due the sparsity information and less meaningful syntax. It also has great practical implications, such as vulnerability and malware detection. We find that pretrained LLMs are mostly capable of detecting similar binary code, even with zero-shot setting. Our main contributions and findings to provide several supervised fine-tuning methods that, when combined, significantly surpass zero-shot L.LMs and state-of-the-art binary code similarity detection methods. Specifically, we up-train the model through data augmentation, translation-style causal learning, LeM2 and cumulative GTE loss. With complete ablation study, we show that our training method can transform a generic language model into powerful binary similarity expert, and also robust and general enough for cross-optimization, cross-architecture, and cross-obfuscation detection.
Introduction
LLMs are almost ubiquitous in everyday life, fueling many powerful applications. Cybersecurity and software researchers have utilized language models to solve complex tasks, such as code generation, summarization, and similarity detection. In this paper, we investigate the benefits that LLMs can bring to binary code modeling. In particular, binary code similarity detection (BCSD) and retrieval difficult due the sparsity of information, loss of semantics and structures, and its limited syntax compared to source code. Moreover, the same source code can be compiled into many formats of binary code using different compiler settings and environments, including optimization levels, machine architectures, compiler software, and obfuscation techniques. Existing LLMs [Roziere et al. 2023 Radford 2018 can somewhat perform binary code analysis with their large training data of trillions of natural language and code tokens, but the proportion of binary code is small. Downstream tasks like similarity detection and vulnerability detection are still difficult and require fine-tuning. Some open-source coder models like CodeT5 [Wang el al.) [021] and GrapRTodeBERT (Guo el al. 2020 even lack the training of binary code, making zero-shor useless for binary analysis.
'Equal contribution.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Many binary code models are trained from scratch [Yang et al. 2021] Tian et al. 2021 Yu et al. 2020] and their sizes are much smaller compared to LLMs. They typically focus on opossioptiiz retrieval and fail to generalize to diverse compiler settings. Other works take pre-trained L.MMs (Tan et al.) 2024, Wang et al. 2022 2024] and apply custom fine-tuning techniques for binary code matching. However, most of these approaches rely on either closed-source models like GPT Radford 2018] or large model sizes. This leads to scalability issues when the computational resource is a constraint, which realistically is the case with small research labs even companies. We want to explore effective and efficient LLM fine-tuning for binary code embedding and matching.
In this work, we address the aforementioned problems and propose EBM (Effective Binary Matching), our novel training framework, including carefully chosen data augmentation and fine-tuning processes, to uptrain a generic LLM into a binary code embedding and matching expert. We show in our experiments that L.MMs have become dominant enough that even zero-shot models can surpass well- trained binary code models. With our fine-tuning applied, EBM can significantly increase the mean reciprocal rank (MRR) by 10% to 70%, depending on the tasks. We also provide a comprehensive ablation study to prove and emphasize the importance of each training process. The dataset and code can be accessed on Github Our major contributions are:
We propose multi-training framework to utilize the power of generic LLMs specifi- cally for BCSD. We target different compiler settings, including cross- coossoopiimiztiinn,, cross- architecture, and cross-obfuscation, to build a general and effective similarity retriever.
To combat the lack of assembly code LLMs, we uptrain the generic LLM to binary code-specific model. Particularly in cross- aresitecche similarity detection, this allows for significantly better translation between different syntaxes.
We utilize Ve to build refinement of the assembly tokens, which encodes better semantics through a masked next token prediction task.
In the downstream contrastive learning task, we propose an enhanced version of InfoNCE loss to utilize all available samples within the batch. This is particularly useful when computing resources are limited and large models are trained.
We build and compare various baselines to evaluate the effect of different language/coder models and state-of-the-ee of-te-arr BCSD models. In our similarity retrieval evaluation, ap- proach outperforms all benchmarks for both datasets.
We conduct thorough ablation studies and in-depth analysis to investigate all our training tasks and how they contribute to the similarity retrieval result. We show that all our training tasks are essential and can improve retrieval performance.
2 Related Works
Traditional BCSD Without data-driven or learning-based methods, code similarity detection is traditionally conducted using static analysis, dynamic analysis, code-based algorithms. Static analysis usually involves graph matching (Dullien and Rolles] 2005) [ourquin el al. 2013]. where control flow graphs are extracted from assembly code and compared using algorithms or user-defined heuristics. Dynamic analysis instead leverages runtime or symbolic execution to investigate program behavior [Pewny et al.) 2015: Xu et al., 2020 Egele et al. 2014 Moser et al. 2007]. Execution traces or paths can be profiled for either manual or automated comparison using distance- or statistics- based methods. Code-based algorithms rely on patterns of the actual binary code, which can be opcode or instruction code. Various ways of analyzing code strings include distance metrics Waattt---tttaannnna Levenshtein distance) [ao et al. 2008). N-Gram matching [Rosenblum et al.)
2008 and frequency analysis [Santos et al. 2013).
Machine Learning-basee BCSD is effective when a large amount of training data is available. The methods can generally be categorized into text-based, structure- based, or combined. based models treat assembly instructions opcodes tokens and feed them into various models such as unsupervised [Ding et al. 2019], RNN-based [Massarelli et al. [019b]] Yang et al.) 2021: Tian et al.
2021
BERT-based Koo et al. [222 202T:AM Ahn et al. 2022 Li et al. [202a]] and LLMM-baed Tan et al.
2024
Wang et al. 2022 Liu et al. 2023 Wang et al. 2024 Such models focus on the semantics
Github Link
2

---

and syntax of the binary code to encode binary programs. In structure-based models, some form of graph structures, such abstract syntax trees, control flow graphs, or custom graphs. is parsed from binary code. Then appropriate machine learning techniques can be applied, including tree-L.TT'M [Tai
al. 2015] Yang ef al. 2021] and graph neural network [Xu el al.) 2017 Li et al. 2019]. Instead of utilizing complicated language models for word embedding, these models often apply Word2
style [Mikolov et al. 2013) representation learning to obtain node features, Other research works
combine code and structure information into streamlined learning process [Massarelli et al. [201 9a
Gao et al. 2018 Yu et at. 2020). When training the networks, contrastive learning is the most popular technique for supervised learning models differentiate positive and negative samples, which fed into the network as a pair of input data. Many approaches also utilize a siamese architecture to simultaneously encode pairs under the same set of model parameters. Recently, many similarity detection methods have adopted the InfoNCE loss [Oord et al. 2018 8) to enhance the discriminative power of negative pairs by maximizing the mutual information. it has been shown to be quite effective and can sometimes lead to large performance gains, especially for LLMs where batch size tends to be small. Readers may refer Appendix. [ for additional information about LLM architecture and foundation models.
3 Methodologies
We define the task of BCSD as retrieving the most similar function from pool of binary functions. Similar binary functions created from the same source code with different compiler settings. In our case, the settings can be aptimization-basee compiler-based. The input binary function and the output is list of functions, ranked by cosine similarity. For evaluation purposes, the pool contains exactly one similar function. The complete training undergoes four processes: data augmentation, translation learning, embedding training, and contrastive learning.
obfuscation-base based, architecture-basee
and
BLK
a0
gec
a3
mapp
BLK
d23
< 608 >>
Target
Cousal Model Max
Input
conect
11
2/12
BLK
Zin
o0
B86
o3
mips
Z21
BLK
F2n
X1
Language Tokens of X;
Language Tokms of X,
X2
Figure 1: The input and target of causal uptraining are shown here. The input contains sampled pairs of semantically identical assembly functions, where BLK tokens added for structure awareness, and language tokens are added for translation awareness. The training follows an autoregressive manner, where previous tokens are used to predict the next unseen token.
3.1 Enabling Translation and Structure Awareness
Assembly code is intermediate code acquired from disassembler, such IDAPRO To clean the data, all addresses, strings, and bytes are replaced by special tokens: addr, byte, str. The special tokens help reduce noise when addresses or bytes are arbitrary and also reduce context length after tokenization. We flatten the original data structure of assembly code, which contains basic blocks and instruction streams, into single sentence. Doing this removes the requirement for hierarchical architecture and layered attention to process the inputs. This allows us to directly use existing pre-trained language and coder models. The drawback that a single sentence no longer contains structural information regarding the semantically bound sections of the code (i.c.. basic blocks). We propose to simply add special BLK token between basic blocks to increase structure awareness during training. This enables the model to consider assembly structure, which can be easily learned in our uptraining processes.
httpss//hexxrrys.co//ddep
3

---

Cross- aoosstarchitectre similarity detection is a difficult task that, to our knowledge, has not been ex- tensively studied along with other compiler settings like cross-opiimization. Assembly code can have a completely different syntax based on the machine architecture during compilation. Different architectures can be seen as different "languages" in this context. Inspired by [Conneau and I Lample) 2019), we add several additional tokens to indicate the "language" information of each binary function, which corresponds to the optimization level, compiler, obfuscation, and architecture. Adding such information has proven to be effective, especially for cross-architecture detection.
3.2 Binary Translation Continual Training
Modern coder models aim to assist humans in coding tasks like code generation and summarization. Although assembly code is included in some of the existing models, it is not as well understood to the extent of source code languages like Python Java. A cheap and effective way uptrain such coder models with assembly code is to use an autoregressive setup, which trains the model to predict the next tokens.
We treat the task similarly to translation in natural language, as different compilers produce various syntaxes depending on the settings. This especially true in cross cross- architecture similarity detection, where it is often not possible to retrieve similar code between machines with architectures like x86 and PowerPC. We only sample semantically identical function pairs (X1; 1X1;X) X2) and concatenate them into a single sequence X = concat(X1, X2) as imput to the model. A pre-trained coder AR model is initialized with weights W and fine-tuned with the causal objective and update rule:
n
LAR Σ
log P(a xee))
W(++) = = V (xLAR(X:W())
(2)
Where 17 is the learning rate. As discussed in section 3.1 we manually add "language" tokens to all assembly functions as an indication of their compiler settings. For training the translation task, we place these tokens in between the concatenation of function pairs, acting as transition. We give an illustration of the input and output format in Figure |
It is important to let the model access these tokens during training before predicting the second sentence, X2, as provides generalization due to knowing what settings to expect and predict. It is important to reiterate that these language tokens are unavailable during the inference phase.
3.3 LLM2Vec
BERT Devlin 2018 models have been proven effective in embedding tasks, largely due to their bidirectional attention mechanism, which enables them capture contextual information from the whole sequence [Reimers] 2019; Shi et al.) 20223. However, the landscape of NLP has shifted significantly in recent years to LLMS trained with next token prediction, such as GPT-2 [Radford et al.) 2019], Llama [Touvron et al. 2023). and their successors. These models, typically based on unidirectional architectures, have achieved state-of-the-ee performance across a wide range of tasks, including embedding tasks, often surpassing traditional bidirectional models in scalability and generalization (Wang et 2023a].
masked
LNTTP
Σ
P(x:) log P(,.)))....,
(3)
To bridge the gap between the strengths of causal models and the benefits of bidirectional architectures.
we adopt an innovative approach called LLM2Vee [BehnamGhader et al.) 2024]. As depicted in Figure [ this method leverages the pre-trained knowledge of causal models, while enabling bidirectional attention and being trained through masked next token prediction task (MNTP). Specifically, we initialize the model with the same weights from a pre-trained causal model and enable bidirectional attention during forward propagation. The model is then fine-tuned using MNTP loss, defined Equation where random token the sequence masked and predicted based on the surrounding context.

---

3 3
0
I1
T2
mask
E4
Figure 2: An overview of the Ve approach. The method adds bidirectional attention (red lines) based on future context to the original causal model (gray lines). fine-tuned with masked next token prediction (MNTP). The model can be further fine-tuned with contrastive learning approaches such InfoNCE and GTE losses, for specific downstream tasks.
In essence, this approach constructs a BERT-like model from pre-trained causal model rather than training a BERT model from scratch. This strategy offers several key advantages. First, it allows us to leverage the existing pre-trained L.LMs, reducing the need for costly and time-consuming pre-training. Most importantly, it achieves better performance than training a BERT-style model from scratch.
3.4 Cumulative GTE Loss
The InfoNCE loss (Equation 4) is widely used objective function in contrastive learning. encourages the model learn distinctive representations by contrasting in-batch negative ones, Incorporating additional contrastive pairs can further enhance the learning process [Li al. 2023b Wang al. 2022]. We denote k; be the embedding the source binary function (key) in batch i, to be the embedding of the target binary function (query) in batch i, and as hyperparameter for temperature.
positive pair against
1 N
exp(sim((s) qs ks)/T)
LInONNE
log
Σ) exp(sim ky)/T)
The GTE (general text embedding) loss generalizes the InfoNCE loss by introducing additional contrastive terms, which include query-key pairs as well as query-query, key-key, and key-query pairs. This broader set of contrasts helps the model capture richer relationships within the data. The GTE loss is defined as:
1 LOTE
exp(sim(qi: kT)/T)
log
N
Z
Z
ks)/T
(5)
sim(gg,k:/T
si)(k)
iti
j
Based on the GTE loss, propose the cumulative GTE loss (cGTE), which aggregates embed- ding representations from multiple distributed models and accumulated inputs. The cGTE loss is formulated as:
g 11.. 1 k: 11..111
(6)
LEGTE GTE(g, k))
where the superscript denotes accumulated vector representation from distributed models batches. The backpropagation is halted until number of inputs are met. The cGTE loss syncs gradient backpropagation generated from asynchronous input batches and distributed models the same model weights. cGTE creates more contrastive pairs with limited computing resources.
5

---

4 Training and Experiments Setup
4.1 Backbone Model and Baselines
We use Qwen2.5-Coder-0.55 (Hui al. 2024] the backbone model for all training processes.
We compare against existing non-LLM methods, including PalmTree [Li et al. 202T]. SAFE [ass]
sarelli et al. 2019b]. OrderMatters (Yu et al. 2020]. and Vem2 (Ding al. 2019]. We also
evaluate several state-of ofrthh-rrt large coder models, such as GraphCodeBERR [Guo et al. 2020
CodeT55 [Wang et al. 2023b), Owen2.5-Coder: 1.5B [Hui et al.)
| 2024]. Qwen3 Yang et al. 2025
and CodeGemma [Team et al. 2024]. All baseline models, except for Qwen2.5-Emm Emm,, are fine-tuned on the same training set using contrastive learning. F.ven,.--Emb, being a pre-trained embedding model, is included as a baseline to indicate the differences between programming language code and binary code. As model is trained on Qwen2.5-Coder-O.nm the Swen2.5-Coder- baseline is good candidate for an ablation study.
4.2 Dataset
We use two datasets for training and evaluation. The first contains multiple libraries written in C and is used by other existing works (Ding et al.) 2019]. We manually compile each library using different optimization levels (O0, 01, O2, and O3), compilers (GCC and Clang), architectures (x86, PowerPC, Arm, and MIPS), and obfuscations (none, substitution, flatten, bogus control flow, and all). The training libraries used include BusyBox, Coreutils, Curl, ImageMagick, PuTTY, and SQLite. The evaluation libraries are GMP, LibTomCrypt, and OpenSSL. We separate the libraries for the purpose of size for the retrieval evaluation is 1,000. The second dataset is BinaryCorp [Wang et al.) [022]. which is constructed based on ArchLinux and Arch User Repository BinaryCorp only contains optsmiiptinn functions. We use the same training and testing splits and pool size of 10,000 as the original paper for comparison.
evaluation, which a common practice in binary code retrieval. The pool
4.3 Training Details
We train our models on a machine with NVIDIA RTX 6000 GPUs (24 GB). Each computing card can contain one batch of 4 entries with 512 tokens as the sequence length. We use AdamW as the training optimizer and learning rate of 10-5, Every model is trained with epochs and warmed up for 1000/ /batch size] steps. The best model weights are based on the best validation contrastive loss.
5 Evaluation
In this section, we evaluate both datasets with LLM--bsedd and non-LLMM baselines. The evaluation metries are mean reciprocal rank (MRR) and recall Due to limited space and large number of combinations for our evaluation setup, we only include a subset of the retrieval tasks. The remaining evaluation details, including additional tasks, training, and inference time, are the Appendix B
MRR
1ecall@
Models
00.03
00:01
00,02
01,03
02,03
Avg.
00,03
00.01
00,02
01,03
02,03
Avg.
SAFE
0.189
0.189
0.171
0.193
0.063
0.063
PalmTTrce
0.023
0.020
0.019
0.314
0.878
0.251
0.008
0.006
0.007
0.184
0.676
0.176
Asm2Vec
0.444
0.494
0.460
0.535
0.563
0.499
0,234
0.290
0.252
0.343
0.376
0.299
OnderMatters
0.006
0.006
0.008
0.006
0.006
0.006
0.000
0.001
0.002
0.001
0.000
0.001
GmphCodeBERT (125M)
0.636
0.757
0.673
0.792
0.920
0.756
0.560
0.694
0.602
0.722
0.895
0.695
CodeTS- (110M)
0.604
0,650
0.629
0.830
0.893
0.721
0.532
0.572
0.552
0.783
0.869
0.662
Qwen2,5-Emb (1.58)
0.569
0.648
0.573
0.773
0:907
0.694
0.498
0.578
0.505
0.699
0.875
0.631
Qwen2.5-Crder (1.5B)
0.758
0,881
0.864
0.936
0.849
0.706
0.842
0.757
CodoGemma (2B)
0.763
0.888
0.833
0.866
0.931
0.856
0.696
0.840
0.778
0.821
0.905
0,808
RBM (0.5R)
0.850
0.942
0.902
0.933
0.955
0.916
0.793
0.903
0.850
0.887
0.929
0.872
Table 1: Evaluation on cross-optimization settings (00, 01, 02, and 03) with pool size of 1,000.
hhups//arrchiiuux. wwwp://www..g////.eee/////
"htpss//aur.archlinu.

---

5.1 Cross-optimization Evaluation
Table I highlligts the performance comparison between various baselines and our approach for cross-optimization retrieval. L.Mss, with zero-shot setting, can outperform existing non-LLM methods, which have been state-of-the-ar in the past for binary similarity detection. However, they still struggle with more difficult retrieval tasks like [O0, 03]. EBM can consistently outperform other models across all optimization settings, achieving the highest MRR and Recall @ scores, while also improving [O0, O3] significantly.
5.2 Cross-architecture
Pross-architectrry has been shown to be the most difficult task for both LLMs and non-LLM baselines, as shown in T Even with contrastive fine-tuning, Qwen-2.5- 2.5-Corr,, which has 1.5 billion parameters, fails to effectively retrieve cross- arosttcrcite functions. Due to the difference in syntax, matching the semantics across several languages is nearly impossible with limited training. Our method outperforms all baselines by large margin. This further proves the significance of the causal training process that contributes to the improvement in cross-architecture generalization. We will show firm results in ablation studies.
Models
MRR
Recall@1
Arm, x64
PowerPC, x64
MIPS, x64
Avg.
Arm, x64
PowerfC, x64
MIPS, x64
Avg.
SAFE
0.239
0.187
0.196
0.208
0.063
0.063
0.063
0.063
Palm'Tree
0.037
0.036
0.018
0.031
0.017
Asm2Vee
0.242
0.293
0.417
0.317
0.085
0.113
0.231
0.143
OrderMatters
0.007
0.007
0.007
0.007
0.002
0.000
0.001
0.001
GraphCodeBERT (125M)
0.067
0.269
0.495
0.277
0.037
0.204
0.419
0.220
CodeT5+ (110M)
0.056
0,303
0.462
0.274
0.035
0.227
0.392
0.218
Qwen2.5- 5-mb (1.SB)
0.039
0.059
0.409
0.169
0.031
0.035
0.331
0.132
Qwen2.5 Coder (1.5B)
0.256
0.481
0.428
0.334
CodeGemma (2B)
0.293
0.581
0.548
0.474
0.208
0.479
0.432
0.373
EBM (0.5B)
0.783
0.792
0.859
0.811
0.675
0.703
0.784
0.721
Table 2: Evaluation on cross-architecture settings (Arm, x86-64, PowerPC, and MIPS) with a pool
size of 1,000.
5.3 Cross-obfuscation
Obfuscation techniques introduce complex and confusing variants to binary function and considered the most difficult task for binary retrieval. In Table 3 all models have shown undesirable results for [all, none], which retrieves vanilla functions from functions with all three obfuscation techniques applied. Our method still outperforms all other baselines. While investigating the functions in this task, we often found that the obfuscated function can be 10x as large the vanilla function. With limited sequence length, this result is somewhat expected. We suspect a much longer sequence length can alleviate this constraint and improve performance.
5.4 BinaryCorp
Table 4 shows the evaluation of the BinaryCorp dataset. Among all benchmarks, j'rrass [Wag et al. 2022] and CLAP [Wang et al. [024] are recent LLM-based models for BCSD. They have shown great results compared to the other methods. CLAP has the closest performance compared to method, though has a few drawbacks. Firstly, CLAP relies on GPT3.5, closed-source model, to generate natural language explanations. It is also costly and difffiuult to scale the model. CLAP uses Llama 13B and 30B as the backbone model for supervised fine-tuning, which can become a constraint for resource- limited scenarios.
In conclusion, EBM is general approach that can adapt any pre-trained coder model, such as 0.5B model used in our study, for binary embedding matching tasks. This methodology eliminates the need for external expert models, such as GPT, or auxiliary inputs, such as control flow graphs Thus, it simplifies the data pipeline and training process and makes it more accessible and easy to use.
7

---

MRR
@
Models
all, none
none, beff
sub,
Avg.
all, none
none, bef
sub,
Avg.
SAFE
0.256
0.181
0.264
0.234
0.0625
0.0625
0.125
0.083
PalmTree
0.122
0.289
0.215
0.209
0.060
0.200
0.083
0.114
Asm2Vec
0.200
0.181
0.264
0.215
0.069
0.063
0.125
0.086
OrderMatters
0.008
0.006
0.007
0.007
0.001
0.001
0.001
0.001
GraphCodeBERT (125M)
0.230
0,648
0,479
0,452
0.163
0.557
0.391
0,370
CodeT5+ (110M)
0.176
0.619
0,372
0.389
0.118
0.539
0.291
0,316
Qwen2.5- 5mb (1.5B)
0.288
0.630
0,466
0.461
0.213
0.538
0,375
0.375
Qwen2.5-Coder (1.5B)
0.391
0.719
0,580
0,563
0.301
0.637
0,491
0.476
CodeGemma (2B)
0.454
0.796
0,571
0.607
0.356
0.735
0.473
0.521
EBM (0.5B)
0.531
0.815
0.784
0.710
0.454
0.738
0.713
0.635
Table 3: Evaluation on cross-obfuscation settings (all obfuscations, bogus control flow, flattened, and substitution) with a pool size of 1,000. The proposed EBM model outperforms all baselines by an absolute margin of over 15% in both MRR and Recall @1 metrics.
MRR
Recall
Models
00,03
01.03
02,03
00,Os
02,0s
Avg.
00,03
01,03
02,03
00,0s
02,0s
Avg.
Gemini
0.037
0.161
0.416
0.049
0.195
0.172
0.024
0.122
0.367
0.030
0.151
0.139
SAFE
0.127
0.345
0.643
0.147
0.377
0.328
0.068
0.247
0.575
0.283
0.250
OrderMatters
0,062
0,319
0.600
0.075
0.233
0.258
0.040
0.248
0.535
0.040
0.158
0.204
Vee
0.072
0.449
0.357
0.046
0.367
0.589
0.296
PalmTree
0,130
0.403
0.677
0.152
0.496
0.372
0.080
0.326
0.609
0.097
0.420
0.306
Trans (Zero Shot)
0.137
0,490
0.693
0.182
0.513
0.403
0.088
0.412
0.622
0.122
0.430
0,335
Trans (Finetune)
0.475
0.663
0.731
0.539
0.664
0.614
0.376
0.580
0.661
0.443
0.585
0.529
CLAP
0.764
0.903
0.941
0.813
0.877
0.860
0.719
0.875
0.920
0.774
0.847
0.827
0.779
0.911
0.955
0.909
0.872
0.725
0.882
0.849
Table 4: Evaluation on the BinaryCorp-33 3M dataset with pool size=10,000
5.5 Similarity Threshold
In the preceding sections, we assumed the presence of at least one matching pair in the search pool. However, in real-world scenarios. top-1 selection often yields high false positive rates. To address this limitation, we introduce a similarity threshold based on the distribution of similarity scores in the training data. reports the mean and standard deviation for both similar and non-similar optimization pairs. We set the threshold at 'mean std' for each model. The accuracy is then computed as the proportion of queries without matching pairs that correctly fall below this threshold, formulated as "# of queries with no matching pair/ / total number of test queries." EBM continues to outperform the best baseline model. The results for the remaining experiments are in the Appendix.
Source
Dest
Accuracy
Mean + STD Similar pairs EBM
Mean STD of Non-similar pairs EBM
CodeGemma
EBM
CodeGemma
CodeGemma
60
0l
0.472
0.719
0.904 = 0.136
0.910 # 0.106
0.207 # 0.220
0.180 = 0.180
60
02
0,387
0.676
0.868 = 0.155
0.879 # 020
0.208 a: 0.212
0.178 # 0.2
60
03
0.236
0.458
0.830 = 0.195
0.833 # 0.173
0.210 # 0.213
0.184 # 0.4
01
02
0.675
0.831
0.937 = 0.092
0.952 2 0.076
0.176 + 0.213
0.164 = 0.173
0l
03
0l438
0.675
0.899
+ 0.151
0.916 = 0.134
0.168 # 0.221
0.170 0.176
02
03
0.618
0.793
0.955 0.1
0.965 # 0.133
0.183 + 0.217
0.174 = 0.173
Table 5; Model Accuracy for non-match and Similarity Metries Comparison
5.6
Comparison of Base and EBM-trained Models
To further assess the effectiveness of EBM, we perform the same evaluation on the base model for similarity detection with Qwen2.5-0.5B and Qwen3-1.7B models to understand the difference between pre- and post-finetuning. Table. [6 illlstrrates the average MRR and Recall @ across cross- optimization, arssttrchite and cross-offuscation. Without training, the base model struggles to
8

---

understand the semantics of assembly code, especially for cross cross- architecture and cross-obfuscation tasks. EBM clearly enables significant uplift for similarity detection and boosts the MRR up to 16X and Recall @ I up to 30X.
Models
Average
Average Recalle
Cros--Otimization
Cross Obfoscation
CressAArchitectue
Cros-Opeimirationn
Cross- Obfoscation
CoossArchinecture
Qwm22 10121-5Coe F-Coder-O 5B (Base)
0.485
0.226
0.050
0.444
0,184
0.026
Owen2.5 5 Coder 0.58 (EBM)
0,926
0.885
0.668
0.733
Qwen3-1.77
0.284
0.111
0.015
Table 6: Evaluation on the Base Model and Finetuned Model
Ablation
We illustrate ablation studies that investigate the effects of data augmentation and fine-tuning pro- cesses. All parts our method must be included, as they provide either generalization or significant performance improvement for tuned LLM understand the context, semantics, and structures of assembly code.
0.8
d all abi.nane
MMR
H
0.2
00,03
ar, am,mips
arm,32
armumpss
orm.powerpc
powerpc.6
64,mips
augmentation
w/b causal
E3M
llm2vec
EDM
(a) Causal training ablation study
(b) Data augmentation, LLM2 and cGTE ablation.
Figure 3: 3a shows the MRR for two versions of EBM, with and without causal training. Cross- architecture retrieval is particularly sensitive to causal training, as it enables better translation between different languages. In (3b), EBM significantly improves the MRR compared to ablated models. LLM2Vec contributes the most the increase.
6.1 Assembly Code Data Augmentation
We provide an ablation study on the data augmentation process, which includes data cleaning and adding special tokens for translation and structure awareness. The details can be found in Section 3.1 In Figure 3b we plot the MRR when data augmentation removed. has a major improvement in arsstarchitecture and cross-obfuscation retrieval. Since these tasks considered more difficult than cross- optssssottiiztiionn retrieval, enhancing the quality of input data with structural and language information is key process. The assembly code also becomes less noisy after the tokenization, which is often a constraint for cross-obfuscation detection.
6.2 Causal Training
Causal uptraining is the first process to fine-tune generic LLM. In our analysis, it has an evident impact on improving cross-acchitecture retrieval and less impact on other retrieval tasks. In Figure Ba we plot the cross-architecture evaluation for the two versions of our model, with and without causal uptraining. The MRR difference varies from 2% to 6%, which significant improvement.
6.3 LLM2Vec and eGTE
M.L2Vec is our BERT-like uptraining that converts a pre-trained decoder model to an embedding model by enabling bidirectional attention. cGTE is enhanced version of InfoNCE loss during

---

09232
(a) EBM
(b) w/o causal
(c) w/o LLM2Vec
(d) w/o cGTE
Figure 4: t-SNE visualization of embedding distributions. Each color corresponds to binary function under different settings, represented by distinct shapes. That means, embeddings for the same function should cluster by color instead of shapes.
contrastive learning that introduces additional contrastive terms. Both are crucial to improving the retrieval performance. as shown Figure 2 We show the difference in MRR for [O0,03], [arm, and [all obf,none]. as they represent pross-optimization, cross- pooss-rcchttectre, and cross- obfuscation, respectively. Overall. LLM2Vec contributes the most to the performance gains for our training and can increase MRR by 2 to 4 times. cGTE is useful in complex tasks like cross-obfuscation, where introducing more negative contrastive pairs enables efficient training.
6.4 Embedding distributions
We provide visualizations for the embeddings of similar binary functions. The t-distributed Stochastic Neighbor Embedding (t-SNE) is used to reduce the embedding dimension In Figure we plot 20 functions that are compiled using different optimization levels and architectures, represented in different shapes. Similar functions are grouped by colors, whereas the compilation setting is grouped by shapes. After dimensionality reduction, EBM produces tight groups that can be easily separated from the others. This illustrates that our final embeddings are effective and contain rich semantics.
Discussions and Limitations
LLMs have proven be powerful for many learning tasks. To bridge such technological gap in the binary software and security domain, we propose EBM, a multi-phase uptraining framework for binary code embedding and similarity detection. EBM is flexible framework for any generic language or coder models, and is effective in outperforming all state-of-the-an benchmarks. Our data augmentation is carefully engineered to provide enhanced awareness of the language and structure information. The causal uptraining module leverages such information and pairs of data to translate different compilation settings specifically enable cross-architecture learning phrased as translation task. M.LM2Vec performs a BERT-like task to learn the context and provide semantic-rich embeddings. Lastly, custom cumulative GTE loss efficiently capture more negative relationships during contrastive learning, significantly improving the semantics embedding in a resource-limited training environment. Our comprehensive evaluation shows that EBM outperforms all benchmarks in all tasks. The ablation study further indicates the effectiveness of all training processes.
Limitations Our limitations include the lack of evaluation on larger LLMs (due to our limited computational resources), relatively small training data compared to state-of-the- coder models, and lack of potential downstream tasks for binary code, such malware and software vulnerability detection. We would like to lay a foundation for future work to apply our method or variants of that address these limitations. We believe that such approach has large potential in achieving the state-of-the-as going forward.
10

---

References
Sunwoo Ahn, Seonggwan Ahn, Hyungjoon Koo, and Yunheung Pack. Practical binary code similarity detection with bert-based transferable similarity learning. In Proceedings of the 38th Annual Computer Security Applications Conference, pages 361-374, 2022.
Parishad BehnamGhader, Vaibhav Adlakha, Marius Mosbach, Dzmitry Bahdanau, Nicolas Chapados, and Siva Reddy. LIm2vec: Large language models are secretly powerful text encoders. arXiv preprint arXiv:2404.05961 2024.
Martial Bourquin, Andy King, and Edward Robbins. Binslayer: accurate comparison of binary exe- cutables. In Proceedings of the 2nd ACM SIGPLAN Program Protection and Reverse Engineering Workshop, pages 1-10, 2013.
K Clark. Electra: Pre- Pre-training text encoders as discriminators rather than generators. arXiv preprint arXiv:2003. 10555, 2020.
Alexis Conneau and Guillaume Lample. Cross-lingual language model pretraining. Advances in neural information processing systems, 32, 2019.
Jacob Devlin. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint 1xXi:18110008003 2018
Steven HH Ding, Benjamin CM Fung, and Philippe Charland. Asm2vec: Boosting static representa- tion robustness for binary clone search against code obfuscation and compiler optimization. In 2019 ieee symposium security and privacy (sp), pages 472-489 IEEE, 2019.
Thomas Dullien and Rolf Rolles. Graph-based comparison of executable objects (english version). Sstic, 5(1):3, 2005.
Manuel Egele, Maverick Woo, Peter Chapman, and David Brumley. Blanket execution: Dynamic similarity testing for program binaries and components. In 23rd USENIX Security Symposium (USENIX Security 14), pages 303-317, 2014.
Debin Gao, Michael K Reiter, and Dawn Song. Binhunt: Automatically finding semantic differences in binary programs. In Information and Communications Security: 10th International Conference, ICICS 2008 Birmingham, UK, October 20-22, 2008 Proceedings 10, pages 238-255 Springer, 2008
Jian Gao, Xin Yang, Ying Fu, Yu Jiang, and Jiaguang Sun. Vulseeker; A semantic learning based vulnerability seeker for cross- pross-platform binary. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, pages 896-899, 2018.
Daya Guo. Shuo Ren, Shuai Lu, Zhangyin Feng. Duyu Tang, Shujie Liu, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, et al. Graphcodebert: Pre-training code representations with data flow. arXiv preprint 2009.00000000666 2020.
Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409. 12186, 2024.
Hyungjoon Koo, Soyeon Park, Daejin Choi, and Taesoo Kim. Semantic-aware binary code represen- tation with bert. arXiv preprint mxXiv:2106.0544 2021.
Zhenzhong Lan. Albert: A lite bert for self-supervised learning of language representations, arXiv preprint /90v: /942, 2019.
Yujia Li, Chenjie Gu, Thomas Dullien, Oriol Vinyals, and Pushmeet Kohli. Graph matching networks for learning the similarity of graph structured objects. In International conference on machine learning, pages 3835-3845. PMLR, 2019.
Xuezixiang Li, Yu Qu, and Heng Yin. Palmtree: Learning an assembly language model for in- struction embedding. Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, pages 3236-3251, 2021.

---

Li Tao Li, Steven HH Ding, and Philippe Charland. Gental: Generative denoising skip gram trans- former for unsupervised binary code similarity detection. In 2023 International Joint Conference on Neural Networks (IJCNN), pages 1-8. IEEE, 2023.
Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. Towards general text embeddings with multi-stage contrastive learning. arXiv preprint arXiv:2308.03288) 03281,,, 2023.
Guangming Liu, Xin Zhou, Jianmin Pang, Feng Yue, Wenfu Liu, and Junchao Wang. Codeformer: A gnn-nested transformer model for binary code similarity detection. Electronics, 12(7):1722, 2023,
Yinhan Liu. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 364, 2019.
Luca Massarelli, Giuseppe A Di Luna, Fabio Petroni, Leonardo Querzoni, Roberto Baldoni, et al. Investigating graph embedding neural networks with unsupervised features extraction for binary analysis. In Proceedings of the 2nd Workshop on Binary Analysis Research (BAR), pages 1-11, 2019.
Luca Massarelli, Giuseppe Antonio Di Luna, Fabio Petroni, Roberto Baldoni, and Leonardo Querzoni. Safe: Self-attentive function embeddings for binary similarity. In Detection of Intrusions and Malware, and Vulnerability Assessment: 16th International Conference, DIMVA 2019, Gothenburg. Sweden, June 19-20, 2019, Proceedings 16, 2019.
Tomas Mikolov, 1lya Sutskever, Kai Chen, Greg S Corrado. and Jeff Dean. Distributed representations of words and phrases and their compositionality. Advances in neural information processing systems, 26, 2013.
Andreas Moser, Christopher Kruegel, and Engin Kirda. Exploring multiple execution paths for malware analysis. In 2007 IEEE Symposium on Security and Privacy (SP'07), pages 231-245.
IEEE, 2007.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprini
2018.
Jannik Pewny, Behrad Garmany, Robert Gawlik, Christian Rossow, and Thorsten Holz. Cross- architecture bug search binary executables. 2015 IEEE Symposium on Security and Privacy, pages 709-724. IEEE, 2015.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Alec Radford. Improving language understanding by generative pre- vre-training. 2018.
N Reimers. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint urXiv:/908. 10084, 2019.
Nathan E Rosenblum, Xiaojin Zhu, Barton Miller, and Karen Hunt. Learning to analyze binary computer code. In AAAI, pages 798-804 0048804,25 2008
Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, et al. Code llama: Open foundation models for code. arXiv preprint (tt.::303./2950000 2023.
Igor Santos, Felix Brezo, Xabier Ugarte- Pedreeo, and Pablo G Bringas. Opcode sequences as representation of executables for data-mining-baar based unknown malware detection. information Sciences, 231:64 82, 2013.
Ensheng Shi, Yanlin Wang. Wenchao Gu, Lun Du, Hongyu Zhang, Shi Han, Dongmei Zhang, and Hongbin Sun. Cocosoda: Effectivee contrastive learning for code search. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), pages 1198-2210. IEEE, 2023.
Kai Sheng Tai, Richard Socher, and Christopher Manning. Improved semantic representations from tree-structured long short-term memory networks. arXiv preprint arXiv:1503.00m 2015.
12

---

Hanzhuo Tan, Qi Luo. Jing Li, and Yuqun Zhang. LIm4decompile: Decompiling binary code with large language models. arXiv preprint arXiv:2403.05286. 2024.
CodeGemma Team, Heri Zhao, Jeffrey Hui, Joshua Howland, Nam Nguyen, Siqi Zuo, Andrea Hu, Christopher A Choquette- -hhtte Jingyue Shen, Joe Kelley, et al. Codegemma: Open code models based on gemma. arXiv preprint arXiv:2406. 11442400..144,,, 1409, 2024.
Donghai Tian, Xiaoqi Jia, Rui Ma, Shuke Liu, Wenjing Liu, and Changzhen Hu, Bindeep: A deep learning approach to binary code similarity detection. Expert Systems with Applications, 168: 14348, 2021.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302. 1397,,
A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
Yue Wang. Weishi Wang, Shafiq Joty, and Steven CH Hoi. Codet5: Identifier-awan unified pre-trained encoder-decoder models for code understanding and generation. arXiv preprint arXiv:2109.00859, 2021.
Hao Wang, Wenjie Qu, Gilad Katz, Wenyu Zhu, Zeyu Gao, Han Qiu, Jianwei Zhuge. and Chao Zhang. Jtrans: Jump-aware transformer for binary code similarity detection. In Proceedings of the 31st ACM SIGSOFT International Symposium Software Testing and Analysis, pages 1-13, 2022.
Liang Wang, Nan Yang, Xiaolong Huang. Linjun Yang, Rangan Majumder, and Furu Wei. Improving text embeddings with large language models. arXiv preprint arXiv:240J.cm 00368, 2023.
Yue Wang, Hung Le, Akhilesh Deepak Gotmare, Nghi DQ Bui, Junnan Li, and Steven CH Hoi. Codet5+: Open code large language models for code understanding and generation. arXiv preprint arXiv:2305.07992, 2023.
Hao Wang. Zeyu Gao, Chao Zhang, Zihan Sha, Mingyang Sun, Yuchen Zhou, Wenyu Zhu, Wenju Sun, Han Qiu, and Xi Xiao. Clap: Learning transferable binary code representations with natural language supervision. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis, pages 503-515, 2024.
Xiaojun Xu, Chang Liu, Qian Feng, Heng Yin, Le Song, and Dawn Song. Neural network-based graph embedding for cross-plattonn binary code similarity detection. In Proceedings of the 2017 ACM SIGSAC conference on computer and communications security, pages 363-376, 2017.
Yifei Xu, Zhengzi Xu, Bihuan Chen, Fu Song, Yang Liu, and Ting Liu. Patch based vulnerability matching for binary programs. In Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis, pages 376-387, 2020.
Shouguo Yang, Long Cheng, Yicheng Zeng, Zhe Lang. Hongsong Zhu, and Zhiqiang Shi. Asteria: Deep learning- based ast-encoding for cross platffrm binary code similarity detection. In 2021 51st Annual IEFF/IFIP International Conference on Dependable Systems and Networks (DSN), pages 224 236. IEEE, 2021.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXhv:2505.09388, 2025.
Zeping Yu, Rui Cao, Qiyi Tang, Sen Nie, Junzhou Huang, and Shi Wu. Order matters: Semantic- aware neural networks for binary code similarity detection. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 1145-1152, 2020.
13

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justificcton right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No] ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: We believe the scope is accurate and the contributions are reflected in the subsequent sections.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
14

---

Justification:
We discuss the limitations of the work in the Conclusion section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approxinations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proof'
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer:
[NA]
Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We disclose the source code to reproduce the result. And the proposed methods are general enough and easy reproduce.
Guidelines:
15

---

The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibilityy In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
We provide instructions in the README.md file in the code repository.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
oorroodeeeammssiooooooooooooeeee ssiiooolccc)) for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https CodeeSumiiii www.iisssooosoiiiio
c/nips.cc/publi///guu
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
16

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer:
[Yes]
Justification:
We discuss about the details in Training Details section.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: All the results clearly show the statistical significance of the experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We include the details about the training setup in this paper.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
17

---

The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics htatps///eerrpps..cppullbcebiiie
Answer: [Yes]
Justification: Yes, do.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is societal impact of the work performed.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
18

---

Released models that have high risk for misuse or dual-use use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We citep the original papers for the datasets used in this paper, and comply with the open-source license.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should citep the original paper that produced the code package or dataset.
The authors should state which version asset used and, possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the pataasess.paperswtt paperswi thcode. cow/coo////ooooo /atasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We provide README, comments alongside the published code.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
19

---

Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the L.M used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: We did not any large language models in this paper.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our L.M for what should or should not be described.
20

---

A More Related Work on Large Language Models
LLM Architectures. In the foundation LLMs, transformer [Vaswani] 2017 is the architecture used to capture contextual relationships for long text sequences. The original transformer contains both encoder and decoder modules, where they work tandem encode input sequences into fixed-length vector and output predictions for new sequence. Such architecture can be used for machine translation, question answering, etc. Encoder models like BERT [Devlin] 2018) and its variants [Liu 2019 Lan 2019 Clark 2020] perform pre-training using masked language modeling with a large amount of text data and generate contextual representations. BERT-style models use bi-directional information that conditions the entire sentence. The generated embeddings can be used for transfer learning or supervised fine-tuning in other downstream tasks. Decoder autoregressive (AR) models such GPT [aadford] 2018 and Llama [Tourron al. 2023 perform the next token prediction based on previously seen tokens and generate new sequences.
Foundation Coder Models are trained on multiple programming languages and large codebases. They are capable of code generation, summarization, and translation. A list of modern open source coder models include Code Llama [Roziere el al.) [003]] codeT5 (Wang et al. [02I]. Qwen- coder [Hui et al., 2024), GraphCodeBERRT [Guo el | 2020), and many more. However, most of coder models are only trained using source code such as Python and Java. Assembly code has not attracted great attention from L.LMs, thus some form of transfer learning required to learn both the syntax and semantics of assembly code.
B Additional Results
This section contains additional results for cross-compiler (Table 7 cross-architecttres (Table 9 and cross-obfuscation (Table 11}. We also plot the t-SNE distributions for all LLM models in Figure
MRR
Recall @
Models
clang.gce
clang-gec
SAFE
0.200
0.063
PalmTrre
0,423
0.178
Asm2Vec
0.523
0.328
OrderMatters
0.007
0,001
GraphCodeBERT (125M)
0.611
0,540
CodeTT+ (110M)
0.635
0,569
Qwen2.5-Embb (1.5B)
0.667
0.593
Qwen2.5-Coder (1.5B)
0.867
0.807
EBM (0.5B)
0.946
0.912
Table 7: Experimental results for cross-compiler between Clang and GCC. As a fairly easy task, Qwen2.5-Emg achieves 0.667 MRR with zero-shot setting. EBM outperforms all the baselines by over 10%.
Models
MRR
Arm, PowerPC
Ann, x32
Arm, MIPS
PowerPC, x32
PowerPC, MIPS
X32, x64
X32, MIPS
SAFE
0.180
0.191
0.189
0.165
0.110
0.196
0.152
Palmiree
0.021
0.008
0.019
0.008
0.028
0.018
0.013
Asm2Vce
0.270
0.236
0.223
0.265
0.240
0.417
0.307
OrderMatters
0.008
0.008
0.007
0.008
0.006
0.007
0.008
GraphCodeBERT (125M)
0.094
0.075
0.256
0.215
0.495
0.239
CodeT5+ (1 (0M)
0.057
0.043
0.048
0.287
0.259
0.461
0.259
Qwea2.5-Emb (1.5B)
0.014
0.030
0.032
0.142
0.095
0.409
0.079
Owen2.5-Coder (I.5B)
0.269
0.319
0.374
0.490
0.480
0.757
0.522
EBM (0.5B)
0.718
0,818
0.770
0.836
0.803
0.906
0.883
Table Additional MRR results for cross-architecture settings.
21

---

(a) CodeT5+
Qwen2.5-Emb (zero-shot)
o0
ol
o3
(c) GraphCodeBERT
(d) Qwen2.5-Coder
03
(e) EBM
Figure 5: t-SNE visualization of embedding distripytions 100 functions generated by LLM models. Each color corresponds to binary function under different optimization settings, represented by distinct shapes. Embeddings for the same function should cluster by color instead of shapes. Note that some colors visually similar but represent different functions. It is evident that the embedding representations generated by EBM distribute more evenly the space than in all others.

---

Recall@1
Models
Arm, PowerPC
Amn x32
Arm, MIPS
PowerPC, x32
PowerPC. MIPS
x32, x64
x32, MIPS
SAFE
0.063
0.000
0.063
0,000
0.000
0.063
0.000
PalmTrre
0.001
Asm2Vec
0.098
0.080
0.072
0,085
0,080
0.231
0.114
OrderMatters
GraphCokeBERT (125M)
0.059
0.034
0.029
0.187
0.146
0.419
0.183
CodeT5+ (110M)
0.023
0.013
0.019
0,223
0.198
0.392
0,205
Qwen2.5. Emb (1.5B)
0.003
0.010
0.013
0.093
0.054
0.331
0,050
Qwen2.5 (1 (L)5))
0.189
0.230
0.280
0.395
0.379
0.682
0.414
(0.5B)
0.771
0.727
0.809
Table 9: Additional recall @ results for cross-architecture settings.
MRR
Models
all, sub
all. fla
all, bef
none, sub
none, fla
sub, bef
fla, bcf
SAFE
0.209
0.148
0.229
0.312
0.200
0.128
0.178
Palm'Tres
0.120
0.254
0.144
0.829
0,260
0.276
0.141
Asm2Vec
0.205
0.238
0.259
0.534
0.332
0.340
0.357
OrderMatters
0.008
0.008
0.007
0.007
0.008
0.008
0.008
GraphCodeBERT (125M)
0.239
0.331
0.290
0,932
0.464
0.621
0.447
CodeTT5+ (110M)
0.174
0.324
0.263
0,873
0.378
0.585
0.465
Qwen2.5-Emb (1,5B)
0.277
0.337
0.355
0.900
0,466
0.618
0.463
Qwen2.5-Coder (1,5B)
0.374
0.417
0.422
0.953
0.583
0.687
0.540
EBM (0.5B)
0.522
0.704
0.614
0.989
0.815
0.817
0.790
Table 10: Additional MRR results for cross-obfuscation settings.
Recall @
Models
all, sub
all, fla
all, hef
none, sub
none, fla
sub, bef
fla, hef
SAFE
0,063
0.000
0.063
0.125
0,063
0,000
0,000
PalmTrree
0.069
0.125
0.072
0.599
0.117
0.186
0.073
Asm2Vec
0.075
0.090
0.114
0,336
0.148
0.161
0.176
OrderMatters
0.001
0.001
0.001
0,000
0.002
0.001
0.002
GraphCodeBERT (125M)
0.186
0.268
0.230
0,901
0.384
0.540
0.375
CodeTS+ (110M)
0.127
0.251
0.196
0.839
0.301
0.511
0.374
Qwen2.5-Embb (1.5B)
0.207
0.262
0.284
0.867
0.375
0.540
0.380
Qwen2.5-Coder (L.5B)
0.292
0.340
0.340
0.921
0.504
0.603
0.454
EBM (0.5B)
0.436
0.624
0.523
0.972
0.751
0.751
0.717
Table 11: Additional recall results for cross-obfuscation settings.
Model
Average Inference Time per Batch (100 samples) in seconds
Average Training Time per Batch
(4 samples) in it/s
CodeTTPP(110M)
0.007
13.90
GraphCodeBERT((25M)
0,005
17.78
Qwen2.5(1.5B)
2,000
2.78
CodeGemma2B
2.540
2.12
EBM(0.5B)
0.014
3.32
Table 12: Model Computational Performance Comparison
23

---

Source
Dest
Accuracy
Mean = STD of Similar pairs
Mean = STD Non-similar pairs
CodeGemma
EBM
CodeGemma
EBM
CodeGemma
EBM
obf all
obf_ none
0.207
0.103
0.615 = 0.232
0.580 # 0.216
0.141 # 0.189
0.244 # 0.156
obf all
obf sub
0.179
0.097
0.607 = 0.235
0.592 # 0.206
0.149 : 0.189
0.254 0.158
obf all
obf
0.057
0.085
0.185
0,221 0.192
0.386 0.172
obf_all
ubf_bef
0.111
0.026
0.651
0.166
0.189
0.163
obf none
obf_sub
0.838
0.890
0.059
0.065
0.181 0.222
0.110
obf_none
obf_
0.169
0.341
0.763
0.746 + 0.161
0.166 0.206
0.230 + 0.167
obf none
obf bef
0.354
0.320
0.149
0,180 0.180
0.233 0.172
ohf sub
obf fla
0.120
0.278
0.734 0.122
0.161 # 0.202
0.235 + 0.165
obf sub
obf bef
0.231
0,265
0.150
0.762 0.113
0.169 = 0.210
0.232 + 0.171
obf fla
obf
0.176
0.273
0.184 0.145 0.166 0.195
0.291 = 0.162
clang
gee
0,529
0,849
0.910
0.257
0.103
powerpe
0.014
0.478
0.727 0.173
0.765
0.278 0.270
x86_32
0.014
0.624
0.732 + 0.149
0.801 +: 0.137
0.290 0.266
0. + 0.
arm
x86 x86_64 64
0.011
0.526
0.766 0.173
0.273 0.266
0.102 0.16]
arm
mips
0.034
0.587
0.757 + 0.149
0.805 + 0.148
0.283 # 0.266
0.094 + 0.159
powerpe
x86 32
0.111
0.529
0.150 0,820 = 0.157
0.307 # 0.267
powerpe
x86 64
0.102
0.424
0.818
0.797 0.195
0.279 + 0.270
0.119 + 0.169
powerpe
mips
0.104
0.474
0.803
0.181
0.114
x86_32 32
x86
0.276
0.712
0.902
0.877 0.114
0.318 0.262
0.121 0.179
x86 x86_32
mips
0.136
0.663
0.797 0.144
0.826 + 0.140
0.280 0.263
0.103 + 0.166
x86_66 64
mips
0.124
0.624
0.800 = 0.145
0.810 0.143
0.286 0.256
0.103 0.165
Table 13: Additional results for Model Accuracy for non-match and Similarity Metrics Comparison
24


Reviews:
[{"review_id": "9AyFdPuj1C", "reviewer": "Reviewer_va3T", "summary": "This paper studies the task of binary executable code similarity detection with help of LLMs. They propose finetuning an LLM model on assembly data and show that this finetuned model outperforms similar models evaluated in zero-shot setting.", "questions": "- \"the same source code can be compiled into many formats of binary code using different compiler settings and environments, including optimization levels, machine architectures, compiler software, and obfuscation techniques. \" - The same is true for source code similarity detection, the same functionality can be written in many ways in source code. While this is a limitation and potential challenge, further motivation is needed to understand why the LLMs that work for source\n\n- On line 193: \"As our model is based upon Qwen2.5-Coder-0.5B, the Qwen2.5-Coder-1.5B baseline is a good candidate for ablation study.\" - not clear why you suggest it is an especially good candidate compared to any other coding model? Why that size of the model?\n\n- Authors mention they finetune all models with their contrastive loss, except Qwen-Emb, which is supposed to illustrate the difference between code & non-code models, but that makes no sense to me. Why also not finetune Qwen-Emb and show that code models perform better compared to non-code ones, even when finetuned? Or perhaps authors meant something else, but it wasn't clear to me.", "limitations": "yes", "rating": 1, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 1, "clarity": 1, "significance": 1, "originality": 1, "strengths_and_weaknesses": "**Weaknesses**\n\n- The paper is poorly written; the motivations behind decisions made in the study and proposed novelty are not spelled out; some of the contributions mentioned, such as \"comparing baselines\", are not sufficient to be considered a contribution in my opinion. There are multiple ambiguous pronoun references which affects the clarity of the paper, e.g. in Conclusion; acronyms are used without prior introduction, e.g. what is AR on line 139? GTE on line 173?\n\n- Key details are missing, e.g. on line 126 the authors say: \"we add several additional tokens to indicate the “language\" information of each binary function, which corresponds to the optimization level, compiler, obfuscation, and architecture. Adding such information has proven to be effective, especially for cross-architecture detection.\" - without specifying what tokens, how and where they are added, at what stage, etc, this information is not as useful for the reader as it could be otherwise. In general, the paper does not seem to be written with AI professional readers in mind, e.g. the authors go into details explaining how autoregressive models learn, but gloss over what their used obfuscation methods are. \n\n- The authors suggest that their proposed solution based on continual training and finetuning of an LLM model is cheap and effective, which is questionable considering approaches like parameter-efficient FT or inference-time demonstrations with larger models are not considered at all. \n\n- The choice of the baseline models and how each one is trained is unclear to me. The authors mix embedding models, such as CodeT5+ and GraphCodeBERT, with generative models, such as Qwen2.5-Coder1.5 for a task of ranking pairs. How are they compared is not clear. This weakens the results because it is not clear what comparisons are being drawn and which of the proposed modifications/design choices are responsible for performance report.\n\n- The prior work section is incomplete when it comes to recent developments, and omits most recent works and models, instead referring back and comparing to 5+ years old models, such as GraphCodeBERT and CodeT5+, which are not state of the art, and weren't intended to be used with assembly either, so not clear why those were chosen over other more modern & stronger code models. To make the paper and it's findings valuable it would need to draw comparisons to more recent models such as DeepSeek models, SantaCoder, WizardCoder, CodeGemma, CodeLlama, etc.. The authors use Qwen-Coder as their base model, which is a strong, state-of-the-art model.\n\n- The motivation behind performing continual training on a generative model and using it as an embedding model is not clear, and most of the design choices are not intuitive, and neither are they explained or supported, which limits the usefullness of the paper since the takeaway messages are not clear. It is also not at all surprising that finetuning an LLM may outperform other, similarly-sized LLMs that are evaluated in zero-shot manner, so the whole evaluation setup needs an overhaul to provide new knowledge for the readers. \n\n\n**Strengths**\n\nThe authors use a comprehensive dataset of cross-architecture, cross-optimization, cross-compiler, cross-obfuscation binaries for their experimentation.", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "YBHA7Axyoq", "reviewer": "Reviewer_NsTz", "summary": "This paper proposes a training strategy to transform an LLM into an assembly code embedding model. It flattens the code and adds structural \"language\" tokens for improved syntax and semantic representation. It treats function pairs like translation tasks to learn cross-variant semantics. It leverages LLM2Vec to enable bidirectional attention in causal models for better embeddings. It also utilizes cumulative GTE loss that enhances training with limited resources. Experiments are conducted on two datasets, and results show that it outperforms baselines.", "questions": "Please refer to the weaknesses.", "limitations": "It may not work when the binary code cannot be disassembled correctly.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. Overall, it's well-written and easy to follow.\n2. The evaluation considers multiple datasets and baselines, and also includes ablation studies on different components.\n3. The results show that the proposed method outperforms baselines.\n\nWeaknesses:\n1. It would be better to include the time cost. Because this paper tries to explore an efficient fine-tuning method, it's suggested to show that the proposed method is more efficient than existing methods.\n2. It would be better to evaluate some other cases. \n\n    2.1 If the pool doesn't contain \"exactly one similar function\", that is, \"no similar function\" or \"multiple similar functions\", what will be the precision and recall?\n\n    2.2 If the user doesn't know the language information of the binary code, or wrong language information is provided, what will happen?\n\n    2.3 When the training configuration is different from the test one, how will the performance change? For example, if the model is only trained on o0, o3, can it work for o1, o3?\n\n3. It would be better to introduce and explain the symbols in each equation. For example, $q_i, k_i, k_j, \\tau, \\eta, (n)$, etc.", "was_revised": true, "final_justification": "The rebuttal addressed some of my concerns and I will keep my score.", "rebuttal": ""}, {"review_id": "M6lvGXt5vq", "reviewer": "Reviewer_CBZw", "summary": "This paper considers the problem of training embedding models for detecting if a pair of code binaries are similar, i.e., generated by compiling the same source file but with different compiler options or architecture backends. For this purpose, the paper presents a method for fine-tune existing code models using a combination of causal language modeling, masked language modeling, and contrastive learning objectives. A model (Qwen2.5-Coder-0.5B) fine-tuned with these objectives is compared with various baselines on similar binary code retrieval task. Experiments show that the fine-tuned model outperforms various baselines.", "questions": "1. Why not consider pairs of binaries that are compiled from different, but semantically equivalent source codes?\n\n2. Does the method require metadata about the binaries such as the hardware architecture, the compiler optimization levels, etc? If so, can this information be extracted directly from binary files? If not, how will the method be used in practice?\n\n3. Equation 6 is not clear. What do you mean by \"multiple distributed models\" and \"accumulated inputs\"?\n\n4. Does the method require disassembling the binary first? Section 3.1 seems to suggest this. It would be good to clarify this.\n\n5. Which dataset is used for results in Fig 3?", "limitations": "The paper has a brief mention of limitations in the conclusion. Perhaps, this could be expanded. One limitation that could be discussed is whether the method is applicable for two binaries that are compiled from different but semantically equivalent source files.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "### Strengths\n1. The empirical results are very promising.\n\n2. The proposed training methods are fairly straightforward, which is a good thing! It seems like the methods should be easily applicable to other baseline models as well.\n\n### Weaknesses\n1. It would have been nice if the experiments included results from fine-tuning another model. At the moment, one cannot say for sure if the presented method will show improvements on other models as well.\n\n2. The experiments only consider similarity between pairs of binaries that are compiled from the same source code. I am very curious what the results look like for a pair of binaries that are compiled from different, but semantically equivalent source codes.\n\n3. If I understand correctly, during inference, the method requires access to metadata about the binaries such as the hardware architecture, the compiler optimization levels, etc. Can such information be extracted from binary files? If not, the method may not practically applicable.", "was_revised": true, "final_justification": "I thank the authors for their response which helped clarify some conceptual issues. I will maintain my score.", "rebuttal": ""}, {"review_id": "nW3hhsAf2z", "reviewer": "Reviewer_W13r", "summary": "This paper explores using LLMs for binary code similarity detection. It demonstrate that pretrained LLMs can detect similar binary code even in a zero-shot setting, and proposes supervised fine-tuning methods, including data augmentation, translation-style learning, LLM2Vec, and cumulative GTE loss, that significantly outperform zero-shot LLMs and state-of-the-art approaches. The approach enhances robustness across different optimizations, architectures, and obfuscations, transforming generic LLMs into effective binary similarity experts.", "questions": "1. How does the backbone model (Qwen2.5-Coder-0.5B) perform without training? (This question is concerned to show the improvement from the approach)\n\n2. How does the approach perform when applied to different backbone models with different scales of parameters? (This question is concerned on the generality of the approach as currently the evaluation is restricted on one small backbone model).\n\n3. What are the downstream application scenarios of the BCSD techniques in practice? (This question is related to the importance and motivation of the studied program, also, it could help justify why the BCSD can be formulated as a retrieval problem). \n\n4. How is the retrieval pool constructed? Why use the size 1000 by default?", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "## Strengths\n+ The evaluation includes diverse scenarios from multiple perspectives (e.g., cross-optimization, cross-architecture, cross-compiler, cross-obfuscation). \n\n## Weaknesses\n- The generality of the approach is not well justified empirically given only one small backbone model is involved. \n- The downstream application tasks of the BCSD techniques are missing (in terms of both the discussion and empirical evaluation). It hurts the significance of interpreting the metric improvements of the proposed approach in practice.", "was_revised": true, "final_justification": "The rebuttal has partially addressed my concerns.\n\nThe experiments are still limited to Qwen models as backbone models, so I have concerns regarding generalization. Moreover, the parameter setting is arbitrary, and the authors acknowledge it.", "rebuttal": ""}, {"review_id": "EcdjJM9udG", "reviewer": "Reviewer_3vSh", "summary": "Vulnerability detection in binary code is a very challenging and the generic large language models are not yet mature enough to reason about similar code. This work proposes uplearning the pretrained LLM to identify the binary code similarity and has shown effectiveness against zero-shot LLMS and the state-of-the-art binary code similarity detection methods.", "questions": "The equations 1 & 2 are not explained", "limitations": "Limitations are included under the conclusion. Limitations include the quantity of the training data set compared to previous work, and the lack of potential downstream tasks, etc. However, this work can be good initial work achieve these limitations.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths :\nThe causal relationship is vital in reasoning the binary code, specifically in understanding the flow. This work is uptrained the LLMS with the causal training which can improve the LLMs capabilities against binary code. \n\nWeaknesses: \nI do not see any specific weakness in this work", "was_revised": false, "final_justification": "", "rebuttal": ""}]

Author rebuttal:

Meta-review: This paper tackles binary code similarity detection (BCSD) and proposes EBM (Effective Binary Matching), a finetuning process combining multiple techniques: data augmentation with additional metadata tokens, translation learning, embedding training with LLM2Vec, and contrastive learning with a variation of GTE loss. Experiments show that this outperforms multiple LLM-based and non-LLM-based baselines for BCSD, and an ablation study shows that each component of the finetuning pipeline contributes substantially to the overall performance.

Reviewers mentioned the following strengths:
* The method is easy to understand and conceptually well-justified
* The experimental setup is thorough: baselines include multiple non-LLM approaches, finetuned generic LLMs, and LLM approaches specific to BCSD; experiments are performed on multiple datasets; an ablation study quantifies the effect of each component of EBM
* The empirical results are strong with EBM clearly outperforming all of the other compared approaches. In particular, Qwen2.5-Coder-**0.5B** finetuned with EBM outperforms the much larger Qwen2.5-Coder-**1.5B** finetuned on the same data with only contrastive learning.

After discussion, some remaining weaknesses include:
* The EBM finetuning approach is only applied to one model, so it is not completely clear how much the approach would generalize to different base models (but there are also no concrete reasons to suspect poor generalization)
* The downstream applications of BCSD, which include vulnerability and malware detection, could use more thorough discussion or experiments to highlight the practical usefulness of the method
* Some parts of the writing are unclear, and there are formatting issues (improper in-text citation format, sub-subsections without subsections) which make reading more difficult

I believe the strong experiments and results outweigh the minor weaknesses, and four of five reviewers are on the side of accept. Thus, I recommend to _accept_ this paper.

_I strongly urge the authors to add needed clarifications to the text and improve the formatting in the next revision._
Decision: Accept (poster)