Guess & Sketch: Language Model Guided Transpilation
Authors: Celine Lee
Abdulrahman Mahmoud
Michal Kurek
Simone Campanoni
David Brooks
Stephen Chong
Gu-Yeon Wei
Alexander M Rush
Venue: ICLR 2024 poster (2024)
Keywords: transpilation
program translation
assembly code
language model
neurosymbolic
machine learning
Primary area: neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
TL;DR: We introduce a neurosymbolic approach to assembly language transpilation that outperforms GPT-4, an engineered transpiler, and fine-tuned language models.

Abstract: Maintaining legacy software requires many software and systems engineering hours. Assembly code programs, which demand low-level control over the computer machine state and have no variable names, are particularly difficult for humans to analyze.
Existing conventional program translators guarantee correctness, but are hand-engineered for the source and target programming languages in question. Learned transpilation, i.e.  automatic translation of code, offers an alternative to manual re-writing and engineering efforts. Automated symbolic program translation approaches guarantee correctness but struggle to scale to longer programs due to the exponentially large search space. Their rigid rule-based systems also limit their expressivity, so they can only reason about a reduced space of programs. Probabilistic neural language models (LMs) produce plausible outputs for every input, but do so at the cost of guaranteed correctness. In this work, we leverage the strengths of LMs and symbolic solvers in a neurosymbolic approach to learned transpilation for assembly code. Assembly code is an appropriate setting for a neurosymbolic approach, since assembly code can be divided into shorter non-branching basic blocks amenable to the use of symbolic methods. Guess & Sketch extracts alignment and confidence information from features of the LM then passes it to a symbolic solver to resolve semantic equivalence of the transpilation input and output. We test Guess & Sketch on three different test sets of assembly transpilation tasks, varying in difficulty, and show that it successfully transpiles 57.6% more examples than GPT-4 and 39.6% more examples than an engineered transpiler. We also share a training and evaluation dataset for this task.

GUESS & SKETCH: TRANSPILATION
LANGUAGE MODEL GUIDED
Celine Lee
Abdulrahman Mahmoud Michal Kurekt Simone Campanoni" Stephen Chong
David Brookst
Gu-Yeon Weit Alexander M. Rush
Cornell University, f Harvard University < Northwestern University
cl 219239ccrne11.ed
ABSTRACT
Maintaining legacy software requires many software and systems engineering hours. Assembly code programs, which demand low-level control over the com- puter machine state and have no variable names, are particularly difficult for hu- mans to analyze. Existing conventional program translators guarantee correctness, but are hand-engineered for the source and target programming languages in ques- tion. Learned transpilation, i.e. automatic translation of code, offers an alternative to manual re- re-writing and engineering efforts. Automated symbolic program trans- lation approaches guarantee correctness but struggle to scale to longer programs due to the exponentially large search space. Their rigid rule-based systems also limit their expressivity, they can only reason about reduced space of programs. Probabilistic neural language models (LMs) produce plausible outputs for every input, but do so the cost of guaranteed correctness. In this work, lever- age the strengths of LMs and symbolic solvers in neurosymbolic approach to learned transpilation for assembly code. Assembly code is an appropriate setting for a neurosymbolic approach, since assembly code can be divided into shorter non-branching basic blocks amenable to the use of symbolic methods. GUESS SKETCH extracts alignment and confidence information from features of the LM then passes a symbolic solver to resolve semantic equivalence of the transpi- lation input and output. We test GUESS & SKETCH on three different test sets of assembly transpilation tasks, varying in difficulty, and show that it successfully transpiles 57.6% more examples than GPT-4 and 39.6% more examples than an engineered transpiler. We also share a training and evaluation dataset for this task.
INTRODUCTION
The increasingly heterogeneous landscape of hardware architectures and their instruction set archi- tectures (ISAs) marks a large and growing need to develop support for cross-ISA software manage- ment. This challenge is especially relevant for hardware-specins legacy software which must be re- written to run on any other hardware. Many high-usage source code files also contain in-lined assem- bly code, which requires porting to alternate hardware architectures. Automated tross-ISA software support has been of interest in the computer architecture community for decades (Armengol-Esse Estapé et al. 2023 Wang al. 2018 Bellard 2005 Ardestani & Renau 2013) Sanchez & Kozyrakis 2013 Emulators, virtual machines, and containerized applications allow users to run software on different host hardware by simulating the architecture of the hardware platform that the software is compiled for. However, this option can be unwieldy and compute-inefficient. Assembly-to-asseml transpilation Ami oce 1989). the process of automatically porting software from one ISA to an- other. offers a way to generate software that can be natively executed on the new hardware. However, current transpilation tools are engineered for the specific source and target hardware architecture. so they scale poorly new ISAs are introduced.
Neural machine learning techniques are natural fit for transpilation. Assembly program translation pairs can be generated by cropiicompi C or C++ programs using different existing compilers and
"Transpiler" describes the general code translation task that our method targets, but we note that the focus of this paper assembly-to--ee to-sseebly transpilation.

---

compiler flags, providing vast amounts of training data. Pairs have the same semantics since they originate from the same high-level program. Assembly code syntax is rigid but simple compared to natural language and most high-level programming languages, settings that existing language
models have been shown to perform well in (Devlin et al. 2019, Feng et al.] 2020 Radford &
[Sutskever 20188] Lewis et al., [0019] Chen et al., 2021). Evaluation this setting can also be done automatically by comparing execution of the input code and the resulting code.
However, key weakness of language models in this setting is their inability to perform long-tail logical reasoning (Kandpal et al. 2022 Miceli-Barone el al. 2023). Assembly code transpilation requires reasoning about the complex semantics of programs. Additionally, specific challenging phenomena, such as differing implementations of mathematical operations on different ISAs. are critical and arise frequently in assembly code.
Motivated by the symbolic properties of logical reasoning in the problem of transpilation, we pro- pose a neurosymbolic method to transpilation. Purely symbolic methods are built on correctness guarantees, but generally can only handle short programs before encountering computational in- tractability. Classical synthesis techniques struggle to scale past ~ lines of assembly code (Hu) et al. 2023). Purely neural language modeling approaches are powerful general translators but have critical failure points that cause program breakdown. We argue for the value of a mixed-method Lc. neurosymbolic, approach that uses probabilistic language models to obtain helpful information for transpilation, then passes such information to an ISA semantics-aware solver to complete the transpilation process.
Our method, GUESS & SKETCH, uses core properties from the language model to extract sym- bolic methods for transpilation. During the neural GUESS phase, a trained language model pro- duces candidate translations for a given input, identifies potential errors in the output, and extracts semantically-aliiee aliigned subsequences from the input and output sequences. Potentially erroneous aligned subsequences are passed to the symbolic SKETCH phase, where the input subsequence is used specification to correct the output subsequence.
We demonstrate the feasibility of our method by porting assembly programs from ARMv8 to RISC- V and vece-versa, but note that our method can generalize to various source and target languages. In order to test our method, we introduce a new benchmark consisting of 3 transpilation problems varying in difficulty and domain. We identify weaknesses engineered symbolic approaches to the task. We also find that existing neural network approaches, using both fine-tuned and pre-trained off- the-shelf large language models, struggle with transpilation. In contrast, our method combines the strengths of both neural and symbolic approaches and successfully transpiles 57.6% more examples than GPT-4, 39.6% more examples than an engineered transpiler, and 13.2% more examples than the most competitive baseline.
2 RELATED WORK
Learned code translation. Code transpilers (or transpilers) translate from one programming lan- guage to another. The core challenge in this space is preserving operational semantics across the source and target language, while operating within the strict syntax and vocabulary of both. One approach to this task is to train neural machine translation systems with paired code sequences for the task, such language model (Lewis al.) 2019 or tree-to-tree neural networks (Chen et al., 2018). Approaches such as Transcoder (Roziere et al. 2020) have also presented an unsupervised approach neural source code-to-source code translation, in which they only require monolingual training data and take advantage of three training objectives: cross-lingual masked language mod- cling, denoising auto-encoding, and back-translation. Follow-up works use the LLVM intermediate representation (Roziere et al. 2022) and seimmattcalll-geerrtte unit tests (Szafraniec et al.) 2023) to further improve this approach. Older statistical approaches have mined parallel code from repos- itories and generated grammar-baa based statistical machine translation models (Nguyen et al.) 2013)
Karaivanov et al. 2014 Koehn et al. 2007). These outputs of these prior learned approaches are the generation directly extracted from the model. GUESS & SKETCHI instead incorporates knowledge of the semantics of the source and target languages in a symbolic solver that improves semantic correct- ness the produced output. Additionally, as far as we aware, approach for learning assembly translation, a lower-level programming language than higher-level programming languages such Python, Java, and even C.
the first to present learned
2

---

Emulators and engineered transpilers. Executing code platform different than the for which it was created a long-desired task. Apple's Rosetta (app) software was designed to case the transition of applications between hardwares by automatically translating binary executables from the previously supported to the new ISA. Specifically, Rosetta in 2006 supported the transition from PowerPC Intel x86 processors. Rosetta released in 2020 enabled translation from x86-64 based processors to support by ARM64-based Apple silicon. Emulators and virtualizers allow users to exccute code designed for another target hardware by simulating the target hardware ISA atop the host hardware. QEMU (Bellard 2005) is one popular emulator and virtualizer that can emulate various architectures on certain host architectures. Other assembly transpilers have been written to translate assembly from one language another, such as from ARM to RISC-V (Schorr et al., 2020). However, these emulators and transpilers take years to develop. GUESS & SKETCH, on the other hand, leverages the translation abilities of learned model to perform bulk of the transpilation.
Neurosymbolic program synthesis. Program synthesis is the task of generating computer pro- grams according to some correctness specification (Lee et al. 2021). the context of program translation, the correctness specification is the semantics of the input program itself. We discuss here some works that take combined neural and symbolic approach to the program synthesis task, similar to own approach. Nye et al. (2019) train LSTM-base model to generate program sketches from some input specification, then use the generated sketch and specification to search for satisfying program. [uo et al. (2022) devise top-down grammar-based method to selectively expand nonterminals in a program syntax tree. The incomplete program tree is converted to a sketch that is passed the symbolic sketch solver generate full program. Unlike these previous works, our method infers the sketch using attributes of single autoregressive language model. The benefit of our approach is over directly producing the sketch or generating based on grammar is that we avoid encoding specific sketch and language technicalities into the training process.
BACKGROUND
3.1
TRANSPILATION
The task of transpilation is take input program Pz, represented sequence of tokens æ, and produce the cemantically-equivleeee program Py represented as sequence of tokens Let D be the domain of all program inputs. For simplicity we represent programs as functions that map inputs to deterministic measurable output, either an integer or program failure: P. : D (Z U L). Semantic equivalence can be measured by checking that for all imputs in D, both programs produce the same execution outputs:
y Vd E D (d) = (d). In practice, we test the full programs on feasible subset of DD determined by the objective of the source program.
When working with programs, we will also assume we can partition the tokens into B, non- overlapping subsequences æ = Ib...... Ibjs,1 where each b B, defines a span over x. Sub- sequences are defined so that they can individually be converted to programs Pz,. Details for identi- fying such subsequences for assembly and translating them into a program representation conducive for symbolic reasoning in a sketch solver are shared in Appendix A.I
3.2 GENERATIVE LANGUAGE MODELS
Let (x.y) E EV=,, y* denote an input and output sequence pair where V is the shared vocabulary of tokens and L. is the maximum length. The objective of a (conditional) generative language model to autoregressively produce the correct output y from input x::
II P(uelB<e.20
arg max VEVL
Modern language models are based on the Transformer architecture /Vawwani et al. 2017). Trans- formers attention (Parikh et al. 2016). routing mechanism that provides a distribution over the input tokens used for predicting the next word. Intuitively, attention learns to indicate which part of the input weigh more for each output. We extract the model's attention between the input
3

---

LM
GUESS
a5 22288
attn. map
BOV
NO, 20125
SKETCH
11 aS. 26094 addi 2029
addd nD, a5.
52
In the GUESS (top) phase, the full input sequence x (blue) is passed to trained lan- guage model (LM), which produces candidate translation y (orange), identifies potential mistakes (red), and extracts subsequence alignment (purple) from attention between the input and output (attn. map). the SKETCH (bottom) phase, aligned input and output subsequences are passed sym- bolic solver À to correct errors identified in the GUESS phase. The final output y' is constructed by recombining corrected subsequences.
Figure 1:
sequence x output sequence y as series of stochastic matrices at each layer mapping every output index probability distribution over input indices M E Allx]a
3.3 SKETCHING
Sketching (Solar-Lezama in which a partial program outlines the high-level implementation, then synthesizer populates the omitted low-level details by ensuring that the resulting code passes some given correctness specifica- tion. Partial programs are expressed in procedural programming language augmented with single added construct: a symbolic constant expressed as hole, denoted *. Programs expressed this form, with holes as placeholders for concrete values, are sketches. In our notation, the partial pro- gram sequence is composed of tokens from the vocabulary and an added hole token: S = {V) compiled by semantics-aware translator into representations Pa in the procedural programming language understandable by the solver.
2009
Solar-Lezama 2006a) is approach program synthesis
Program sequences
The correctness specification is set by source program Pa. The goal of the synthesizer is to identify the mapping & : S + V* that populates the holes of the partial program sequence s to produce the full program sequence d(s) whose corresponding program is semantically equivalent to the source program: vd E D Po(e)(d) = (d)
The synthesis engine reduces the resulting programmatic sketch representation constraint sat- isfaction problem solved using counterexample guided inductive synthesis (Solar-Lezama et al. 2006b) find values for the holes.
NEUROSYMBOLIC TRANSPILATION: GUESS & SKETCH
Given an input program Pa represented as sequence x E VL, our goal is to learn to generate a semantcalll--equiveenn output sequence y V which represents program py: P. Py- Programs are comprised of function definitions that are generally independent from one another, so functions individually translated then stitched back together. See details in Appendix A
The challenge of our neurosymbolic approach is that language models operate on prefixes, perform- ing inference by producing one token at time, while sketch-based methods reason with partially complete sequences. To meaningfully pass information between the language model and the symbolic solver, we must extract relevant sequence-level information from the language model
In encoder-decoder models this comes from cross-attention, for decoder-only models by renormalizing self-attention.
4

---

for the solver to their semantic alignment in the input.
with. Specifically, the solver needs candidate output translations and
Our method breaks the problem into stages that can be better solved by the complementary strengths of neural and symbolic methods: a probabilistic machine learning language model produces candi- date translations, then alignment and confidence information is extracted and passed semantics- aware solver to filter the search spaces for a correct solution. The pipeline for the GUESS & SKETCH approach is illustrated in Figure I
4.1
GUESS: STRUCTURED CANDIDATES FROM GENERATIVE MODEL
The GUESS phase produces guesses tuples. For an input sequence 2, GUESS produces tuples composed of: candidate transpilation y, alignments between subsequences: A B%yy and poten- tial token-level errors in the prediction: E' {0, 131x1,
Candidates. To produce candidate sequences we follow a standard generative approach. We first train generative language model on paired source language and target language program sequences. Once trained, candidate transpilations are produced by querying the model:
E top k p(ylz)
(1)
yEVE
Aligument. Since the input and target output sequences are intended to be globally semantically equivalent, we assume output sequences locally align to input sequences. While there is not a one-to-one equivalence between to- kens, subsequences of the two programs be matched. We use this subsequence matching and the transformer attention to determine the alignment used by the sketch system. A sample extracted alignment matrix, along with the truth alignment matrix, is shown in Figure 2
Alignment is represented as a vector between subse- quences: A. To extract the alignment from the lan- guage model, we average the transformer attention matri- ces connecting and y single layer to form a stochas- tic matrix M A ||]x||=
Figure 2: True subsequence alignment
(l), attention (r), and projected subse- quence alignment (r) from the GUESS model.
We then set the alignment
Ab, = b; for the input subsequence with the highest aggregate attention score. Aggregate atten- tion score is given by norm of the submatrices i... Yby E B2 Mb, 2 &b
Guesses and Errors. The generative model also used identify tokens where it is most likely guessing. First we check if the output token j is predicted with probability less than some value
ply, |e x) <>
(2)
These low-confidence prediction points correlate long-tail code phenomena, i.e. instances that arise rarely in the data distribution, and are where the model may have made translation mistake. The second case is if the general model confident, but the program violates a domain specific heuristic, specifically if the token or its aligned input subsequence reference some entity not de- scribed in scope. If either of these conditions are satisfied, the tokens question are marked as potentially erroneous: E (0, 1]l
4.2 SKETCH: REASON OVER ALIGNED CANDIDATES
The SKETCH phase produces a full synthesized transpilation using symbolic program solver meth- ods and information from the GUESS phase. Note that determining full program equivalence an undecidable problem,
focus solving for errors in individual subsequences By,
We create a sketch s for each subsequence b E B, that has an possible error from the first stage. The sketch is created from ya by replacing each position in j that also
Create the sketch.
5

---

Algorithm GUESS & SKETCH Pseudocode
procedure GUESS & SKETCH(X)
for y, A, E E (UESS(2) do for b in By do
D produce candidates, alignments, potential errors
P. = Pr then return y
if E for any E b then f Ab
identify potential error D- get aligned input index produce sketch sequence D solve for solution (synthesizer)
8 < PLACE_HOLES!! Φ + arg m, 1(Ps., = Pa(s))
if Φ success then
y < UUPDATE(1, (()))
D update subseq.
satisfies Eú A O with a hole e, i.c potentially erroneous tokensn The correctness specification is the program represented by the aligned input subsequence Ti, where As bę- Correctness specifications must be based on complete semantics, so for input subsequences with out-of-scope references, we extract the definition of the referenced entity from the full program. The retrieved entity definition is used to complete the semantics of the correctness specification.
semantics-awasd aware translator lifts the sketch and correctness specifications into their sketch solver programmatic representations P, and Pz.,, respectively. Details about this translation process for our assembly language experiments are shared Appendix A.I
Solve the sketch. To solve the sketch is to find mapping that correctly populates all holes of the partial program sequence to satisfy the correctness specification: Vd E D Pa.,, (d) = Po(e) (d).
solution populating all holes of the partial program sequences found by the sketch solver, it is applied to and the updated subsequence @(s) replaces the subsequence in the full program sequence. the subsequence had an out-of-scope reference, the solver would have also resolved a definition of the referenced entity. The resolved referenced entity definition is also updated in the full program. cases where sketching solution cannot be found, GUESS & SKETCH resorts to the original prediction. Since our method always least defaults to the original generation, the correctness of GUESS & SKETCH is lower-bounded by the correctness of the initial guess. This full process is summarized in Algorithm
EXPERIMENTAL SETUP
Dataset Our experiments focus on transpilation between real programs compiled to different ISAs, specifically the ARMv8 and RISC-V assembly languages. ARMv8 and RISC-V are both reduced in- struction set architectures (ISAs), and have some similarities in instructions (Hennessy Patterson 2011]. We construct training and evaluation datasets for this task.
Training data is composed of 307,910 ARMv8 and RISC-V assembly file pairs compiled from C code files from The Stack (Ko- cetkov al., (022]. All selected source C files can be independently compiled to as- sembly using the standard C libraries (c.g. stdlib, stdio). The C files are compiled both ARMv8 and RISC-V target archi- tecture assembly files under the -00, -01, -02, and -03 optimization flags using cross- compilers aarch64-1inm -nnua------------ gcc and 1.scv64-1inux-gu--
Test Dataset
Avg len
In
Out
Unix Commands
11
96
Project Euler
45
159
Benchmarks
484
/
Test sets for transpilation. Length is measured as number of lines the assembly file, and is averaged across both ARMv8 and RISC-V architectures under the -00 optimization flag.
Figure 3:
The resulting dataset shared on HuggingFacd
Inference of the system is evaluated on 3 different test sets, summarized in Table Code is emulated in Docker images with QEMurBlllrrd
Bellard (2005). Project Euler constructed from 45 C implementa-
httpss//uuuggiiisssccooaaascccct arm risc

---

RISC-V ARMv8
ARMv8 RISC-V
Method
Proj. Euler
Benchmx
Unix Cmds
Proj. Euler
Benchmx
Unix Cmds
Few-shot (GPT4)
11.1%
18.2%
4.44%
0
27.3%
Transpiler FT StarCoder
24.4%
12.5%
54.5%
8.9%
36,4%
8.9%
36.4%
FT CodeLLaMA
11.1%
36,4%
2.2%
0
36.4%
Encoder-Decoder
68.9%
6,3% 18.8 %
36,4%
66.7%
6.25%
81.2%
GUESS & SKETCH
80% %
81.2%
75.6% %
25.0 %
81.2%
Table 1: Main Transpilation results on full program accuracy (Project Euler, Benchmarks, and Unix Commands test sets). Bold shows best results with p < 0.01 significance.
tions of Project Fuler mathematical challenge problems Benchmarks is 16 C implementations of programs in The Computer Language 23.03 Benchmarks Game Unix Commands is 11 C imple- mentations of Basic Unix commands
For verification, all test sets are cross-compiled to the ARMv8 and RISC-V architectures under the -00 flag. System performance is measured by execution output match. We sample the top 100 candidate guesses for given full assembly file.
System We experiment with two different types of generative language models: a smaller trans- former encoder-decoder model with a bidirectional encoder and autoregressive decoder based on the BART architecture (Lewis et al.) 2019), and larger transformer decoder-only models pre-trained on code (Li et al., 2023] Rozière et al. 2023). The first model class is trained from seratch where the second is pretrained. All language models
trained one NVIDIA RTX A6000 GPU. The encoder-decoder models are trained for 156 hours total and the pre-trained decoder-only models are fine-tuned for 240 hours total. Pre-trained models fine-tuned with LoRA (Hu et al., 2022) De- tails of training are shown in Table 4 All resulting models are shared on Huggingface We use confidence threshold y 0.9, although we found that it was critical for accuracy. Additional experiments are in Appendix A
The symbolic solver is built with Rosette (Torlak Bodik 2013). a programming language for synthesis and verification built on top of the Z3 (de Moura & Bjørner 2008) SMT solver. The input space restricted to 16-bit bitvectors, consistent with the register sizes of the ARMv8 and RISC-V architectures used.
Baselines We consider several alternate approaches to assembly transpilation.. With Few-shot learning (Brown al. 2020), we prompt GPT-4 (OpenAl 2023) with instructions and a couple of examplar oututt assembly pairs to obtain a transpilation for a given input assembly file. See details of the specific prompt in Appendix[Di. [DI] Transpilers
manually-engineee that convert the given source assembly to the given target assembly. These are programmatically for source-target hardware pairs for which
transpilers
written for the specified source-to-targe to-target-hardwart cannot find a transpiler, we cannot obtain numbers for this baseline. We use the engineered Arm V8- 10-RISCV64 transpiler written by members of the IBM Research Haifa team transpiler from RISC-V to ARMv8. LM only methods, FT StarCoder (Li et al. 2023). FT CodeL- LaMA (Roziere et al. 2023). Sncooer-Deccode (Lewis et al. 2019). are the purely neural approaches to machine translation, in which we train or fine-tune language model with the paired assembly data. The Encoder-Deccoder method is equivalent to just the GUESS method of our approach.
We did not find
RESULTS AND ANALYSIS
Performance of our methods the test are shown in Table 1 GUESS & SKETCH outper- forms all alternative approaches with 0.01 significance level The Few-shot approach, even with
hhtps://uuggingface wwwceliieeee/baararrrrroo (anaaaaaaagaaamttttiiiccsssaaaan 4lo2222008 cloze2048
hhtpp://githhbb... wwwpittuu...omsccecm
According sample z-test.

---

Few-shot
Starcoder
CodeLLama
Transpiler
Enc-Dec
GUESS & SKETCH
Process
Length Failure
7
6
0
0
34
0
Compile
50
2
References
5
5
11
1
Semantics
Copying
Logic
5
3
3
3
Memory
10
10
9
0
2
2
Math
3
0
2
3
Correct
10
6
11
61
70
Table 2: Analysis of failures by different transpilation methods. Collected on the Project Euler test set. Categories are listed in order of bottleneck precedence.
the largest existing language model today, GPT-4, cannot successfully perform most transpilations. GUESS & SKETCH even outperforms the engineered Transpiler, which fails to translate programs for which it cannot recognize even one instruction. We run several GuEsS-only models, compar- ing from-scratch training to pre-trained models. Interestingly, the fine-tuned pre-trained large lan- guage models perform much worse than even just the trained smaller encoder-decoder model. The best best-performing baselines is the Encoder-
approach, which we use for the full GUESS & SKETCH. Further experiments testing the performance gain of GUESS SKETCH over the Encoder- Decoder approach on more test programs are shared in Appendix B and support the same 10% increase in correct transpilations.
Error Analysis Table 2 clas- sifies assembly transpilation er- rors under one of several cat- egories, determined by bottle- neck failure reason: mathe- matic, copying, ISA, references, logic, memory, and length. See descriptions of each in Ap- pendix C and examples in Ap- pendix
Input
Ieput:
stteing
01,
cep caRl
Output: RISC-V
Output: RISC-V
Ene-Dsc Output:
Guess & Skatch:
Enc-DDe
Target
.align
why
:LC3:
The encoder-decoder model (GUESS) makes few ISA mis- takes, but runs into number of emros semantics and out-of- scope references, some of which are resolved by the solver in GUESS SKETCH. However, unless the semantics of all of its erroneous subsequences are resolved, an incorrect transpi- lation is not corrected. That is, even though mathematically erroneous subsequences are being resolved across the examples in the test sets, if the bottleneck problem is not resolved not all errors are properly aligned and solved, the transpilation still fails.
47, -221
Lia
bie
ad, 2b,
SONEUM
0145
fid
fal,
Lavid
E40, Fub
Eavid
E40, Jub
call
powepte
oxll
pootatt
<no double-word
JC02
mamory
.end
2075252254
Figure 4: Example outputs.
Interestingly the other approaches fail to transpile compile before even reaching semantics. For few-shot, the model generates invalid instructions, despite the prompt including a translation instruc- tions well as multiple exemplar transpilations. Fine-tuning models generate invalid assembly from pretraining despite the fine-tuning phase. On the other hand, the manually engineered transpiler is unable to process many examples all.
Figure shows two example outputs. The left shows a guess that is resolved. The language model output (bottom, left) predicts tokens for the incorrect global memory reference, highlighted in yel-
8

---

Project Euler
RISC-V ARMv8 ARMv8 to RISC-V
Encoddr--ecoddr
30.1
34.3
GUESS & SKETCH
21.3
25.3
Table 3: Average number of samples used by the encoder-decoder and GUESS & SKETCH ap- proaches for the Project Euler test set. The range is [1, 100). (Lower is better.)
low. According to the model cross-attention, these tokens most align to those of the corresponding fmov instruction in the input assembly (top), highlighted in purple. However, in the predicted full memory location is produced with the double-word IEEE representation for the desired float 5 Ce+0. After resolution with GUESS SKETCH, a correct memory location generated and the memory reference is updated (bottom, right), highlighted in green. The example on the right shows a problem that GUESS & SKETCU does not resolve, The LM output (bottom, left) predicts tokens for the register values with low confidence, highlighted in red. A correct solution is shown (bottom, right). The register use and logic flow is inconsistent.
assembly program,
Sampling Aside from solving more examples in the test dataset, GUESS & SKETCH also reduces the number of samples needed from the underlying LM. For of test examples, they are correctly transpiled using the encoder-decoder approach only after sufficiently many samples. Using GUESS & SKETCH, a handful of these are successfully transpiled with fewer samples. Table [3 shows the average number of samples from the LM used by the encoder-decoder approach and the GUESS & SKETCH approach during evaluation the Project Euler test set. Examples that achieve a correct transpilation after the htth sample logged to use is samples, and examples that do not achieve correct transpilation within 100 samples use 100 samples.
LIMITATIONS
While GUESS SKETCH is significantly more effective than the baseline approaches, there are still several remaining open challenges.
The SKETCH method is dependent on alignment with the source sequence. GUESS fails to provide accurate alignment than the sketch may be unable correct the output issue.
Memory management issues are hard for the sketch solver. These include reasoning about val- on the stack any given point in the program, register choice decisions that incorrectly propagated during autoregressive generation, and loading memory addresses into the register.
The best performing model is a mid-size encoder-decoder, which is strong pattern matching, but likely cannot perform programmatic reasoning. Potentially larger code models could better solve some of the symbolic transpilation issues, if instruction hallucinations could be reduced.
GUESS SKETCH limited in length by the context length generative language models. Using convolutional methods such as SLeD (Ivgi et al.) 2022) could resolve these mistakes in practice.
We have no formal proof of equivalence, only checking on a small finite set of inputs.
CONCLUSION
In this work, we present GUESS & SKETCH, neurosymbolic approach assembly-to-assembby transpilation. GUESS & SKETCH extracts alignment and confidence information from a language model to guide a symbolic solver. We demonstrate the efficacy of this approach on three different test sets of assembly programs in the ARMv8 and RISC-V architectures. Future work to build on this approach to identify and use patterns in the decoder attention of the language model that may be helpful for the solver, such live variable analysis (Aho al. (0066) patterns. Other future work may include transpiling to or from higher levels code optimization and devising a mechanism to reason about more elements of the machine state, such values the stack.

---

ACKNOWLEDGMENTS
We thank Justin Chiu, Amrit Baveja, Hao Tang, Yair Schiff, Omer Gul, Kevin Ellis, Ameesh Shah, Sahil Bhatia, and Adwait Godbole for helpful conversations and feedback throughout the project. This work is supported in part by the National Science Foundation (NSF) grant CCF-1704834, CL and AMR are sponsored by NSF Grant DRL-2229873 and 2037519.
REFERENCES
Aus BASIC mach C:B to C transpiler. Amiga-Magazin, 1988(6):001
URL.
licon/
about-the- tow.--------sttttttt-ssssaaa-aaa
Icee standard for binary floating- point arithmetic. ANSI/EEEE Std 754-1985, pp. 1-20, 1985. doi: 10.1109/EEESTD. 19A.EESSD.1199..88 1995588922
The occam transpiler. Byte Magazine, 14(13):350, 1989.
Alfred V. Aho, Monica S. Lam, Ravi Sethi, and Jeffrey D. Ullman. Compilers: Principles, Tech- niques, and Tools (2nd Edition). Addison- Silison-Weseyy Longman Publishing Co., Inc., USA, 2006. ISBN 0321486811 1..
Ehsan Ardestani and Jose Renau. Esesc: fast multicore simulator using time-based sampling. In 2013 IEEE 19th International Symposium on High Performance Computer Architecture (HPCA), pp. 448-459, 2013.
doi: 10. 10000//PAAA2221155255550
Jordi Armengol-Esste Estapé, Jackson Woodruff, Chris Cummins, and Michael F. P. O'Boyle. Slade: A portable small language model decompiler for optimized assembler, 2023.
Fabrice Bellard. Qemu, a fast and portable dynamic translator. In Proceedings of the Annual Con- ference on USENIX Annual Technical Conference, ATEC '05, pp. 41. USA, 2005. USENIX Association.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neclakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Vors Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners, 2020.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pondé de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavar- ian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plap- pert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voos William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji. Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Joshua Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Pe- Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojeiech Zaremba. Evaluating large language models trained on code. CoRR, abs/2107.03374, 2021. URL http:/ttts/axxi.../t/ httpr//aas///113///33
Xinyun Chen, Chang Liu, and Dawn Song. Tree-to-tree neural networks for program translation, 2018. URL httptttsss//oeeeeeiiiieee net. net./forum?idd=ka=
Leonardo de Moura and Nikolaj Bjomer. Z3:
An efficient smt solver. In C. R. Ramakrishnan and Jakob Rehof (eds.), Tools and Algorithms for the Construction and Analysis of Systems, pp. 337-340, Berlin, Heidelberg. 2008. Springer Berlin Heidelherg. ISBN 978-3- 978-3-540-78800-3
10

---

Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter the Association for Computational Linguistics: Human Language Technologies, Volume (ong and Short Papers), pp. 4171-4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10. 18653/11/N19-1425
URL https:
//aclanthology wwwpethhoooo.o.oo/...... 1423
Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. Codebert: A pre-trained model for pro- gramming and natural languages. In Findings of the Association for Computational Lingnis- tics: EMNLP 2020, pp. 1536-1547, Online, November 2020, Association for Computational URL https:://accattttttttt
Linguistics. doi: 10. 12.18663/1/2000.mmmg www.dinggseeenl....... org/2020. www.ffiiinnnnnemmnnnm emnlp 139
Daya Guo, Alexey Svyatkovskiy. Jian Yin, Nan Duan, Mare Brockschmidt, and Miltiadis Allamanis. Learning to complete code with sketches. In International Conference on Learning Representa- tions, 2022. httttt:s////:teeereeee
www.:ntt//ommmdagggeeooo (ttp:/iiiig999s022333 9uMSC6ZBT
John L. Hennessy and David A. Patterson. Computer Architecture, Fifth Edition: A Quantitative Approuch. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 5th edition, 2011. ISBN 012383872X.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang. and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Con- ference on Learning Representations, 2022. URL
net/ forrmm?
id=nZevKeeFYf9
Jingmei Hu, Eric Lu, David A. Holland, Ming Kawaguchi, Stephen Chong, and Margo Seltzer. Towards porting operating systems with program synthesis. ACM Trans. Program. Lang. Syst. 45(1), mar 2023. ISSN 0164-0925. doi: 10.1145/3563943. URL http::/ //o::///ooi.. orq/10. 1145 1545/3353943)
Maor Ivgi, Uri Shaham, and Jonathan Berant. Efficient long-text understanding with short-text models. 2022.
Daniel D. Johnson, Daniel Tarlow, and Christian Walder. R-u-sure? uncertainty-awant code sugges- tions by maximizing utility across random user intents. In Proceedings of the 40th International Conference on Machine Learning, ICML'23, JMLR.org. 2023.
Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. Large language models struggle to learn long-tail knowledge, 2022.
Svetoslav Karaivanov, Veselin Raychev, and Martin Vechev. Phrase-based statistical translation of programming languages. In Proceedings of the 2014 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming & Software, Onward! 2014, pp. 173-184, New York, NY. USA, 2014. Association for Computing Machinery. ISBN 9781450332101. doi: 10.1145/2661 1002266613362200 148. URL https:///e:: .owooiioorg/00/ 1o.g/10.1145 1145/ 2661 36. 2661148
Denis Kocetkov, Raymond Li, Loubna Ben Allal, Jia Li, Chenghao Mou, Carlos Muñoz Ferrandis, Yacine Jernite, Margaret Mitchell, Sean Hughes. Thomas Wolf, Dzmitry Bahdanau, Leandro von Werra, and Harm de Vries. The stack: of permissively licensed source code. Preprint, 2022.
Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondřej Bo- jar, Alexandra Constantin, and Evan Herbst. Moses: Open source toolkit for statistical ma- chine translation. In Proceedings of the 45th Annual Meeting of the Association for Com- putational Linguistics Companion Volume Proceedings of the Demo and Poster Sessions, pp. 177-180, Prague, Czech Republic, June 2007. Association for Computational Linguistics. URL
Celine Lee, Justin Gottschlich, and Dan Roth. Toward code generation: A survey and lessons from semantic parsing. 2021.
11

---

Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. Bart: Denoising srquence-t--sqqueee pre- training for natural language generation, translation, and comprehension. arXiv preprint arXiv: 13461, 2019.
Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Mare Marone, Christopher Akiki, Jia Li, Jenny Chim, Qian Liu, Evgenii Zheltonozhskii. Terry Yue Zhuo, Thomas Wang, Olivier Dehaene, Mishig Davaadorj, Joel Lamy-Poirier, João Monteiro, Oleh Shliazhko, Nicolas Gontier, Nicholas Meade, Armel Zebaze, Ming-Ho Yee. Lo- gesh Kumar Umapathi, Jian Zhu, Benjamin Lipkin, Muhtasham Obiokulov, Zhiruo Wang, Rudra Murthy, Jason Stillerman, Siva Sankalp Patel, Dmitry Abulkhanov, Marco Zocca, Manan Dey, Zhihan Zhang, Nour Fahmy, Urvashi Bhattacharyya, Wenhao Yu, Swayam Singh, Sasha Luc- cioni, Paulo Villegas, Maxim Kunakov, Fedor Zhdanov, Manuel Romero, Tony Lee, Nadav Timor, Jennifer Ding, Claire Schlesinger, Hailey Schoelkopf, Jan Ebert, Tri Dao, Mayank Mishra, Alex Gu, Jennifer Robinson, Carolyn Jane Anderson, Brendan Dolan-Gavitt, Danish Contractor, Siva Reddy, Daniel Fried, Dzmitry Bahdanau, Yacine Jernite, Carlos Muñoz Ferrandis, Sean Hughes, Thomas Wolf, Arjun Guha, Leandro von Werra, and Harm de Vries. Starcoder: may the source be with you!, 2023.
Antonio Valerio Breeii-Barone, Fazl Barez, Ioannis Konstas, and Shay B. Cohen. The larger they are, the harder they fail: Language models do not recognize identifier swaps in python, 2023.
Niels Möller and Torbjorn Granlund. Improved division by invariant integers. JEEE Transactions Computers, 60(2):165-177 2011. doi: 1.11109..200..1444
Anh Tuan Nguyen, Tung Thanh Nguyen, and Tien N. Nguyen. Lexical statistical machine trans- lation for language migration. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2013, pp. 651-654, New York, NY, USA, 2013. Associa- tion for Computing Machinery, ISBN 9781 4503223777
doi: 00.1145/2291411.24455844 URL
https::/doi... http//oi..ggg//// 11t.//...111/211111111 htt.:///..0.11114/991111111112.1 2494584
Maxwell I. Nye, Luke B. Hewitt, Joshua B. Tenenbaum, and Armando Solar-Lezama. Learning to infer program sketches. CoRR, abs/1902.06349, 2019. URL
1902.06349
OpenAl. Gpt-4 technical report, 2023.
Ankur Parikh, Oscar Täckström, Dipanjan Das, and Jakob Uszkoreit. A decomposable atten- tion model for natural language inference. In Proceedings of the 2016 Conference on Em- pirical Methods Natural Language Processing, pp. 2249 2255, Austin, Texas, November 2016. Association for Computational Linguistics. doi: 10.18653/v1/D16-112 1244. URL htthttss:
David A. Patterson and John Hennessy. Computer Architecture: Quantitative Approach. Mor- gan Kaufmann Publishers Inc., San Francisco, CA, USA, 1990. ISBN 1558800698.
Alec Radford and Ilya Sutskever. Improving language understanding by generative pre-training. In arxiv, 2018
Baptiste Roziere, Marie-Anne Lachaux, Lowik Chanussot, and Guillaume Lample. Unsupervised translation of programming languages. Advances in Neural Information Processing Systems, 33, 2020.
Baptiste Roziere, Jie Zhang, Francois Charton, Mark Harman, Gabriel Synnaeve, and Guillaume Lample. Leveraging automated unit tests for unsupervised code translation. In International Conference on Leurning Representations, 2022.
fformm?
11-cmt-6KtR4p
Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, Jérémy Rapin, Artyom Kozhevnikov, Ivan Evtimov, Joanna Bitton, Manish Bhatt, Cristian Canton Ferrer, Aaron Grattafiori, Wenhan Xiong, Alexandre Défossez, Jade Copet, Faisal Azhar, Hugo Touvron, Louis Martin, Nicolas Usunier, Thomas Scialom, and Gabriel Synnaeve. Code llama: Open foundation models for code, 2023.
12

---

Daniel Sanchez and Christos Kozyrakis. Zsim: Fast and accurate microarchitectural simulation of thousand-core systems. In Proceedings of the 40th Anmual International Symposium on Computer Architecture, ISCA 13, pp. 475-486, New York. NY, USA. 2013. Association for Computing Machinery. ISBN 978 450320795.
doi: 10. 10.114552245592224486666
URL https://doi.
org/10.1145/222 10000.145//20 1tt.1144/22822.2222
Moshe Schorr, Matan lvgi, Hillel Mendelsonl, Shay Aviv, Herann Theiler, and Tom Kolan. arm2riscv. https: http://gtttuu..tto www.hhww.oommeecoooomoeooeeeoeee hwtpm///cooor//aaaae att.orrrrmmmmmmrsc....omm 2020.
Armando Solar-Lezama.
The sketching approach to program synthesis. In Zhenjiang Hu (ed.), Programming Languages and Systems, pp. 4-13, Berlin, Heidelberg, 2009. Springer Berlin Hei- delberg. ISBN 978-3-642-10672--9
Armando Solar-Lezama, Liviu Tancau, Rastislav Bodik, Sanjit Seshia, and Vijay Saraswat, Com- binatorial sketching for finite programs. SIGARCH Comput. Archit. News, 34(5):40 oct 2006a. ISSN 0163-5964. doi: 10.1145/1169919.11 168919 URL httpp:://dii..re 1145/1168911 1168907
Armando Solar-Lezama, Liviu Tancau, Rastislav Bodik, Sanjit Seshia, and Vijay Saraswat. Com- binatorial sketching for finite programs. In Proceedings of the 12th International Confer- ence on Architectural Support for Programming Languages and Operating Systems, ASP- LOS XII, pp. 404-415, New York, NY, USA, 2006b. Association for Computing Machinery. ISBN 1595934510. doi:
URL httpttttt///.ll... 0tt.//0000.00/1111115555
1168857. 1169077
Mare Szafraniec. Baptiste Roziere, Hugh James Leather, Patrick Labatut, Francois Charton, and Gabriel Synnaeve. Code translation with compiler representations. The Eleventh International Conference on Learning Representations, 2023. TRL/tttps://opereveaie net forum?
Id=XXoEUUeNeeSQ
Emina Torlak and Rastislav Bodik. Growing solver-aided languages with rosette. Onward! 2013, pp. 135-152, New York, NY, USA, 2013. Association for Computing Machinery. ISBN 9781450324724. doi: 10.1145/2509578 2509556.
URL http::://eei..ogg//.///
2509578. 2509586
Helena Vasconcelos, Gagan Bansal, Adam Fourney, Q Vera Liao, and Jennifer Wortman Vaughan. Generation probabilities are not enough: Exploring the effectiveness of uncertainty highlighting in ai-powered code completions, 2023.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention all you need. Luxburg, S. Bengio. H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Ad- vances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL E1le/35fee22335777ee11)1133d...2
Guyon, U. Von
cctppper.pppp..eeeeeeeeeeeee.eee filefilllle pdf
Wenwen Wang, Stephen McCamant, Antonia Zhai, and Pen-Chung Yew. Enhancing cross-isa dbt through automatically learned translation rules. SIGPLAN Not., 53(2):84-97, mar 2018. ISSN 0362-1340. doi: 10.1 10.1145/2999957.377760. 3296957.31774
URL
A IMPLEMENTATION DETAILS OF GUESS & SKETCH FOR ASSEMBLY
Function boundaries. The length of assembly files often well exceeds the context window size of the language model. To handle this issue, we perform translation through the language model by separating functions from one another and translating them individually. This decision is grounded in the fact that for the ISAs tested, most information in the functions is independent of instructions in other functions. This is especially true with regard to the general structure of the computations rather than specific levelevel details and values. The language models trained these separated assembly functions. In inference, the models are passed separated assembly functions, and the resulting function translations concatenated back together compose the full assembly program.
13

---

Confidence threshold: y The underlying language model can be very confident about incorrect
predictions (Johnson et al. 2023; Vasconcelos et al. 2023). In the assembly translation setting, this often happens for example when referencing out Section 4.1 This is why domain specific heuristics can help the GUESS & SKETCH system iden- tify which basic blocks correct. To evaluate the effect of y system performance,
entities, as described at the end of
sweep [0.8, 0.9, 0.95] with the Project Euler test set. A lower would flag fewer potential errors for correction, which may reduce or maintain the number of instances of sketching. Fewer sketching instances may result in fewer corrections, but has the benefit of reduced computation time. We find that across these y values, the number of corrected programs is the same, but the inference runtime increases with 7. From 0.8 to 0.95, the inference time increases by 2.2x.
A.1 ALIGNED SEQUENCES IN ASSEMBLY: PURE BASIC BLOCKS
Assembly basic blocks are sequences of code lines that have a single entry point and single exit point. That is, there are no branching operations within the code sequence (Patterson & Hennessy) 1990). We introduce pure basic blocks, subset of basic blocks defined sequences of assembly code lines that have a single entry point, a single exit point, and no memory or stack management within the code sequence. This constrains pure basic blocks to be code sequences in which all data is either passed in via values already loaded into registers, or constant values coded into the sequence. This decision remove memory operations and other control flow instructions greatly simplifies the equivalence relation between source and target subsequences.
Identifying out-of-scope references. In the context of assembly, out-of-scope references as po- tential mistakes are classified as any piece of code that use or reference global memory. Examples include the ll instruction in the RISC-V architecture or custom string or function definitions.
Extract pure basic blocks. From given token in the sequence, we identify the surrounding pure basic block by inspecting the neighboring assembly lines. We greedily search lines upward and downward from the given token until one matches section boundary definition, branching. memory management, or stack management operation. The enclosing lines comprise the pure basic block.
We identify pure basic block inputs and outputs as values in relevant registers upon input and upon exit. Free registers in the basic block are registers that are read from before they are assigned to, and are considered inputs to the pure basic block. Values in the final registers of aligned pure basic blocks are considered the outputs of the pure basic block.
For pure basic blocks with global references, semantics of the referenced entities are extracted from the full program sequence by performing a string-matching search for the referenced label and its following definition.
Translating pure basic blocks. We lift assembly blocks from their corresponding hardware lan- guages into an intermediate form usable by the synthesis engine. In this work, pure basic blocks that may be marked as potentially erroneous can be marked due to either global references or low- confidence token predictions.
Potential errors due to global references are solved using a custom solver designed for resolving global references. Pure basic blocks with global references must include the definition of the refer- enced entity in its semantics. The aligned entity on the input side, whether retrieved from its global definition or directly obtained from the input pure basic block, is translated into its bitvector repre- sentation. The pure basic block sequence and the bitvector representation of the correct entity value are passed to the global reference solver.
Potential errors due to low-confidence token predictions are solved using the Rosette (Torlak & Bodik] 2013) program synthesis engine. Aligned input and output sketch subsequences Ty, and are lifted into Rosette functions Pæyz and P., where Pa is a partial program with holes replaced by Rosette symbolic constants. The lifting is done by mapping each assembly line its Rosette counterpart according to the semantics of the corresponding assembly hardware ISA.
Solving the sketch. The global reference solver solves for hole mappings in output pure basic block sketch by either resolving the global reference label used directly translating the entity
14

---

ARMv8 Rosette
RISC-V Rosette
Finow ed, irm
Led liazi]
11
Exd
estu rd,
End lsign-ertend ra teteveetor 65331
Exd tesgn-exteedd (bitveotor 6411
nove rd, het, lel ims
End
tcunest (extracct 65 32 nnt (bvor
lbsel->bitvenke (bvalt inm))]
toxtzact 31 zdi tovahl box 1mn
Lsl iim
Lavahi ims)
lax Lim
taviabs 22 1xs1]
axal
Ibwashz
lbwwahr ims1]
ed, cal
Led. Ibvadd zal 2o2)
rladd xol,
Ibvadd
12n0e ed,ca
[zd. Ibvnnq sal;
dlang rosette
langs
<aro input rogo? <SEo, ou:put reg?
(define (bitvector
idefiie (reg54
(bv 1 reg64711
requence>
(late FS rosetto, Siquenoe
serc_output_reg
(xtgt (etgt_ igg__irut_reee
<tgt. Inget rege> <tat cotp_couuu_eeee zog?
13
12
aactolicmmmms comttrentte
(late (iggl_ role_oosttteeee sequenee
<igl_nittst_ree) rege))
14
lestine-rvtolle neqp47)
<ept_ rww__rootttteeeeee requenee>
idefise
17
Lsynthestze
18
#:forall 1iist xi
#ipuarantee (ove) lange (etgt_lang- xib11>
Figure 5: Assembly instructions are mapped to Rosette instructions according to the semantics of the corresponding assembly hardware ISA (sample shown at top). Holes in the sequence (indicated in dashed red rectangles) are translated into Rosette symbolic constants. The resulting Rosette structions, along with the input and output registers, are plugged into Rosette function template (bottom) to generate full Rosette program whose solution produces a corrected mapping from holes to values.
Model (# params)
L.R.
Batch
No. Steps
LoRA
LoRA Modules
Quant.
Enc-Decoder (400M)
3e-5
520k
Mlarcoder-Base (15.5B)
5e-6
16
2.9k
16
c projj c atttn, gatn __projy__proj
int8
CodeLlama (13B)
5e-6
16
2.9k
16
int8
Table Training details for language models used.
in the block. If the erroneous token in the output pure basic block is reference label, the solver searches for entity definitions in the full generated program sequence whose bitvector representation matches the desired bitvector value identified definition replaces the hole left in the sketch. If the solver does not find a match, it creates new global definition with a unique label, and uses that label to replace the hole left the sketch. If the erroneous token in the output pure basic block is numerical value, the solver translates the desired bitvector value set by the input sequence into the representation expected by the ISA and replaces the hole left in the sketch with the resulting value.
by the input sequence. If it finds a match, the label of the
Sketches for errors due low-confidence tokens are solved by Rosette. Rosette solves for the hole mappings by ensuring that for all program inputs, the two functions are equivalent. This process is shown in Figure
A.2 MODEL TRAINING DETAILS
Details about training the generative language models are shared in Table
B ADDITIONAL EXPERIMENTS
To further test the benefit of GUESS & SKETCH over just the language model approach, we run experiments with more Project Fuler examples. We collect solutions to 82 additional unique Project Euler problems implemented in C TD and compile them to the ARMv8 and RISC-V ISAs under
15

---

Method
RISC-V V to ARMv8
ARMv8 RISC-V
Encoder-Decoder GUESS & SKETCH
37.8% 34.1%
41.5%
51.2%
Table 5: Performance on More Project Euler problems.
the -00 optimization flag. The average number of lines in these programs is 246. The results of running the strongest baseline and our method are shown in Table GUESS SKETCH continues to provide performance gains averaging approximately 10%.
C CATEGORIZATION OF FAILED TRANSPILATIONS
Failed transpilations are categorized under one of several bottleneck failure reasons, listed in order of precedence. Process failures include length and process failure, which the very process of transpilation fails on the given input. If an example does not encounter process failure, the next category compilation failures including using the incorrect ISA instructions or global references. If the example successfully compiles, the next category of failures it may encounter is semantic failures including mathematic reasoning, copying, operational logic, and memory mis-management. These categories further described below.
Length. Some transpilation methods suffer from long input and output sequences. For example, current attention-based language models generally have context window limit, sequences that exceed that context window length will not be able to be processed by the language model.
Process failure. Examples that fall under this category are ones where the transpilation process fails when processing the input, such as the rules-based transpiler that breaks down upon receiving an input that it cannot parse.
Incorrect ISA. In assembly transpilation, the produced sequences must use exactly the instruc- tions and entities available to the hardware in question. Failure examples that fall under this category produce sequences mistakenly use syntax that is incorrect or that actually belongs to different ISA.
Global references. Assembly programs might make references to entities that are invalid, or oth- erwise use or define global reference labels incorrectly. In these cases, the program will fail.
Mathematic. Math errors are ones in which the translation process fails to correctly perform the required mathematic reasoning for translation. Examples include translating code idioms such as different implementations of division (Möller & Granlund, 201T), addition and subtraction of large constants, and translation of float values to their IEEE 754 representations
(iee 1985).
Copying errors are ones in which part of the input sequence fails to be copied to the out- put sequence. Examples include copying of constant strings, constant numeric values, and custom function names.
Copying.
Incorrect operation or register logic. The produced assembly sequence may use syntactically valid but semantically incorrect logic. These logical errors involve incorrect register or operation use, and the subsequent propagation of such mistakes.
Memory mis-management. Assembly code must be able to reason about values in memory and manage memory access. Errors this category are indicated by attempts to access memory at incorrect invalid stack memory locations, which may yield stash smashing, stack overflow, or segmentation faults the latter, and unexpected values either case.
C.1 EXAMPLE ERRONEOUS TRANSPILATIONS
In this section, include example erroneous transpilations from different methods.
16

---

LLM output: RISC-V
ma main:
addi
sp, Sp,
addi
a4, s0,
sd
ra, 8264 (sp)
........
1bu
0 (a4ta5) a5, 64 -64 (s0)
sd
r0, 8256 (sp)
sub
sd
s1, 8248 (sp)
1w
addi
sp, 8272
(a) Arguments are invalid.
(b) Offiet values are out of range.
Figure 6: The fine-tuned pre-trained code models tend to instructions from ISAs other than the one which it is directed to use. Underlined arguments indicate invalid productions.
Pre-trained code language models, even after fine-tuning with examples in domain, tend to make more ISA mistakes than do other methods. Figure 6 shows two examples of erroneous generated code from fine-tuned Starcoder-Base method. Figure ba shows an example of the fine-tuned Starcoder-Bas Base method producing code that is largely correct, but violates syntactic rules of the target hardware (RISC-V) by using added-register offsets for the lbu instructions. The syntax of RISC-V 64 does not allow register value addition for loading unsigned bytes by address. It also only allows subtraction by a specified register value rather than an immediate. Figure 6b shows code that allocates then uses large stack space, but in doing so actually violates syntactic rules of the target hardware (RISC-V) by using an immediate value outside the legal 12-bit immediate ranges for the addi and sd instructions.
Mistakes from fine-tuned code LLMs.
D BASELINE IMPLEMENTATION DETAILS
D.1 PROMPTING GPT-4
The prompt used extract translations from GPT-4 for Arm to RISC-V is as follows. For function translations:
You are able to translate assembly code from ARMv8 to RISC-V 64.
ARMv8:
main:
n n. LFBO: n),
x30, [sp. -48]!)a),
cfi.def.cfa.offse L. 29, 148 [x0, [x0,
30, -40),n
[imov ta)p)
[x0]],n(tstr(xxx, [sp. 40] ( ('n))tarrp\tx0 :
_Ld0 tprinit sp. 24%
1000,x0,10000 :10100000000)
tx0 x0. (tt:::/lhhttllt..ioooosssaaaaa ".n
[sp. 2
(tw1, 34953)nn 0x8888, Isl t6) wO, tlsr)
tx1, x1, 32) wo,
wl, t,n) wO. 31
(tsub)twl. wl. tm) ||00)))\,mul]] wl. t [sp 2 16 (smulllsmll)) wI, [Is]]tss]xxx x0, 32). wl, w tasr tww wO, 41n)taar))tu tww, wO, 4|m)tsub)(ww) w0, :ls wo, [in]tsub] tw2, wl.
[sp. 24]] ,
0x8888, 1sl
wl, 31]n]tsub]te2 w2, w0) w
(mo,) t2|m) w0, 2|n]tadd))tw) w0, tl] w0
3)n) 32) lwd wl. tstr [sp. 36][n][]/dr)|w1 [sp. [a)
[sp. 321
tx0, [tdd) x0. :102::1.hhht00011pppinnmmmmmooop 0)
imo tad [tt.::.....cccoooooaaccaaaaaaa1.
[x0, #:
[sp. 100
[x0]]n) [
tsubs (xx3, x3, (tt.iiooww...uuullqp....1111....
(.3:)))mmov(ee wl'n t1dp) x30. [sp]. 48.p n)
17

---

RISC-V 64:
main: A (tadii.ssp.............. .sp.s...8pp... (ndd (ttp:..2pppnnnnaddiiill..
| || (s (sa) -
s0,-00)
a3
a5
ccccckecck__ee caik_fil@plllp
|0ga (00 taddi
48 18)n)tjr)tre
ARMv8:
main: n. LFL6: ofi.sstartprroc]]][ssppp1aa x30, [sp. -64]]\n)t
cfi_def_cfa_offee cfi 29, -64.n\t.ccfiooffset 30, -56),
(tmov\ bt =:g:::_ssacccchkkouuudaa/.______
[x0,
[x0]],n]tstr(xx1, _ [sp. 56])n
#:got lo12: stack chk |
\tmov)txl,
tprintf(an/ sp. 200 100.x00.11
1x0 x0. wt111:0:000000010.00000000001110 (00
:0, sp. 10
[sp (9).
.LC6),)) (mm))
[.d0::.............0.00100111110 sp. 1
x0,
11110,da00000111111111d000..., x0. :10l::
x0
http::1.|attppprrr.............0 x0. :lo12 :.
[x0, #:got_lo12
tmo
twl, :0: chk gu) [x0,
tsubs' \x3 x3, [ [b [x
56]],n)
.L3:)n (n:ontmooorwww. wldp x30, [sp], t.) n),
refi.rettore offf t,
RISC-V 64:
main: htgp./pp_.ooolltt...a.....aaaaa. (s)) IsO
[ti))aa,
tal |ashhhaa....Cchhhcacarrppaaalaa0 [Fiieee||hliiii.....aaaaa3333,aa
.. ..LCS [.tphhechhhhnmlll/lllll0l00/o.ll ta5, mag.shhhuuu..00000000113344..4.. .20% wwtpa/awwbhhccllggprpprtldassssa . http://cchhhccclurrrrreeiiiiiaaa tas a3, [ag.a//hhhhiia....0iihaiaaaass._
t stack chk htt 48ds)))aa/addd))pppppp....
ARMv8:
18

---

b:|n zero (88) global (tebal(te)a)) (e) aliga (13 \a\ in) type ttc, ( 8
RISC-V 64: b: b:|n zero 18 | g align t.. @object/att.sszeell
8
ARMv8:
foo: n N...FLL (fai
x30. Isp. -16]!)
cfi.offset cfi.def.cfa_offsee 1..n 29, -.ooott.cfi.oofsset 30, -8
tmov\ sp)n)ttarpp.tx glo tbar tadrp) g 2 x0,
x0, :loll:gllball]n(.bb]
nxttarp)xxx, (bl (ald tbar talrrp global.6'nj x0, o10 ::o02::gobbll.6'aa(bb1...
[x0, :: #:g00_oo12:gloall.311
global_3/p](udd)(000) 100 :o2::
n)\ inooop) tldp (x22 1x29, x30, [sp]. [f t. ctn.tt..fi.dfffffff.
RISC-V 64:
foo: taddi (aaddi\spp....-10........
_8(sp) tsd 1s0
tsd _0(sp) (n) tadii 1so sp
.166 (ta, :
tllla ( g|o
teall
thar@ paar@plt)nntll (aann)/aaaaaaa.a.a1aaaaa global 3 gloohal
. tala 1lla)tto global 5 5)
tcall (baa@ba@rprp.. (bbar@pl))a)tlla ta0, global (gn)tcall\(bbl pnoopp tld tra gglloba
((s (s0 (0sp) taddi 16 t)
ARMv8:
{ finsert input code to translate}
RISC-V 64:
For outer file translations:
You are able translate assembly code from ARMv8 to RISC-V 64.
ARMv8:
\t arch armv8-a)n)t file 1 tt section
rodata )n) blt. :t"ter Enne your age: "(n)t. align) are % your grade:
years odl.:
(t"Your grade is: (
your gpa string (t"ig saaappaassss..... is: "tt""""""."..."..."""""""""""""" your name: text
name is n).t. tmaik. /function\nf main }. LFE6: siz smzc) [ (Ubuntu 11.3.0-lubumi 22.04) n.. [:: GNUost.kU ""N=sotck."".@prrobbiis"""" @prrogbits
RISC-V 64:
(t. (fflel/"progrrmm19920222 "0m pic text
section t, textion a a.i \nEeer your age: ". 13
rodata
years o oddl ( are %d
your grade: "\n string) your gpa: string (tagg(""i Your gpa is: %1f { ||,""""||000"tt.iinnn"""""""""""
Your grade is: "c
"in't. http://hhhh..........aaaaaa"""a1
[1:" name is ()) @ @function (n {
your name: text n size Imain -main) mainkn)tt.idmm LCGCC: (Ubuntu I1.3.0-lubuntul -22.04) (n). section note GNU-stack "" "m".@roobits'pn
ARMv8:
(t. a file (te file
program 2490936. 2490936.c*(ply .006...""n) te
rodata n align 1.3. string Enter the distance the van
19

---

has travelled: :")
"" n
a.aign 13)
LC2;)n)t. string ( Amount te (2 n global Anit. type type\tmain
t.globall tmain tize) - -main n ident "GCC: (Ubuntu 11.3.0-1 (22.04)
note GNU- " @roogbits)nn
RISC-V 64:
lt. file ftleeiee"eeeee p (""rrogram2249090 1e1366..mmmni to text
(t. rodata
align LCO. s..iininng (""Eeer
Enter the distance the van has "t"rigg align glll nn)t. "GCC: (
travelled:" : (t
LC2:)
m tmain, n.. section note -Nt-stack
type \ ttmann @function (omcctionn/mmai///t.... (maii
Ubuntu
ubbutul
11.3.0" s
@ @progbits)n
ARMv8:
tt. arch armv8-a)n)t, www.fiieltttt pooe(ttpppgra110079000 (0 nn text next) global globalltbbnii
bss bst align type bobject\n't ttb ( Ho 1. size size
(Ubuntu 11.3.0-lubuntul
22.04) 11.3.0" 11.3.0"(ntt.seere section note GNU--ttcck @progbits
RISC-V 64:
(t. file \t" [""rogrram40790777 .0007...."l1 nnIn)t option pic text globl) t bss aliga (3) tbb @o size size (ee) size ltf t.. I'CC: (Ubuntu 11.3.0- ubuntul 222004) 11.3.0") nit. section note GNU-stack, not....NUssst...stcc
@@progbits),
ARMv8:
\t arch armv8-a)n) (tt.flleep"orraa 77740889 .c")n't "748089.c"''''
t.. (13
string) ('%f()nnk n\ (1 (13 I.
L.t:)n)t..triin "" text) t.iig g.
(n)t.
time) --main t. ident) L'GCC: (Ubuntu 11.3.0-1ubuttu 222.44) 11.3.0" sectionn (t. .NU- stack "" @progbits)
type tmain Sfunction (
RISC-V 64:
(t. file file)tt pror''"proamm777880000 . t. option pickn t.cllntttextii
rodata (ntt. align I.C n))t. string ( (*% | (n%f"(nnt align I.CII
section (t. t
(t.string \t.string\""'if"" ("" aligalligni aliganiiigl textx)))))
(....llooolll tmain'tminns (a) t.) t. tmain @ (aauctiion)(mmmin]]]] size (main. -main - iddlt. I (Ubuntu 11.300-1ubuntu 22.04) 11.3.0" progbiits
note settcion
section
stu-stack,""
ARMv8: (insert input code to translate]
RISC-V 64:
The reverse direction reverses source and target language specifications accordingly.
20


Reviews:
[{"review_id": "PZAO8ki9Ov", "reviewer": "Reviewer_nDKt", "summary": "The paper presents an approach for machine language translation. They attempt to utilise a generative language model with a confidence score to identify uncertain blocks or \"guesses\" which can then be symbolically solved using a neuro-symbolic solver. They rely on Sketch (Solar-Lexama  et al)  prior work for handling the expansion/completion of the uncertain tokens. The authors perform experiments on other three datasets (Unix, Euler, Benchmarks) outperforming or equal (in rare cases) in all test settings.", "questions": "- Explain more how the confidence is applied and used is it based on prior work as there is significant literature in this area\n- Does the Lamda hyper-parameter effect the output, was this ablated on, but not included?\n- Why do you choose only the region of error to be solved? Did you consider using a buffer before and after as well to increase the consistency across the section and provide context?", "limitations": "", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "- While a simple concept, the method outperforms prior work\n- The concept of uncertainty is a good mapping to identify holes in the generated program\n- Evaluation is robust and thorough, providing analysis of failure cases\n- Authors identify a setting for Neuro-symbolic approaches to work stably and outperform prior works", "weaknesses": "- Novelty within this approach is quite limited the translation is a standard approach the confidence is simple (see below), and they use an existing neuro-symbolic solver therefore, it is more on the sole idea of putting these together. This is the main criticism. However, they outperform prior work, and the idea is interesting and technically sound. \n- Confidence is very trivially explained. In general, deep models are very confident even when they are wrong. It isn't clear how this was implemented and is critical to the method. As the author's rely on this to identify potential errors for solving.\n- The parameter lambda is not ablated on as the threshold for identifying blocks. It is unclear if this is set low to allow more errors i.e. false negatives but to make the result more robust.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Z4w14pVdEj", "reviewer": "Reviewer_awtu", "summary": "This paper presents an approach for translating low-level assembly programs into higher-level code for the purpose of analysis and human understanding. The approach is based on a combination of neural processing and symbolic program translation. The proposed approach,  called GUESS & SKETCH, extracts alignment information from features of the neural language model and passes it to a symbolic solver to perform \"transpilation\". The paper also presents experiments illustrating the benefits of GUESS & SKETCH as compared to GPT-4 and an engineered \"transpiler\".", "questions": "Please see above?", "limitations": "", "rating": 8, "confidence": 2, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The paper is well written and presents a clear contribution. The combination of generative language models and program synthesis by sketching is new and it is shown to be effective as compared to state of the art techniques.", "weaknesses": "I could not understand the correctness guarantee provided by the approach. The authors say \"the correctness of GUESS & SKETCH is always lower-bounded by the correctness of the initial guess\" -- the authors should explain what they mean by lower bound here. If the translation is incorrect, how can it be useful in practice?\n\nThe scalability is unclear.  What s the largest program that has been translated using the approach presented here?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "geOOiN11eZ", "reviewer": "Reviewer_LBov", "summary": "this work presents a way of transpilation: turning one assembly code into another functionally equivalent assembly code.\n\nthe main techniques consists of: using a LM to generate candidate programs. from the internal values of the LM 1) alignment/attention and 2) uncertainty, generate localized sketches with holes for the candidate program. this localized sketch is then solved, with the holes resolved to values that are provably equivalent to the source code's corresponding fragments. the fragments are then stitched together, finishing the transpilation process.\n\nresults show the proposed method beats a reasonable set of baselines -- a heuristic based transpiler, and few-shot using gpt4.", "questions": "## program equivalence?\n\nAs I understand program equivalence is an undecidable problem. If I recall correctly, synthesis systems like sketch does not have a way to fully verify the SKETCH and the SPEC are identical over all input-outputs, but only over a bounded domain?\n\nIs this an issue for your work? Or is it because everything is bounded to begin with, as we're working over assembly and we only need to account for, for instance, 16 bits integers or similar ? Or is it some Z3 theory magic that allows for a DSL which programs can be reasoned for full equivalence?\n\nAt any rate, this should probably be clarified in the paper.\n\n## successfully compile = ?\n\nIf I read correctly, success is measured over a set of input-output test cases to see if the input code runs the same as the compiled output code. Is this related to the program equivalence problem above somehow? Is this comprehensive enough to make sure the transpiling is not mistaken?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "## the good part of quality: that it worked\nThe presented method works, on a domain of highly structured translation task (i.e. highly stylized texts), something a language model should perform very well at, and it shows. The extra care taken to correct the translation locally is a reasonable yet good idea to complement the weakness of the language model.\n\nThe benchmark is thorough, and the evaluation (on what is being shown) is solid. \n\n## clarity\nI am very grateful how this work is able to encapsulate the domain specific aspects of compiler and assembly, so that the top level algorithm remains accessible to the ML audience. Thank you!\n\n## novelty : fair\nI think it is a straight forward paper, and it outlined reasonable decompositions of the transpiling tasks to LLM and a symbolic solver.", "weaknesses": "## the not so good part of quality:\n\n### evaluation set is small \nThis work can be significantly beefed up with a synthetic test set. Evaluation on mere 100s of programs is likely not sufficient. Since it is possible to compile C into both architectures, and since test generation / fuzzing is a well established approach, this work can benefit from an artificial/synthetic test set consists of about ~1k programs, to evaluate the correctness of the transpiler more thoroughly. \n\n### lack of statistic tests\nAt least we should see confidence intervals of the results, or some kind of t-test to make sure that the proposes method is better than the baseline not due to noise. Kindly ask your colleagues in statistics to look over your tables and give recommendations on how it could be made bullet proof.\n\nI would love to see this update in the rebuttal period.\n\n## fit of venue\nWhile I think this is a good paper, it might be a better fit at PLDI. As I am unsure what is the AI/ML lessons gained from this work, other than it is possible to build such a system, and some relatively detailed finding on how well language models are at learning over a highly stylized text (assembly code) when compared to English sentences.\n\nHowever, as other application papers of the compiler flavour has a precedence of appearing at ICLR, this is not a major concern.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)