Original: Learning C to x86 Translation: An Experiment in Neural Compilation
Revised: Learning C to x86 Translation: An Experiment in Neural Compilation

Original: Abstract
Revised: Abstract

Original: Deep learning has had a significant impact on many fields.
Revised: Deep learning has had a significant impact on many fields.

Original: Recently, code-to-code neural models have been used in code translation, code refinement and decompilation.
Revised: Recently, code-to-code neural models have been used in code translation, code refinement and decompilation.

Original: However, the question of whether these models can automate compilation has yet to be investigated.
Revised: However, the question of whether these models can automate compilation has yet to be investigated.

Original: In this work, we explore neural compilation, building and evaluating Transformer models that learn how to produce x86 assembler from C code.
Revised: In this work, we explore neural compilation, building and evaluating Transformer models that learn how to produce x86 assembler from C code.

Original: Although preliminary results are relatively weak, we make our data, models and code publicly available to encourage further research in this area
Revised: Although preliminary results are relatively weak, we make our data, models and code publicly available to encourage further research in this area

Original:  Introduction
Revised:  Introduction

Original: Machine learning based compilation has been explored for over a decade [Wang and O’Boyle, 2018].
Revised: Machine learning based compilation has been explored for over a decade [Wang and O’Boyle, 2018].

Original: Early work focused on learning profitability heuristics while more recently, deep learning models have been used to build code-to-code models, for translating or decompiling code.
Revised: Early work focused on learning profitability heuristics while more recently, deep learning models have been used to build code-to-code models, for translating or decompiling code.

Original: However, to the best of our knowledge, there has been no prior work on using machine learning to entirely automate compilation, i.e given a high level source code program generate the equivalent assembler code.
Revised: However, to the best of our knowledge, there has been no prior work on using machine learning to entirely automate compilation, i.e given a high level source code program generate the equivalent assembler code.

Original: Compilers are large, complex objects [Lattner and Adve, 2004] and automating their behavior represents a significant research challenge.
Revised: Compilers are large, complex objects [Lattner and Adve, 2004] and automating their behavior represents a significant research challenge.

Original: In this paper, we investigate whether it is possible to learn an end-to-end machine compiler using neural machine translation.
Revised: In this paper, we investigate whether it is possible to learn an end-to-end machine compiler using neural machine translation.

Original: In particular, we focus on the translation of small C functions to x assembler.
Revised: In particular, we focus on the translation of small C functions to x assembler.

Original: Given that there has been over 50 years research into developing reliable compiler technology, it may seem unnecessary to expend further effort in learning a solved problem.
Revised: Given that there has been over 50 years research into developing reliable compiler technology, it may seem unnecessary to expend further effort in learning a solved problem.

Original: However, using a neural translation approach opens up the possibility of unsupervised compilation from a language to an ISA (instructon set architecture/machine code).
Revised: However, using a neural translation approach opens up the possibility of unsupervised compilation from a language to an ISA (instructon set architecture/machine code).

Original: This means we may be able to automatically generate compilers for new programming languages and new hardware.
Revised: This means we may be able to automatically generate compilers for new programming languages and new hardware.

Original: If true, this enables programming language researchers and hardware architects to rapidly explore new designs and will have a transformational impact on both domains.
Revised: If true, this enables programming language researchers and hardware architects to rapidly explore new designs and will have a transformational impact on both domains.

Original: Indeed, there is already work in unsupervised translation between programming languages
Revised: Indeed, there is already work in unsupervised translation between programming languages

Original: [Lachaux et al., 2020].
Revised: [Lachaux et al., 2020].

Original: However, before we can begin to consider unsupervised compilation, we first have to determione whether supervised neural compilation is feasible.
Revised: However, before we can begin to consider unsupervised compilation, we first have to determione whether supervised neural compilation is feasible.

Original: To learn this C → x86 translation, we use an existing function-level C corpus, Anghabench [da
Revised: To learn this C → x86 translation, we use an existing function-level C corpus, Anghabench [da

Original: Silva et al., 2021], to build a parallel C-x86 assembler corpus.
Revised: Silva et al., 2021], to build a parallel C-x86 assembler corpus.

Original: Then, we model the compilation task as a sequence-to-sequence task (akin to machine translation) with the Transformer architecture [Vaswani et al., 2017].
Revised: Then, we model the compilation task as a sequence-to-sequence task (akin to machine translation) with the Transformer architecture [Vaswani et al., 2017].

Original: We study the effect of modifying different settings by varying training data size, model size, number of epochs, and other hyperparameters.
Revised: We study the effect of modifying different settings by varying training data size, model size, number of epochs, and other hyperparameters.

Original: While we can successfully generate syntactically correct assembler over 80% of the time and obtain high BLEU scores (c. 90 BLEU in some benchmarks), generating semantically correct assembler is more challenging.
Revised: While we can successfully generate syntactically correct assembler over 80% of the time and obtain high BLEU scores (c. 90 BLEU in some benchmarks), generating semantically correct assembler is more challenging.

Original: The best model can only compile correctly about 33% of the functions in a benchmark built from an existing program synthesis evaluation set
Revised: The best model can only compile correctly about 33% of the functions in a benchmark built from an existing program synthesis evaluation set

Original: [Collie et al., 2020]; it specially struggles to compile functions with numerous arguments and arrays.
Revised: [Collie et al., 2020]; it specially struggles to compile functions with numerous arguments and arrays.

Original: The article is structured as follows.
Revised: The article is structured as follows.

Original: In Section 2, we briefly summarise related work in NLP and machine learning for code.
Revised: In Section 2, we briefly summarise related work in NLP and machine learning for code.

Original: In Section 3, we formalize the task of machine compilation and propose how to effectively build neural compilers and fairly evaluate them.
Revised: In Section 3, we formalize the task of machine compilation and propose how to effectively build neural compilers and fairly evaluate them.

Original: Then, in Section 4 we establish our experimental framework and report results.
Revised: Then, in Section 4 we establish our experimental framework and report results.

Original: Finally, we discuss our approach and conclude, in Section 5.
Revised: Finally, we discuss our approach and conclude, in sections 5 and 6, respectively.

Original:  Related Work
Revised:  Related Work

Original: Natural Language Processing and Machine Translation In Natural Language Processing (NLP), the current state-of-the-art typically involves using some variant of the Transformer architecture [Vaswani et al., 2017] together with some form of subword tokenization
Revised: Natural Language Processing and Machine Translation In Natural Language Processing (NLP), the current state-of-the-art typically involves using some variant of the Transformer architecture [Vaswani et al., 2017] together with some form of subword tokenization

Original: [Sennrich et al., 2015].
Revised: [Sennrich et al., 2015].

Original: In this work, we omit the prolific literature in unsupervised NLP (most famously, BERT [Devlin et al., 2018]) and machine translation because we focus on a supervised setting.
Revised: In this work, we omit the prolific literature in unsupervised NLP (most famously, BERT [Devlin et al., 2018]) and machine translation because we focus on a supervised setting.

Original: Deep Learning for Code and Symbolic Data
Revised: Deep Learning for Code and Symbolic Data

Original: Recent works have proposed to use the encoderdecoder Transformer architecture out of the box for symbolic mathematics [Saxton et al., 2019,
Revised: Recent works have proposed to use the encoderdecoder Transformer architecture out of the box for symbolic mathematics [Saxton et al., 2019,

Original: Lample and Charton, 2019], or even for automated symbolic proving with decoder-only Transformers [Polu and Sutskever, 2020].
Revised: Lample and Charton, 2019], or even for automated symbolic proving with decoder-only Transformers [Polu and Sutskever, 2020].

Original: The state-of-art NLP systems for unsupervised pretraining have also been with successfully applied to code, as in CodeBERT [Feng et al., 2020].
Revised: The state-of-art NLP systems for unsupervised pretraining have also been with successfully applied to code, as in CodeBERT [Feng et al., 2020].

Original: However, other research lines explore the use of alternative modeling strategies for code instead of flat sequences, such as trees to leverage the grammar [Chen et al., 2018] or other kinds of graphs for data flow analysis [Cummins et al., 2021].
Revised: However, other research lines explore the use of alternative modeling strategies for code instead of flat sequences, such as trees to leverage the grammar [Chen et al., 2018] or other kinds of graphs for data flow analysis [Cummins et al., 2021].

Original: Machine Learning for Compilers
Revised: Machine Learning for Compilers

Original: Many works have proposed the use of machine learning for performance improvement[Emani and O’Boyle, 2015] or code optimization [Leather and Cummins, 2020].
Revised: Many works have proposed the use of machine learning for performance improvement[Emani and O’Boyle, 2015] or code optimization [Leather and Cummins, 2020].

Original: The field is gaining momentum with recent works such as the CompilerGym [Cummins et al., 2020], a reinforcement learning environment for compilers optimization.
Revised: The field is gaining momentum with recent works such as the CompilerGym [Cummins et al., 2020], a reinforcement learning environment for compilers optimization.

Original: However, the common approach is to use machine learning for decision-making e.g. Cavazos et al.
Revised: However, the common approach is to use machine learning for decision-making e.g. Cavazos et al.

Original: [2006], not to directly generate assembler with a machine learning decoder.
Revised: [2006], not to directly generate assembler with a machine learning decoder.

Original: Code Translation and Code-to-Code models Code-to-code models have been applied in tasks such as 1.
Revised: Code Translation and Code-to-Code models Code-to-code models have been applied in tasks such as 1.

Original: programming language translation Drissi et al.
Revised: programming language translation Drissi et al.

Original: [2018], even in unsupervised settings
Revised: [2018], even in unsupervised settings

Original: Lachaux et al.
Revised: Lachaux et al.

Original: code refinement Tufano et al.
Revised: code refinement Tufano et al.

Original: [2018], or 3.
Revised: [2018], or 3.

Original: decompilation Katz et al.
Revised: decompilation Katz et al.

Original: The latter is the inverse task of the one we are posing, and in this specific work it was constrained to a highly restricted subset of C with a maximum of 5 statements.
Revised: The latter is the inverse task of the one we are posing, and in this specific work it was constrained to a highly restricted subset of C with a maximum of 5 statements.

Original: To the best of our knowledge, no previous work has addressed the task of machine compilation.
Revised: To the best of our knowledge, no previous work has addressed the task of machine compilation.

Original: One specific additional challenge we note, is that the target sequences are considerably longer (being assembler) instead of a similar size (as usual in machine translation) or considerably shorter (as usual in summarization or decompilation).
Revised: One specific additional challenge we note, is that the target sequences are considerably longer (being assembler) instead of a similar size (as usual in machine translation) or considerably shorter (as usual in summarization or decompilation).

Original:  Methods
Revised:  Methods

Original: We pose machine compilation as a sequence-to-sequence task.
Revised: We pose machine compilation as a sequence-to-sequence task.

Original: Akin to machine translation, machine compilation is the task of translating code into assembler language.
Revised: Akin to machine translation, machine compilation is the task of translating code into assembler language.

Original: More formally, given a dataset D with N pairs ( x i , y i ) , where x i is an input program
Revised: More formally, given a dataset D with N pairs ( x i , y i ) , where x i is an input program

Original: and y i is the corresponding assembler code, the system is trained with max likelihood estimation: ℓ ( θ , D )
Revised: and y i is the corresponding assembler code, the system is trained with max likelihood estimation: ℓ ( θ , D )

Original: = (cid:80) ( x i , y i ∈ D ) lnp
Revised: = (cid:80) ( x i , y i ∈ D ) lnp

Original: ( y i | x i , θ ) .
Revised: ( y i | x i , θ ) .

Original: Posing the task as a sequence-to-sequence task conditions both the data generation and the model building.
Revised: Posing the task as a sequence-to-sequence task conditions both the data generation and the model building.

Original: 3.1 Training data
Revised: 3.1 Training data

Original: Regarding the granularity, as a first approach we decided to consider functions, following Lachaux et al.
Revised: Regarding the granularity, as a first approach we decided to consider functions, following Lachaux et al.

Original: Functions, unlike statements, are standalone units of meaning that can be translated, but at the same they are shorter and easier to test (unit tests) than a whole program (integration tests).
Revised: Functions, unlike statements, are standalone units of meaning that can be translated, but at the same they are shorter and easier to test (unit tests) than a whole program (integration tests).

Original: Since we investigate a supervised setting, we need pairs (C functions, x86 assembler).
Revised: Since we investigate a supervised setting, we need pairs (C functions, x86 assembler).

Original: However, C functions cannot be directly compiled; they typically need additional context (inclusion of headers, type definitions, constant definitions).
Revised: However, C functions cannot be directly compiled; they typically need additional context (inclusion of headers, type definitions, constant definitions).

Original: Thus, even if we have pre-exiting C compilers, generating these data pairs is not trivial.
Revised: Thus, even if we have pre-exiting C compilers, generating these data pairs is not trivial.

Original: For this work, we base our dataset on Anghabench [da Silva et al., 2021], a benchmark of around 1 million C functions coupled with the required minimal C code to compile them.
Revised: For this work, we base our dataset on Anghabench [da Silva et al., 2021], a benchmark of around 1 million C functions coupled with the required minimal C code to compile them.

Original: Anghabench is built by crawling C code from Github repositories.
Revised: Anghabench is built by crawling C code from Github repositories.

Original: The authors extracted individual functions, and applied type-inference to reconstruct the missing definitions required to compile them (e.g., declarations of auxiliary functions, type definitions, etc).
Revised: The authors extracted individual functions, and applied type-inference to reconstruct the missing definitions required to compile them (e.g., declarations of auxiliary functions, type definitions, etc).

Original: However, while these reconstructions makes the functions compilable, they are not executable.
Revised: However, while these reconstructions makes the functions compilable, they are not executable.

Original: Apart from not necessarily having a main function and input/output calls, the declared auxiliary functions are not defined.
Revised: Apart from not necessarily having a main function and input/output calls, the declared auxiliary functions are not defined.

Original: This, among other issues, prevents execution.
Revised: This, among other issues, prevents execution.

Original: It is not practical to directly use this dataset for neural compilation.
Revised: It is not practical to directly use this dataset for neural compilation.

Original: The inclusion of headers and type definitions while necessary for GCC, adds noise to the machine translation task.
Revised: The inclusion of headers and type definitions while necessary for GCC, adds noise to the machine translation task.

Original: We refer to
Revised: We refer to

Original: Appendix A for the description of our preprocessing pipeline together with the statistics (Appendix
Revised: Appendix A for the description of our preprocessing pipeline together with the statistics (Appendix

Original: B) of the resulting dataset, which we call Angha-Par (Angha Parallel).
Revised: B) of the resulting dataset, which we call Angha-Par (Angha Parallel).

Original: After filtering for length, we kept as many as 500k programs (Angha-Par500k) and a subset (250k) of those for an ablation study (Angha-Par250k).
Revised: After filtering for length, we kept as many as 500k programs (Angha-Par500k) and a subset (250k) of those for an ablation study (Angha-Par250k).

Original: 3.2 Evaluation
Revised: 3.2 Evaluation

Original: BLEU Machine translation is usually evaluated with BLEU score [Papineni et al., 2002], based on n-gram overlaps between the generated sequence and the ground truth one (in our case, the GCC assembler).
Revised: BLEU Machine translation is usually evaluated with BLEU score [Papineni et al., 2002], based on n-gram overlaps between the generated sequence and the ground truth one (in our case, the GCC assembler).

Original: This metric does not take into account syntactical or semantic correctness.
Revised: This metric does not take into account syntactical or semantic correctness.

Original: However, it is easy to compute for all cases.
Revised: However, it is easy to compute for all cases.

Original: Syntactic accuracy We use GCC to check if the assembler generated is syntactically correct, by asking it to generate object code from the assembler.
Revised: Syntactic accuracy We use GCC to check if the assembler generated is syntactically correct, by asking it to generate object code from the assembler.

Original: This metric is more relevant and even easier to compute.
Revised: This metric is more relevant and even easier to compute.

Original: IO accuracy We evaluate semantic correctness using observational equivalence or IO accuracy between the reference GCC assembler and the one output by the model, following recent works on program translation [Lachaux et al., 2020].
Revised: IO accuracy We evaluate semantic correctness using observational equivalence or IO accuracy between the reference GCC assembler and the one output by the model, following recent works on program translation [Lachaux et al., 2020].

Original: That is, we check whether for a given set of inputs, the assembler predicted by the models have the same output as the reference GCC compilation (in other words, we evaluated whether the assembler functions generated by the models pass the available unit tests).
Revised: That is, we check whether for a given set of inputs, the assembler predicted by the models have the same output as the reference GCC compilation (in other words, we evaluated whether the assembler functions generated by the models pass the available unit tests).

Original: While this is no proof that the two programs are formally equivalent, in practice it is a high indicator that it is.
Revised: While this is no proof that the two programs are formally equivalent, in practice it is a high indicator that it is.

Original: This is the most relevant metric and the one we use for selecting the best model and assessing its real performance.
Revised: This is the most relevant metric and the one we use for selecting the best model and assessing its real performance.

Original: However, Anghabench programs, while compilable, are not executable, since the function dependencies are not included.
Revised: However, Anghabench programs, while compilable, are not executable, since the function dependencies are not included.

Original: Thus, we cannot run unit tests on them.
Revised: Thus, we cannot run unit tests on them.

Original: For this reason, we take a subset 1 of 64 functions extracted from the program synthesis benchmark collated in Collie et al.
Revised: For this reason, we take a subset 1 of 64 functions extracted from the program synthesis benchmark collated in Collie et al.

Original: We then add a main function with the required input/output calls to execute them with randomly generated input/output pairs (referred as IO examples, from now on).
Revised: We then add a main function with the required input/output calls to execute them with randomly generated input/output pairs (referred as IO examples, from now on).

Original: We refer to this benchmark as Synthesis-bench.
Revised: We refer to this benchmark as Synthesis-bench.

Original: 3.3 Model
Revised: 3.3 Model

Original: Following previous work on machine translation and deep learning for symbolic mathematics and source code modeling, we use a Transformer model (encoder-decoder) in different settings.
Revised: Following previous work on machine translation and deep learning for symbolic mathematics and source code modeling, we use a Transformer model (encoder-decoder) in different settings.

Original: We implement all models with Fairseq [Ott et al., 2019], a PyTorch [Paszke et al., 2019] sequence modeling library.
Revised: We implement all models with Fairseq [Ott et al., 2019], a PyTorch [Paszke et al., 2019] sequence modeling library.

Original: As usual in sequence-to-sequence models, we train with teacher forcing and use a special token to denote the end of the sequence, which is also predicted by the model.
Revised: As usual in sequence-to-sequence models, we train with teacher forcing and use a special token to denote the end of the sequence, which is also predicted by the model.

Original: 3.4 Code and Data Availability
Revised: 3.4 Code and Data Availability

Original: We release 2 both the code and the data used in this work for the sake of reproducibility.
Revised: We release 2 both the code and the data used in this work for the sake of reproducibility.

Original: S YNTHESIS
Revised: S YNTHESIS

Original: A NGHA -P AR
Revised: A NGHA -P AR

Original: [Table]
Revised: [Table]

Original: ARAMS
Revised: ARAMS

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 2x
Revised: 2x

Original: [Table]
Revised: [Table]

Original: 1e2x w.
Revised: 1e2x w.

Original: [Table]
Revised: [Table]

Original: Big + 1/2 epochs) as per the most relevant variant, correct
Revised: Big + 1/2 epochs) as per the most relevant variant, correct

Original: IO examples.
Revised: IO examples.

Original:  Experiments
Revised:  Experiments

Original: We experiment with 4 Transformer sizes (Trans-Small, Trans-Med-Trans-Big, Trans-Big+).
Revised: We experiment with 4 Transformer sizes (Trans-Small, Trans-Med-Trans-Big, Trans-Big+).

Original: With
Revised: With

Original: Trans-Big we further experiment with different vocabulary and data sizes, and number of epochs.
Revised: Trans-Big we further experiment with different vocabulary and data sizes, and number of epochs.

Original: See Table 1 for the parameter count of each of the models, and Appendix C for more details on the models.
Revised: See Table 1 for the parameter count of each of the models, and Appendix C for more details on the models.

Original: We train all models with the same data (except from the ones that have a different vocabulary, which use a different tokenizer, and the one that uses half of data) and the same number of epochs,(except for the model additionally trained for 5 more epochs).
Revised: We train all models with the same data (except from the ones that have a different vocabulary, which use a different tokenizer, and the one that uses half of data) and the same number of epochs,(except for the model additionally trained for 5 more epochs).

Original: Regarding other hyperparameters, all models are trained with the Adam optimizer [Kingma and Ba, 2017].
Revised: Regarding other hyperparameters, all models are trained with the Adam optimizer [Kingma and Ba, 2017].

Original: We refer to Fairseq and our source code for additional details.
Revised: We refer to Fairseq and our source code for additional details.

Original: We do not conduct any hyperparameter search, aside from the different configurations reported in Table 1.
Revised: We do not conduct any hyperparameter search, aside from the different configurations reported in Table 1.

Original: We then evaluate as described in Section 3.2, always with beam search ( k = 5 ) and taking the best hypothesis among the top 5.
Revised: We then evaluate as described in Section 3.2, always with beam search ( k = 5 ) and taking the best hypothesis among the top 5.

Original: Table 1 shows the results summary of each model, together with their respective size.
Revised: Table 1 shows the results summary of each model, together with their respective size.

Original: The best model, as per the most relevant metric (IO evaluation, that is, observational equivalence) is the
Revised: The best model, as per the most relevant metric (IO evaluation, that is, observational equivalence) is the

Original: Transformer-Big trained for 10 epochs.
Revised: Transformer-Big trained for 10 epochs.

Original:  Discussion and Conclusion
Revised:  Discussion and Conclusion

Original:  Transformer-Big+1/2x epochs is the best model in terms of the most relevant metric, IO accuracy (observational equivalence).
Revised: Results Transformer-Big+1/2x epochs is the best model in terms of the most relevant metric, IO accuracy (observational equivalence).

Original: It is the best model in terms of syntactical accuracy in Synthesis-Bench and BLEU score in the Angha-Par test.
Revised: It is the best model in terms of syntactical accuracy in Synthesis-Bench and BLEU score in the Angha-Par test.

Original: The smallest model clearly underfits the task of machine compilation, while all reasonably sized models achieve similar enough results (except the model trained with half of the data, which performs considerably worse).
Revised: The smallest model clearly underfits the task of machine compilation, while all reasonably sized models achieve similar enough results (except the model trained with half of the data, which performs considerably worse).

Original:  The smallest model variant,Transformer-Small obtains a surprisingly high syntactic accuracy in
Revised: On the surprisingly high syntax accuracy of the small model The smallest model variant,

Original:  Angha-Par, as shown in Table 1, given that that its outputs are abnormally long (see Table 7.
Revised: Transformer-Small obtains a surprisingly high syntactic accuracy in Angha-Par, as shown in Table 1, given that that its outputs are abnormally long (see Table 7.

Original: On inspection the outputs do not correlate with the inputs.
Revised: On inspection the outputs do not correlate with the inputs.

Original: Instead, the model behaves as a nunconditional assembler language model.
Revised: Instead, the model behaves as a nunconditional assembler language model.

Original: Furthermore, we observe repeated outputs with different inputs.
Revised: Furthermore, we observe repeated outputs with different inputs.

Original: This phenomenon is reminiscent of the hallucinations described in other Sequence-to-Sequence models [Raunak et al., 2021] and the mode collapse of some generative models [Thanh-Tung et al., 2018].
Revised: This phenomenon is reminiscent of the hallucinations described in other Sequence-to-Sequence models [Raunak et al., 2021] and the mode collapse of some generative models [Thanh-Tung et al., 2018].

Original:  Focusing on the outputs of the best model, we observe that 1.
Revised: Error analysis Focusing on the outputs of the best model, we observe that 1.

Original: if a model has one correct IO example for a given function, then it is highly likely that other IO examples are correct2.
Revised: if a model has one correct IO example for a given function, then it is highly likely that other IO examples are correct

Original: many syntactical errors occur because of a premature end of the hypothesis, 3.
Revised: many syntactical errors occur because of a premature end of the hypothesis, 3.

Original: IO accuracy does not correlate with cyclomatic complexity, 3 but with the number of function arguments and pointer variables in the function (as shown in Table 6), 4.
Revised: IO accuracy does not correlate with cyclomatic complexity, 3 but with the number of function arguments and pointer variables in the function (as shown in Table 6), 4.

Original: models fail and succeed in the same functions, 5.
Revised: models fail and succeed in the same functions, 5.

Original: correct model outputs look very similar to the GCC ones although not necessarily identical, 6.
Revised: correct model outputs look very similar to the GCC ones although not necessarily identical, 6.

Original: there are some trivial errors, such as true and false being confused with variable names instead of boolean values.
Revised: there are some trivial errors, such as true and false being confused with variable names instead of boolean values.

Original: For a more complete error analysis, we refer to Appendix E, and for samples of the model, to Appendix F. In Appendix D, we break down the results for each of the functions in Synthesis-Bench, include the average length of the outputs of the different models, and report the most frequent syntactical errors and error intersections between statistics between the different models.
Revised: For a more complete error analysis, we refer to Appendix E, and for samples of the model, to Appendix F. In Appendix D, we break down the results for each of the functions in Synthesis-Bench, include the average length of the outputs of the different models, and report the most frequent syntactical errors and error intersections between statistics between the different models.

Original:  There is no compelling reason to believe that neural networks would not scale with data, model size, and compute in a similar way to other domains [Kaplan et al., 2020].
Revised: Scaling There is no compelling reason to believe that neural networks would not scale with data, model size, and compute in a similar way to other domains [Kaplan et al., 2020].

Original: Indeed, the models generally perform better with more data, compute, and parameters, even though the largest model we trained was not the best.
Revised: Indeed, the models generally perform better with more data, compute, and parameters, even though the largest model we trained was not the best.

Original: This may be due to insufficient training data or a sub-optimal training procedure e.g. insufficient updates.
Revised: This may be due to insufficient training data or a sub-optimal training procedure e.g. insufficient updates.

Original: However, unlike other domains, code quality is evaluated in a binary fashion, correct or illegal which may cause sharp accuracy curves.
Revised: However, unlike other domains, code quality is evaluated in a binary fashion, correct or illegal which may cause sharp accuracy curves.

Original:  Our best model can correctly compile less than half of the examples in the IO evaluation.
Revised: Limits Our best model can correctly compile less than half of the examples in the IO evaluation.

Original: It is, therefore, far from being usable in practice.
Revised: It is, therefore, far from being usable in practice.

Original: Furthermore, we have no control over the output space, and we operate on small functions instead of entire programs.
Revised: Furthermore, we have no control over the output space, and we operate on small functions instead of entire programs.

Original: In this work, apart from using code tokenizers and IO evaluation, we have not included any domain knowledge.
Revised: In this work, apart from using code tokenizers and IO evaluation, we have not included any domain knowledge.

Original: Given the large amount of prior syntatctic and semantic information available for source and target, an obvious next step is to incorporate this in to the translation scheme.
Revised: Given the large amount of prior syntatctic and semantic information available for source and target, an obvious next step is to incorporate this in to the translation scheme.

Original:  Finally, we remark on the potential ethical and legal implications of training models on Github data, an emerging topic in the machine learning for code community due to the release of OpenAI’s Codex Chen et al.
Revised: Ethical concerns regarding crawling data Finally, we remark on the potential ethical and legal implications of training models on Github data, an emerging topic in the machine learning for code community due to the release of OpenAI’s Codex Chen et al.

Original: This is not an immediate concern, as we limit our training to an already published dataset.
Revised: This is not an immediate concern, as we limit our training to an already published dataset.

Original: Any future work, however, which accesses Github code, needs to address these issues and not to access repositories with restrictive licenses.
Revised: Any future work, however, which accesses Github code, needs to address these issues and not to access repositories with restrictive licenses.

Original: We conclude that our neural compilation approach shows that sequence-to-sequence deep learning models can, indeed, learn to compile end-to-end.
Revised: We conclude that our neural compilation approach shows that sequence-to-sequence deep learning models can, indeed, learn to compile end-to-end.

Original: Nevertheless, the performance is far from ideal and the restrictions make it still far from being usable in practice.
Revised: Nevertheless, the performance is far from ideal and the restrictions make it still far from being usable in practice.

Original: The task presents many challenges, such as output length or hard syntactic and correctness constraints, that were not explicitly tackled in this work.
Revised: The task presents many challenges, such as output length or hard syntactic and correctness constraints, that were not explicitly tackled in this work.

Original: As future work, we suggest 1.
Revised: As future work, we suggest 1.

Original: scaling up our approach, in terms of data, compute, and model parameters, 2.
Revised: scaling up our approach, in terms of data, compute, and model parameters, 2.

Original: investigating how to incorporate domain knowledge in form of inductive biases or alternative data representations and inputs, and 3.
Revised: investigating how to incorporate domain knowledge in form of inductive biases or alternative data representations and inputs, and 3.

Original: researching unsupervised techniques to leverage unlabelled (i.e., not parallel) code or assembler.
Revised: researching unsupervised techniques to leverage unlabelled (i.e., not parallel) code or assembler.

Original: A Preprocessing pipeline
Revised: A Preprocessing pipeline

Original: Our preprocessing pipeline is composed of the following steps:
Revised: Our preprocessing pipeline is composed of the following steps:

Original: Compilation: We use the GCC compiler to compile the C code into x86 assembler.
Revised: Compilation: We use the GCC compiler to compile the C code into x86 assembler.

Original: We do not apply any optimizations (-O0).
Revised: We do not apply any optimizations (-O0).

Original: Boilerplate removal: We remove the headers and type and constant definitions.
Revised: Boilerplate removal: We remove the headers and type and constant definitions.

Original: Likewise, we remove the header and footer of the assembler.
Revised: Likewise, we remove the header and footer of the assembler.

Original: In both cases, we believe those inject noise and make sequences longer than need be.
Revised: In both cases, we believe those inject noise and make sequences longer than need be.

Original: Pre-tokenization: We use the GCC C and x86 assembler (GAS) tokenizers with the Pygments 4 library.
Revised: Pre-tokenization: We use the GCC C and x86 assembler (GAS) tokenizers with the Pygments 4 library.

Original: In C, new lines are meaningless and just used to make code more human readable, but in GAS end of lines delimits the end of each instruction.
Revised: In C, new lines are meaningless and just used to make code more human readable, but in GAS end of lines delimits the end of each instruction.

Original: Thus, in the latter we replace end of lines by a special token <newline> . 4.
Revised: Thus, in the latter we replace end of lines by a special token <newline> . 4.

Original: Length filtering:
Revised: Length filtering:

Original: Due to computational restrictions and potentially easing the task, we discard the (C, assembler) pairs such that when summing the length of tokens of the C code and assembler we get more than 314 tokens.
Revised: Due to computational restrictions and potentially easing the task, we discard the (C, assembler) pairs such that when summing the length of tokens of the C code and assembler we get more than 314 tokens.

Original: Train-valid-test split: We randomly split the pairs into training, validation, and test sets, with 2k programs for validation and test and the rest for training.
Revised: Train-valid-test split: We randomly split the pairs into training, validation, and test sets, with 2k programs for validation and test and the rest for training.

Original: Subword tokenization: We use subword encoding to automatically split tokens into further tokens based on n-gram frequencies in the train set.
Revised: Subword tokenization: We use subword encoding to automatically split tokens into further tokens based on n-gram frequencies in the train set.

Original: Specifically, subword-nmt [Sennrich et al., 2015] .
Revised: Specifically, subword-nmt [Sennrich et al., 2015] .

Original: This has the benefit of decreasing the vocabulary size while making out-ofvocabulary tokens virtually impossible (since unknown tokens can be reconstructed from ASCII characters or other subwords present in the vocabulary).
Revised: This has the benefit of decreasing the vocabulary size while making out-ofvocabulary tokens virtually impossible (since unknown tokens can be reconstructed from ASCII characters or other subwords present in the vocabulary).

Original: Formatting: We write each C and assembler programs in plain text files, such that we have one program for each line.
Revised: Formatting: We write each C and assembler programs in plain text files, such that we have one program for each line.

Original: B Data
Revised: B Data

Original: [Table]
Revised: [Table]

Original: Filter
Revised: Filter

Original: [Table]
Revised: [Table]

Original: IO
Revised: IO

Original: [Table]
Revised: [Table]

Original: Par500k
Revised: Par500k

Original: 1.044M
Revised: 1.044M

Original: Max.
Revised: Max.

Original: length 500k ✗
Revised: length 500k ✗

Original: [Table]
Revised: [Table]

Original: Par250k
Revised: Par250k

Original: 1.044M
Revised: 1.044M

Original: Max.
Revised: Max.

Original: length + random 250k ✗
Revised: length + random 250k ✗

Original: Synthesis-Bench
Revised: Synthesis-Bench

Original: Manual (difficulty)
Revised: Manual (difficulty)

Original: [Table] [Table]
Revised: [Table] [Table]

Original: V
Revised: V

Original: S UBW / TOK C ( AVG LEN )
Revised: S UBW / TOK C ( AVG LEN )

Original: S UBW / TOKEN ASM ( AVG LEN )
Revised: S UBW / TOKEN ASM ( AVG LEN )

Original: C OVERAGE
Revised: C OVERAGE

Original: [Table]
Revised: [Table]

Original: C Experiments
Revised: C Experiments

Original: We experiment with the following models:
Revised: We experiment with the following models:

Original: • Transformer-Small: The small model follows the transformer_iwslt_de_en configuration in Fairseq, that is, 6 encoder layers and 6 decoder layers, an embedding size of 512 and 4 attention heads.
Revised: • Transformer-Small: The small model follows the transformer_iwslt_de_en configuration in Fairseq, that is, 6 encoder layers and 6 decoder layers, an embedding size of 512 and 4 attention heads.

Original: • Transformer-Big (base): The big model follows the transformer_wmt_en_de_big_t2t configuration in Fairseq, with 6 encoder layers and 6 decoder layers, an embedding size of 1024 and 16 attention heads – -50% data: Transformer-Big trained with Angha-Par250k instead of Angha-Par500k.
Revised: • Transformer-Big (base): The big model follows the transformer_wmt_en_de_big_t2t configuration in Fairseq, with 6 encoder layers and 6 decoder layers, an embedding size of 1024 and 16 attention heads – -50% data: Transformer-Big trained with Angha-Par250k instead of Angha-Par500k.

Original: – -1/2x vocab:
Revised: – -1/2x vocab:

Original: Transformer-Big trained with a vocabulary of 4k tokens (instead of 8k tokens).
Revised: Transformer-Big trained with a vocabulary of 4k tokens (instead of 8k tokens).

Original: – +1/2x vocab: Transformer-Big trained with a vocabulary of 16k tokens (instead of 8k tokens).
Revised: – +1/2x vocab: Transformer-Big trained with a vocabulary of 16k tokens (instead of 8k tokens).

Original: – +1e2x weight-decay: Transformer-Big further regularized (a weight decay of 0.
Revised: – +1e2x weight-decay: Transformer-Big further regularized (a weight decay of 0.

Original: instead of 0.0001) – +1/2 epochs: Transformer-Big trained for a total of 10 epochs (instead of 5).
Revised: instead of 0.0001) – +1/2 epochs: Transformer-Big trained for a total of 10 epochs (instead of 5).

Original: • Transformer-Med:
Revised: • Transformer-Med:

Original: The medium-size model roughly follows the Transformer-Big configuration, but with 8 attention heads (instead of 16) and a Feed-Forward hidden size of(instead of 4096).
Revised: The medium-size model roughly follows the Transformer-Big configuration, but with 8 attention heads (instead of 16) and a Feed-Forward hidden size of(instead of 4096).

Original: • Transformer-Big+: This model has the same configuration as Transformer-Big, but withadditional layers for both the encoder and the decoder.
Revised: • Transformer-Big+: This model has the same configuration as Transformer-Big, but withadditional layers for both the encoder and the decoder.

Original: D Expanded results
Revised: D Expanded results

Original: We report the fine-grained IO evaluation for the best model in Table 5, together with other metrics to ease the analysis of the results.
Revised: We report the fine-grained IO evaluation for the best model in Table 5, together with other metrics to ease the analysis of the results.

Original: Specifically, apart from the aforementioned syntactic accuracy and
Revised: Specifically, apart from the aforementioned syntactic accuracy and

Original: BLEU scores, we also report:
Revised: BLEU scores, we also report:

Original: LOC : Lines of Code, the number of lines of the C implementation.
Revised: LOC : Lines of Code, the number of lines of the C implementation.

Original: Tokens:
Revised: Tokens:

Original: The number of tokens of the C implementation.
Revised: The number of tokens of the C implementation.

Original: Cyclo :
Revised: Cyclo :

Original: The cyclomatic complexity : Cyclomatic complexity =
Revised: The cyclomatic complexity : Cyclomatic complexity =

Original: E − N +2 × P where E is the number of edges in the flow graph, N is the number of nodes in the flow graph, and P is the number of nodes that have exit points.
Revised: E − N +2 × P where E is the number of edges in the flow graph, N is the number of nodes in the flow graph, and P is the number of nodes that have exit points.

Original: Params:
Revised: Params:

Original: The number of parameters of the C function.
Revised: The number of parameters of the C function.

Original: Pointers:
Revised: Pointers:

Original: The number of pointer parameters (typically arrays) of the C function.
Revised: The number of pointer parameters (typically arrays) of the C function.

Original: Finally, Tables 6, 7, 8, 9 show the correlations between IO errors and other metrics, the mean output length of each model, the most frequent syntactical errors, and the most frequent IO errors, respectively.
Revised: Finally, Tables 6, 7, 8, 9 show the correlations between IO errors and other metrics, the mean output length of each model, the most frequent syntactical errors, and the most frequent IO errors, respectively.

Original: F UNC
Revised: F UNC

Original: IO
Revised: IO

Original: S YNTAX
Revised: S YNTAX

Original: BLEU
Revised: BLEU

Original: LOC
Revised: LOC

Original: TOKENS
Revised: TOKENS

Original: CYCLO
Revised: CYCLO

Original: PARAMS
Revised: PARAMS

Original: POIN add ✗ ✓ 85.
Revised: POIN add ✗ ✓ 85.

Original: array_inc ✗ ✓ 87.
Revised: array_inc ✗ ✓ 87.

Original: array_prod ✓ ✓ 97. array_sum ✓ ✓ 97.
Revised: array_prod ✓ ✓ 97. array_sum ✓ ✓ 97.

Original: binary_digits ✓ ✓ 97.
Revised: binary_digits ✓ ✓ 97.

Original: binary_mul_sum ✗ ✗ 50.
Revised: binary_mul_sum ✗ ✗ 50.

Original: clamp ✗ ✓ 96.
Revised: clamp ✗ ✓ 96.

Original: collatz ✓ ✓ 98.
Revised: collatz ✓ ✓ 98.

Original: count_odds ✓ ✓ 87.
Revised: count_odds ✓ ✓ 87.

Original: cube_in_place ✗ ✗ 65.
Revised: cube_in_place ✗ ✗ 65.

Original: digit_prod ✗ ✗ 59.
Revised: digit_prod ✗ ✗ 59.

Original: digits ✓ ✓ 82.
Revised: digits ✓ ✓ 82.

Original: diveq ✗ ✓ 75.
Revised: diveq ✗ ✓ 75.

Original: diveq_sca ✗ ✓ 82.
Revised: diveq_sca ✗ ✓ 82.

Original: dot ✗ ✓ 97.
Revised: dot ✗ ✓ 97.

Original: elementwise_ ✗ ✗ 3.
Revised: elementwise_ ✗ ✗ 3.

Original: [Table]
Revised: [Table]

Original: sum
Revised: sum

Original: [Table]
Revised: [Table]

Original: of
Revised: of

Original: [Table]
Revised: [Table]

Original: negated_sum_ _and
Revised: negated_sum_ _and

Original: [Table]
Revised: [Table]

Original: max eq ✗ ✗ 81.
Revised: max eq ✗ ✗ 81.

Original: fact ✓ ✓ 96. fact_fact ✓ ✓ 96.
Revised: fact ✓ ✓ 96. fact_fact ✓ ✓ 96.

Original: fib_n ✓ ✓ 97.
Revised: fib_n ✓ ✓ 97.

Original: fourth_in_place ✗ ✗ 45.
Revised: fourth_in_place ✗ ✗ 45.

Original: int_sqrt ✓ ✓ 86.
Revised: int_sqrt ✓ ✓ 86.

Original: last_elem ✓ ✓ 97.
Revised: last_elem ✓ ✓ 97.

Original: last_zero_idx ✓ ✓ 98.
Revised: last_zero_idx ✓ ✓ 98.

Original: length ✗ ✗ 41.
Revised: length ✗ ✗ 41.

Original: max ✗ ✗ 79.
Revised: max ✗ ✗ 79.

Original: max_elt ✗ ✗ 87.
Revised: max_elt ✗ ✗ 87.

Original: min ✗ ✗ 80.
Revised: min ✗ ✗ 80.

Original: min_elt ✗ ✗ 88.
Revised: min_elt ✗ ✗ 88.

Original: min_so_far_ ✗ ✓ 0.
Revised: min_so_far_ ✗ ✓ 0.

Original: _subtractedmirror_image ✗ ✗ 77.
Revised: _subtractedmirror_image ✗ ✗ 77.

Original: muleq ✗ ✓ 73.
Revised: muleq ✗ ✓ 73.

Original: muleq_sca ✗ ✓ 85.
Revised: muleq_sca ✗ ✓ 85.

Original: negate ✗ ✓ 87.
Revised: negate ✗ ✓ 87.

Original: pluseq ✗ ✗ 76.
Revised: pluseq ✗ ✗ 76.

Original: prod_elts ✓ ✓ 97.
Revised: prod_elts ✓ ✓ 97.

Original: prod_n_squared ✓ ✓ 97.
Revised: prod_n_squared ✓ ✓ 97.

Original: prod_sq_elts ✗ ✗ 85.
Revised: prod_sq_elts ✗ ✗ 85.

Original: replace_first ✗ ✗ 79.
Revised: replace_first ✗ ✗ 79.

Original: replace_last ✗ ✗ 79.
Revised: replace_last ✗ ✗ 79.

Original: reverse ✗ ✗ 55.
Revised: reverse ✗ ✗ 55.

Original: reverse_int ✗ ✗ 61.
Revised: reverse_int ✗ ✗ 61.

Original: search ✓ ✓ 95.
Revised: search ✓ ✓ 95.

Original: sort ✗ ✗ 33.
Revised: sort ✗ ✗ 33.

Original: subeq ✗ ✗ 74.
Revised: subeq ✗ ✗ 74.

Original: subeq_sca ✗ ✓ 89.
Revised: subeq_sca ✗ ✓ 89.

Original: subtract_of_ ✗ ✗ 46.
Revised: subtract_of_ ✗ ✗ 46.

Original: _min_reversesum_abs ✗ ✗ 59.
Revised: _min_reversesum_abs ✗ ✗ 59.

Original: sum_elts ✓ ✓ 97.
Revised: sum_elts ✓ ✓ 97.

Original: sum_n ✓ ✓ 96.
Revised: sum_n ✓ ✓ 96.

Original: sum_n_squared ✓ ✓ 92.
Revised: sum_n_squared ✓ ✓ 92.

Original: sum
Revised: sum

Original: [Table]
Revised: [Table]

Original: of_lists_ ✗ ✗ 18.
Revised: of_lists_ ✗ ✗ 18.

Original: [Table]
Revised: [Table]

Original: multiplied
Revised: multiplied

Original: [Table]
Revised: [Table]

Original: after
Revised: after

Original: [Table]
Revised: [Table]

Original: dividing
Revised: dividing

Original: [Table]
Revised: [Table]

Original: sum
Revised: sum

Original: [Table]
Revised: [Table]

Original: positives ✗ ✗ 44.
Revised: positives ✗ ✗ 44.

Original: sum
Revised: sum

Original: [Table]
Revised: [Table]

Original: ✓ ✓ 98.
Revised: ✓ ✓ 98.

Original: triangle_prod ✓ ✓ 97.
Revised: triangle_prod ✓ ✓ 97.

Original: [Table]
Revised: [Table]

Original: E Expanded error analysis
Revised: E Expanded error analysis

Original: Focusing on the outputs of the best model
Revised: Focusing on the outputs of the best model

Original: [Table]
Revised: [Table]

Original: we observe:
Revised: we observe:

Original: • When the model has one correct IO test in a given function, it is likely that the others will be also correct, as shown in Table 9.
Revised: • When the model has one correct IO test in a given function, it is likely that the others will be also correct, as shown in Table 9.

Original: The probability of generating a program that only passes one unit test by chance is, indeed, very low.
Revised: The probability of generating a program that only passes one unit test by chance is, indeed, very low.

Original: • After manually inspecting the most frequent syntactical errors (Table 8), we find that most of these occur because the output finishes prematurely.
Revised: • After manually inspecting the most frequent syntactical errors (Table 8), we find that most of these occur because the output finishes prematurely.

Original: For instance, it is common to find outputs with operators with unbalanced parentheses as the last instruction, not because the model has not learned the syntax, but because the decoding terminated in the middle of the program.
Revised: For instance, it is common to find outputs with operators with unbalanced parentheses as the last instruction, not because the model has not learned the syntax, but because the decoding terminated in the middle of the program.

Original: This occurs when outputs are long and the model predicts the end of the program special token prematurely.
Revised: This occurs when outputs are long and the model predicts the end of the program special token prematurely.

Original: • In our experiments, IO accuracy does not correlate with cyclomatic complexity, as shown in Table 6.
Revised: • In our experiments, IO accuracy does not correlate with cyclomatic complexity, as shown in Table 6.

Original: We see two potential reasons for that, namely, 1.
Revised: We see two potential reasons for that, namely, 1.

Original: in Synthesis-Bench there are not enough functions to observe sufficient variability in cyclomatic complexity to observe the expected correlation, or 2.
Revised: in Synthesis-Bench there are not enough functions to observe sufficient variability in cyclomatic complexity to observe the expected correlation, or 2.

Original: the sources of the errors are more simple (e.g., the mere presence of an array) than the complexity captured by cyclomatic complexity..
Revised: the sources of the errors are more simple (e.g., the mere presence of an array) than the complexity captured by cyclomatic complexity..

Original: In fact, the number of function parameters and the number of points seems to be indeed negatively correlated with the IO accuracy.
Revised: In fact, the number of function parameters and the number of points seems to be indeed negatively correlated with the IO accuracy.

Original: Thus, we conclude that the more function parameters and more pointers, the more difficult is for neural models to correctly interpret C and generate the corresponding assembler.
Revised: Thus, we conclude that the more function parameters and more pointers, the more difficult is for neural models to correctly interpret C and generate the corresponding assembler.

Original: Finally, with no surprise, syntactical accuracy and BLEU score positively correlate with IO accuracy, since correct solutions are clearly syntactically correct and, with a lesser degree, lexically similar to the GCC solution.
Revised: Finally, with no surprise, syntactical accuracy and BLEU score positively correlate with IO accuracy, since correct solutions are clearly syntactically correct and, with a lesser degree, lexically similar to the GCC solution.

Original: However, the correlation is not strong enough for these metrics to be used as reliable proxies of the IO accuracy in case unit tests are not available.
Revised: However, the correlation is not strong enough for these metrics to be used as reliable proxies of the IO accuracy in case unit tests are not available.

Original: • All models fail in the same functions: Table 10 shows that the intersection of IO errors between the different models is almost full, meaning that errors are related to some intrinsic difficulty of these functions (at least to neural compilers) and not to randomness in the training process.
Revised: • All models fail in the same functions: Table 10 shows that the intersection of IO errors between the different models is almost full, meaning that errors are related to some intrinsic difficulty of these functions (at least to neural compilers) and not to randomness in the training process.

Original: • Model outputs do appear like GCC outputs, but with some artifacts such as unnecessary nop operations in some cases (see supplementary material).
Revised: • Model outputs do appear like GCC outputs, but with some artifacts such as unnecessary nop operations in some cases (see supplementary material).

Original: • We observe some trivial errors.
Revised: • We observe some trivial errors.

Original: For instance, true and false (boolean values from stdbool ) are confused with variable names.
Revised: For instance, true and false (boolean values from stdbool ) are confused with variable names.

Original: If they are manually replaced with 1 and 0, the models usually generate a correct output.
Revised: If they are manually replaced with 1 and 0, the models usually generate a correct output.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: open CFI at the end of file; missing .cfi_endproc directive expecting operand after ’,’; got nothing unbalanced brackets in operand 1.
Revised: open CFI at the end of file; missing .cfi_endproc directive expecting operand after ’,’; got nothing unbalanced brackets in operand 1.

Original: number of operands mismatch for ‘mov’ number of operands mismatch for ‘add’ unbalanced brackets in operand 2.
Revised: number of operands mismatch for ‘mov’ number of operands mismatch for ‘add’ unbalanced brackets in operand 2.

Original: bad or irreducible absolute expression
Revised: bad or irreducible absolute expression

Original: CFI instruction used without previous .cfi_startproc
Revised: CFI instruction used without previous .cfi_startproc

Original: junk at end of line, first unrecognised character is ‘%’ symbol ‘.L3’ is already defined number of operands mismatch for ‘cmp’ symbol ‘.L5’ is already defined number
Revised: junk at end of line, first unrecognised character is ‘%’ symbol ‘.L3’ is already defined number of operands mismatch for ‘cmp’ symbol ‘.L5’ is already defined number

Original: of operands mismatch for ‘movq’ number of operands mismatch for ‘lea’ .cfi_endproc
Revised: of operands mismatch for ‘movq’ number of operands mismatch for ‘lea’ .cfi_endproc

Original: without corresponding .cfi_startproc
Revised: without corresponding .cfi_startproc

Original: symbol ‘.L4’ is already defined operand type mismatch for ‘sar’ number of operands mismatch for ‘pop’ number of operands mismatch for ‘sal’ number of operands mismatch for ‘pxor’ number of operands mismatch for ‘movslq’ .
Revised: symbol ‘.L4’ is already defined operand type mismatch for ‘sar’ number of operands mismatch for ‘pop’ number of operands mismatch for ‘sal’ number of operands mismatch for ‘pxor’ number of operands mismatch for ‘movslq’ .

Original: size expression for sum_n does not evaluate to a constant
Revised: size expression for sum_n does not evaluate to a constant

Original: [Table] [Table]
Revised: [Table] [Table]

Original: F Output samples
Revised: F Output samples

Original: We include some model output samples, together with the reference GCC output and the original C, selected as follows:
Revised: We include some model output samples, together with the reference GCC output and the original C, selected as follows:

Original: • All outputs belong to the best model, specifically they are one of the top 5 hypothesis.
Revised: • All outputs belong to the best model, specifically they are one of the top 5 hypothesis.

Original: • All outputs belong to Synthesis-Bench.
Revised: • All outputs belong to Synthesis-Bench.

Original: • They have not been randomly sampled, or cherry-picked to show the best outputs only, but manually curated to try to provide representative outputs of different cases, which may not appear proportionally.
Revised: • They have not been randomly sampled, or cherry-picked to show the best outputs only, but manually curated to try to provide representative outputs of different cases, which may not appear proportionally.

Original: Listing 1: C input
Revised: Listing 1: C input

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: Listing 3: GCC assembler
Revised: Listing 3: GCC assembler

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Figure]
Revised: [Figure]

Original: int search ( int *arr , int v, int n) { int idx = −1; for ( int i = 0; i < n; ++i ) { if ( idx == −1 &
Revised: int search ( int *arr , int v, int n) { int idx = −1; for ( int i = 0; i < n; ++i ) { if ( idx == −1 &

Original: [Figure]
Revised: [Figure]

Original: arr [ i ] == v) { idx = i ; } }return idx ; }
Revised: arr [ i ] == v) { idx = i ; } }return idx ; }

Original: [Table]
Revised: [Table]

Original: Listing
Revised: Listing

Original: GCC
Revised: GCC

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: Listing void min_so_far_subtracted ( int *arr , int *arr2 , int *result , int n) { int min_so_far_array [n]; int min_so_far = 1; for ( int i = 0; i < n; i ++) { min_so_far = ( min_so_far <
Revised: Listing void min_so_far_subtracted ( int *arr , int *arr2 , int *result , int n) { int min_so_far_array [n]; int min_so_far = 1; for ( int i = 0; i < n; i ++) { min_so_far = ( min_so_far <

Original: arr [ i ]) ? min_so_far : 1; min_so_far_array
Revised: arr [ i ]) ? min_so_far : 1; min_so_far_array

Original: [ i ] = min_so_far ; }int max_of_min_so_far _and_other_array [n]; for ( int i = 0; i < n; i ++) { int m1 = min_so_far _array [ i ]; int m2 = arr2 [ i ]; int max = (m1 > m ?
Revised: [ i ] = min_so_far ; }int max_of_min_so_far _and_other_array [n]; for ( int i = 0; i < n; i ++) { int m1 = min_so_far _array [ i ]; int m2 = arr2 [ i ]; int max = (m1 > m ?

Original: m1 : m2); max_of_min_so_far_and_other_array [ i ] = max; } for ( int i = 0; i < n; i ++) { result [ i ] = min_so_far
Revised: m1 : m2); max_of_min_so_far_and_other_array [ i ] = max; } for ( int i = 0; i < n; i ++) { result [ i ] = min_so_far

Original: _array[i]−max_of_min_so _far_and_other_array [i]; } }
Revised: _array[i]−max_of_min_so _far_and_other_array [i]; } }

Original: Listing
Revised: Listing

Original: Model assembler min
Revised: Model assembler min

Original: [Figure]
Revised: [Figure]

Original: Listing
Revised: Listing

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: correct, yet meaningless output.
Revised: correct, yet meaningless output.

Original: This case follows a common pattern we have observed: when the input C is too complex for the model, the model just outputs a short, syntactically correct assembler that semantically has almost nothing to do with the input C.
Revised: This case follows a common pattern we have observed: when the input C is too complex for the model, the model just outputs a short, syntactically correct assembler that semantically has almost nothing to do with the input C.

Original: The model seems to behave like an unconditional assembler language model.
Revised: The model seems to behave like an unconditional assembler language model.

Original: Listing
Revised: Listing

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: Listing 14: Model assembler
Revised: Listing 14: Model assembler

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]