Program Translation via Code Distillation
Authors: Yufan Huang
Mengnan Qi
Yongqiang Yao
Maoquan Wang
Bin Gu
Colin Clement
Neel Sundaresan
Venue: EMNLP 2023 Main (2023)
Keywords: Program Translation
Intermediate Representations
Neural Machine Translation
Multilingual Code Generation
Pre-training
TL;DR: We proposes a state-of-the-art program translation model called CoDist with code distillation that provide a translation pivot for programming languages.

Abstract: Software version migration and program translation are an important and costly part of the lifecycle of large codebases. Traditional machine translation relies on parallel corpora for supervised translation, which is not feasible for program translation due to a dearth of aligned data. Recent unsupervised neural machine translation techniques have overcome data limitations by included techniques such as back translation and low level compiler intermediate representations (IR). These methods face significant challenges due to the noise in code snippet alignment and the diversity of IRs respectively. In this paper we propose a novel model called Code Distillation (CoDist) whereby we capture the semantic and structural equivalence of code in a language agnostic intermediate representation. Distilled code serves as a translation pivot for any programming language, leading by construction to parallel corpora which scale to all available source code by simply applying the distillation compiler. We demonstrate that our approach achieves state-of-the-art performance on CodeXGLUE and TransCoder GeeksForGeeks translation benchmarks, with an average absolute increase of 12.7% on the TransCoder GeeksforGeeks translation benchmark compare to TransCoder-ST.

Program Translation via Code Distillation
Yufan Huang 1. Mengnan Qi' Yonggiang Yao Maoquan Wang
Bin Gu
Colin Clement' Neel Sundaresan'
MMicrosoft Cloud and Al
School of Artificial Intelligence, Jilin University
3 Mohamed bin Zayed University of Artificial Intelligence
(yufanhuang, mengnangiinnnnii
yoogggiagyaa maoquanwang @emicrooottcom
Abstract
code libraries that widely used in banks and other financial institutions need to be migrated updated languages for casier maintenance and de- velopment. Additionally, adaptation to different hardware platforms and operating systems often requires projects to be implemented in different high-level languages or different versions of the same language. An automatic program translation system, such as Java Swift, blessing for Android developers who want adapt their appli- cations to the iOS operating system.
Software version migration and program trans- lation are important and costly part of the lifecycle of large codebases. Traditional ma- chine translation relies on parallel corpora for supervised translation, which is not feasible for program translation due to a dearth of aligned data. Recent unsupervised neural machine translation techniques have overcome data lim- itations by included techniques such as back translation and low level compiler intermediate representations (IR). These methods face signif- icant challenges due to noise code snippet alignment and the diversity of IRs respectively. this paper we propose a novel model called Code Distillation (CoDist) whereby we capture the semantic and structural equivalence of code a language agnostic intermediate represen- tation. Distilled code serves translation pivot for any programming language, leading by construction to paraliel corpora which scale all available source code by simply apply- ing the distillation compiler. We demonstrate that our approach achieves state-of of-the-ar art per- formance CodeXGLUE and TransCoder GeeksForGeeks translation benchmarks, with an average absolute increase of 12.7% on the TransCoder GeeksforGeeks translation bench- mark compare to TransCoder-ST.
Natural language translation typically relies heavily on high-quality parallel corpora. However, creating parallel data for program translation tasks can be prohibitively expensive as it requires the generated code to be aligned possible. Unlike natural language, even minor variations missing tokens result in significant errors in code lan- guages. Additionally, perfect alignment of tokens does not guarantee accuracy due to unique syntax rules and restrictions present each code language. For example, when translating Java code to C++, the variable name "xor" in Java cannot be directly mapped as variable name in C++ because it is a reserved word in C++. Therefore, program trans- lation is challenging task due to the difficulty in obtaining high-quality parallel code data.
1 Introduction
Recent research has proposed unsupervised methods for neural machine translation in pro- gram translation tasks. PLBART (Ahmad et al., 2021) randomly adds various types of noise (to- ken deletion, token shuffling, etc.) to multilin- gual code snippets and forces the model to re- cover them. This strategy not only helps close the gap between different programming languages but also improves the alignment ability of the model. TransCoder (Roziere et al., 2020), introduced unsupervised approach for translating source code by constructing pseudo-paralle corpus using to augment translation data pairs. Another system, TransCoder-ST (Roziere et al.,
Program translation is the process of converting code written in one programming language an- other. Unlike compilers that only convert high- level languages to low-level machine code, pro- gram translation can generate translations between complex high-level programming languages. The industry demand for high-quality program transla- tion is ever- ever-present, many companies require the migration of project codes from older programming languages to new ones with better support, features, and a modern workforce. For example, COBOL
"Jont first author

---

2021b), employs open-source automatic test gener- ation tools to develop unit tests for commonly used programming languages. This system eliminates potentially invalid data pairs in tack-translation and refines the unsupervised model using high- confidence data examples that have passed the tests, thus enhancing the overall level of confidence in the machine translation process.
translation pivot, we introduce a novel multilin- gual program translation model called CoDist that unifies multiple translation pairs into a single dis- tilled code decompiler task. Through the specific language token switch, the distilled code derived from source language code snippet can be seam- lessly converted into any target language supported by the model. Our experiments demonstrate that our method surpasses tfte-offthhearr techniques the CodeXGLUE (Lu et al., 2021) and TransCoder GeeksforGeeks benchmark datasets. Furthermore, our model exhibits zero-shot and few-shot abili- ties that are competitive in low-resource code lan- guages, To summarize, our main contributions are:
These methods, however, face significant chal- lenges, which currently limit their ability to serve as perfect method of translation. Due the random noise, PLBART may end up paying more attention to irrelevant information in the input, overlooking the crucial information needed for alignment, re- sulting in slower convergence speed and reduced performance. TransCoder models can be biased towards certain data patterns, which can result in poor performance on new or unseen data. Besides, the implementation of back translation in language alignment can introduce randomness and ambigui- ties between program languages, which ultimately slows the convergence of the model and presents challenges when aligning low-frequency words. To alleviate this problem, TransCoder-ST filter out these errors during back translation though unit tests, but the ensuing challenge is the cost of con- structing unit tests, which can be particularly high in industrial projects where the code requires multi- ple dependencies. This poses a significant obstacle the capturing of appropriate test data for code fragments in complex scenarios and at times, make it infeasible.
1. In this study, we introduce pivot language in programming translation called distilled code. The code distillation technique a lossy compression, extracting the core semantics of any code snippet of high-level programming languages into a language agnostic format. Distilled code unifies all basic morphemes across languages. For high-level semantics (e.g., complex API calls), code distillation extracts the core part of program logic, and decomposes concatenated object names into bag-of- wo-ds representations.
Through our novel multilingual program trans- lation model, we unify multiple translation pairs into a single distilled code decompiler task. To build a code translation system that support N programming languages transla- tion, our system only need to train one neural network model instead of N22 sequence-tu sequence model tasks by traditional transla- tion techniques.
In this paper, we introduce code distillation, which overcomes the limitations of parallel data and alignment in different languages by leverag- ing a custom compiler to simplified intermediate representation which contains only the essential logical aspects and object names. We train model decompile the distilled code, so that the pro- cess of translation is to compile the source snippet to the distilled representation, and decompile to the target language using the trained model. We systematically discovered which elements of each source language are necessary for code understand- ing, eliminating these language-specific elements from the distilled code representation. Further, our distillation unifies basic morphemes. blurring gram- matical differences between languages, and uses more abstract form to merge high-level semantic ex- pressions between different languages. To demon- strate the effectiveness of using distilled code as
3. Our method can leverage massive unsuper- vised code corpora through self-supervised pre-training tasks, and our final pre-training form is identical to the program translation form, which reduces the gap between pre- training and fine-tuning. It is noteworthy in that our training does not rely on parallel cor- pora, and can produce translations of much higher quality than traditional methods.
4. Our method outperforms current state-of- the-art techniques on the CodeXGLUE and TransCoder GeeksforGeeks benchmark datasets, and it is competitive in low lowrresoure code languages.

---

2
Related Work
guage another without the need for parallel data or human supervision. TransCoder (Roziere et al., 2020) combines cross-lingual masked language modeling (Lample and Conneau, 2019), denois- ing auto-encoding, and back-translation (Artetxe et al., 2017) and applies them to the source code setting. However, back translation creates many pseudo-parallel code pairs with lot of noise, which restricts the upper limit of the model's abil- ity. TransCoder-ST (Roziere et al., 2021b) adds an automated unit-testing system to get high-quality pairs in back translation. Af- ter filtering by unit test, the model can align on some common basic elements, but it still cannot generate correctly in complex scenarios such as industrial projects where the code requires multiple dependencies.
The lack of parallel translation pairs poses a signif- icant challenge for automated program translation. To overcome this challenge, many recent works have focused on designing pre-training tasks or us- ing various unsupervised methods. In this section, we will discuss the benefits and limitations of these approaches.
2.1 Pre-training for Program Translation
The goal the paradigm large-scale pre-training is to stimulate the potential of models pro- gram translation by designing well-structured pre- training tasks. CodeBERT (Feng et al., 2020) pre- trains the masked language modeling task proposed by BERT (Devlin et al., 2018) code corpora, then adds decoder for end-to-end training on pro- gram translation. However, it treats the code as ordinary text and loses the structural information of the code. Some works incorporate intrinsic fea- tures of programming languages to overcome this shortcomings. GraphCodeBERT (Guo et al., 2020) improves on CodeBERT (Feng et al., 2020), adding data flow graphs extracted from source code to improve the model's understanding of code struc- ture. StructCoder (Tipimeni et al., 2022) improves the transformer model make encoder decoders structure-un aware, and introduces abstract syntax tree and data flow graph information. Most of these models are encoder-only pre-trained models that fo- cus on code understanding, the lack of pre-training in the decoder consistently restricts the model's 's performance on the task of program translation. Some researchers begun to investigate the joint pre-training of the encoder and decoder. PLBART (Ahmad et al., 2021) builds on the existing natu- ral language translation model BART (Lewis et al., 2019), continuing the same pre-training on code do- main data. They incorporate various forms of noise, including token masking, deletion, and infilling, into the input code and force the model to recon- struct the original code at the decoder. MuST-PT (Zhu et al., 2022b) pre-trained on multilingual code snippets for translation task. Those pre-training tasks is still disparate from program translation task, making them still rely on fine-tuning using parallel program translation data sets.
3 Distilled Code Compiler
Since these approaches mentioned in the "Related Work" section still have their own limitations, we choose an alternative approach that involves build- ing translation pivot. Instead of one-to-one trans- lation processes, we first compile any source lan- guage program to a distilled code representation and then decompile it into target program using a trained model. We provide a unified front-end for compilation, which improves information density and narrows the language gap by distilling core mantics from the original code. In this section, we will briefly introduce translation pivots and explain the construction process of our compiler.
3.1 Brief Review of Translation Pivot
TransCoder-IR (Szafranicc et al., 2022) explored a new path for program translation, it utilizes a low- level intermediate representation (IR), provided by LLVM (Lattner and Adve, 2004), be the pivot be- tween the source code and the target code. LLVM IR is a low-level programming language used as an intermediate step in the compilation process between high-level programming language and machine code, provides a relatively uniform rep- resentation for different programming languages. In their experiment, the source code is converted into LLVM IR through the compiler, and LLVM IR is restored into the target code. Its great ad- vantage is that, the process of translating source language code the L.VMM IR does not require additional neural network training, and training a neural decompiler should suffice to obtain the tar-
2.2 Unsupervised for Program Translation
Unsupervised program translation aims to automat- ically translate code from one programming lan-

---

get language code.
code components on the semantic representations of the data. It is concluded, based on the findings in Appendix A, that moderate elimination of non- central information will not significantly affect the semantic understanding of the input code by pre- trained models such as PLBART. As a result, we have implemented a pruning technique to filter out non-significant syntax tree nodes while retaining crucial components such as variable names, func- tion names, data types, and reserved keywords. De- spite some loss of information during the pruning process, the utilization enables us to restore it in the decompilation stage. Unify Basic Morphemes While the logical flow of the code components are retained in pruned syntax trees are preserved, various high-level program- ming languages
According to (Szafraniec et al., 2022), the ex- perimental results were limited by the huge varia- tions between different IR dialects in LLVM. The LLVM ecosystem is jointly developed by many teams, which leads to different IRs for each lan- guage. On the other hand, the original intention of LLVM IR design is not just to align different lan- guages, there is lot of redundant information that has weak correlation with program translation (such as for garbage collection and debugging). These factors lead to LLVM IR to be extremely verbose, so that even short programs can consume the entire limited context window of LLMs.
robust language model
3.2 Our Distilled Code Compiler
A well-designed translation pivot should be inde- pendent from specific languages and still be able to capture the semanties of the input as much as pos- sible. We design new set of compilers called dis- tilled code compilers for languages such as Python, C++. Java and C#. Compare with the intermediate representations from the low-level compiler like LLVM, our distilled code contains less information from the source language, while preserving overall logical flow and data type relations, which makes it more consistent with the design principles of translation pivot.
distinct set of symbols and keywords to express semantics. In Appendix B, the design scheme for mapping the basic morphemes in Python, Java, C++, and C# languages into unified expression form is presented and discussed in de- tail. Additionally, the existing program translation model faces obstacles related to the unique expres- sion form of each language. A case in point is the switch expression in Java, which is not supported by the current Python and must be substituted with an if-else structure achieve the same result. We also unify these specialized expressions from vari- ous languages into a unified form.
Specifically, our distilled code compiler first con- vert origin codes into abstract syntax tree, and then unimportant language-specifts elements are removed from the tree and reassembled into code forms that embody the original logic and data flow. Figure depicts the comprehensive procedure of code distillation.
Fuzzing Remaining Variations Different lan- guages often have library and API names which have similar sub-word elements but differ in subtle ways like sub-word order or choice. We overcome this difference by representing function and ob- ject names by bag of words representation of the snake and camel-case segmented elements. Further, we randomize the order of these during decom- piler training to ensure the model not sensitive to language-specific orderings except in the decompi- lation stage.
3.2.1 Distillation
The abstract syntax tree (AST) serves as a graphi- cal representation of the grammatical structure of a programming language, which is a natural medium for expressing distillation operations on different code parts. We compute the AST by using a parser generator tool called TreeSitter'. Code distillation compresses language-specific information and re- duces variations in programming languages while preserving the input code semantics the greatest extent possible. This section will describe the dis- tillation process in details.
3.2.2 Reassembling
The final step involves reassembling the syntax tree into a sequence format that is acceptable for use with the language model. Special string templates are designed for morphemes shared by all program- ming languages such as for loops, if statements, and while loops. These templates take into account the design patterns of different languages, retaining causal, nested, referring and other logical relation- ships between different components in the syntax tree, resulting in a comprehensive, expressive, and
Syntax Tree Pruning The experiments docu- mented in Appendix A reveal the impact of various
hhttps://tree- sittrrsssttttgggooo thub. 10/ sree-sitter/

---

Parsing
Distilaaioon
Reassembling
Source Code
Syntax Tree
Distilled Tree
Distilled Code
function
function
Franc prisi ln seculattyy
Specetuced queiet
signature
body
Aignature
body
modiler
Aatt caiee
modifer
funs core
call
cal
comment
comment
Prune
Friched
Enthed
Fuzzy
Unity
Ine
Fratfed
Systemsout pripp
fpoh
Fethed
prit
CreateCliont]
BOW: dient Croate
Figure 1: The overall pipeline for converting source code into distilled code. In the first step, the source code is represented as syntax tree. Next, we align different languages to translation pivots by removing language-specific elements which are not important the logical and semantic function. Finally, the nodes are combined in string templates to yield distilled code strings suitable for large language model training.
extensible representation.
of program translation is generate target code function Y in code language Cy.
4 Distilled Code Decompiler
4.2 Representation Pre-training
After source code is transformed into the transla- tion pivot through the distilled code compiler, next we need build a decompiler that can output the target code. We introduce our multilingual dis- tilled code decompiler model, which is a encoder- decoder transformer based model pre-trained on large-scale multilingual code corpora. Under the framework of multilingual models, all languages share the same encoder and decoder and map to the same latent space. In this section, introduce several self-supervised pre-training tasks that implemented. The first two tasks provide good rep- resentations for our distilled code and original code. The last pre-training task similar to the final de- compiler task, which further aligns the distilled code and target code.
There are a substantial amount of high-quality code corpora available for different program languages on internet. However, these monolingual codes are not aligned across programming languages, which presents a significant obstacle for conventional supervised training methods to effectively utilize them for translation. To overcome this challenge, propose set of unsupervised pre-training meth- ods that aim mine the hidden semantic infor- mation in these data. The pre-training methods discussed in this subsection allow the model to ob- tain a more robust embedding for translation tasks. Masked Language Modeling First introduced in (Devlin et al., 2018), Masked Language Model (MLM) pre-training task is self-supervised learn- ing task designed to train models understand context and semantics in natural language. In this work, we extend the application of the aforemen- tioned method to our translation pivot, the distilled code. We propose to train our model simultane- ously on multiple programming languages, lever- aging the distilled code as unified representation. This approach has the potential to improve the per- formance of our model on cross-lingual program- ming tasks. Specifically, we first obtain the distilled codes for the multilingual program language codes through the distilled code compiler, concatenate with source code and then train the encoder to predict randomly masked tokens.
4.1 Problem Setting
In this work, the primary focus will be program translation at the function level, these sequences embody minimal semantic program of an appro- priate length. Suppose our model supports the fol- lowing set of code languages = [CI,...,CC, Cx where each C₁ is the monolingual code corpora in language i. Giving a source code function X = {x1, Z2,,.... 27) in code language C, which consists of I code snippets. It is first input to the distilled code compiler to obtain its corresponding
distilled code D = {d()) d()) "...,.." dfx)), the goal

---

Marked Disiileed Code
Madiee
Masked Language Modeling
Resse
priet
mes Ledi eneeyereeeetnn
Tenear
Transformer Encoder
IntenCe
Timoor
intente
Narued Destiled Code
Helled Code
Denoising Auto-Encoding
Origin Dutiiiee Oode
Origin Cose
[wel
(wel printag)
Hol lcrwont sxy
Transformer Encoder
Transformer Decoder
Fast
Feet indie"orrr
sprre prietic
bedt
platte cirr
Manday Boart
Multiliingul Program Generation
Noboed Dutilled Code
Origin
CV
rool)
fossale.leitht teeg"trro"")
2f ((brre"))
Foo) (eintled)
vold foot) "rats""tror"cooeee "coeddi)
G-
Transformer Encoder
Transformer Decoder
Java
niblie vold (ac()
tea arintan ('Errse
Python
(rnor""
Figure 2: The whole process of pre-training, From top to bottom, we implemented three pre-training tasks step step. Our pre- pre-traine training first allows the model fully understand the domain data, and then moves closer to the downstream task decompiler form.
Denoising Auto-Encoding Inspired by the BART model, our approach incorporates various types of program corruption on both the word and sen- tence level, such as random word shuffle, random word dropout, and sentence permutation. Both the source code and the corresponding distilled code are intentionally corrupted, and then combined into single sequence. The model is then trained to reconstruct the original sequence. To enhance the robustness of the model decompiler, we specifically add more noise to the code snippets stored in the bag of words the distilled code.
pre-training stage, model utilizes programming language code and its corresponding distilled code in each monolingual corpus as training data pair. In order to aid the model in accurately translating to the corresponding language, special language sym- bols (e.g., <java>, <python>) are added as a switch on the decoder. There are subtle differences between translation pivots distilled from different languages. To narrow the gap among them, add noise during the training process. After training. the model able to uniformly understand distilled codes in different languages and generate corre- sponding target codes on the decoder side.
4.3 Training for Translation
Experiments
Upon completion of the pre-training phase de- scribed in the previous subsection, our model has already obtained a good embedding representa- tion for distilled codes from different program lan- guages. The pre-training task outlined in this sec- tion further assists the model in aligning distilled code and various target codes.
5.1 Training Details
Training Datasets We select C#, C++ Java, and Python files from projects with more than 5 stars in the GitHub public repositories and CodeNet projects (Puri al., 2021). We extract complete, structurally sound code at the function level and preprocess the data by deleting docstrings, com- ments, and dead code blocks. We also employed XLCoST (Zhu et al., 2022a), benchmark dataset
Multilingual Program Generation We start from the distilled code to generate the complete code across various programming languages. During the

---

that contains fine-grained parallel data in seven commonly used programming languages. Evaluation Studies focusing on code translation typically use the CodeXGLUE dataset (Lu et al., 2021) (Java to C#, C# to Java) as the evaluation data. We follow them and utilize the two metrics offered by the benchmark, BLEU (Papineni et al.. 2002) and CodeBLEU (Ren et al., 2020), to assess the performance of our model. These metrics can measure n n-gram overlap, code syntax, and seman- tic equivalence between the generated code and the target code. Compared with these metrics that re- flect the quality of the generated code, sometimes humans expect to intuitively feel whether the trans- lation result can run correctly and return the same result the ground truth. GeeksforGeeks is an on- line platform with computer science and program- ming articles, which gathers many coding problems and presents solutions in several programming lan- guages. The TransCoder model provides test envi- ronments and test samples for the C++-Python-Java code pairs collected from GeeksforGeeks. They evaluate whether the generated function returns the same output as the reference when given the same input. Top-N (CA@N) will check whether any of the top-N translations generated by the model passes the test. Following TransCoder, CA@1 metric computed with beam size 10. Experimental Details Our model is a sequence- to (o-sequence (seq2seq) transformer model with 12 layers (6 in the encoder and 6 in the decoder), 8 attention heads, and dimension of 1024. all pro- gramming language share single encoder and single decode, At training time, we alternate be- tween each of programming language corpora. For multilingual translation language modeling task, we mask 15% of the token. In multilingual transla- tion auto-encoding task, the implementation ratios the three noise addition schemes, random token mask, random token dropout, and sentence permu- tation are 0.3, 0.3, and 0.2, respectively, and they are 0.5, 0.5, and 0 in the bag of words of distilled code. We optimize our model with the Adam opti- mizer and polynomial decay learning rate sched- uler, with an initial learning rate of 10-5 most of our experiments. Our experiments use mixed precision accelerated training and are conducted in 8 V V000 GPUs.
Java Cava-ICH
Java
Method
BLEU
CodeBLEU
BLEU
CodeBLEU
Naive copy
18.54
18.69
PLBART
83.02
87.92
78.35
85.27
StruetCoder
85.02
88.42
80.66
86.03
CoDist 85.80
88.75
83.19
86.20
Table Results CodeXGLUE translation task. Our model achieves state-of-the-an performance BLEU score C#-Java and both BLEU and CodeBLEU Java-Cif.
5.2 Results
Performance CodeXGLUE We conducted evaluation of proposed model on the CodeXGLUE benchmark, comparing it two isting works, PLBART and StructCoder. PLBART, strong baseline model, utilizes auto denoising pre- tre-training tasks on Java and Python methods and natural language texts before fine-tuning on the task of program translation. StructCoder, on the other hand, employs abstract syntax trees and data flow graphs to make the encoder-decoder structure- aware, and currently holds the top position. The "naive copy" baseline simply replicates the input source code as the output translation, illustrating how closely similar two programming languages are. Table 1 presents our model's results which indicate its state-of-the-art performance in terms of BLEU score and CodeBLEU for both Java-C# and C#-Java translation tasks. This showcases the exceptional alignment capabilities of our method at the n-gram level.
use the
Performance on GeeksforGeeks The exper- imental setup of TransCoder was followed, using the CA@1 I metric calculated with a beam size of 10, for comparison of our model with TransCoder, TransCoder-ST,, TransCoder-IR on Python-Java-C++ program translation pairs. Our model proved to have excellent results across various sub-translation tasks, shown in Table 2, particularly regards to starting from Python code. This success is mainly attributed the distilled code design style which is more geared towards C++ and Java code, reducing the difficulty aligning the target code from the Python code for the decompiler model. In contrast, TransCoder-ST requires the construction of a more precise automated unit-testing system climinate invalid translations, whereas our model does require unit-testing and can be easily expanded to include more programming languages and other program translation models. We also compare with large language models InstructGPT (text-davinei-003

---

Method
C++->Jay
C++ Python
Java-C++
Java Pytton
Python +C++ Python -Java
TransCoder
65.1%
47.1%
79.8%
49.0%
32.6%
36.6%
TransCoder-ST
68.0%
61.3%
84.6%
68.9%
56.7%
58.2%
TransCoder-IR
62.9%
74.5%
text-davinci-003
77.8%
74.7%
72.1%
79.8%
74.1%
69.6%
gpt-3.5-turbo
88.6 %
85.2 %
80.6%
88.7 %
85.67%
80.9%
CoDist
82.1 1%
67.9%
87.9%
68.1
86.9
81.1 %
Table Results on the TransCoder GeeksforGeeke Geeks translation task. We follow the experimental configuration TransCoder ensure that all ground-truth test files Python, Java and C++ output correct results advance.
Method Codist w/o MPG
C+++Java JJva
1++-Python
Java
Dava-Pythom
Python
Python-JJava
82.14%
67.89%
87.85%
68.10%
86.94%
81.12%
33.61%
22.62%
45.61%
23.92%
42.61%
30.71%
w/o MLM
70.12%
61.21%
80.08%
60.78%
77.94%
71.36%
w/o MLM and DAB
69.08%
58.41
79.44%
57.11%
76.44%
68.46%
Table 3: Abalation study of pretraining task TransCoder GeeksforGeeks. The setting of w/o MPG indicates our model removes the multilingual program generation task, while w/o ML.M and DAE means neither masked language model task nor the denoising auto-encoding task are used. The rest the configuration can be deduced by analogy.
version) (Ouyang et al., 2022) and ChatGPT (gpt-3.5-turbo version) (OpenAI., 2022) using the OpenAl official recommended translation prompt template, and turn off sampling to generate the final code. We cannot be sure of the generalization of OpenAI models as their test sets are proprictary, so we cannot rule out the influence of data leakage. While our model size is significant smaller than those models. ChatGPT potentially has state of the art translation rates for half the language pairs in GeeksForGeeks, but even still CoDist has the best performance on the other half of langauge pairs.
Conclusion
In this paper, we propose novel method for pro- gram translation with distilled code. Our distilled code has the characteristics of high information density and general applicability, making it ideal candidate for serving as translation pivot. We have developed distilled code compilers for C#. Java, Python, and C++, and a new multilingual pro- gram translation model. This model unifies multi- ple translation pairs into the distilled code decom- piler task. Our results demonstrate the competitive- ness of the proposed approach on the CodeXGLUE and GeeksforGeeks datasets, and highlights the potential for building good translation pivots in program translation tasks. We argue that our dis- tillation idea has universality and is a highly effi- cient information compression method for struc- tured texts like high-level programming languages, documents and tables with minimal data loss.
5.3 Analysis
We utilize a model that incorporates pre-training tasks (MLM, DAE and MPG) to enhance the performance of our distilled code decompiler. To examine the contribution of each pre-training task, we select the TransCoder GeeksforGeeks dataset and conduct experiments by removing some of the tasks. As demonstrated in Table 3, the results show that MLM and DAE tasks help model better understand semantics of distilled code and programming languages, while MPG task form and downstream translation task form are unified, which narrows the gap between the pre-training and finetune stages, and significantly improves the effect of the model. Our research shows that judicious combination of these pre-training tasks can lead to more advanced and efficient code decompile.
Limitations
LLMs frequently encounter the issue of hallucina- tions, where generated text invents facts, objects or relationships. Our approach is not devoid of this problem, especially when method invocation has complex web of dependencies. The insufficient availability of factual knowledge and world knowl- edge in the training corpus is primarily responsible for the current limitations of our model. In fu- ture work we will incorporate a Enhhacee mechanism to obtain missing information from ex- tensive databases in order to enhance the model.

---

References
Kishore Papineni. Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: a method for automatic evalu- ation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computa- tional Linguistics, pages 311-318.
Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, and Kai- Wei Chang, 2021. Unified pre-training for program understanding and generation. arXiv preprint 10330233..
Ruchir Puri, David S Kung, Geert Janssen, Wei Zhang. Giacomo Domeniconi, Vladimir Zolotov, Julian Dolby, Jie Chen, Mihir Choudhury. Lindsey Decker, et al. 2021. Codenet: A large-scale ai for code dataset for learning diversity of coding tasks. arXiv preprint arXiv:2105. 12655.
Mikel Artetxe, Gorka Labaka, Eneko Agime, and Kyunghyun Cho. 2017. Unsupervised neural ma- chine translation. arXiv preprint arXiv:1710.1104. 1104..
Jacob Devlin, Ming-Wei Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understand- ing. arXiv preprint arXiv:1810.04800
Shuo Ren, Daya Guo, Shuai Lu, Long Zhou, Shujie Liu, Duyu Tang, Neel Sundaresan, Ming Zhou, Ambrosio Blanco, and Shuai Ma. 2020. Codebleu: a method for automatic evaluation of code synthesis. arXiv preprint arXiv:2009.1022
Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xi- aocheng Feng. Ming Gong. Linjum Shou, Bing Qin, Ting Liu, Daxin Jiang, et al. 2020. Codehert: pre-trained model for programming and natural lan- guages. arXiv preprint 200.055002..8155.
Baptiste Roziere, Maric-Anne Lachaux, Lowik Chanus- sot, and Guillaume Lample. 2020. Unsupervised translation of programming languages. Advances Neural Information Processing Systems, 33.
Daya Guo, Shuai Lu, Nan Duan, Yanlin Wang, Ming Zhou, and Jian Yin. 2022. Unixcoder: Unified cross- modal pre-training for code representation. arXiv preprint ar arXiv:2203.00 220.00380.
Baptiste Roziere, Marie-Anne Lachaux, Mare Szafraniec, and Guillaume Lample. 2021a. Dobf: A deobfuscation pre-training objective for program- ming languages. arXiv preprint arXiv:2102.07492.
Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, et al. 2020. Graphcode- bert: Pre-training code representations with data flow. arXiv preprint arXivv2009.08830 083666
Baptiste Roziere, Jie M Zhang, Francois Charton. Mark Harman, Gabriel Synnaeve, and Guillaume Lample. 2021b. Leveraging automated unit tests for unsupervised code translation. arXiv preprint 11X.v:2110.06773.
Guillaume Lample and Alexis Conneau. 2019. Cross- lingual language model pretraining. arXiv preprint 191.07900...721..
Marc Szafraniec, Baptiste Roziere, Hugh Leather Fran- cois Charton, Patrick Labatut, and Gabriel Synnaeve. 2022. Code translation with compiler representations. arXiv preprint arXiv:2207.0337
Chris Lattner and Vikram Adve. 2004. Llvm: com- pilation framework for lifelong program analysis & transformation. In International symposium on code generation and optimization, 2004. CGO 2004., pages 75-86. IEEE.
Sindhu Tipimeni, Ming Zhu, and Chandan K Reddy, 2022. Structcoder: seuucture-wwar transformer for code generation. arXiv preprint 2rXiv:2206.0522
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zeutleoyer.. 2019. Bart: De- noising sequence-tes to- to-sequence pre- prettrining for natural language generation, translation, and comprehension. arXiv preprint arXiv: 13461.
Ming Zhu, Aneesh Jain, Karthik Suresh, Roshan Ravin- dran, Sindhu Tipimeni, and Chandan K Reddy. 2022a. Xlcost: A benchmark dataset for cross-lingua code intelligence. arXiv preprint 2206. 084744
Shuai La, Daya Guo, Shuo Ren, Junjie Huang. Alexey Svyatkovskiy, Ambrosio Blanco, Colin Clement, Dawn Drain, Daxin Jiang, Duyu Tang, et al. 2021. Codexglue: A machine learning benchmark dataset for code understanding and generation. arXiv preprint (tt::2122.0666
Ming Zhu, Karthik Suresh, and Chandan K Reddy. 2022b. Multilingual code snippets training for pro- gram translation. In Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 36, pages 11783-11790.
OpenAl. 2022. Chatgpt:
Optimizing language models for dialogue. https: / openai. com/ blog/ chatgpt. Blog post.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang. Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models follow instruc- tions with human feedback. Advances Neural Information Processing Systems, 35:27730-27744,

---

A Code Representation Analysis
/* oricinat=/
publlie inl besSan (in. [ man, ind Langel)
Napeinteges, lnteges> langth: t+f) for Hitt
Code developers often use descriptive and informa- tive identifiers when writing programs to enhance the code's rcadability and maintainability. This practice of using informative identifiers helps to maintain the rich semantics of the code. In this section, we aim to investigate the influence of vari- ous code (onponentssidentiferss, program reserved words, token order etc.) on the semantic under- standing. The findings of this analysis guides us in designing of distilled code.
(iin.cemainnnin (romet return now int (omnss())) 11:
else dic.put (targot - aunstil, 1)
return mull/
1 obfearatedd? public int Func_0 0fn:: VAR_3, int VnR MapeTrragurr , Thlegurs VMR new FaahhMpp---1 ():
for int VAR_ = VAR < VAS 0.longhh: +*VAB_3)
LE (VAR_ R.mmitaii akky D9A8 A[VARAAR (1)) retarn new int II 1VA8 gog EVAR oIVAR_>1>, VAR_ else
Valk 2.50t IAA
VAN O[VAK 31, VAR 31:
mull;
Our research endeavors to identify which nodes of the abstract syntax tree can be removed without compromising the model's ability comprehend the code. In the context of code pre-training mod- els, our experiments aim to investigate the impact of adding noise to a specific code part. If the per- formance of these models significantly decreases as a result, it provides evidence that the semantics implied in the code part have a substantial influ- ence on the model's ability to comprehend the code. The worse the performance of the model with noisy code parts, the more likely these parts damaged by noise contains core information and should be kept in our distilled code.
Full obfuscation version of the two-sum solution. Full obfuscation of a two-sum solution which finds the two numbers in list that will sum up a target number in Java. The original code is shown above the obfuscated code.
Figure 3:
11 shaffle Linost/ HapeInteger, Integar> die new HastMap<>0
for (int
length: ++1)
dis-put (larget mastlll il: zetuen new inl[] (dic.got Inena [i]), 1)r return pull;
Le (dio dddooocatesnokeeyaaae (inns
public intfll tvsemttict() nuns, int targot) (
/* shaffle cokenet/
C. dic int null for comsiinntiy 1. if Map got [ [i] Tnbegar, int. -ei mim new die Fal wlas
int
die
twoSun
targst () RashMtap
Specifically, we inject noise into some code parts by replacing the identifiers in code, which is pro- posed in DOBF (Roziere et al., 2021a), shuffling the original code, which is actually represented by the positional embeddings fed the models, delet- ing the reserved keyword: (e.g. 'assert', 'const', and 'if') and deleting the structure related symbols including brackets or certain punctuation symbols (i.e. 'C. ')', 'T' 'I', " < ", and ":).. We select Java and C# as our target programming lan- guages and use the corresponding datasets from CodeNet (Puri et al., 2021). Noticing that prob- lems in CodeNet with larger indexes tend to have less valid solutions, we filter the dataset by select- ing the first 1000 problems and randomly sampling 10 solutions for each problem, and also filter out the problems that are too short which are possibly wrong answers and remove the comments and the documents in the codes to avoid influences from natural languages in the codes. Finally the filtered dataset is composed of 7454 Java solutions and 4508 C# solutions. We obfuscate the identifiers Java codes with tools borrowed from (Roziere et al., 2021a). Each identifier is replaced with a corresponding token like 'FUNC00' 0' and 'VAR_0', After such changes, the code examples still pre-
nuns dic longth target ) Intogor : > public
( 2 1
Figure 4: Shuffled version the two-sum solution. The code with all lines shuffled is shown above the code with all tokens shuffled.
serve the functionality of the original codes. We show some examples of the obfuscated code, shuf- fled code and deleted code are shown separately in Figure 3, Figure 4 and Figure 5.
We expect some probe tasks to evaluate whether well well-behaved code representation model is more confused about the noised code compare with the original code. We hypothesize that if the specific code part is damaged and the model's performance drops significantly on these tasks, it indicates that the model relies on this piece of semantic informa- tion to understand the overall code.
The first probe task is the zero-shot code-to-code search task proposed in UniXcoder (Guo et al., 2022), which retrieve codes with the same seman- tics from a collection of candidates in a zero-shot setting and rank the candidates with these similari- tics. Denote the obfuscation transform from code space W to Wohfusenten by fary- In practice, we

---

Denote the original code of the jth solution to the 1th problem by Wij and its embedding from 24 remover bepwordarr
model can tolerate the noise we add when design- ing the translation pivot to flur different high-level languages. Meanwhile replacing identifiers and deleting structure- duces the model's 's effectiveness. This indicates that identifiers and structure-related symbols are core information that should be kept in the translation pivot much as possible.
1 resdew C. I none , Parget
Sap s intoger integor
baaheap
(itee contatnatley mow
1 4 iit got ( nume [ asc . put target - ZURO [1] 1
symbols significantly re-
ya removen pertain nothollar/
penilie tor hendinn nom harger
Hap inbeger intogez
new Hashztup <.
die crntalnstley
retura new lnt dio got nund 1
eive
die put Larpet ZURx
Another probe task falls on the semantic analysis. Here our study aims to analyze the code semantic understanding of the model by analysing the distri- bution of cosine similarity on positive and negative code pairs, where each positive and negative ex- amples serve as the query and the other positive and negative examples in the candidate set serve as the candidate. We collect all the positive code pairs that are composed of any two solutions to the same question, and the negative pairs is composed of any two solutions to different questions. In the experiment, we inject noise into the positive and negative examples and observe the changes of their density plots of cosine similarities. We should try to ensure that adding noise to the model will not prevent it from distinguishing between positive and negative.
Figure 5: Deleted keywords Version of the two-sum solution. The code with keywords removed is shown above the code with structural symbols replaced with blank spaces.
can use the original code wi, as query and the ob- fuscated codes (os)(Wi,s) as candidates, and vice versa. For an input code Why which is the jth solu- tion to the 7th problem, we consider all solutions for the same problem as the correct retrieval results, i.e. wiy and Joof (w.j'). and all the solutions for other problems incorrect retrieval results, i.e. toy and sobs (w where i' +
muxcoder-basee
qurey-cand
bleu@1
p@ 10
map
mrr
Java-Java
53,49
23.81
28.78
36.03
Java- (ava-Javasshuffle)
73.58
22.5
26.88
41.97
83.09
24.3
29.49
40.21
19.9
21.91
26.38
40.38
a pre-trained model by hag- We apply average pooling to the outputs from the last layer of the pre-trained model to get the embedding h of the input code Wy.j E W. All the positive code pairs and the negative pairs can be denoted P { (wi.j. w and A {(wr,j wy Hize re- spectively. After computing the cosine similarities of the embeddings of the code pairs in P and N, we plot the histograms of the similarities to observe the changes in the similarity distribution of posi- tive and negative under different noise additions in Figure 6.
Java-Java(dobf)
2.59
8.43
9.62
18.23
Table 4: Compare code-code search task performance appling various noise addition methods. Java-Java means the ability of the Unix-Coder base to scarch for all Java codes with the same semantics from the candidate codes using certain Java code as a query. Java(shufile), lava(keywords) Java(symbols) and Java(dobf) represents the noise of adding shuffle code. delete the reserved keywords, delete the structure related symbols and replace the identifiers to all the codes in the candidate library.
Table 4 shows the interference of four types of noise on the zero-shot code-to-code search task. The BLEU@I is the BLEU score between the query and the candidate with the highest similarity. The P@ 10 is the percentage of the queries that the correct code is in the top 10 most similar codes. The MAP is the average of the average precision of the correct code for each query. The MRR is the av- erage of the reciprocal rank of the correct code for each query. These four metries comprehensively measure the quality of the retrieved results. Com- pared with the baseline, shuffling the original code and deleting reserved keywords does not interfere with the model's performance, which means the
Figure 6 shows the similarity distribution of pos- itive and negative code pairs under different noise additions. The green histogram and yellow his- togram in each subfigure represents the similarity score density distribution of positive pairs and neg- ative pairs. Ideally, the similarity distribution of negative examples should tend to 0 as whole, while the similarity distribution of positive exam- ples should tend to as a whole. Therefore, the larger the overlap between the yellow histogram and the green histogram, the more confused the code model is in distinguishing the semantic dif-

---

supali and pralie Latr
regalimo 144 paclue dee. PVN
04
cone umarrrty
Figure 6: Similarity distribution of positive and negative code pairs under different noise additions. The three subfigures study represent Java2Java (no noise added), Java2Java-dobf (dobf noise added to candidate set), and Java-dob22lavaddof (dobf noise added to both query and candidate) from left to right. The yellow histogram in each subfigure represents the cosine similarity distribution between each negative example and the rest of the negative examples. We calculated the proportion of code pairs occupying different similarities and formed the similarity score density distribution of the overall negative example pairs. The green histogram in each subfigure represents the similarity score density distribution of positive pairs.
ference between positive and negative examples. To analyze this, we introduced noise such as re- placing the identifiers in code (dobf), and gradually increased this noise in the query and candidate parts in the sub-images from left right. Our re- sults showed that as more dobf noise was added to code pairs, the similarity overlap between positive and negative code pairs increased, and the overall similarity distribution gradually moved closer to 1. This suggests that the semantic information in the identifiers part is important for the model to correctly understand and distinguish codes with different semantics.
Unitial Furn int float
C++
Fass
Python
int
ist
inf. string
strin a
stllstring a
String
bool
busl a
booltam
bad ai
bool
char Vectonc-
chara Voctorrth
char
Vectorec
sell
nupco sitrrppc
HichMapez a
Didhomrycs B
a= aequese Queacl
HashSeter
Queup-tr
Queupec
doqueccaa a
sudodaque-
Dequeio
Andoquet)
Table 6: Unify part of common data types.
Uniral Fimms
Pytkee sudt
Klab agoie
Makaattia
legin)
legis
Math logla
Marhl Mathaggi
mads logio
Beoria)
Beerial
Mata Boooa medonttaai-oo
Mats
mai Bloora
me00.0
maden madiino
Syeem out peiztia
Creacts webko))
priatl cndtr")
pisteed
Spaterorn pirtleed
aclineial
postaa
Choraterto monele
Cher Tul nollno)))
excplactaa)
contacessh
exephaciah)
Aloggh)
Likegte))
Clergis
lestal
B Unify Basic Morphemes
Table 7: Unify part of common built-in methods.
Here we demonstrate part of morphemes with equivalent semantics across the Python, Java, C++. and C# programming languages, and present these morphemes in their unified expression forms. To aid in models pretrained on common code corpora capable of comprehending the semantics of these morphemes, we often adopt their unified represen- tations from the symbols and reserved words found in various high-level programming languages.
Table 5, Table 6 and Table 7 are presented the unified forms of operators, data types, and built-in methods partly. Here strive to align the vast ma- jority of common basic morphemes in C++, Java, C#, and Python language. For those that we do not yet support, alignment will be handled by the fuzzy strategy in the compiler and the multilingual code snippet generation task in the decompiler.
Unified Form
C++
Ca
Python uth
ath
a4h
a4h
a-b
a-b
a-b
a-b
a-b a*b
atb
a*b
a*b
a*b
wb a/b
aSh w/b
a/b
a/b
aSh
With a//b
int(a/b)
int(a/b) pow(s.b)
int(a/b) Math.pow(a..)
int(a/b)
pow(s.b)
Math pow..
a
n&&b
&&
a& &b
a&&b
a and
allb
allb
allb
allb
a or b
la
not
Table 5: Unify part of common operators.


Reviews:
[{"review_id": "XuXsUQREBi", "reviewer": "Reviewer_U8tt", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you time and energy to review our manuscript! We read your opinions carefully and we want to explain further about these points of opinions.\n\n1. Some ablation studies analizes should be conducted to verify the effectiveness of the components of the method.Therefore, the description of the method should be simplified.\n\nSorry, As we consider that code translation tasks are relatively minor in EMNLP submissions, and our method introduce some complex concepts. Therefore, we would like to elaborate in detail and thoughtfulness to reduce the difficulty of understanding for non code translation reviewers. Taking into account the opinions of the other two reviewers, we have added the following ablation experimental results, please refer to the experiments we added. We will also supplement these experiments in the official version.\n\n1.1. Analyze the role of the code distillation steps(syntax tree pruning, unify basic morphemes, fuzzing remaining variations) in building a translation pivot\n\n|distilled code type  |BLEU|CodeBLEU|\n|------|------|------|\n|tree pruning|74|68|\n|tree pruning + fuzzing|76|71|\n|All(tree pruning + fuzzing+unify)|81|78|\n\nWe take the pair code of C++ and Java in the Transcoder geeksforgeeks evaluation data, and we use three strategies to extract their distiiled code (only syntax tree pruning, syntax tree pruning + fuzziness, and all strategies are used), and calculate the difference between the pair of distilled code. We measured their \"distance\" using BLEU and CodeBLEU, and the results are shown in the table above, and we can see that the addition of these three strategies can gradually bridge the differences between different language codes and bring them closer like a perfect translation pivot.\n\n1.2 2.Analyze the role of the code distillation steps(syntax tree pruning, unify basic morphemes, fuzzing remaining variations) in finall translation task\n\nComputation Accuracy on TransCoder GeeksforGeeks dataset:\n\n|distilled code type  |C++2Java|C++2Python|Java2C++\t|Java2Python|Python2C++|Python2Java|\n|------|------|------|------|------|------|------|\n|tree pruning|63.90%|44.61%|74.95%|50.86%|51.82%|52.90%|\n|tree pruning + fuzzing|68.46%|50.43%|76.02%|55.30%|57.68%|60.17%|\n|All(tree pruning + fuzzing+unify)|82.14%|67.89%|87.85%|68.10%|86.94%|81.12%|\n| Compared model|||||\n|TransCoder|65.1%|47.1%|79.8%|49.0%|32.6%|36.6%|\n\nBLEU & CodeBLEU on CodeXGLUE dataset\n\n|distilled code type|C#2Java BLEU|C#2Java CodeBLEU|Java2C# BLEU|Java2C# CodeBLEU|\n|------|------|------|------|------|\n|tree pruning|75.26|79.52|77.98|82.26|\n|tree pruning + fuzzing|83.01|82.98|83.14|86.58|\n|All(tree pruning + fuzzing+unify)|83.19|86.12|85.8|88.75|\n| Compared model|||\n|CodeBERT|72.14|79.41|79.92|85.10|\n\nWe can observe that the degree of contribution of these three parts is different in different types of code translation datasets. The Transcoder Geeksforgeeks dataset comes from programming test websites, the code of Geeksforgeeks most leverage on primative data types and expressions, so unify play the most important part of the translation; The code in CodexGLUE comes from the industry's github repro. There are a large number of cross-function and even cross-folder level API calls, so our fuzzy method can significantly improve the effect.\n\n1.2.3 add more pretrain task abalation study\n\n|pretrain method  |C++2Java|C++2Python|Java2C++\t|Java2Python|Python2C++|Python2Java|\n|------|------|------|------|------|------|------|\n|Codist|82.14%|67.89%|87.85%|68.10%|86.94%|81.12%|\n|w/o DAE|71.18%|64.01%|80.94%|64.22%|79.44%|70.54%|", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- I like the idea of improving code translation by utilizing a language-agnostic IR as the translation pivot. I think creating the proposed IR that  captures the semantic and structural equivalence of code is very interesting and useful.\n- All implementation details are presented clearly and resonably. The proposed method looks very convincing and solid.\n- The proposed method achieved very good performance, and the main SOTA methos are included in baselines. It’s surprising that the model’s performance is close to GPT-3.5-turbo.\n", "reasons_to_reject": "- Some ablation studies analizes should be conducted to verify the effectiveness of the components of the method.Therefore, the description of the method should be simplified.", "questions_to_authors": "", "paper_topic_and_main_contributions": "To overcome the the noise in code snippet  alignment and the diversity of IRs for code translation, the authors propose a novel model called  Code Distillation (CoDist) whereby they capture the semantic and structural equivalence of code in a language agnostic intermediate representation. Distilled code serves as a translation pivot for any programming language, leading by construction to parallel corpora which scale to all available source code by simply applying the distillation compiler. And the experiments show the proposed approach achieves state-of-the-art performance on CodeXGLUE and TransCoder GeeksForGeeks translation benchmarks.\n", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Yv1XhV2NYo", "reviewer": "Reviewer_KDdW", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you time and energy to review our manuscript! We read your opinions carefully and we want to explain further about these points of opinions.\n\n1. The invented language is limited to single object-oriented paradigm in programming, while there are other programming languages following different paradigms.\n\nIn fact, our code distillation method is not limited to object-oriented programming languages. In theory, all of the programming languages which can be parsing into syntax trees can be implemented through our method. we use Treesitter (supporting 113 program languages) as parsing tools.\n\nThe current limitation to our approach is rooted in the available evaluation datasets within this code translation domain. Regrettably, the current datasets (TransCoder GeeksforGeeks and CodeXGLUE)  does not encompass all programming language types. This is the main reason we presently apply this approach only to object-oriented languages. This is just a coincidence and not a limitation of our code distillation method. It's easy to extend this approach to other classes of languages, such as procedural-oriented languages.  In order to better understanding, we have given the following examples:\n\n1.1 Process-oriented code example\n\nC implementation of bubble sort:\n```\nvoid swap(int arr[], int i, int j) {\n    int temp = arr[i];\n    arr[i] = arr[j];\n    arr[j] = temp;\n}\n\nvoid bubbleSort(int arr[], int size) {\n    for (int i = 0; i < size - 1; i++) {\n        for (int j = 0; j < size - i - 1; j++) {\n            if (arr[j] > arr[j + 1]) {\n                swap(arr, j, j + 1);\n            }\n        }\n    }\n}\n```\nAccording to CoDist's compilation rules, the distilled code of this code should be\n```\nswap(arr, i, j){\n    temp = arr[i];\n    arr[i] = arr[j];\n    arr[j] = temp;\n}\n\nbubbleSort(arr, size){\n    for(i=0; i<size-1; i=i+1){\n        for(j=0; j<size-i-1; j=j+1){\n            if(arr[j]>arr[j+1]){\n                BOW: swap ( arr , j , j + 1 )\n            }\n        }\n    }\n}\n```\n\n1.2. Functional programming code example\n\nScheme  implementation of bubble sort:\n```\n(define (swap arr i j)\n  (let ((temp (vector-ref arr i)))\n    (vector-set! arr i (vector-ref arr j))\n    (vector-set! arr j temp)))\n\n(define (bubble-sort arr size)\n  (do ((i 0 (+ i 1)))\n      ((< i (- size 1)))\n    (do ((j 0 (+ j 1)))\n        ((< j (- size i 1)))\n      (if (> (vector-ref arr j) (vector-ref arr (+ j 1)))\n          (swap arr j (+ j 1)))))\n  arr)\n```\nIn Scheme, prefix notation is used to represent computational processes. Here at Distilled Code, we will uniformly convert it into infix notation to align with other languages. Then according to CoDist's compilation rules, the distilled code of this code should be：\n```\nswap(arr, i, j){\n    temp = arr[i]\n    arr[i] = arr[j]\n    arr[j] = temp\n}\n\nbubble-sort(arr, size){\n    for(i=0; i<size-1; i=i+1){\n        for(j=0; j<size-i-1; j=j+1){\n            if(arr[j]>arr[j+1){\n                BOW: swap ( arr , j , j + 1 );\n            }\n        }\n    }\n    return arr;\n}\n```\n1.3. Object-oriented code example\n\nPython implementation of bubble sort:\n```\ndef swap(arr, i, j):\n    temp = arr[i]\n    arr[i] = arr[j]\n    arr[j] = temp\n\ndef bubble_sort(arr):\n    for i in range(len(arr) - 1):\n        for j in range(len(arr) - i - 1):\n            if arr[j] > arr[j + 1]:\n                swap(arr, j, j + 1)\n```\nAccording to CoDist's compilation rules, the distilled code of this code should be\n```\nswap(arr, i, j){\n    temp = arr[i];\n    arr[i] = arr[j];\n    arr[j] = temp;\n}\n\nbubbleSort(arr){\n    for(i=0; i<len(arr)-1; i=i+1){\n        for(j=0; j<len(arr)-i-1; j=j+1){\n            if(arr[j]>arr[j+1]){\n                BOW: swap ( arr , j , j + 1 )\n            }\n        }\n    }\n}\n```\nHere, we've chosen three different programming paradigms for demonstration: procedural (C), functional (Scheme), and object-oriented (Python). We implemented the bubble sort algorithm in these languages and converted them to our Distilled Code. Despite the significant syntax differences among all languages, our approach can provide a highly unified representation. This demonstrates our method's ability to generalize across different programming styles.\n\nThanks again for your perspective. This brings inspiration to our future work, and we may expand our method to other types of languages ​​in the future work(datset building and evaluation).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The authors in this paper are reached the holy grail for NLP in the different domain, the grail I mentioned is interlingua, a universal language. The authors developed interlingua for the programming languages and demonstrated its usefulness.", "reasons_to_reject": "The invented language is limited to single object-oriented paradigm in programming, while there are other programming languages following different paradigms.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The authors present a novel approach to code translation using intermediate code representation. The main idea is to use a synthetic language code as intermediate representation. The main contribution are the method of intermediate code representation and a model for the translation using this representation.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "0Jb64muG6N", "reviewer": "Reviewer_soRb", "summary": "", "questions": "Why is there no experiment conducted on w/o DAE in Table 3?\n\n", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you time and energy to review our manuscript! We read your opinions carefully and we want to explain further about these points of opinions.\n\n1. The paper lacks sufficient ablation analysis concerning the distilled code methods. For instance, it does not explore the impact of excluding syntax tree pruning, unify basic morphemes, or fuzzing remaining variations individually.\n\nWe have added some experiments here for your reference.\n\n1.1. Analyze the role of the three steps in building a translation pivot\n\n|distilled code type  |BLEU|CodeBLEU|\n|------|------|------|\n|tree pruning|74|68|\n|tree pruning + fuzzing|76|71|\n|All(tree pruning + fuzzing+unify)|81|78|\n\nWe take the pair code of C++ and Java in the Transcoder geeksforgeeks evaluation data, and we use three strategies to extract their distiiled code (only syntax tree pruning, syntax tree pruning + fuzziness, and all strategies are used), and calculate the difference between the distilled code of the pair pair code between them. We measured their \"distance\" using BLEU and CodeBLEU, and the results are shown in the table above, and we can see that the addition of these three strategies can gradually bridge the differences between different language codes and bring them closer like a perfect translation pivot.\n\n1.2 2.Analyze the role of the three steps in finall translation task\n\nComputation Accuracy on TransCoder GeeksforGeeks dataset:\n\n|distilled code type  |C++2Java|C++2Python|Java2C++\t|Java2Python|Python2C++|Python2Java|\n|------|------|------|------|------|------|------|\n|tree pruning|63.90%|44.61%|74.95%|50.86%|51.82%|52.90%|\n|tree pruning + fuzzing|68.46%|50.43%|76.02%|55.30%|57.68%|60.17%|\n|All(tree pruning + fuzzing+unify)|82.14%|67.89%|87.85%|68.10%|86.94%|81.12%|\n| Compared model|||||\n|TransCoder|65.1%|47.1%|79.8%|49.0%|32.6%|36.6%|\n\nBLEU & CodeBLEU on CodeXGLUE dataset\n\n|distilled code type|C#2Java BLEU|C#2Java CodeBLEU|Java2C# BLEU|Java2C# CodeBLEU|\n|------|------|------|------|------|\n|tree pruning|75.26|79.52|77.98|82.26|\n|tree pruning + fuzzing|83.01|82.98|83.14|86.58|\n|All(tree pruning + fuzzing+unify)|83.19|86.12|85.8|88.75|\n| Compared model|||\n|CodeBERT|72.14|79.41|79.92|85.10|\n\nWe can observe that the degree of contribution of these three parts is different in different types of code translation datasets. The Transcoder Geeksforgeeks dataset comes from programming test websites, and its titles rarely involve complex external API calls, so fuzzing remaining variations such as on the type like method invocation are relatively rare; The code in CodexGLUE comes from the industry's github repro, so there are a large number of cross-function and even cross-folder level API calls, so our fuzzy method can significantly improve the effect.\n\n2. The unify basic morphemes approach necessitates numerous manual operations, making it challenging to extend its applicability to additional languages.\n\nThe appendix B of the paper and our feedback here can be used for your reference. Our unified basic themes here mainly include operators, primitive data types, and build-in methods parts. The first two parts can be easily enumerated and their concepts are commonly shared by most programming languages. The build-in methods are also enumerated and shared across all popular languages (e.g., print, sort, etc.)\n\nIn the codist method, for non enumerable code, we actually rely on fuzzy part of code and parsing tree prune to complete intermediate language extraction, which completely avoids manual components and can automatically generate samples without human intervention. \n\nIn short, Codist is designed to be an unsupervised algorithm. Its main body alignment is achieved through syntax tree pruning and fuzzification operations. Manual alignment is limited and enumerable, yet can reduce the noise significantly and relieve the neural networking understanding burdens. \n\n3. Why is there no experiment conducted on w/o DAE in Table 3?\n\nWe have added some experiments here for your reference.\n\n|pretrain method  |C++2Java|C++2Python|Java2C++\t|Java2Python|Python2C++|Python2Java|\n|------|------|------|------|------|------|------|\n|Codist|82.14%|67.89%|87.85%|68.10%|86.94%|81.12%|\n|w/o DAE|71.18%|64.01%|80.94%|64.22%|79.44%|70.54%|", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper introduces a novel pivot language for code translation and investigates three distinct self-supervised pre-training tasks.\n2. The motivation behind the study is lucid, and the paper is thoughtfully composed and well-written.", "reasons_to_reject": "1. The paper lacks sufficient ablation analysis concerning the distilled code methods. For instance, it does not explore the impact of excluding syntax tree pruning, unify basic morphemes, or fuzzing remaining variations individually.\n2. The unify basic morphemes approach necessitates numerous manual operations, making it challenging to extend its applicability to additional languages.", "questions_to_authors": "Why is there no experiment conducted on w/o DAE in Table 3?\n\n", "paper_topic_and_main_contributions": "The paper introduces a novel model named Code Distillation (CoDist) for code translation. This model effectively captures the semantic and structural equivalence of code in a language-agnostic intermediate representation, enabling it to serve as a translation pivot for any programming language. Experimental results demonstrate that the proposed model outperforms the baselines on both the CodeXGLUE and GeeksForGeeks translation benchmarks. The main contributions of this research include: (1) the introduction of a new programming translation pivot language called distilled code, and (2) the utilization of multiple unsupervised code corpora through three distinct self-supervised pre-training tasks.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "N/A", "Typos_Grammar_Style_and_Presentation_Improvements": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper addresses program translation and proposes an intermediate language (called distilled code) that prunes language-specific details but preserves the semantics of a program. (It's noticed that _distilling_ here has a different meaning from knowledge distilling).

Reviewers generally find the idea interesting - having an intermediate language hasn't been work really well for translating natural language but may be a good direction to go for translating programming language. Overall, the approach is well designed and achieves high performance.
Decision: Accept-Main