Patent Description:
The disclosure herein generally relates to the field of code translation, and, more particularly, to a method and system for the translation of programming code based on the semantic similarity.

Code translation is an evolving field which was existent over years. In the recent times, due to the advancements in the infrastructure and compute power, this field has got a renewed interest. Availability of skilled people who understand the technology is limited and this has also fueled the need for code translations to technologies which are cloud native. Though one to one translation which were once ignored are getting renewed interests due to the cloud advantage, there are many more open issues to be addressed. This type of one to one translation still carries the technical debt to the target side also and are not architecturally native to the target environment. Recent advancements in machine learning (ML) based translations either attempt one to one translations or constrained translations where they are dependent on naming conventions of methods and variables.

Legacy codes have evolved over the years and are maintained by multiple developers. Each maintenance change done by a new developer other than the owner on the code might introduce technical debt to the original architecture. Due to this, there might be many ways in which the same functionality is duplicated / coded and possibility of dead assignments and dead codes in the code are enormous. Though static dead code identification algorithms exist, their relevance in execution context is always questionable. All the above issues make the direct one to one translation of code or the constrained translation techniques difficult to maintain and a solution that involves a complete reverse engineering and forward engineering is also time and effort intensive. In all the above scenarios, the ability to understand the context beyond a statement and group the statements as blocks to make the step of translation is essential.

The existing capabilities of code translation do not go beyond statement to statement translation and hence these methods do not understand the semantics of the code. Syntactically same but semantically similar code understanding does not happen. As the complete code is translated, the technical debt, the unused code, the duplicate code etc. are all translated too and hence is difficult to maintain. Architecturally the translated code resembles the source code which makes it alien to the target environment.

Document<NPL>, discloses a few basic ideas that can be used to construct translators for a wide variety of languages and machines. Besides compilers, the principles and techniques for compiler design are applicable to so many other domains that they are likely to be reused many times in the career of a computer scientist. The study of compiler writing touches upon programming languages, machine architecture, language theory, algorithms, and software engineering.

Document <NPL>, discloses many of the techniques used by an aggressive optimizing compiler to make a program faster and smaller.

Document<NPL>, discloses leveraging recent approaches in unsupervised machine translation to train a fully unsupervised neural transcompiler. The model is trained on source code from open source GitHub projects, and shown that it can translate functions between C++, Java, and Python with high accuracy. The method relies exclusively on monolingual source code, requires no expertise in the source or target languages, and can easily be generalized to other programming languages. The model can also build and release a test set composed of <NUM> parallel functions, along with unit tests to check the correctness of translations.

Document<NPL>, introduces the concept of program representation learning based on functional equivalence. It describes a self-supervised representation learning algorithm that uses source-to-source compiler transformation techniques (e.g., dead code elimination, obfuscation and constant folding) to generate syntactically diverse but functionally equivalent program snippets. These equivalent program snippets are used by a model to identify equivalent programs out of a large dataset of distractors.

Embodiments of the present disclosure present technological improvements as solutions to one or more of the above-mentioned technical problems recognized by the inventors in conventional systems. The invention is defined in appended claim set <NUM> to <NUM>.

Code translation is an evolving field and due to the advancements in the infrastructure and compute power, this field has got a renewed interest. The few of the existing methods for code translation involve one-to-one translation or constrained translations where they are dependent on naming conventions of methods and variable. In those cases, dead code is repeated, and architecture becomes heavy. Moreover, same code gets translated repeatedly which results in consumption of more time and effort. The above mentioned issues make the direct one to one translation of code difficult to maintain and a solution that involves a complete reverse engineering and forward engineering is time and effort intensive.

According to some embodiment of the disclosure, a method and a system for translation of codes based on the semantic similarity have been provided to solve the technical problems of the prior art. A machine learning (ML) model is developed, that understands and encapsulates the semantics of the code in the source side and translates the semantic equivalent code which is more maintainable and efficient compared to one to one translation.

The system is configured to group a plurality of statements present in the source code together into blocks of code and comprehend the semantics of the block. The translation is made to the block and not individual statements. The system is trained to understand syntactically different but semantically similar statements. While understanding the semantics of the block and translating, the unused / duplicate code is eliminated and the technical debt that passes from the source to the target during the translation is reduced. As the system understands the semantics of the source system, it tries to group the components also based on defined architectures. The translated code is better architected and native to the target environment.

<FIG> illustrates a network diagram of a system <NUM> for translation of codes based on the semantic similarity. The codes are normally referred as source codes and are taken from the software program. Although the present disclosure is explained considering that the system <NUM> is implemented on a server, it may also be present elsewhere such as a local machine. It may be understood that the system <NUM> comprises one or more computing devices <NUM>, such as a laptop computer, a desktop computer, a notebook, a workstation, a cloud-based computing environment and the like. It will be understood that the system <NUM> may be accessed through one or more input/output interfaces <NUM>-<NUM>, <NUM>-<NUM>. <NUM>-N, collectively referred to as I/O interface <NUM>. Examples of the I/O interface <NUM> may include, but are not limited to, a user interface, a portable computer, a personal digital assistant, a handheld device, a smartphone, a tablet computer, a workstation and the like. The I/O interface <NUM> are communicatively coupled to the system <NUM> through a network <NUM>.

In an embodiment, the network <NUM> may be a wireless or a wired network, or a combination thereof. In an example, the network <NUM> can be implemented as a computer network, as one of the different types of networks, such as virtual private network (VPN), intranet, local area network (LAN), wide area network (WAN), the internet, and such. The network <NUM> may either be a dedicated network or a shared network, which represents an association of the different types of networks that use a variety of protocols, for example, Hypertext Transfer Protocol (HTTP), Transmission Control Protocol/Internet Protocol (TCP/IP), and Wireless Application Protocol (WAP), to communicate with each other. Further, the network <NUM> may include a variety of network devices, including routers, bridges, servers, computing devices, storage devices. The network devices within the network <NUM> may interact with the system <NUM> through communication links.

The system <NUM> may be implemented in a workstation, a mainframe computer, a server, and a network server. In an embodiment, the computing device <NUM> further comprises one or more hardware processors <NUM>, one or more memory <NUM>, hereinafter referred as a memory <NUM> and a data repository <NUM>, for example, a repository <NUM>. The memory <NUM> is in communication with the one or more hardware processors <NUM>, wherein the one or more hardware processors <NUM> are configured to execute programmed instructions stored in the memory <NUM>, to perform various functions as explained in the later part of the disclosure. The repository <NUM> may store data processed, received, and generated by the system <NUM>. The memory <NUM> further comprises a plurality of modules. The plurality of modules is configured to perform various functions. The plurality of modules comprises a block vectorization module <NUM>, a semantic code understanding module <NUM>, and a semantically similar pattern learning module <NUM>.

The system <NUM> supports various connectivity options such as BLUETOOTH®, USB, ZigBee and other cellular services. The network environment enables connection of various components of the system <NUM> using any communication link including Internet, WAN, MAN, and so on. In an exemplary embodiment, the system <NUM> is implemented to operate as a stand-alone device. In another embodiment, the system <NUM> may be implemented to work as a loosely coupled device to a smart computing environment. The components and functionalities of the system <NUM> are described further in detail.

According to an embodiment of the disclosure, a functional block diagram of the system <NUM> for translation of codes based on the semantic similarity is shown in <FIG>. A software program is normally written in a certain programming language which comprises a number of lines of codes. These codes are referred as the source codes. As shown in <FIG>, the source code is provided as the input and this input source code is split into a plurality of blocks. The plurality of blocks is then given to the block vectorization module <NUM>. Further, the semantically similar pattern learning module <NUM> takes the output from the block vectorization module <NUM> and the semantic code understanding module <NUM> as input and learns to represent the related blocks of code which are semantically similar. These semantically similar related blocks of code are then translated into a target language selected by the user.

<FIG> illustrates an example flow chart of a method <NUM> for translation of the codes based on the semantic similarity, in accordance with an example embodiment of the present disclosure. The method <NUM> depicted in the flow chart may be executed by a system, for example, the system <NUM> of <FIG>. In an example embodiment, the system <NUM> may be embodied in a computing device.

Operations of the flowchart, and combinations of operations in the flowchart, may be implemented by various means, such as hardware, firmware, processor, circuitry and/or other device associated with execution of software including one or more computer program instructions. For example, one or more of the procedures described in various embodiments may be embodied by computer program instructions. In an example embodiment, the computer program instructions, which embody the procedures, described in various embodiments may be stored by at least one memory device of a system and executed by at least one processor in the system. Any such computer program instructions may be loaded onto a computer or other programmable system (for example, hardware) to produce a machine, such that the resulting computer or other programmable system embody means for implementing the operations specified in the flowchart. It will be noted herein that the operations of the method <NUM> are described with help of system <NUM>. However, the operations of the method <NUM> can be described and/or practiced by using any other system.

Initially at step <NUM>, the source code is provided for translation as the input. At step <NUM>, the source code is parsed using a parser. The use of any existing parser is well within the scope of this disclosure. The control flow of the program along with the data dependencies that can be technically extracted from the code are extracted in this step. In this step only the variable level information is extracted.

Further at step <NUM>, a program flow graph is created using the parsed source code, wherein the program flow graph is used to establish a set of relations between a plurality of statements present in the source code. In an example, there are three type of statement relations;.

Further, at step <NUM>, a matrix of the established set of relations is created between the plurality of statements. The establish set of relations derived from the previous step is used to create a matrix of relationship between the statements. This information is further processed to get the statements that are closer to each other / compared to other statements irrespective of their physical presence in the code.

At step <NUM>, the source code is split into the plurality of blocks using the matrix, wherein the statements in one block are closer to each other as compared to other statements irrespective of their physical presence in the source code. An example of source code splitting is provided in the later part of the disclosure.

At step <NUM>, the plurality of blocks is vectorized to get a vectorized plurality of blocks using the block vectorization module <NUM>. A block diagram of the block vectorization module <NUM> is shown in <FIG>. In any software program, there could be one or more programs, say program <NUM>, program <NUM>, program n etc. Each program has one or more data elements and one or more method. Further, each method comprises one or more statements, and these statements are referred as the plurality of blocks. The vectorized plurality of blocks are generated based on multiple properties of the source code such as usage, value associated, scope of usage, data types/ structure/ size, relation with other elements, coding pattern and domain pattern etc..

At step <NUM>, a model is trained to understand a semantic equivalence between the plurality of blocks of code even if they are syntactically coded in different manner using the semantic code understanding module <NUM>. The main objective is to have the model having a semantically enriched encoder/decoder which drives translation between the source and the target language. A semantically enhanced encoder is trained on source code with the context of semantic equivalence. This gives an edge over understanding different semantical construction within a given language.

At step <NUM>, the method <NUM> includes identifying semantically similar statements out of the vectorized plurality of block using the trained model utilizing the semantically similar pattern learning module <NUM>. The identification of semantically similar statements leads to understanding blocks of semantically equivalent code which is present throughout the source inventory. This is also a feature in translation wherein the technical debt and duplications that are present in the source inventory is completely optimized and eliminated.

At step <NUM>, a target language is selected by the user using the user interface <NUM>, in which the source code needs to be translated. The translation is completely specific to the target language. And finally, at step <NUM>, the source code is translated into the target language using a decoder (not shown in the figure). The decoder is a pretrained machine learning model and configured to choose semantically same, but syntactically different statements. The choice between semantically same, but syntactically different statements is chosen by the decoder itself. For example, while decoding a looping logic, which loop should be used, a "for" loop or a "while" loop - such decisions are made by the decoder.

Final step of <NUM> can also be explained with the help of another example as follows. compute vs add functions can be used to describe semantically similar, but syntactically different statements.

Following are the benefits due to the model learning such similarities while translating and decoding.

The translated target code is not just line to line translation as in any available methods. In the translated code, the technical debts and duplication/dead code are removed through the semantic equivalence analysis. The target decoder also ensures a translation of functionally equivalent code in the target that is native to the target architecture.

Further, semantically similar, but syntactically different statements detection comprises: A supervised dataset is created consisting of varieties of syntactically different but semantically same statements in respective languages. This is performed to ensure that the encoder and decoder have the necessary information to vectorize code blocks in an efficient manner.

The supervised dataset is then preprocessed, wherein variable names are normalized to ensure lesser vocabulary handling hence making it easier for the model to learn. A tokenizer for this model is trained on the dataset and arrives at a lower number of vocabulary due to the repetitive nature of code syntax words. Then, a transformers based model is pre-trained on the respective source code language using a language modelling technique. The model takes in lines of dataset wherein some words/tokens (<NUM>% of the input words or tokens) are masked and the model tries to predict what the masked token/word is. Through this pretraining technique, a language model is obtained which now has learnt embeddings of syntax of the associated language.

This model is then further fine-tuned on the created supervised dataset which will train the model to generate similar vectors for semantically similar but syntactically different statements. These statements are sent as pairs together to the model and trained to output mathematically closer vectors for such pairs and distant vectors for unlike pairs. The attention mechanism in the transformer model learns the patterns behind such statements to produce mathematically closer vectors which is then used in further processes. In an example, the transformer based architecture is using: no of attention heads - <NUM> and no of hidden layers - <NUM>.

According to an embodiment of the disclosure, <FIG> shows a block diagram of the semantic code understanding module <NUM>. The semantic code understanding module <NUM> encompasses a Machine Learning (ML) based architecture which when given two blocks will try to represent them based on the semantic similarity irrespective of syntax. For each of the block, variable properties are taken into account and alongside the semantics of the block. The model which is a neural network generates vectors for the blocks based on semantic similarity. This is further utilized by the decoder to translate it into the target language.

In the neural network model, the transformer based model is employed in which the source language code snippet is taken as an input and then encode it using an encoder-based architecture to create a language independent vector format and then the target language-based decoder model is used and finetuned to get the semantically equivalent output for the provided input code snippet.

According to an embodiment of the disclosure, the system <NUM> can also be explained with the help of following examples:.

Example for generation of plurality of blocks:.

Example Code Snippet:
<IMG>
Thus, example of generated one block out of the plurality of blocks is
<IMG>
<IMG>.

The dotted lines in the above matrix denotes the sequence of permutations which are not significant according to the relationships.

In the above example, ACCOUNT-ID, MULTIPLYING-FACTOR and INTEREST and three different logics. Above is an example of the INTEREST block separated. Likewise, the code is split into blocks of code that pertain to a functionality based on the above method.

Example of identifying semantic equivalence. <IMG>
<IMG>.

In the above example, there are two blocks which are performing certain function, until the value of A reaches <NUM>. This can be done in two separate ways as shown in Block <NUM> and Block <NUM>. Thus, both the blocks are performing same function, in other words, both are semantically similar, even though the syntax is different for block <NUM> and Block <NUM>.

The scope of the subject matter embodiments is defined by the claims.

The disclosure herein addresses unresolved problem maintaining direct one to one translation of code difficult time and effort efficient. The embodiment thus provides the method and system for translation of codes based on the semantic similarity.

The hardware device can be any kind of device which can be programmed including e.g. any kind of computer like a server or a personal computer, or the like, or any combination thereof. The device may also include means which could be e.g. hardware means like e.g. an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a combination of hardware and software means, e.g. an ASIC and an FPGA, or at least one microprocessor and at least one memory with software processing components located therein. Alternatively, the embodiments may be implemented on different hardware devices, e.g. using a plurality of CPUs, GPUs etc..

Alternatives within the scope of the claims will be apparent to persons skilled in the relevant art(s) based on the teachings contained herein.

Claim 1:
A processor implemented method (<NUM>) for translation of codes based on a semantic similarity, the method comprising:
providing, via a user interface, a source code for translation as an input (<NUM>);
parsing, via one or more hardware processors (<NUM>), the source code using a parser (<NUM>);
creating, via the one or more hardware processors (<NUM>), a program flow graph using the parsed source code, wherein the program flow graph is used to establish a set of relations between a plurality of statements present in the source code (<NUM>);
creating, via the one or more hardware processors (<NUM>), a matrix of the established set of relations between the plurality of statements (<NUM>);
splitting, via the one or more hardware processors (<NUM>), the source code into a plurality of blocks using the created matrix;
training, via the one or more hardware processors (<NUM>), a semantically enhanced encoder with a context of a semantic equivalence between the plurality of blocks of the source code irrespective of a manner in which the plurality of blocks is syntactically coded (<NUM>);
identifying, via the one or more hardware processors (<NUM>), semantically similar statements out of the plurality of blocks using the semantically enhanced encoder (<NUM>);
selecting, via the user interface, a target language in which the source code needs to be translated (<NUM>);
translating, via the one or more hardware processors (<NUM>), the source code into the target language using a decoder based on the identified semantically similar statements, wherein the decoder is a pretrained machine learning model, wherein the encoder and decoder represent a transformer model, wherein the transformer model is fine-tuned on a supervised data set to generate one or more similar vectors for semantically similar but syntactically different statements, wherein similar vectors are mathematically close, and wherein the transformer model understands and encapsulates code semantics and translates a semantic equivalent code, wherein an attention mechanism in the transformer model learns patterns to produce one or more mathematically closer vectors, and the decoder utilizes the generated vectors to translate the generated vectors into the target language; (<NUM>) and
eliminating, via the one or more hardware processors, during translation at least one of unused and duplicate codes from the source code using the set of relations and by semantic equivalence analysis.