Enhancing Cross-Language Code Translation via Task-Specific Embedding Alignment in Retrieval-Augmented Generation (2025)

Abstract: We introduce a novel method to enhance cross-language code translation from Fortran to C++ by integrating task-specific embedding alignment into a Retrieval-Augmented Generation (RAG) framework. Unlike conventional retrieval approaches that utilize generic embeddings agnostic to the downstream task, our strategy aligns the retrieval model directly with the objective of maximizing translation quality, as quantified by the CodeBLEU metric. This alignment ensures that the embeddings are semantically and syntactically meaningful for the specific code translation task. Our methodology involves constructing a dataset of 25,000 Fortran code snippets sourced from Stack-V2 dataset and generating their corresponding C++ translations using the LLaMA 3.1-8B language model. We compute pairwise CodeBLEU scores between the generated translations and ground truth examples to capture fine-grained similarities. These scores serve as supervision signals in a contrastive learning framework, where we optimize the embedding model to retrieve Fortran-C++ pairs that are most beneficial for improving the language model’s translation performance. By integrating these CodeBLEU-optimized embeddings into the RAG framework, our approach significantly enhances both retrieval accuracy and code generation quality over methods employing generic embeddings. On the HPC Fortran2C++ dataset, our method elevates the average CodeBLEU score from 0.64 to 0.73, achieving a 14% relative improvement. On the Numerical Recipes dataset, we observe an increase from 0.52 to 0.60, marking a 15% relative improvement. Importantly, these gains are realized without any fine-tuning of the language model, underscoring the efficiency and practicality of our approach.

## introduction cross-language code translation is a critical task in modern software development, especially as legacy programming languages, such as fortran, continue to be prevalent in scientific computing, while more contemporary languages like c++ are favored for their performance and versatility in production environments. the goal of automatic translation from fortran to c++ is to preserve the functionality and structure of legacy code while benefiting from the optimizations and ecosystem of c++. however, achieving high-quality translations that adhere to the syntax and semantic norms of the target language remains a challenging problem, particularly when there is a lack of large, aligned datasets or evaluation metrics that cover both source and target languages effectively. traditional approaches to cross-language translation, such as retrieval-augmented generation (rag) @xcite typically involve two phases: first, retrieving relevant examples from a database, followed by a language model generating code conditioned on both the query and the retrieved examples. in prior efforts, the retrieval models in rag systems have relied on generalpurpose embedding models @xcite , which are not tailored to the specific nuances of code translation. these embeddings aim to retrieve relevant pairs from the source and target languages but do not directly optimize for the quality of the generated code. as a result, while the retrieved examples may be relevant in a broad sense, they often fail to guide the language model towards producing translations that maximize fidelity to the ground truth in the target language. given the scarcity of high-quality parallel fortran-c++ data, we generate synthetic c++ translations from abundant fortran code using an llm to create a pseudo-parallel corpus. although these synthetic translations may not be flawless, they provide a robust similarity signal that enables effective alignment of fortran code embeddings. this "pseudo-alignment" enhances the retrieval of relevant examples in our rag framework, lead-ing to significant improvements in downstream translation quality as demonstrated by a consistent 14-15% gain in codebleu scores. we collect a dataset of 25,000 fortran code examples from stack v2 @xcite and use the llama 3.1-8b @xcite model to generate corresponding c++ translations. in the absence of ground truth c++ translations, we evaluate the quality of these translations using pairwise codebleu similarity scores. this metric captures both syntactic correctness and semantic fidelity, providing a robust signal for aligning the retrieval model through contrastive learning. the proposed approach aims to addresses the shortcomings of general-purpose embedding models by integrating task-specific metrics into the retrieval optimization process. by aligning the retrieval model with the downstream task of producing high-quality c++ code, our method ensures that the examples retrieved during inference are not just broadly similar but are semantically and syntactically aligned in a way that enhances the llm's generative performance. the result is a significant improvement in translation quality, as measured by codebleu, over previous methods that lack such alignment. our contribution is twofold: first, we demonstrate the effectiveness of contrastive learning for fine-tuning retrieval models in the context of crosslanguage code translation, using a task-specific metric to guide alignment. second, we show that optimizing retrieval for downstream generation tasks can lead to state-of-the-art results, particularly in cases where aligned datasets are not readily available for both source and target languages. this work not only advances the field of code translation but also opens up new possibilities for applying similar techniques to other language pairs and domains where task-specific evaluation metrics are available for only one side of the translation. ## related work historically, code translation strategies before the advent of llms relied heavily on rule-based and statistical machine translation (smt) systems @xcite . these systems used predefined rules or statistical mappings between the source and target programming languages, such as tree-based translation approaches that mapped syntax trees between languages. while these methods provided structured and interpretable outputs, they were limited in their ability to handle the semantic complexities of different programming languages and struggled with code diversity, edge cases, and idiomatic translations. with the rise of deep learning and llms, finetuning models on large datasets became the goto method for improving code translation. models like codebert @xcite and codex @xcite , when fine-tuned on specific language pairs, improved translation quality by leveraging vast amounts of parallel code data. however, the main limitation of llm fine-tuning lies in the resource-intensive process. fine-tuning requires substantial amounts of labeled data and computational resources, making it impractical for niche or legacy languages like fortran, where parallel data may be scarce. as a next step, task-specific alignment of llms emerged to improve translation by better guiding the model's output. while alignment techniques help improve output fidelity, they still necessitate fine-tuning or explicit modification of the llm itself, which can be resource-intensive and may still fall short of generalization when translating between languages with significant structural differences @xcite . rag introduced a more flexible approach by allowing llms to retrieve and condition their outputs on example pairs from a relevant dataset. while rag improves translation by augmenting the model's input, the effectiveness of this strategy depends on the quality and relevance of the retrieved examples. in an example case @xcite , the retrieval step relies on generalpurpose embeddings like nomic-embed or code-bert, which, although effective at retrieving semantically similar code, are not optimized for specific downstream metrics like codebleu. as a result, the llm might not always retrieve the examples that would best assist in producing translations aligned with target-specific quality metrics. the approach we propose offers a significant advantage by focusing on semantic alignment of the retrieval mechanism without the need to finetune the llm itself. through contrastive learning, we optimize the embedding model to retrieve fortran-c++ pairs that are more likely to maximize the downstream metric (e.g., codebleu) when used by the llm for generation. this strategy ensures that the most relevant examples are retrieved for each translation task, improving the generation quality without requiring computationally expensive fine-tuning of the llm. this retrieval alignment makes rag more efficient and better suited for translating between languages where high-quality paired datasets may not be available. by concentrating on improving the quality of retrieved examples, our method achieves high-quality translation with minimal additional model training, leveraging existing llm capabilities more effectively. ## methods this section provides the technical description of our proposed method. ## experiments and results in our study, we utilized three datasets to enhance code translation through rag and embedding alignment. the hpc fortran2cpp dataset @xcite , comprising 315 fortran-c++ code pairs, and the numerical recipes dataset @xcite , containing 298 fortran-c++ pairs, were employed for rag retrieval and evaluation with llms. additionally, we used the stack-v2 dataset @xcite , which includes over 500,000 fortran code snippets, for rag alignment. from stack-v2, we sampled 25,000 high-quality and diverse fortran code snippets by selecting files larger than 500 bytes and prioritizing those with the highest combined star and fork counts, indicating relevance and popularity. since stack-v2 lacks fortran-c++ pairs, we extracted files containing metadata, code, and comments, and utilized the llama 3.1-70b instruct model to extract executable fortran code, discarding other metadata. we selected the starcoder model @xcite with 125m parameters as the embedding backbone for our rag pipeline and aligned it using contrastive learning on the stack-v2 dataset. initially, we use the llama 3.1-8b model to translate the cleaned fortran code snippets into corresponding c++ code. after code translaton, we computed pairwise codebleu scores between the generated c++ code snippets to quantify the syntactic and semantic similarities of their translations. leveraging these codebleu metrics and the embeddings from the fortran codes, we employed the proposed soft-infonce loss function with a temperature of 0.1 to align the embeddings, effectively training the embedding model to map semantically similar code snippets closer in the embedding space. the embedding model was trained using the adam optimizer with a learning rate of 10 -3 and a batch size of 128 per gpu, sampling approximately 1,280,000 code pairs for alignment. this training process was distributed across 256 gh200 gpus to accelerate the process, though it can also be performed on fewer gpus at a significantly slower pace. training on 256 gh200 gpus took approximately 15 minutes per epoch, with early stopping at epoch 20. this scales to around 60 minutes per epoch with 64 gpus and 2 hours per epoch with 32 gpus. while training a rag model does incur computational overhead, it remains significantly less expensive than fine-tuning a multibillion-parameter llm. after alignment, we integrated the embedding model into the rag pipeline, storing fortran-c++ pairs along with their fortran embeddings in a vector database. we then evaluated the performance using the llama 3.1-8b, llama 3.1-70b, mistral123b, and mixtral 8x22b models-all instruct-tuned-under zero-shot, 1shot, 2-shot, and 3-shot settings. the evaluation was conducted on the benchmark datasets hpc fortran2c++ and numerical recipes, following the setup described by @xcite . the codebleu scores for both the aligned and unaligned models were obtained by comparing the rag-augmented generated c++ translations against the ground truth c++ code. figure 2 shows scatter-plots of codebleu scores for code samples produced using rag retrieval with aligned versus unaligned embeddings derived from starencoder. symbols crosses, pluses and triangles respectively indicate whether the sample was evaluated using a 1-shot, 2-shot, or 3-shot method. the red dashed lines delineates the boundary where the aligned samples have the same code-bleu score as the non-aligned ones, and across all four tested datasets, we observed a majority of samples above the red line, indicating that the aligned model produces translated codes closer to ground truth. in other words, the results in figure 2 demonstrate that aligned embeddings significantly improve translation quality for each fortran-to-c++ code translation task. specifically, on the hpc for-tran2c++ dataset, averaged over all shot counts and models, the aligned embeddings achieved an average codebleu score of 0.73, whereas unaligned embeddings achieve 0.64. figure 3 further corroborates these findings by presenting the distribution of codebleu scores across various experimental configurations. the box plots reveal that aligned embeddings not only increase the median scores but also reduce performance variability. this indicates that our approach consistently enhances translation quality and leads to more reliable code translations. the consistent improvements across different model sizes (8b and 70b parameters) and datasets demonstrate the robustness and scalability of our method. a key observation is that the aligned embedding figure 3 : box plots illustrating the distribution of code-bleu scores across various shot counts (1-shot, 2-shot, 3-shot) for both unaligned and aligned models. the results are presented for two models (llama3.1 70b and llama3.1 8b) across two datasets (numerical recipe and hpc fortran2c++ dataset) models consistently achieve higher codebleu scores compared to unaligned models when transitioning from zero-shot to few-shot settings. for instance, on the hpc fortran2c++ dataset with the llama3.1 70b model, the aligned model improves from 0.364 to 0.710 (+0.346) in the 1shot setting, surpassing the unaligned model's improvement from 0.364 to 0.626 (+0.262). similar trends are observed with the llama3.1 8b model, where the aligned model increases from 0.342 to 0.688 (+0.346), compared to the unaligned model's increase from 0.342 to 0.579 (+0.237). the mistral 13b and mixtral 8x22b models also exhibit greater improvements with aligned embeddings in few-shot settings, confirming the benefit of embedding alignment across different architectures. on the numerical recipes dataset, the aligned models again demonstrate superior improvements over unaligned models. for example, the llama3.1 70b aligned model improves from 0.280 to 0.593 (+0.313) in the 1-shot setting, exceeding the unaligned model's increase from 0.280 to 0.512 (+0.232). this consistent pattern across datasets reinforces the advantage of embedding alignment in enhancing code translation performance. we acknowledge that codebleu may not capture all functional nuances. therefore, we performed a small-scale manual check (appendix a) on a subset of translations. while we observed that a majority compiled and produced the expected outputs, further large-scale functional evaluation remains an important future direction. these results indicate that embedding align-ment significantly enhances the models' capacity to exploit few-shot prompts, leading to superior code translation performance as measured by code-bleu scores. alignment optimizes the embedding space to better capture the syntactic and semantic nuances of code translation tasks, thereby augmenting the models' few-shot learning capabilities. additionally, larger models tend to outperform smaller ones. the llama3.1 70b model consistently achieves higher codebleu scores than the llama3.1 8b model across both datasets and embedding types. the strong performance of the mixtral 8x22b model, which combines multiple experts, highlights the benefits of increased model capacity. furthermore, diminishing marginal gains are observed when increasing the number of shots beyond two, suggesting that the majority of performance improvements are realized with just one or two examples. this indicates that while fewshot examples are beneficial, adding more beyond a certain point yields limited additional gains. ## conclusion we introduced a novel method for enhancing crosslanguage code translation from fortran to c++ by aligning embeddings within a rag framework. by leveraging contrastive learning based on code-bleu similarity scores, we aligned the fortran embedding model so that code snippets yielding highquality translations are positioned closer in the embedding space. this alignment enables the rag system to retrieve semantically meaningful examples that effectively guide th llm during code generation. our experimental results demonstrate substantial improvements in translation quality without the need for fine-tuning the llm. specifically, using aligned embeddings increased the average codebleu score from 0.64 to 0.73 on the hpc fortran2c++ dataset and from 0.52 to 0.60 on the numerical recipes dataset, representing relative improvements of approximately 14% and 15%, respectively. the larger model (llama3.1 70b) consistently outperformed the smaller model (llama3.1 8b), indicating that increased model capacity enhances the effectiveness of our approach. additionally, we observed diminishing returns beyond two-shot prompting, suggesting that most performance gains are achieved with just one or two examples. thus, our approach significantly improves code translation performance by optimizing the retrieval mechanism through task-specific em-bedding alignment, rather than relying on computationally expensive fine-tuning of the llm. this method is computationally efficient, scalable, and adaptable to other code translation tasks, particularly when aligned datasets are scarce or evaluation metrics like codebleu are critical. future work could extend this alignment strategy to additional programming languages and explore integrating other evaluation metrics to further enhance translation quality.