Title: QiMeng-MuPa: Mutual-Supervised Learning for Sequential-to-Parallel Code Translation
Venue: NeurIPS 2025
Accepted: yes
Keywords: mutual-supervised learning

code generation

HPC code translation
Abstract: The rise of GPU-based high-performance computing (HPC) has driven the widespread adoption of parallel programming models such as CUDA. Yet, the inherent complexity of parallel programming creates a demand for the automated sequential-to-parallel approaches.
However, data scarcity poses a significant challenge for machine learning-based sequential-to-parallel code translation. Although recent back-translation methods show promise, they still fail to ensure functional equivalence in the translated code. In this paper, we propose \textbf{QiMeng-MuPa}, a novel \textbf{Mu}tual-Supervised Learning framework for Sequential-to-\textbf{Pa}rallel code translation, to address the functional equivalence issue. QiMeng-MuPa consists of two models, a Translator and a Tester. Through an iterative loop consisting of Co-verify and Co-evolve steps, the Translator and the Tester mutually generate data for each other and improve collectively. The Tester generates unit tests to verify and filter functionally equivalent translated code, thereby evolving the Translator, while the Translator generates translated code as augmented input to evolve the Tester. Experimental results demonstrate that QiMeng-MuPa significantly enhances the performance of the base models: when applied to Qwen2.5-Coder, it not only improves Pass@1 by up to 28.91\% and boosts Tester performance by 68.90\%, but also outperforms the previous state-of-the-art method CodeRosetta by 1.56 and 6.92 in BLEU and CodeBLEU scores, while achieving performance comparable to DeepSeek-R1 and GPT-4.1. Our code is available at \url{https://github.com/kcxain/mupa}.


Peer reviews:
comments: SUMMARY:
This paper addresses the challenge of ensuring functional equivalence in sequential-to-parallel code translation, where the scarcity of parallelization data makes equivalence guarantees difficult. It proposes a Mutual-Supervised Learning (MSL) framework that enhances equivalence verification and model refinement by establishing an iterative closed loop between a Translator and a Tester.

STRENGTHS AND WEAKNESSES:
Automatically translating existing sequential (CPU) code into efficient parallel (GPU/CUDA) code—while still guaranteeing functional equivalence in the face of scarce data—is indeed a problem worth solving. The authors’ development of a data engine combined with joint supervised fine-tuning to tackle this practical challenge represents a meaningful contribution.

The main concern is that the loop of using filtered data to train a model—and then using the improved model to further filter data—has already been explored in many previous works, so it isn’t particularly novel.

QUESTIONS:
Overall, I don’t have any particular suggestions. I find the paper solid if somewhat conventional—its innovation is moderate, but it addresses a very practical problem. I believe it’s necessary for someone to apply a general method to a real-world domain. Therefore, I give it a weak accept.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 2
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-07-01

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Mutual-Supervised Learning (MuSL), a framework for sequential to parallel code translation, specifically targeting C-to-CUDA generation. The core contribution is a self-supervised, iterative training loop designed to overcome the scarcity of parallel data while enforcing functional correctness. The framework employs two models: a Translator that proposes a parallel code version from a sequential source and a Tester that generates unit tests to verify the functional equivalence of the Translator's output. The process alternates between a \"Co-verify\" step, where the Tester validates the translation, and a \"Co-evolve\" step, where code pairs that pass verification are used as high-quality training data to fine-tune both the Translator and the Tester. The authors report that this mutual supervision loop significantly improves the pass rate of translations compared to strong existing baselines, addressing  the need for semantic correctness in automated code generation.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- The work addresses a problem of high significance for the code translation community.\n\n- The core concept of an iterative, self-supervised loop for data generation and verification is theoretically elegant. \n\n- The work demonstrates good originality in its specific formulation and application.\n\n- The paper is generally well-written and organized, with the core MuSL framework being presented in an intuitive manner. The diagrams and high-level descriptions effectively convey the workflow.\n\n- The reported empirical results, showing a substantial jump in pass rates, are promising and demonstrate the potential of this direction.\n\n-  While using LLMs for code translation and test generation are individually established ideas, the proposed mutual-supervision framework, where the two models iteratively improve each other, is a novel and clever combination of existing techniques. The self-supervised approach to generating a verified, parallel dataset is a creative solution to the data scarcity problem.\n\n\nWeaknesses:\n\n\n- In my opinion, the paper's central claim of ensuring \"functional equivalence\" is not technically sound. It rests on the premise that passing a finite set of generated unit tests is a sufficient proxy for true semantic equivalence, which is undecidable by Rice's Theorem. This is the paper's most significant conceptual weakness. The framework's success is entirely contingent on the Tester's ability to generate comprehensive tests, a capability that is assumed rather than proven.\n\n- Key analyses are missing, rendering the evaluation incomplete. First, there is no systematic evaluation of the performance of the generated parallel code, a critical success metric for any parallelization task. Second, the computational cost and scalability of this iterative compile-and-run loop are not analyzed, which is a major practical concern.\n\n- The paper states the Tester is \"enhanced\" by fine-tuning on verified code pairs, but it fails to explain how observing a correct (source, parallel_code) pair provides a direct learning signal to generate better, more challenging tests for a new, unseen source program. This feedback loop is crucial to the \"co-evolution\" claim, and its underspecified nature weakens the paper's argument.\n\n- The originality is slightly diminished by the omission of a direct comparison to highly relevant prior art like TransCoder-ST, which also used a test-based verification step (albeit with a non-learned test generator). Such a comparison would have better isolated the novel contribution of the learned Tester.\n\nQUESTIONS:\n- Can you provide a quantitative analysis of the quality and coverage (e.g., line, branch, mutation score) of the tests generated by the Tester? A positive response here, demonstrating that the Tester generates non-trivial and comprehensive tests, would significantly increase my confidence in the framework's core verification step. It might be worth stating it in the main text.\n\n- Could you please provide a more precise explanation for the Tester's improvement signal? How does fine-tuning on correct (source, translation) pairs improve its ability to generate more discerning tests for new functions, rather than simply learning to generate tests for a wider variety of functions? A convincing answer is critical for the \"co-evolution\" claim. In short, why this helps generate better tests?\n\n- Could you provide a systematic analysis of the execution-time speedup of the generated CUDA code versus the sequential C originals across your entire benchmark? A positive result showing that correctness does not come at the cost of performance would substantially strengthen the paper's practical significance.\n\n- What is the wall-clock time and total computational resource requirement for a full MuSL run on your benchmark, and how does this scale with the number of programs and iterations? This is essential for understanding the method's practicality.\n\n- Could you justify the omission of a direct comparison to TransCoder-ST? My evaluation would improve if you could demonstrate the superiority of your learned Tester against a strong, off-the-shelf test generation engine like EvoSuite used in that work.\n\nLIMITATIONS:\nWhile they briefly mention expanding capabilities in the appendix, they fail to discuss the most critical limitations of their work in the main paper. Constructive suggestions for improvement include:\n\n- A dedicated discussion on the fundamental gap between passing unit tests and guaranteeing true functional equivalence. The authors should be upfront that their method is a powerful heuristic for improving confidence in correctness, not a formal guarantee.\n\n- An analysis of the potential failure modes, such as the Tester generating weak or trivial tests, which could lead to a polluted training set of \"false positives.\"\n\n- A transparent discussion of the significant computational cost and scalability challenges, which are a major barrier to the practical application of this method.\n\n- The authors are not fully transparent about the limitations of their work. While they acknowledge some future directions, they do not grapple with the fundamental gap between test-passing and correctness, nor the potential for the Tester to generate trivial or non-discriminating tests.\n\n- Without a clear path to ensuring more robust correctness checks and managing the prohibitive computational cost, it is unlikely that practitioners or researchers will be able to build directly upon this method in its current form.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

comments: SUMMARY:
This paper introduces a “Mutual-Supervised Learning” (MSL) framework for translating sequential code (C) to parallel code (CUDA) by pairing a Translator model and a Tester model. The Translator generates candidate code while the Tester produces unit tests, and they iteratively filter and improve each other via a closed-loop process.

STRENGTHS AND WEAKNESSES:
Strengths
- Focus on missing functional correctness considerations in prior work on C - CUDA translation
- Adding test cases to BabelTower would be a valuable contribution 
- Leveraging back-translation with test-driven 
- Extensive experiments across model backbones, metrics, and thorough ablations 

Weaknesses
- Vacuous tests. The `Tester` model can cheat and only generate trivial inputs (like all zeros), which might fail to identify bugs during equivalence testing. It is unclear how the approach avoids this failure more.
- Contamination analysis. It is unclear if the monolingual training sets are decontaminated. 
- Evaluation suite tests. The test suites only contain 5 test cases which might be less for comlpex C/Cuda functions.

QUESTIONS:
1. Performing equivalence testing. 

Can the authors explain how the equivalence testing is performed from the raw inputs? Are the function signatures aligned for the inputs? Are the outputs from program execution serialized before comparison? 

2. Vacuous tests discussion.

Can the authors explain how the vacuous tests issue mentioned above is prevented?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 2
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a Mutual-Supervised Learning (MuSL) framework for translating sequential code into parallel code. The framework consists of two components, a Translator and a Tester, that iteratively generate data for each other and improve jointly through co-evolution. Experimental results demonstrate the effectiveness of the MuSL framework in enhancing both translation accuracy and functional correctness.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n+ The topic of code translation, particularly from sequential to parallel code, is important to high-performance computing and software engineering.\n+ The proposed Mutual-Supervised Learning framework is language-agnostic, suggesting potential applicability beyond C and CUDA to other programming languages.\n+ The experimental evaluation demonstrates the effectiveness of the proposed approach across multiple metrics.\n\n**Weaknesses**\n- The framework is claimed to be language-agnostic, but experiments only involve C and CUDA. \n- The effectiveness of the proposed method requires further evaluation. It is unclear whether the observed performance improvements are primarily due to the proposed Mutual-Supervised Learning framework, or simply the result of training/fine-tuning the models on the dataset. As shown in Table 1, models with similar parameter sizes such as TransCoder and CodeRosetta achieve comparable results to the proposed approach, raising questions about the contribution of the framework itself versus the impact of supervised training data.\n- The evaluation settings should be described in more detail, including specifics such as the hardware configurations. Additionally, the training overhead introduced by the proposed method, especially due to the iterative co-verification and co-evolution steps, should be reported and discussed to assess its practicality and scalability.\n\nQUESTIONS:\n1. The paper states that \"while it is possible to train separate Translators and Testers for C and CUDA, we find that language-agnostic models perform better\", and attributes this to the larger amount of training data available when combining multiple languages. Could the authors provide empirical evidence or quantitative results to support this claim? Are there quantitative results comparing language-specific and language-agnostic models? Additionally, if more training data from other languages were included, would the performance improve further?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}