reviews: 
Comments: This paper explores whether separating highly conflicting objectives into different optimization levels or keeping them unified is more effective for multilingual, multi-task learning in automatic speech recognition (ASR) and speech-to-text translation (S2TT). It introduces three Multi-Objective Optimization (MOO) methods—VS-ASR, VC-ASR, and VM-ASR—designed to resolve conflicts between objectives at various optimization stages. These approaches are compared against baseline methods like pre-training and fine-tuning (PT+FT) without MOO. Through experiments on LibriSpeech, AISHELL v1, and CoVoST v2 datasets, the study demonstrates that applying MOO across multiple optimization levels yields consistent improvements in both self-supervised and supervised tasks. A key insight is that segregating highly conflicting objectives enhances performance for ASR and S2TT, and the order of optimization significantly affects ASR accuracy. The study also emphasizes the importance of penalty parameters in Multi-Level Optimization (MLO) training, showing robustness across different model sizes. These findings offer valuable insights for optimizing conflicting objectives in multilingual and multi-task speech systems, with broader implications for the development of more efficient ASR and S2TT models.
Strengths: 1. Effective Handling of Conflicting Objectives: This paper directly addresses the issue of conflicting objectives by utilizing conflict-avoidant update directions, overcoming the limitations of traditional methods like static weighting and constrained optimization.
2. Comprehensive Exploration of MOO Techniques: Unlike related work (e.g., "reward soups" by Rame et al., 2024), which focuses on linear scalarization for Pareto-optimality, this paper explores a broader range of Multi-Objective Optimization (MOO) techniques to effectively manage conflicts in multilingual, multi-task learning.
3. Transparency and Reproducibility: The authors provide accessible code, ensuring reproducibility, and clearly outline their experimental setup, facilitating verification.
4. Thorough Evaluation: Experiments are conducted in both task-based and language-based MLO settings using consistent hyperparameters, and considerations such as memory consumption and training time are discussed, adding depth to the evaluation.
Weaknesses: 1. Code Quality: The provided code requires cleaning and refinement for better readability and maintainability.
2. Organization and Flow:
a. Important conclusions and interesting findings are relegated to the appendix, disrupting the reading flow.
b. Key sections (e.g., Section 5.2) should be integrated into the main text for coherence.
3. Abbreviation Clarity:
a. Certain abbreviations (e.g., FT, VC-ASR in the introduction) are used without proper introduction or explanation.
b. A clear glossary or introductory explanations would enhance readability.
4. Limited Evaluation:
a. Only two models (Conformer 58M and 100M) are evaluated.
b. Broader evaluation across diverse architectures and sizes is necessary to convincingly demonstrate the method's effectiveness.
5. Theoretical Analysis: Further theoretical analysis is needed (Notes as Future Work in Conclusion)  to comprehensively understand the capabilities and limitations of the proposed method.
Recommendation: 6
Reviewer confidence: 4
Soundness/correctness: 3
Clarity: 2