Title: Imbalances in Neurosymbolic Learning: Characterization and Mitigating Strategies
Venue: NeurIPS 2025
Accepted: yes
Keywords: neurosymbolic learning

learning theory

learning imbalances
Abstract: We study one of the most popular problems in **neurosymbolic learning** (NSL), that of learning neural classifiers given only the result of applying a symbolic component $\sigma$ to the gold labels of the elements of a vector $\mathbf x$. The gold labels of the elements in $\mathbf x$ are unknown to the learner. We make multiple contributions, theoretical and practical, to address a problem that has not been studied so far in this context, that of characterizing and mitigating *learning imbalances*, i.e., major differences in the errors that occur when classifying instances of different classes (aka **class-specific risks**). Our theoretical reveals a unique phenomenon: that $\sigma$ can greatly impact learning imbalances. This result sharply contrasts with previous research on supervised and weakly supervised learning, which only studies learning imbalances under data imbalances. On the practical side, we introduce a technique for estimating the marginal of the hidden gold labels using weakly supervised data. Then, we introduce algorithms that mitigate imbalances at training and testing time by treating the marginal of the hidden labels as a constraint. We demonstrate the effectiveness of our techniques using strong baselines from NSL and long-tailed learning, suggesting performance improvements of up to 14\%.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the problem of mitigating class-specific error disparities in NeSy classification tasks. The analysis is sound and highlights how the concept-extractor function significantly influences these imbalances and propose mitigation strategies applicable at both training and test time. This paper support the approaches with experiments demonstrating improvements in classification accuracy.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**:\n\nThe paper presents a solid theoretical analysis that extends prior work by providing a bound on the risk associated with class-specific learning imbalances. This result is well-founded and offers a promising direction for improving the trustworthiness of NeSy models. Additionally, the authors propose a mitigation technique applicable during both training and test time.\n\n------\n\n**Weaknesses**:\n\nThe proposed mitigation strategies rely on linear programming and optimization procedures, which I believe may be challenging to scale in practice. Moreover, it is unclear how addressing class-specific imbalances interacts with existing methods aimed at solving reasoning shortcuts. Finally, while the theoretical contributions are valuable, the empirical evaluation is limited to relatively simple datasets such as MNIST and CIFAR.\n\nQUESTIONS:\nThe topic of learning imbalances in NeSy is novel and relevant. However, I find it difficult to grasp the practical implications of mitigating these imbalances, especially in comparison to existing strategies for addressing Reasoning Shortcuts [1]. Could you provide concrete and realistic examples where your proposed strategy offers clear advantages? I believe that including a compelling example would significantly strengthen the paper.\n\nFurthermore, how computationally demanding is the proposed linear programming approach during training? While the method appears to improve model accuracy, does it also lead to improvements in concept accuracy? The same question applies to the proposed test-time mitigation strategy.\n\nLastly, do you think the mitigation strategies you are proposing can be paired with those based on annotation acquisition like [2]?\n\n------\n\n**References**:\n\n[1] Emanuele Marconato, Stefano Teso, Antonio Vergari, and Andrea Passerini. Not all neuro-symbolic concepts\nare created equal: Analysis and mitigation of reasoning shortcuts. In NeurIPS, 2023\n[2] Emanuele Marconato, Samuele Bortolotti, Emile van Krieken, Antonio Vergari, Andrea\nPasserini, and Stefano Teso. BEARS Make Neuro-Symbolic Models Aware of their Reasoning\nShortcuts. In UAI, 2024\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-20"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper focuses on a neglected core issue in neurosymbolic learning (NSL)—learning imbalance, which refers to systematic differences in classification errors across different classes. Through theoretical analysis and algorithmic design, it reveals the independent influence of symbolic components on class imbalance in NSL. The authors propose two algorithms (LP and CAROT) to mitigate imbalance during training and testing phases, and validate their effectiveness through experiments. These algorithms achieve significant performance improvements over existing baseline methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\nThe paper provide a class-specific error bound in the context of NESY and show the effect of $\\sigma$ on learning imbalances.\n\nThe authors propose two algorithms to mitigate imbalances during training and testing time.\n\nWeaknesses\n\n1.The dependence of CAROT on the accuracy of $r$ and the dependence of the LP method on batch size and $\\sigma$ should be supplemented in the Discussion or Conclusion section with an analysis of these limitations.\n\n2. The LP-based method requires solving a linear program for each training batch, which may incur substantial computational overhead, especially for large batch sizes. It is recommended to report the runtime and resource consumption in the main text and to provide a discussion on the scalability of the approach.\n\n3. The main text (e.g., Section 2) references \"Tables 6 and 7,\" but these tables are absent in the main manuscript. While they may be included in the appendix, this is not explicitly stated. Section 4 introduces numerous mathematical symbols, yet their explanations are deferred to Table 7, creating a disconnect that hampers readability.\n\n4. The structure is densely packed, and transitions between concepts are abrupt. In particular, the shift from Section 3 to Section 4 is rather sudden. It is recommended to include a brief transitional paragraph to improve the flow.\n\nQUESTIONS:\nHow is the NP-hardness of LP mitigated on large-scale datasets? Are approximation algorithms or parallelization considered?\n\nLIMITATIONS:\nThe computational complexity of nonlinear programming can be further discussed, particularly regarding its scalability in high-dimensional scenarios.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates learning imbalances in neurosymbolic learning (NSL), specifically focusing on class-specific risks that emerge when neural classifiers are trained with hidden gold labels revealed only through a symbolic component. Unlike previous research, this work highlights how the symbolic component itself can cause imbalances, not just data imbalances. The authors propose a method to estimate hidden gold label marginals and introduce algorithms to mitigate these imbalances during both training and testing, achieving up to 14% performance improvement over existing baselines.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n1. The authors continuously use simplified and intuitive examples to illustrate the important comcepts and strategies, making the paper easy to read.\n2. The paper introduces and addresses the unexplored problem of learning imbalances in Neurosymbolic Learning (NESY), which is novel and applicable.\n3. This paper has strong theoretical foundations, offering stricter bounds and makes fewer assumptions compared to existing work.\n4. Empirical experiments validates the effectiveness of the proposed CAROT algorithm.\n\n## Weaknesses\n\n1. Consider Case 1 in Example 3.2. The solution of (2), though tighter than exsiting bounds, are still worst-case upper bounds. It might not be appropriate to claim that class 0 is the hardest to learn by comparing the worst-case solutions of classification risks.\n2. A minor issue: missing noun in line 8: \"Our theoretical (?) reveals a unique phenomenon\".\n\nQUESTIONS:\nI notice that in the experiment part, you use only MAX-M to validate the effectiveness of your algorithm. Are there any other choices of $\\sigma$ in practical scenarios? Have you considered the approximation of $\\hat{\\boldsymbol{r}}$ and effectiveness of CAROT in different choices of $\\sigma$?\n\nLIMITATIONS:\nYes, the authors have adequately addressed the limitations and potential negative societal impact of their work.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the challenge of mitigating class imbalance in neurosymbolic learning (NESY) systems by proposing CAROT, a distributionally robust optimization framework that adjusts for class-specific risk at both training and test time. The authors formalize class imbalance as a risk reweighting problem using a linear program and propose a batch-level approximation algorithm (Algorithm 1) to compute instance weights dynamically. The framework is evaluated across synthetic and real-world benchmarks (MNIST, CIFAR-10), and claims to improve balanced accuracy in low-data and long-tail settings. The theoretical contributions include bounding excess risk in class-imbalanced settings and analyzing generalization under distribution shift.\n\nSTRENGTHS AND WEAKNESSES:\nLet's begin with Strengths:\nThe underlying issue of how NESY systems overfit to dominant symbolic patterns due to class imbalance in supervisory signals is an inadequately explored issue and I am glad the authors have done so in this paper. CAROT appears to be a thoughtful adaptation of DRO principles to symbolic learning. Of note, CAROT consistently improves balanced accuracy across synthetic and real-world datasets. In particular, it appears more robust than prior approaches like RECORDS and LA in long-tail settings. From a theoretical perspective,the generalization bounds in Section 3 provide formal support for the class-level risk minimization approach and the decomposability of the loss into class partitions seems well-leveraged.\n\nNow, let's get into the Weaknesses:\n1. Methodological Limitations\n- The LP in Equation (5) can have exponential constraint growth (via σ⁻¹ enumeration). No concrete approximation strategies are proposed, which limits scalability. Possible alternatives (e.g., sampling-based surrogates, hierarchical grouping of symbolic types, streaming approximations) are not discussed.\n- The softmax reparameterization in Algorithm 1 may cause optimization instability, particularly in skewed or underdetermined settings. No ablation is provided for this choice.\n- CAROT assumes known symbolic partitions, which is restrictive. Real-world NESY systems often operate with evolving or noisy taxonomies. This limits generalizability.\n\n2. Theoretical Gaps\n- The assumption that misclassification depends only on class is strong. Symbolic tasks typically involve multi-factor dependencies (e.g., depth of composition, operator type). This abstraction may misrepresent error dynamics.\n- The paper assumes i.i.d. symbolic structures, yet most NESY settings include structured dependencies (e.g., scene graphs, logic programs). It would strengthen the paper to either relax this assumption or justify it empirically.\n- The tightness and operational utility of the theoretical bounds (Propositions 3.1–3.4) is unclear. Do they guide parameter selection or batch size decisions? No empirical validation is provided.\n\n3. Experimental Limitations\n- Evaluation is done on overly synthetic symbolic tasks (MAX, SUM, etc.) that may not reflect real-world NESY generalization challenges. The authors may benefit from considering evaluation on more complex symbolic reasoning datasets like CLEVRER, ProofWriter, or GQA.\n- Only 3 seeds are used, which is insufficient given the observed variance across some datasets (e.g., CIFAR-10). For high-variance metrics like balanced accuracy, ≥10 seeds would improve statistical reliability.\n- Baseline selection could be expanded. While RECORDS and LA are relevant, stronger fairness-focused baselines like IRM (“Invariant Risk Minimization”, Arjovsky et al., 2019) or Group DRO (“An Investigation of Why Overparameterization Exacerbates Spurious Correlations”, Sagawa et al., 2020) are notably absent. Even basic class-reweighting schemes are omitted.\n- No computational benchmarks are provided. Training time overhead, LP solver configuration, and memory usage are critical in assessing CAROT’s deployability.\n\nQUESTIONS:\n1. Can the authors relax the i.i.d. assumption to allow for structural dependencies among instances (e.g., compositional overlap in symbolic trees)? How might this affect the guarantees in Propositions 3.1–3.4?\n2. What is the sensitivity of Algorithm 1 to initializations, learning rate choices, or softmax reparameterization temperature? Were convergence pathologies (e.g., oscillations, mode collapse) observed?\n3. Given that CIFAR-10 and MNIST are not symbolic reasoning datasets, what modifications would be needed to deploy CAROT on real NESY benchmarks (e.g., CLEVRER, NLVR, or hybrid QA tasks)? Have the authors considered adapting the framework to hierarchical or graph-structured label spaces?\n\nLIMITATIONS:\nThe authors acknowledge several limitations (computational overhead, symbolic domain coverage). However, the assumptions regarding instance independence, class-only misclassification risk, and the representativeness of synthetic benchmarks warrant deeper discussion. In particular, the societal risks of deploying fairness-motivated reweighting without proper calibration or robustness checks in high-stakes symbolic systems such as legal or medical rule-based AI deserve consideration.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}