Title: Contrastive Representations for Temporal Reasoning
Venue: NeurIPS 2025
Accepted: yes
Keywords: Contrastive Learning

Representation Learning

Combinatorial Reasoning
Abstract: In classical AI, perception relies on learning state-based representations, while planning --- temporal reasoning over action sequences --- 
is typically achieved through search. We study whether such reasoning can instead emerge from representations that capture both perceptual and temporal structure.  We show that standard temporal contrastive learning, despite its popularity, often fails to capture temporal structure due to its reliance on spurious features. To address this, we introduce Contrastive Representations for Temporal Reasoning (CRTR), a method that uses a negative sampling scheme to provably remove these spurious features and facilitate temporal reasoning. CRTR achieves strong results on domains with complex temporal structure, such as Sokoban and Rubik’s Cube. In particular, for the Rubik’s Cube, CRTR learns representations that generalize across all initial states and allow it to solve the puzzle using fewer search steps than BestFS — though with longer solutions. To our knowledge, this is the first method that efficiently solves arbitrary Cube states using only learned representations, without relying on an external search algorithm.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces CR² (Contrastive Representations for Combinatorial Reasoning) to address limitations of standard contrastive learning (CRL) when applied to combinatorial problems. The authors identify that CRL fails in combinatorial domains because it learns to exploit superficial context features (like wall layouts in Sokoban) rather than temporal structure. CR² addresses this by incorporating \"in-trajectory negatives\" - negative samples from the same trajectory rather than just across different trajectories. By including negatives that have similar context (similar wall layouts), the model is forced to learn temporal structure instead of just memorizing context. This is done by repeating trajectories in the batch, so there is a higher chance that negatives come from trajectories with similar context.\n\n\nMain Claims of Contributions:\n\n- Failure Mode Identification: Standard CRL fails on combinatorial problems by learning context-dependent representations that ignore temporal structure\n- Novel Algorithm: CR² optimizes I(X; X⁺) - I(X⁺; C), encouraging representations that maximize temporal mutual information while minimizing context dependence\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- The paper addresses an important problem in representation learning: capturing temporal dependencies to enable better planning and improve sample efficiency. The combination of contrastive learning with novel approaches shows promise for learning effective representations.\n\nWeaknesses:\n\nThis work requires substantial development (sufficient novelty and additional contributions) before publication. The main issues are:\n\n \n- The core insight about negative sample selection is reasonable but offers minimal novelty. The proposed solution appears to be an ad-hoc fix rather than a principled algorithm, with weak connections between the theoretical objective and actual implementation.\n\n- The method makes strong, unrealistic assumptions. It only works if we can cleanly separate \"context\" from \"temporal dynamics\" - a strong assumption that breaks down in many domains. The paper provides no systematic approach for identifying context in practice.\n\n- Inadequate Experimental Validation: The evaluation doesn't support the broad claims made. Despite the title suggesting general \"combinatorial reasoning,\" testing is limited to puzzle games with no evidence of effectiveness on real combinatorial problems (traveling salesman, vehicle routing, etc.). Additional issues include: No results across multiple random seeds, insufficient analysis of learned representations beyond basic correlation, and missing comparisons to state-of-the-art methods (Graph Neural Networks, neural combinatorial optimization, Monte Carlo Tree Search variants).\n\n\n- Fig. 3 highlights the difference in the code for CRL and CR2 (which is unusual to add in the main paper). This again suggests an engineering modification rather than a research contribution. \n\n- The related work section is superficial and fails to clearly position the work within existing literature.\n\nOverall Assessment:\n\nThe work appears more like an engineering hack than a novel research contribution and needs significant theoretical development and experimental validation before being ready for publication.\n\nQUESTIONS:\nQuestions:\n\n- The \"repetition factor\" hack seems unprincipled - why this specific approach?\n- All experiments use BestFS. How do the learned representations perform with other search algorithms (A*, MCTS, etc.)? Is the improvement specific to greedy search strategies?\n- The definition of context (Definition 4.1) works well for Sokoban where static elements are clearly separable, but how would you formally define context in domains like Rubik's Cube where you mention \"pseudo-context\"? Could you provide a more general mathematical framework for identifying context across different combinatorial domains?\n- The conditional independence assumption is crucial to your theoretical analysis. Have you empirically validated this assumption across your test domains? What happens to your theoretical guarantees when this assumption is violated?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nContrastive learning fails on combinatorial reasoning tasks because it learns to exploit superficial contextual features that disregard relevant temporal information (Fig. 1). The failure occurs when there's a constant \"context\" (like wall patterns in Sokoban) that makes it trivial to distinguish between trajectories. Standard contrastive learning samples negatives from different trajectories, so models can achieve perfect accuracy by identifying the context rather than learning meaningful temporal relationships (Fig. 5-6). To resolve this, CR^2 samples negative examples (temporally distant states) from within the same trajectory (in-trajectory negatives) rather than only from different trajectories. \n\nCR^2 optimises $I(X; X_+) - I(X_+; C)$, encouraging representations that capture temporal structure while being invariant to context. Practically, this is done via a simple modification (Fig. 3) to how the data is loads; repeat trajectory indices in batches with a \"repetition factor\" (typically 2). Evaluation across 5 different settings (Sokoban, Rubik's Cube, N-Puzzle, Lights Out, Digit Jumper) shows that CR² consistently outperforms standard contrastive learning (Fig. 4) and matches supervised baselines using Best-First Search planning.\n\nSTRENGTHS AND WEAKNESSES:\nStrength: Overall, I like this paper – it provides a nice example of why standard contrastive learning fail on combinatorial problems, proposes a simple fix using in-trajectory negatives and shows consistent improvements across multiple domains. Improves search efficiency when combined with Best-First Search planning. The code is made public for future extensions. \n\nLimitation:  Despite the well-structured evaluations, there are some limitations. The problems considered are quite small-scale and evaluation on more realistic problem settings might be useful to evaluate the significance of the contribution. Some of the results presented seem to be for limited (i.e., 3) or single seeds, making it difficult to evaluate their robustness. The restrictive Assumption 4.2 (X_i ⊥ C | X_j) may not hold in many real-world domains with partial observability or stochastic dynamics, limiting applicability beyond clean combinatorial puzzles. Furthermore, the paper shows that despite the improvements in representation, CR^2 still cannot replace search entirely – particularly in limited budget settings. Table 1 shows solve rates near zero for complex problems without search (15-puzzle). This suggests we may be hitting fundamental limits of what these types of representations can achieve for combinatorial reasoning.\n\nQUESTIONS:\n1.\tIn the setting with partial observability, stochastic dynamics – Assumption 4.2 would not hold. How sensitive is the method to these violations?  Can we characterise exactly which MDP structures satisfy this assumption?\n2.\tWere other search algorithms evaluated apart from best-first search – that weren’t included in the manuscript? Example, A* ?\n3.\tWe can see that the training objective minimises I(X⁺; C) i.e., mutual information between representations and context, should lead to context invariance. However, this is not shown empirically. Therefore, could we see how much the mutual information decreases during training for the different settings.\n\nWe can see that search is necessary to get appropriate solutions -- with certain environments requiring more of the search budget (6000 nodes). \n\n4.\tCan we quantify the relationship between representation quality (measured by correlation with true distances) and required search budget? \n5.\tDoes a 10% improvement in correlation translate to a predictable reduction in nodes explored? \n6.\tHow do these trade-offs change with problem size (15-puzzle vs. 24-puzzle)?\n7.\tIs this relationship linear, logarithmic, or domain-dependent?\n8.\tIs there a threshold correlation value below which search becomes ineffective?\n\n\n9.\tFor Fig.8 would it be possible to include confidence intervals? this would allow a concrete evaluation of the CRL and CR^2 and importantly the robustness of the evaluation. \n10.\tThe results show that having higher repetition factor degrade performance. Is this because the higher R is increasing too much noise in the training signal?\n11.\tMinor point: Last equality of equation 4 is repeated. It would be good to remove it?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces CR2, a contrastive representation learning method for solving multi-step combinatorial problems. The paper demonstrates the failure of standard contrastive learning approaches in solving combinatorial problems due to the presence of excessive non-essential information, commonly referred to as \"common context,\" in all instances of the same trajectory. Based on this observation, the paper goes on to propose a method to introduce in-trajectory negatives. This is achieved simply by replicating the trajectories a fixed number of times in the same batch and leaving the rest of the CL pipeline as is, which has the effect of randomly introducing negatives from the same trajectory. The experiments demonstrate that CR2 improves significantly over the naive CRL objective. It is also competitive with supervised value-based methods.\n\nSTRENGTHS AND WEAKNESSES:\n**Quality**: All claims made in the paper are well-supported by targeted experiments. The limitations of the current work, i.e., the use of deterministic known dynamics and noiseless states, are clearly stated. \n\n**Clarity**: The paper is well written and easy to follow. The intuition provided using probabilistic arguments is helpful in understanding the general principle behind the method in a task-agnostic manner.\n\n**Significance**: Using deterministic dynamics and noiseless states makes the experimental setup seem somewhat toy-like. The paper's impact could be greatly enhanced if experiments on more realistic tasks were also included.\n\n**Originality**: Although the idea in the paper closely relates to the concept of \"hard negatives\" in general contrastive representation learning literature, I believe the way the idea is presented and instantiated is original.\n\nQUESTIONS:\n1. Isn't repeating trajectories (as described in Figure 3) in a batch equivalent to randomly (perhaps not uniformly randomly) sampling in-trajectory \"hard negatives\"? \n\n2. Why does the training objective saturate for CR2 in Figure 5? Is it to be expected due to in-trajectory negatives?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}

comments: SUMMARY:
The paper presents CR$^2$ (Contrastive Representations for Combinatorial Reasoning), which extends temporal contrastive learning with in-trajectory negatives to learn embeddings that double as effective search heuristics. 
By revealing how standard contrastive RL collapses onto static context and then maximizing conditional mutual information via a simple “repetition factor,” the method preserves the temporal signal crucial for planning. Evaluated on several tasks like complex puzzles and control problems, CR2’s embeddings align closely with true state-space distances, lifting correlation metrics and solve rates and, in many settings, matching the performance of heavyweight search-centric systems. However, further analysis confirms that explicit search is still required on the harder instances.

STRENGTHS AND WEAKNESSES:
**Strenghts**

- The paper is well written and easy to follow. 
- The paper identifies an important flaw with standard contrastive RL approaches, especially for combinatorial reasoning tasks, and provides a solution -- using CR$^2$ that learns to ignore the context and focus on the temporal and causal structure. 
- The paper provides further analysis and experiments where they investigate CR$^2$ such as testing if search is still needed, experimenting in cases where the context is dynamic. 

**Weaknesses**

- The conclusions from the t-SNE plots are not very clear, and the claim that the plots for CRL form clusters, indicating they learn the context (walls), while CR$^2$ seems to be a bit hand-wavy. Perhaps this argument could be strengthened.


Overall, I think the paper makes a good contribution, and I recommend acceptance

QUESTIONS:
None

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 4
IMPACT: 4
ORIGINALITY: 3
DATE: 2025-07-06