From: ICLR.cc/2023/Conference/Program_Chairs
title: Paper Decision
decision: Accept: poster
metareview:_summary,_strengths_and_weaknesses: The paper studies the out-of-support (OOS) extrapolation problem, whose goal is to make prediction on a sample that is outside the support of the training distribution. The authors tackle the problem by reducing the OOS problem into an out-of-combination problem, which is a special case of the OOS problem, using a bilinear transduction scheme. The authors provides generalization bound on the proposed bilinear transduction scheme, as well as experimental results on multiple synthetic problems which proves the efficacy of the proposed method. Further experimental validation on robot control problems shows that the proposed method can be effective in certain real-world settings.

The paper initially received split reviews. All reviewers agreed that the studied OOS extrapolation problem is interesting and challenging, and the proposed solution to the problem is novel. The reviewers also agreed that the method could be effective when certain assumptions are satisfied. 

However, there was a large disagreement among the reviewers on whether the assumptions the paper is based on is reasonable or not (bilinear transduction, stationary property) for real-world applications. Some of the reviewers argue that the proposed assumption may be too restrictive and does not apply to all OOS problems, as we can always find counterexamples where the proposed assumption does not hold (e.g. test images with unseen camera positions and lighting conditions for image classification tasks). On the other hand, other reviewers did not consider it as a critical weakness as long as the authors clearly describe the types of problems for which the proposed extrapolation scheme works. After an in-depth internal discussion on the topic, there was consensus among the reviewers to accept the paper, as the merits outweigh the limitations.

However, since there exist a clear limitation, I suggest the authors clearly discuss the limitation of the proposed method, and describe scenarios that can benefit from it, reflecting the suggestions from the reviewers.
justification_for_why_not_higher_score: There exists a clear limitation of the work, in that it is based on a very strong assumption (regularity in the task space). Also, the paper did not fully explore what types of out-of-support extrapolation problems can benefit from the proposed method. 
justification_for_why_not_lower_score: The paper proposes an interesting idea that could lead to further research on out-of-support extrapolation problems.
summary_of_AC-reviewer_meeting: This was a borderline paper at first, but the reviewers reached a consensus before having a scheduled AC meeting, and thus it was unnecessary to have the zoom meeting.
note_from_PC: if the above contains the word "oral" or "spotlight" please see: "oral" presentation means -> notable-top-5% and "spotlight" means -> notable-top-25%. As stated in our emails, we are disassociating presentation type from AC recommendations

From: ICLR.cc/2023/Conference/Paper3647/Reviewer_dNKc
title: Response to rebuttal
comment: Thanks for updating the introduction to emphasize the assumptions of the current work. 

1. Comparison with SE(3)-equivariant model clearly showcases the strength and generality of the proposed method in that it performs well OOS for different transformations without including more specific inductive biases like translation and scale equivariance. 

2. Thank you for clarifying that the Meta-world (and Adroit) tasks are not translation invariant/equivariant to the target location. Nevertheless, a comparison with a translation equivariant model could have emphasized this (similar to the SE(3) model above), but is not mandatory. I am not too familiar with these tasks to come up with an appropriate equivariant architecture, but I believe it should be possible. 

_"They are not equivariant to the target location as the trajectory is composed of two parts: grasping an object and then pushing or placing it to the target position, the first part does not change as a function of the target."_  
What about the second part of pushing it to the target position? I am assuming that is not equivariant either. 

Since the paper is aimed at a general audience, I believe it will be beneficial to describe the Meta-world and Adroit tasks in more detail in Appendix D. Showing a single ground-truth trajectory sampled from in-distribution data and comparing it with a ground truth out-of-sample trajectory would be helpful to visualize the extrapolation task. Same for the Slider task: I think Figure 14 should depict one ground truth in-distribution and one ground truth out-of-sample trajectory while keeping everything constant (i.e., all positions) except the mass.

From: ICLR.cc/2023/Conference/Paper3647/Authors
title: Update: comparison with tensor field networks completed for all objects
comment: Please note that we have edited response 2/3 to Reviewer *dNKc*. The comparison to SE(3)-equivariant tensor field networks now includes results on all objects.

From: ICLR.cc/2023/Conference/Paper3647/Authors
title: General Response
comment: In this work, we attempt to learn predictors using the expressivity of Neural Networks (NNs) and evaluate these predictors in out-of-distribution (OOD) settings. Without assumptions, NNs will fail OOD, and every set of assumptions will come with a different set of limitations since no machine learning algorithm is universally better than another.

For instance, for OOD rotations, one can build SE(3) equivariance into the architecture to capture the transformation between train and test. We achieve competitive domain-specific performance in this case, e.g. for grasp point prediction from object point clouds under OOD rotations (Table 2 in the revised submission), and even go beyond it, as our assumptions capture several domains. We have demonstrated success on domains ranging from invariant and equivariant analytic functions to robotic tasks such as grasp point prediction for different rotations and translations and reaching for different target positions.

We can also capture domains where properties such as invariance and equivariance are not immediately evident and it is challenging to build such architectures. E.g. grasp point prediction for various scales, and analytic functions that are neither invariant nor equivariant. In Meta-World push and Adroit reach, the trajectories are composed of two segments: grasping an object and reaching a target. They are also not invariant nor equivariant to the target position. Furthermore, in such domains where action spaces are multi-dimensional, drawing the connection between target shifts and actions is challenging. In these domains, further research would be required to build architectures that capture distribution shifts.

The assumptions we make in this paper allow us to perform OOD generalization in scenarios where prior methods require domain-specific knowledge and in scenarios where it is not clear how to build in this knowledge. Our method has its own limitation which we clearly state — it can achieve success OOD only under the assumptions we provide. We have provided our assumptions and guarantees concretely in (updated) Section 3.2 and Appendix B and demonstrated our success in a number of robotics tasks. We hope that this work inspires future work on further domains that satisfy the same assumptions.

We highlight the main **revisions** we made to the submission:

- The Introduction Section has been updated to better reflect our objective.
- The assumptions have moved from Appendix B to Section 3.2.
- A discussion about our assumptions regarding the representation space has been added to Section 3.3.
- The Discussion Section was updated regarding future exploration of real-world domains that satisfy our assumptions.
- Appendix C.3 now contains a comparison between bilinear transduction and SE(3)-equivariant tensor field networks for mugs (Table 2).

Proposed **future revisions**:

- We propose to compare our method with tensor field networks on the remaining objects (bottle and teapot) and to the Vector Neurons framework.

From: ICLR.cc/2023/Conference/Paper3647/Authors
title: Author Response to Reviewer gXKu
comment: Thank you for the clarification questions and for acknowledging the detailed analysis and success of our method on the demonstrated domains.

**Re Weaknesses**:

(1) **Real-world domains** 

>*It looks like to me the proposed method only works for a very limited scenario, in which o.o.s can be easily converted to o.o.c. For example, in the tasks chosen by authors, the data is almost stationary with respect to the anchor point, meaning the data distribution now is only on delta x.*
    
If we understand correctly, the reviewer is implying that our method can only extrapolate to in-distribution $\Delta x$s? 
A global solution for *any* $\Delta x$ would likely include a strong structure prior like invariance or equivariance. In the cases that we demonstrate, our assumptions allow us to make predictions without these priors for domains that are invariant, equivariant or neither, albeit to a smaller space of out-of-distribution $x$.
We have previously stated this in our assumptions in Appendix B and have updated the main text to emphasize the assumptions in Section 3.2. In Figure 5 (formerly Figure 6), we indeed demonstrate that a limitation of our method is that we can only extrapolate to $\Delta x$s that are in-distribution. 

>*I wonder if any real-world o.o.d problems are like this. For example, in image classification task, the test images can be o.o.d because of unseen lightning conditions, unseen camera angles, unseen background and etc.*
    
A real-world problem that fits our assumptions, which we attempted to address, is grasp point prediction for new object orientations, positions and scales. Since our method can extrapolate to in-distribution $\Delta x$s, we would expect extrapolation to new distributions only if the *differences* between domain points are in distribution and if we can represent their shift by $\Delta x$. Grasp point prediction satisfies our assumptions as the transformed object differences are in distribution.
We are excited about exploring which additional real-world problems satisfy our assumptions, and have added this to the discussion section.

>*I wonder if the proposed method can be extended to any more complex datasets where the stationary property does not hold true.*
    
One might potentially apply our method iteratively to expand beyond the scope of the largest $\Delta x$ by adding the predictions to the train set every iteration, and retraining on a larger and larger set of $x, \Delta x$. 

(2) **Assumptions**

>*The discussion of o.o.c condition is a bit loose. For example the theorem on page 5 (what is an informal theorem anyway) requires the condition to be as depicted in figure 3. I think assumptions in theorem should be defined more rigorously than referring to a figure.*

Thanks for your feedback. Our aim in the submission was to emphasize the “intuition” underlying the structural conditions required for our guarantee to hold. However, reviewer feedback has made it clear that it would have been preferable instead to have the formal assumptions moved from Appendix B (where they were stated rigorously), to the main body. We have therefore included all three assumptions in a formal statement of our theorem in Section 3.2. We stress that all this exposition was included in the submitted manuscript, but we have just moved it all to the main body to accommodate reviewer suggestions. We then deferred further elaboration on the assumptions to Appendix B.5, which includes Figure 9 (formerly Figure 4), and used the additional space available to make precise the connection to MNAR matrix completion.

**Re Quality**:

(1) **Discussion** 
>*Discussion in o.o.c can be improved with more rigorous discussion.*

Please see the updated Section 3.2 and point 2 above.

(2) **Reproducibility**

>*Not reproducible as code is not provided.*

We have provided a detailed description of our implementation and hyperparameter choice in Appendix D and plan to release the codebase in the near future as stated in Section 8.

From: ICLR.cc/2023/Conference/Paper3647/Authors
title: Author Response to Reviewer WsKo
comment: Thank you for the positive feedback and suggestions for clarification!

**Re Weaknesses**:

(1) **Matrix completion**
 
>*The theorem in Section 3.2 is derived by leveraging the techniques in matrix completion. I wonder what are the key differences and similarities between this paper's result and the matrix completion result.*

As described in the related work, there are two general strands of matrix completion research. In the missing-at-random (MAR) and missing-completely-at-random (MCAR) setting, one leverages statistical properties of random sampling to argue that the set of possible low-rank completions of a given matrix is a singleton with high-probability. More strongly (say, in the presence of noise and/or corruption), one can leverage the “restricted isometry property” of random sampling to argue that two low-rank matrices which differ in Frobenius norm also differ when restricted to randomly subsampled entries.
    
Our setting is in the spirit of the “missing-not-at-random  (MNAR)” matrix completion, where entries are not observed according to some probability distribution with large coverage. Instead, the pattern of observed entries is **deterministic**, but has some other structural conditions which allow for completion under low-rank assumptions. Our paper reduces to possibly one of the simplest MNAR scenarios where completion is feasible: that of a matrix partitioned into 4 blocks, where the top- and left- blocks are observed, the top-left block has rank equal to that of the entire matrix, and the bottom-right block is (completely or partially) unobserved. It can be shown that such a matrix admits a unique completion (see e.g. Shah et al. 2020 cited in our paper). It is this result that we leverage, and reduce finite-rank bilinear predictors over possible “continuous distributions” to this “discrete” matrix completion setting via a careful limiting argument.
    
(2) **Scaling training**

>*If I understood correctly, the proposed transductive re-parameterization needs (x−x′) and x′, so the training is on every pair of x′ and x. Will this cause some complexity issues during training when the number of training samples is large (as the number of pairs is almost the square of the training samples)?*

The formal assumptions require covering the marginals of $x$ and $\Delta x$, but only a subset of these (depending on the complexity of the domain) is required to be covered by all combinations ($x$, $\Delta x$), we clarify this in Assumption 3.1 in the revised Section 3.2.
In practice, we haven’t observed significant training efficiency issues for datasets of size 100 and 1000 for Adroit e.g.

From: ICLR.cc/2023/Conference/Paper3647/Authors
title: Author Response to Reviewer dKNc (3/3)
comment: (3) **High-dimensional inputs** 
 
>*I think the approach will have problems with high-dimensional inputs because modeling of each dimension independently is not accurate for most high-dimensional inputs (like images) unless one uses PCA. If PCA or other such techniques is required for the approach, I think this should be discussed along with their approach.*
 
Our method indeed requires a representation space for computing $\Delta x$ by subtracting data points. PCA is one example of extracting such a representation. Alternatively, one could possibly obtain representations via self-supervised learning [b] or pre-trained models [c]. Another research direction is exploring other ways of computing $\Delta x$ that are not via subtraction which may alleviate the need for obtaining such representations. Thank you for pointing out that this was missing from the method description, we have added a discussion regarding this pre-processing step in Section 3.3.

**Re other comments**:

(1) **High-dimensional inputs**

>*The method is related to translation symmetry due to the Δx term and is able to learn translation invariance/equivariance. But it is not clear to me how much more general it is without using additional data augmentations or PCA. For example, can one expect extrapolation to other symmetries (rotation, scaling) without data augmentations and PCA?*

Please refer to 3 above.

(2) **Domains**

>*Most of the experiments consist of symmetry due to which extrapolation is possible. What are other types of out-of-support cases that can be solved with the current approach but do not fit into translation/rotation/scale invariance/equivariance?*

Figure 7 (formerly Figure 8) describes a 1-D regression problem that is not invariant nor equivariant. As described above in point 2, Meta-World push and Adroit reach are also not invariant or equivariant to the target location. We agree that exploring further domains is an exciting direction that we plan to pursue in the future, we added this to the discussion section.

(3) **Clarification test vs train**

>*Please add the difference between train and test distributions for each experiment in the main text.*

We added the train and test splits in Section 4.

(4) **Manipulation baselines**

>*In Meta-world and Adroit benchmarks, please compare with a translation equivariant model.*

Please refer to 2 above.

(5) **Point cloud baselines**

>*For point cloud classification, please compare with methods that use the point clouds as inputs directly and not the low dimensional features.*

Please refer to 2 above.

(6) **Slider dynamics**

>*For Slider experiments, it would be interesting to see the ground truth slider torque dynamics for different object masses to see the difference between train and test, and understand how the method extrapolates.*

We added a visualization of Slider actions in Appendix D.1, Figure 14.

**References**:

[a] Deng et al. “Vector Neurons: A General Framework for SO(3)-Equivariant Networks”, ICCV 2021

[b] Kingma and Welling “Auto-Encoding Variational Bayes”, ICLR 2014

[c] Upchurch et al. “Deep Feature Interpolation for Image Content Changes”, CVPR 2017

From: ICLR.cc/2023/Conference/Paper3647/Authors
title: Author Response to Reviewer dNKc (2/3)
comment: (2) **Domain-specific baselines**
 
>*Many of the experiments (periodic functions, push/reach and other robotic manipulations) only require translation invariance/equivariance for out-of-support extrapolation but such models are not compared with.*
 
Manipulation baseline: 
Meta-World push and Adroit reach actions are neither invariant nor equivariant to the target location. They are not invariant to the target location as they require actions that lead to object placement in different locations. They are not equivariant to the target location as the trajectory is composed of two parts: grasping an object and then pushing or placing it to the target position, the first part does not change as a function of the target. To the best of our knowledge, no existing methods attempt to solve the manipulation tasks we show in Meta-World and Adroit with translation invariance/equivariance. 
For Meta-World reach, we have shown that a linear baseline can capture this task (without equivariance assumptions). 
 
**Q for the reviewer**: We would appreciate it if the reviewer could clarify what invariant/equivariant baselines they are referring to.
 
>*For point-cloud tasks, out-of-support extrapolation requires rotation/translation/scale symmetries but it is hard to judge if these symmetries are learnt automatically from the method as the input to the model is not the point cloud itself, but low dimensional PCA features and the mean of all the points.*
    
PCA learning symmetries: 
PCA captures axes of variation and is a standard dimensionality reduction technique that is not specific to rotation/translation/scale. It is applied similarly to point clouds regardless of their rotation, translation and scale. Our method captures *differences* between data points given representations that allow comparison by subtraction. If the data contains these recurring differences and the representation space provides a means to compare these differences, our method can be used for extrapolation.
 
>*Adding these symmetries for point-cloud tasks has been explored a lot (e.g., [1]) and should be compared with.*
 
[**Edited with full SE(3) results on all objects**] SE(3) equivariant baseline: Thank you for suggesting this baseline. We applied the SE(3) network proposed by Thomas et al. to the mug in our dataset. Specifically, we train the mean of the last equivariant layer in the network used for Tetris object classification to predict a grasping point on 100 rotated or scaled mugs. Following the implementation in Thomas et al., we provide 6 manually labeled key points of the point cloud that uniquely identify the grasp point: 2 points on the handle and mug intersection and 4 points on the mug’s bottom. We then check extrapolation separately to rotated and scaled mugs on 50 orientations and scales sampled from the out-of-sample distribution described in Section 4. Under various rotations, extrapolation is possible for tensor field networks and bilinear transduction. **For scaling however, scale transformation guarantees do not exist in SE(3) architectures, and the error is higher than for bilinear transduction**.
Task|Tensor Field Networks|Bilinear Transduction (Ours)|
|-----|---------------------------|---------------------------|
|mug rotation|$0.009 \pm 0.003$ |$0.01\pm 0.003$|
|mug scale|$0.027 \pm 0.011$|$0.004\pm 0.001$|
|||
|bottle rotation|$0.008 \pm 0.004$ |$0.004\pm 0.002$|
|bottle scale|$0.019 \pm 0.008$|$0.003\pm 0.001$|
|||
|teapot rotation|$0.011 \pm 0.006$ |$0.006\pm 0.002$|
|teapot scale|$0.093\pm 0.039$|$0.003\pm 0.001$|

We have updated only the mug results in Appendix C.3 Table 2 (due to time and memory constraints) and include the full results here. We are also looking into comparing our method with other SE(3) methods, such as Vector Neurons [a].

From: ICLR.cc/2023/Conference/Paper3647/Authors
title: Author Response to Reviewer dNKc (1/3)
comment: Thank you for the suggestions and for acknowledging that the paper is clear and provides a novel way of thinking about extrapolation.

**Re main concerns**:

(1) **Assumptions**

>*While the goal of the work is to explore a “general setting does not assume apriori access to distribution shift or model class”, I believe that their assumption of bilinear transduction is essentially assuming apriori knowledge about the type of distribution shift or model class.*

We agree with the reviewer that our method only provides extrapolation if certain conditions are met. We have previously stated these assumptions rigorously in Appendix B and have updated Section 3.2 to emphasize them in the main script as well. We are sorry that our introduction did not clearly reflect the conditions and we have revised the introduction to better reflect our objective: “In this work, we propose an alternative structural condition under which out-of-support extrapolation is feasible.”

>*While the assumption is weaker than translation invariance/equivariance, it is still strong and is able to extrapolate only in very specific cases.*

We have shown empirically that under our set of assumptions, our method can capture some invariant (Figure 4b, formerly Figure 5b) and equivariant (Figure 6, formerly Figure 7) structures, and also structures that are neither invariant nor equivariant (Figure 7, formerly Figure 8). In each of these domains, we do not assume domain knowledge and can still capture these structures. We argue therefore that while domain-specific assumptions capture inductive biases of specific domains, our method builds on a different set of assumptions, that we have empirically shown can capture several domains, varying in rotation, translation, scale, and mass without providing this knowledge per each domain separately. 
We thank the reviewer again for clarifying this point and have made edits to the introduction and Section 3.2 to reflect our objective and assumptions precisely in the main text.

From: ICLR.cc/2023/Conference/Paper3647/Reviewer_dNKc
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: Paper tackles the out-of-support extrapolation problem where the input at test is out of the support of the training distribution. A bilinear model is trained to predict the output $h(x)$ using $\bar{h}(x’, \Delta x)$ where $x’$ is some chosen anchor point and \Delta x is the distance of $x$ to the anchor point. Given an unseen test point, the model is able to extrapolate (in certain situations) by using an anchor point seen in training and the distance from it. 

strength_and_weaknesses: Out-of-support extrapolation is an interesting but challenging task and the paper provides a novel way of  looking at the problem via a reformulated out-of-combination task. 

My major concerns:

1. While the goal of the work is to explore a “general setting does not assume apriori access to distribution shift or model class”, I believe that their assumption of bilinear transduction is essentially assuming apriori knowledge about the type of distribution shift or model class. While the assumption is weaker than translation invariance/equivariance, it is still strong and is able to extrapolate only in very specific cases. 

2. Many of the experiments (periodic functions, push/reach and other robotic manipulations) only require translation invariance/equivariance for out-of-support extrapolation but such models are not compared with. For point-cloud tasks, out-of-support extrapolation requires rotation/translation/scale symmetries but it is hard to judge if these symmetries are learnt automatically from the method as the input to the model is not the point cloud itself, but low dimensional PCA features and the mean of all the points. Adding these symmetries for point-cloud tasks has been explored a lot (e.g., [1]) and should be compared with. 

3. I think the approach will have problems with high-dimensional inputs because modeling of each dimension independently is not accurate for most high-dimensional inputs (like images) unless one uses PCA. If PCA or other such techniques is required for the approach, I think this should be discussed along with their approach. 



Other questions/comments:

1. The method is related to translation symmetry due to the $\Delta x$ term and is able to learn translation invariance/equivariance. But it is not clear to me how much more general it is without using additional data augmentations or PCA. For example, can one expect extrapolation to other symmetries (rotation, scaling) without data augmentations and PCA? 
2. Most of the experiments consist of symmetry due to which extrapolation is possible. What are other types of out-of-support cases that can be solved with the current approach but do not fit into translation/rotation/scale invariance/equivariance?
3. Please add the difference between train and test distributions for each experiment in the main text. 

4. In Meta-world and Adroit benchmarks, please compare with a translation equivariant model. 
5. For point cloud classification, please compare with methods that use the point clouds as inputs directly and not the low dimensional features. 
6. For Slider experiments, it would be interesting to see the ground truth slider torque dynamics for different object masses to see the difference between train and test, and understand how the method extrapolates. 



References:

[1] Thomas, Nathaniel, et al. "Tensor field networks: Rotation-and translation-equivariant neural networks for 3d point clouds." *arXiv preprint arXiv:1802.08219* (2018).

**Post-rebuttal**
The authors have adequately addressed my concerns and I am increasing my score. As in my original review, I think the paper provides a novel way of looking at the challenging out-of-sample problem, is well-formulated and easy to read. After rebuttal, authors have convinced me that the approach empirically extrapolates in many different settings without just using equivariance. 
clarity,_quality,_novelty_and_reproducibility: The paper is well written and easy to read. The reformulation of an out-of-support problem to a potentially easier out-of-combination problem is novel and interesting. 


summary_of_the_review: While the goal of the work is to explore a “general setting does not assume apriori access to distribution shift or model class”, I believe their assumption of bilinear transduction is still strong and is able to extrapolate only in very specific cases. Most of the experiments essentially require symmetries to solve the out-of-support task (for example translation/rotation/scale invariance/equivariance) but such invariant/equivariant methods are not compared with. 
correctness: 4: All of the claims and statements are well-supported and correct.
technical_novelty_and_significance: 4: The contributions are significant, and do not exist in prior works.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 6: marginally above the acceptance threshold

From: ICLR.cc/2023/Conference/Paper3647/Reviewer_WsKo
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: This paper studies the issue about the mismatch between $\mathcal{D}\_{test}$ and $\mathcal{D}\_{train}$ (i.e., out-of-distribution (OOD)). Different from a common OOD situation that the ratio between the density function of $\mathcal{D}\_{test}$ and $\mathcal{D}\_{train}$ is bounded, the authors focus on out-of-support where the density ratios are not bounded. To solve such a problem, the authors propose a transductive re-parameterization by introducing an anchor point $x'$ for a query point $x$. By leveraging the techniques from low-rank matrix completion, the authors provide a guarantee for OOS extrapolation using such transductive re-parameterization under certain conditions.
strength_and_weaknesses: Strength: The topic is interesting. The result seems novel and convincing. The presentation is relatively clear.

Weaknesses: Some places may need more explanation and justification. I have some detailed questions as follows.

1. The theorem in Section 3.2 is derived by leveraging the techniques in matrix completion. I wonder what are the key differences and similarities between this paper's result and the matrix completion result.

2. If I understood correctly, the proposed transductive re-parameterization needs $(x-x')$ and $x'$, so the training is on every pair of $x'$ and $x$. Will this cause some complexity issues during training when the number of training samples is large (as the number of pairs is almost the square of the training samples)?
clarity,_quality,_novelty_and_reproducibility: This paper is novel and clearly written. The overall quality of this paper is high.
summary_of_the_review: I like this paper in general, due to the novelty of the result and the clearness of representation.
correctness: 4: All of the claims and statements are well-supported and correct.
technical_novelty_and_significance: 4: The contributions are significant, and do not exist in prior works.
empirical_novelty_and_significance: 4: The contributions are significant, and do not exist in prior works.
flag_for_ethics_review: NO.
recommendation: 8: accept, good paper

From: ICLR.cc/2023/Conference/Paper3647/Reviewer_gXKu
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: The paper first discusses out-of-combination scenario, which is a special case of out-of-support scenario. Then the paper propose to use anchor points to convert OOS case to OOC case for improving generalization.
strength_and_weaknesses: Strength:
1. Detailed analysis of different o.o.d scenarios.
2. Under certain constraints, the proposed method of using anchor points works in improving generalization performance.

Weaknesses:
1. It looks like to me the proposed method only works for a very limited scenario, in which o.o.s can be easily converted to o.o.c. For example, in the tasks chosen by authors, the data is almost stationary with respect to the anchor point, meaning the data distribution now is only on delta x. I wonder if any real-world o.o.d problems are like this. For example, in image classification task, the test images can be o.o.d because of unseen lightning conditions, unseen camera angles, unseen background and etc. I wonder if the proposed method can be extended to any more complex datasets where the stationary property does not hold true.
2. The discussion of o.o.c condition is a bit loose. For example the theorem on page 5 (what is an informal theorem anyway) requires the condition to be as depicted in figure 3. I think assumptions in theorem should be defined more rigorously than referring to a figure.
clarity,_quality,_novelty_and_reproducibility: The paper is in general clearly written. Discussion in o.o.c can be improved with more rigorous discussion. In general quality is good. Not reproducible as code is not provided.
summary_of_the_review: The paper has some novelty in recognizing a special case of o.o.s generalization and propose a method to tackle this case. However it is not clear to me how much percentage of real-world o.o.d problems belong to this special case. The proposed method seems to require strong pre-condition ( stationary data distribution w.r.t anchor points) to work
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold