Venue: ICLR 2023 TinyPapers

Reviews:

final_score: rating: Great Start (GS): a submission which meets some of the reviewing criteria but has room for improvement
review_content: summary_of_contributions: The paper presents methods for building classification models that can make predictions on data with label assignments that depend on the data source. The authors explore this less explored area of importance and evaluate multiple methods such as Input Append, Classifier Append, Predict Mapping, Separate Classifier, and Calculate Mapping for learning multiple label assignments dependent on a point’s underlying data source.  
strengths_and_weaknesses: Strengths:
- The paper addresses an important and less explored problem of making predictions on data generated by multiple sources.
- The paper proposes multiple methods for learning multiple label assignment that depend on the data source.
- The paper provides an evaluation of these methods on two public datasets, which adds credibility to the preposed methods.

Weakness:
- What is the source of MNIST and PTB-XL? Did you divide the training set into multiple sources, and set the label space differently for each source in the experiment? There should be an explanation of the experimental setup.
- Is it correct that a person manually calculates the mapping matrix in "calculate mapping"? It would be helpful to have an explanation.
suggested_changes: Suggested changes:

- In classification, the mapping matrix for change of basis should be one-hot per column. It would be good to try representing the output of the mapping model in the “predict mapping” method as one-hot.
- The cell labeled “Model” in the tables and the empty cell below it would look nicer if merged using a LaTeX `Multirow' package.
- "calculate mapping" and "calculated mapping" are being used interchangeably. Please unify them.
- I think the context of the statement that "$C_{S_i}$ can be written as an identity matrix" is not clear. It would be helpful to clarify the statement by indicating the dimensions of $M_{S_i}^{S_j}$ and $C_{S_i}$ and providing a clear explanation.
review_title: null

Meta-review: metareview: Strengths:
- The paper studies a relevant and timely research problem, that of learning with data coming from multiple sources.
- The presentation of the work is good, and proposed concepts are nicely illustrated.

Weaknesses:
- Misses connection with important multi-task and multi-domain learning literature, where related techniques of encoding sources and cross-task mappings have been studied.
- The experiments seem to use random subsets of data, so it is not clear what is learned by the techniques.
confidence: 4: The area chair is confident but not absolutely certain
Decision: Invite to archive