Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 1: Reject
review_content: This paper presents a model that is able to associate objects seen in a new time step with the objects seen in previous frames and therefore consider the cases of adding new objects, dropping object no longer visible, but keeping them in memory in case they reappear in the future. The model assumes that the location of the objects is given and the only task to perform is the correct association of instances. Results shows that the proposed approach is better than hand-crafted features on different simulated tasks. Additionally the propose model is shown to help to reduce the computational cost for question answering task.

I lean to reject this paper because in my opinion the proposed method is just a set of predefined rules to train a network to be able to perform object association between a new time step and a memory. Additionally the experimental evaluation is very weak in several points (see below). 

- Contribution: this approach proposes a set of rules to train a network to be able to learn the correct association  between two set of features. Additionally, the network is used together with a memory to keep track of previously seen objects. In my understanding the proposed work presents a set of hard-coded rules to train a network for tracking. However, the connection with tracking is presented only in related works (at the end of the paper) and no comparison with other tracking approach is presented. Additionally, the network model that is actually used for the experiments (graph network) is not presented in the main paper, which makes more difficult to understand how the network works for the given task.

- Experimental evaluation: Experiments are performed on a very simple, simulated environment. Objects are very simple to recognize and their location (which is often  a difficult component of the problem) is given. As the task is quite straight forward, the model obtains almost 100% on all tests. Comparisons are made only with hand-crafted features. It is quite evident that a learned similarity measure will be better than hand-crafted distances. 

Authors should explain more clearly that the proposed contribution is a set of rules used to learn a distance between features in order to associate object instances. 
A cleared connection with tracking should be provided from the introduction. In the evaluation there should be a comparison on multiple and more challenging tracking datasets. In this way we can compare the proposed training technique with other tracking approaches that also learn to a distance in order to associate object at different time steps. 


review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: With the assumption of object persistence and inspired by the sticky indices, this paper proposed a novel object alignment method for matching arbitrary entities in different sets. The motivation is clear and easy to follow. The simulation experiment with the symbolic dataset gives impressive results. 

I like this idea and have one concern about the experiment. To be specific, all the reported results are obtained in the symbolic dataset to simulate the real-world case. Have you performed the entity alignment in the real-world data, such as the cross-lingual knowledge graphs, or cross-modal analysis? It's expected to experiment with more challenging datasets. By the way, have you released your code? This is important to an ICLR submission. 

review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: 
This paper proposes AlignNet, a bipartite graph network that learns to match to sets of objects. AlignNet has a slot-wise object-based memory that associates an index with each unique object and can discover new and re-appearing objects. Experiments are conducted on a symbolic dataset.

I do not think the paper meets the acceptance threshold, and recommend for weak rejection. While the paper proposes an interesting architecture to address the alignment problem, it has noticeable flaws in its experimental designs.

First, all the experiments are conducted on toy symbolic datasets, where the alignment problem is rather easy to solve. On the other hand, real-world scenarios can be far more complicated. For example, the appearance of the same object can change due to lighting and distance, and it is unreasonable to assume that their features would remain static (apart from simple uniform noises). In addition, the paper only compares against hand-crafted similarity measures (MSE and cosine). It is unfair to compare learned methods only to hand-crafted methods. As a reasonable and fair comparison, the paper should also compare AlignNet against learned similarity measures (such as a neural network supervised with ground-truth labels for alignments).

The toy dataset and simple baselines in this paper raise doubts on whether the proposed method is applicable to more complex scenarios (such as aligning two sets of objects in natural images through their appearance features).
review_title: Official Blind Review #2

Meta-review: This paper proposes a network architecture which labels object with an identifier that it is trained to retain across subsequent instances of that same object.

After discussion, the reviewers agree that the approach is interesting, well-motivated and written, and novel. However, there was unanimous concern about the experimental evaluation, so the paper does not appear to be ready for publication just yet, and I am recommending rejection.
Decision: Reject