Venue: ICLR 2017 workshop

Reviews:

final_score: rating: 7: Good paper, accept
review_content: This paper investigates a set of simple models for generating scene and image graph embeddings. Scene graphs are represented either with count features on their constituent nodes, count features on their constituent nodes and short paths, or convolutionally. A margin objective is then used to learn projections from the space of image features and graph representations into a joint embedding space. This paper finds that on a dataset of scene graphs, the representation based on path counts outperforms the other two approaches both in identifying similar images to the one for which the graph was annotated, and in retrieving the target image.

This is a clean, focused, and well presented contribution. It's an interesting result that the approach based on path counts outperforms the convolutional / GraphNN approach---it seems like count-based models have generally been on the way out (at least in machine translation and language modeling). Presumably it's the relatively small size of the training data that makes them still useful here. It might be useful to mention how you think this approach might scale to larger datasets. How big is the object vocabulary? How many subpaths occur in the test set but not the training set? Are you doing anything else (e.g. smoothing, backoff) to deal with count sparsity?
review_title: Review

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: The submission studies scene graph prediction via joint embeddings of images and graphs. It evaluates two different embeddings (one of which is a simple baseline). The graph embeddings are evaluated in ranking experiments. Interestingly, a representation that is essentially a "bag of small subgraphs" performs very competitively; it substantially outperforms graph network representations. 

Scene graph prediction will likely become an increasingly important topic in computer vision, and this submission presents some string baselines for the problem. Having such baselines is extremely important: so, even though the paper does not introduce new algorithms, I would recommend that this submission is accepted.

It would be interesting to see to what extent these results generalize to larger datasets that have a long tail of relationship and node types, such as VisualGenome; I encourage the authors to perform such experiments for a future full version of this paper.

The submission should probably cite this related work: https://arxiv.org/pdf/1701.02426.pdf

review_title: Strong baselines for scene graph prediction

Decision: Accept