Interpretable Relational Representations for Food Ingredient Recommendation Systems
Authors: Kana Maruyama
Michael Spranger
Venue:  (2021)
Keywords: Metric Learning
Gastronomy
Memory Network
Knowledge Graph
Interpretable

Abstract: Supporting chefs with ingredient recommender systems to create new recipes is challenging, as good ingredient combinations depend on many factors like taste, smell, cuisine style, texture among others. There have been few attempts to address these issues using machine     learning. Importantly, useful models do obviously need to be accurate but importantly -- especially for food professionals -- interpretable. In order to address these issues, we propose the Interpretable Relational Representation Model (IRRM). The main component of the model is a key-value memory network to represent relationships of ingredients. We propose and test two variants of the model.
One can learn latent relational representations over a trainable memory network (Implicit model), and the other can learn explainable relational representations over a pre-trained memory network that integrates an external knowledge base (Explicit model).
The relational representations resulting from the model are interpretable -- they allow to inspect why certain ingredient pairings have been suggested. The Explicit model additionally allows to integrate any number of manually specified constraints.
We conduct experiments on two recipe datasets, including CulinaryDB with 45,772 recipes and Flavornet with 55,001 recipes, respectively. The experimental results show that our models are both predictive and informative.

Under review as a conference paper at ICLR 2021
INTERPRETABLE RELATIONAL REPRESENTATIONS FOR FOOD INGREDIENT RECOMMENDATION SYSTEMS
Anonymous authors Paper under double-blind review
ABSTRACT
Supporting chefs with ingredient recommender systems to create new recipes is challenging, as good ingredient combinations depend on many factors like taste, smell, cuisine style, texture among others. There have been few attempts to ad- dress these issues using machine learning. Useful Machine Learning models do obviously need to be accurate but importantly especially for food profession- als - interpretable. In order to address these issues, we propose the Interpretable Relational Representation Model (IRRM). The main component of the model is key-value memory network to represent relationships of ingredients. We propose and test two variants of the model. One can learn latent relational representations over a trainable memory network (Implicit model), and the other can learn ex- plainable relational representations over pre-trained memory network that inte- grates an external knowledge base (Explicit model). The relational representations resulting from the model are interpretable they allow to inspect why certain in- gredient pairings have been suggested. The Explicit model additionally allows to integrate any number of manually specified constraints. We conduct experiments on two recipe datasets, including CulinaryDB with 45,772 recipes and Flavornet with 55,001 recipes, respectively. The experimental results show that our models are both predictive and informative.
INTRODUCTION
Data mining and machine learning methods play an increasingly prominent role in food preference modeling, food ingredient pairing discovery and new recipe generation. Solving these tasks non- trivial, since the goodness of ingredient combinations depends on many factors like taste, smell, cuisine, texture, and culture. Ahn et al. (011) detected that the number of shared flavor molecules between ingredients is one of important factors for food pairing. They found Western cuisines show a tendency to use ingredient pairs that share many flavor compounds, while East Asian cuisines tend to avoid compound sharing ingredients. Using this idea, [arg et al. (2017) developed rule-based food pairing system which ranks ingredients based on the number of shares of flavor molecules. Recently, Park et al. (2019) suggested neural network approach based flavor molecules and co-occurrence of ingredients in recipes. These approaches focus on one-to-one food pairing. There also research related to many-to-one pairing. De Clereq et al. (016) proposed the Recipe Com- pletion Task which tries to identify matching ingredients for partial list of ingredients (the recipe) using a Matrix Factorization based recommender system. Although efforts have been made to detect good ingredient combinations, there is no current Machine Learning method in this field that allows to interpret why suggested pairs are good.
Our work is targeted at interpretable recommendation systems for food pairing and recipe comple- tion. Given a set of pre-selected ingredients (cardinality 1 or more) by user, the recommender suggests top-N ingredients from set of candidates. For example, suppose user selects apple and chocolate as the pre-selected ingredients, our recommender suggests some good paired ingredients (e.g. cinnamon) and also identifies reasons (e.g. cinnamon is good for apple and chocolate in terms of their flavor affinity).
For this, we propose the Interpretable Relational Representations Model (IRRM) in two variants to address food pairing and recipe completion tasks. The model features key-value memory network (Sukhbaatar et al. (2015), Miller et al. (2016) to represent relationships of ingredients. One variant

---

Under review as a conference paper at ICLR 2021
of the model is trained to learn latent relational representations a trainable memory network (Implicit Model). The other model can learn explainable relational representations over the pre- trained memory network integrating an external knowledge base (Explicit Model). The relational representations are interpretable and can be queried as to the reasons why the ingredients have been suggested. The Explicit model can integrate any number of constraints which can be decided manually based on the characteristics of the desired recommender system. Our contributions are as follows:
1. We model ingredient pairing as a general recommendation task with implicit feedback.
2. We introduce the Interpretable Relational Representations Model and it's two variants: Im- plicit and Explicit. Both of which can learn pair specific relational representations (vectors) for one-to-one (i.e. ingredient to ingredient) and many-to-one (ingredient-set to ingredient) food pairing tasks. The relational vectors are also interpretable.
3. We propose a training procedure learn one-to-one and many-to-one relationships effec- tively using recipes.
4. We evaluate our proposed models in the Recipe Completion Task and the Artificial Food pairing Task on the CulinaryDB and the Flavornet datasets. Our proposed approaches demonstrate competitive results on all datasets, outperforming many other baselines.
5.
We perform qualitative analysis. The results presents our proposed Explicit model is capa- ble of unraveling hidden ingredients structures within recipes.
2 RELATED WORK
There are two related streams of work in recommender systems that are important for this paper: the session-bust based setting and the knowledge-aware systems.
In the session-based -baeed setting, user profile can be constructed from past user behavior: A natural solution to this problem is the item-to-item recommendation approach. variety of methods exist for this problem. For example, Quadrana et al. (2017) models the item sequence using RNNs, Kang & (2118) uses Self-Attention layers, and Wu et al. (2020) uses Transformer layers. While these methods mainly focus how to encode item click-sequence interactions,
target good ingredient pairing using only ingredient attributes and the relationship between a ingredient set and an ingredient based on co-occurrence in recipes. For this develop new architecture integrating set encoders and relational memory with novel loss and score functions.
There are also increasingly methods for integrating knowledge into recommenders. Zhang et al. (016)) and Cheng et al. (2016) directly incorporate user and item features as user profile into neu- ral network models. Huang et al. (2018) and Wang & Cai (0200) integrate them using a pre-trained knowledge graph. These methods try to represent user context using external knowledge base, there- fore, usually these knowledge embeddings are integrated user embeddings. In this work, we corporate knowledge specifically to detect relationships between an ingredient set and an ingredient for interpretation improve recommendation performance.
3 PROBLEM DEFINITION
We first introduce the notations used throughout this paper. We model recipe completion as a recom- mendation scenario with implicit feedback (Huang et al.) 2018 Tay et al., 2018 In such scenarios, a user has interacted with an item and the system infers the item that user will interact next based on the interaction records of the user. We apply this to the food domain by using recipes as interaction records.
Let denote a set of ingredients and (i.,..., i denote a pre-selected ingredient set, where i I is the ingredient and M is the number of ingredients. We call {i...... ias pre-selected ingredient set in this paper. Next, let Ieandidate denotes set of candidate ingredients. Ieandidate depends on each pre-selected ingredient set, that is, Icandidate I {ii..... }.s). In addition, we assume that a knowledge base (KB) of ingredients is also available and the KB contains factors which are related to why some ingredients are good combinations. A KB is defined as set of triplets over
2

---

Under review as a conference paper at ICLR 2021
Scatin
krock
1896
LOGS
Layyer
(a) Implicit model
Explicit model
Figure 1: IRRM architectures
entity set & and relationship set L. A KB triplet (es, L, ea) composed of two entities Co. E E and a relationship E L. where e, an ingredient (e.i. e, E I) and is attribute and e, the attribute value. For instance, (upple, flavorMolecule, (-)-Epicatechin)) denotes that apple contains the (-iitcccatenhin flavor molecule.
Based on these preliminaries, we define the food ingredient recommendation task. Given pre- selected ingredient set (i,....,iM and candidate ingredients Lcandidates we would like to infer the top-N ingredients from Loandidate.
RECOMMENDATIONS WITH MEMORY NETWORKS
In this section, introduce the IRRM architectures. We start with the Implicit model that consists of a trainable key-value memory network. We then augment the Implicit model using key-value memory network which integrates pre-trained entity and relationship vectors with ingredient tributes in the KBs we call this extention the Explicit model. The overall architecture is described Figure The input of our architecture are a pre-selected ingredient set and a candidate ingredient Peandidate E Ioandddate- The output is score. In inference, our recommender uses these scores to rank Leandidate:
4.1 INGREDIENT EMBEDDING LAYER AND INGREDIENT SET ENCODER
Ingredients are represented as one-hot encoding vectors (corresponding to a unique index key be- longing to each ingredient). At the embedding layer, this one-hot encoded vector is converted into a low-dimensional real-valued dense vector representation which multiplied with the embedding matrices Q Ødx which stores ingredient embeddings. is the dimensionality of the ingre- dient embeddings while [Z] is the total number of ingredients. icandidate is converted to using this embedding layer. On the other hand, pre-selected ingredients (f1,..., ias) are encoded by the Ingredient Set Encoder (Figure ( At first, each ingredient i, is converted to vector using the Ingredient Embedding Layer (same ieandidate). As result, i; E Rd vectors are generated. The sum of these vectors converted to the ingredient set vector p using a feed-forward network with single hidden layer, followed by Layer Normalization.
4.2 RELATION ENCODER
Tay et al. 2018) introduced LRAM (Latent Relational Attentive Memory), in order, to generate latent relational vectors between user-item interactions. We expand this module by adding a residual connection, followed by Layer Normalization. This idea is inspired by Vaswani et al. (2017).
Given the pair Relation Encoder first applies p + q to generate the joint embedding of p and q. The generated vector & E Rd is of the same dimension of p and q. Note we also tried other transfer functions here such element-wisst multiplication just using multi- multi-layered perceptron M LPp, q). However, we found that addition performs best. This joint embedding s is used as the input of the memory network. The attention vector E Rd is vector of importance weights over keys which are
pre-selected ingredient set vector and a candidate ingredient vector, (p, q), the
3

---

Under review as a conference paper at ICLR 2021
represented the key matrix K [k1.... ....... kx] | pairs in the memory network and k, E IR is a key vector. Each element of the attention vector a can be defined as = s" ky, where aj E R. In order to normalize the attention vector a to probability distribution, use the Softmax function:Softmaz(())
RNxd where N is the number of key-value
expla;) We generate Σ exp(a..)
the vector m = E Softman(an))n as the summation of weighted value vectors which are represented as the value matrix V = [.)....,, E RR Finally, in order to generate the relational vector r, is added with the joint embedding and Layer Normalization applied as follows r = LayerNorme (s+ + m).
4.2.1 THE EXPLICIT MODEL
In order to improve interpretability and predictive performance, we incorporate ingredient attribute information from given KB into the memory network. Inspired by recent works which integrate memory network with external memories (Huang et al.] (2018)) we propose the Explicit Relational Encoder. Instead of the trainable key matrix K and value matrix V, we pre-train vectors over given KB. We then freeze key and value matrix for training the explicit model. Given pair of a pre- selected ingredient
fi....., 1M} } and candidate ingredient icandidate: {ii...... . leandidate } is converted into the entity vectors using the KB embeddings which provide the entity vectors e E Rd* and the relationship vectors Rdd Note that in case of dkB + d, convert the joint embedding IRd into s' Rd and the relational vector r R into r' Rd with linear projections. We use the TransF (Bordes et al. (2013)) for the KB embeddings. The reason for this choice is that given triplet (es; latt, Ext)) TransF can learn entity vectors and relationship vectors to use
follow eitt = + latt- KB relationships usually correspond to attribute types of entities, the notation last as the attribute type and Eatt as its value. Hence, we set the key matrix as follows:
K K= (att ]
where N depends on the number of attribute types which you want to integrate and K constant through training. The value matrix is initialized as follows:
Vatty
Σ (E( of deandidate)
Calt
Σ
(e: + latt,)
(2)
( feandidate)
V Vatty
Vatty |"
There can be many onc-to-multiple relations in the KB, For instance, an apple has multiple flavor molecules. Therefore, the entity vector Catt should not be an ingredient specific vector and e, + latt instead of using Catt-
4.3 SCORE FUNCTION
Finally, we define our score function as the relationship between the pre-selected ingredient set vector the candidate ingredient vector q, and the relational vector r:
s(p. q. r) = CosSim(p q) + CosSim(p + q.r)
(4)
where CosSim the cosine similarity, and good pairs will have high scores etc.
Figure 2 shows the geometric differences between our score function and other possible ones. Fig- (() (a) tries to place the ingredient set and each ingredient into the same spot in vector space, 2 b) learns to fit the ingredient set and each ingredient with the relational vector r. This score function is often used the domain of collaborative filtering. The score functions and are sug- gested in previous work. TTy et al. (2018) reported - -|pp |pp +
qll could achieve better performance than - 1p - q in the domain of collaborative filtering, which the user-item based recommender. However, in the food pairing task, the result the opposite. It seems if
-llp + q because of r. p and can not be trained properly. As approaches Ô, the performance improved, so the representation of r cannot be leamned. This makes sense, since the ingredient set is mix- ture rather than list of ingredient and ingredient sets can be seen single processed ingredient.
4

---

Under review as a conference paper at ICLR 2021
Recipe A ing
Recipe Ing_b1
Recipe C ing_ ing_ce c2
ing a2
ing_b2 ing_ ing b4
ing_a3
ing
ing. set ?
mo
(2)
9 ng.
(a) -Ipp all
(b) -lpp + 91l
e.g. batch
Ingrodient
Ing 1A)
[ng.a2]
(3) Iegrefien
Ingredient
Ing.a3
Set
Fig. 11, ing (3)
Ingrodiont
kg
Fig.a1
Ingredient set
Ing, 21, ing Mi]
(ng,4t] c1)
(c) Ours: CosSim(p, CosSSimpptq. r)
legredient
kgg
kgg
Figure 2: Geometric comparisons of score func- tions.
Figure batches.
How to generate pairs for mini-
Therefore, we propose to use a score function (Figure c)) that tries to place the ingredient set p and each ingredient q into the same location in vector space and rotate the relationship between p and closer to the same place as a relational vector r which represented using attributes from KB. In addition, since network structure is symmetric with respect to p and q, require symmetric bilinear score function.
4.4 OBJECTIVE FUNCTION
Our objective function defined as:
Batch Pos
exp( 8 Su.
Σ
logl ==1 exp( Py
7 Neg exp( (Pr
(5)
where À is the margin that separates the golden pairs and corrupted pairs, T is a temperature pa- rameter, Batch the mini-batch size, Pos is the number of positive examples, Neg is the number of negative examples for each positive example, and the score function for negative examples take the same relational vector as their positive example. In order, to define this objective function, we combine the Batch All loss (Hermans al. (2017)) ) with the effective triplet loss in Metric Learning and the additive margin in the softmax loss (Wang et al. (2018)) Note that while the hinge loss is also possible, found that the softmax loss has better performance and is more stable.
4.5 TRAINING
Using pre-pr prepproessed recipes (removed all ingredients without attributes and empty recipes), we train our models the following steps (Figure 3): At first, we randomize the order of recipes and its ingredients (Figure 3 ()), and then generate the sequence of ingredients from recipes (Figure (3) After that, we generate pairs between pre-selected ingredient set and candidate ingredient from the sequence (see Figure 3)3) and loosely make pairs from same recipes belong to the same mini- batch. Next, we sample additional positive examples as necessary which are taken from other recipes randomly. Finally, we sample negative examples randomly. In the end, mini-batches are generated. Note, we also trained our models without the recipe restriction. However, performance was worse in both one-to-one and many-to-on pairs.
EVALUATION
We evaluated the the Implicit model and the Explicit model tasks against standard baselines and we analyzed the relational representations in terms of interpretability. We propose two tasks for evaluation
5

---

Under review as a conference paper at ICLR 2021
For evaluation, we first use the same ranking procedure in De Clereg et al. (2016), namely the Recipe Completion Task. In the Recipe Completion Task, the recommender ranks all ingredients to predict one randomly eliminated ingredient, and the remaining ingredients of each recipe used as input to the model- which is the pre-selected ingredient set in our definition. We adopt three evaluation metrics for this task (same as the previous work); Mean Rank: the mean position of the climinated ingredient in the ranked ingredients, Hit Ratio(HR)@ 10: the proportion of correct ingredients ranked the top 10, and ROC-AUC(AUC); the mean area under the ROC curve. This task is useful as it allows us to measure the basic performance of our model in the same setting as existing baselines imany-to-one pairing) and against ground truth data.
In order to understand the model performance further, we tested on second task called Artificial Food Pairing Task. In this task, we generate pairs from existing recipes based on ingredient co- occurrences. Given some ingredients as pre-selected ingredient set and candidate ingredient, pairs that occur in any of recipes are used as a positive example, otherwise as a negative candidate example. The Artificial Food Pairing task consists of positive pairs and negative pairs where the same pre-selected ingredient set always used for both positive and negative pairs but positive and negative examples are randomly taken from its candidates with a pre-specified number and ratio. In this task, the recommender predicts whether pairs are positive or negative examples. We use this task for more detailed analysis. We measure MAP@ 10: the mean average precision in top-10 and AUC as metrics.
We evaluate two datasets. The first dataset is taken from the CulinaryDB (Bagler (017)) p. and the second dataset is from Flavornet (Ahn et al. (201I)). Both datasets contain the recipe set which consist of the list of ingredients and the cuisine categories (e.g. Chinese, Japanese) and the ingredient sets which consist of the names, the food categories (e.g. Meat, Fruit), the flavor molecules the ingredient has. The statistics of datasets shows Table Before training models, the recipe set are divided into train, validation and test set. On the other hand, we generate triplets (Figure 7 from whole ingredient set in order to train TransF, 172,207 triplets for CulinaryDB and 40,952 triplets for Flavornet.
5.1 BASELINES
We compare our Implicit/Explici models to the following baselines.
FREQ:
Frequency predictor that always recommend the most frequently used ingredients of the training recipes. Despite its simplicity it is often a strong baseline in certain domains.
PMI: Recommender based the PMI score.
FFIDFFReeoomendee based on the TF-IDF score.
NMF (De Clereq et al. (2016)) Non-negative matrix factorization based recommender. The model is trained using the train and the validation recipes. It is implemented by our- selves.
NeuMF (He al. (017)): ): Neural matrix factorization model which is based on the Neural collaborative filtering framework. We use Ingredient Embedding layer and Ingredient Set Encoder (Section 4.1 as embedding layers of this model. And the training process is also same as ours (Section 4.5). More details of implementation can be found in section
A.5
WIDE&DEEP (Cheng et al. (016)): Wide & Deep model based recommender. We use a pre-selected ingredient set, a candidate ingredient, and attributes (Table 4 as inputs of this model. And the training process is same as ours (Section 4.5). More details of implemen- tation can be found in section A.5
5.2 RESULTS AND DISCUSSION
Predictive performance Table shows the results on all datasets for all compared baselines in Recipe Completion Task. While both our Implicit and Explicit models outperform all counterparts on all metrics usually with clear margin, our Explicit model is approximately same performance as the Implicit model. On the other hand, NMF the best baseline. Performance is good since the Recipe Completion task requires finding only one missing ingredient even though there should

---

Under review as a conference paper at ICLR 2021
Table 1: Experimental results on the Recipe Completion Task.
Datasets
CulinaryDR Mcan Rank
Flavornet Mean Rank
Mctrics
HR@ 10
HR@20
AUC
HR@10 10
HR@20
AUC
FREQ
473.5
0.119
0.140
0.605
257.9
0.131
0.153
0.621
PMI
612.5
0.055
0.056
0.527
346.9
0.066
0.069
0.531
TFIDF
478.5
0.055
0.055
0.598
261.3
0.055
0.089
0.613
NMF
57.0
0.435
0.559
0,900
36.3
0.479
0.599
0.89
NeuMF
37.5
0.400
0.530
0.943
35.7
0.332
0.509
0.898
WIDF&DEEP
43.5
0.350
0.478
0.929
37.7
0.351
0,493
0.896
[IMMiiilliiit
35.9
0.391
0.367
0.943
28.1
0,485
0.631
0.926
IRRM(Explicit)
35.4
0.397
0.575
0.944
28.3
0,480
0.629
0.925
Table 2: Experimental results on the Artificial Food Pairing Task (CulinaryDBB)
Pair type
one-lo-one
two-lu-one
three-lo-one
Metrics
MAP@I0
AUC
MAP@ T0
AUC
MAP@
AUC
NMF
0.638
0.702
0.604
0.786
0.601
0.794
NeuMF
0.841
0.905
0.745
0.923
0.747
0.919
WIDE&DEEP
0.787
0.919
0.705
0,930
0.726
0.926
W/O-RELENC
0.574
0.736
0.494
0.731
0,499
0.746
(mpl(ipl
0.839
0.904
0.755
0.921
0.792
0.925
(Rpp(Ex)licit
0.842
0.906
0,759
0.922
0.790
0.926
be multiple correct ingredients in the real situation. It appears that approaches effective for many-to-one pairs which are constrained by recipes.
the Artifi- cial Food Pairing Task. We report the results of one-to-one, two-to-one, and three-to-one. We sample positive examples and negative examples according three ratio settings like (Pos, _Ng) E {(20, 80), (50, 50), (80,20 20)}. Beforehand we classify all ingredients into two cat- egories: {Low, High} depending on the number of ingredient occurrences in recipes. Then, pre- selected ingredient sets and positive examples are randomly sampled from each category set to have the same ratio, and negative examples are randomly sampled from all candidates. We adopt this process since generally speaking low frequency ingredients are more difficult to handle than high frequency ingredients. the end, We prepare 4,536 tasks by pre-selected ingredient set size, where each task consists of 100 pairs.
In order to evaluate the pairing skills from one-to-one many-to-one,
As can seen Table our models outperform NMF on one-to-one, two-to-one, and three-to-one test sets. The proposed models show stable performance the number of pre-selected ingredients increases.
Here, we only report the results of the Explicit model since the Implicit model shows the same tendency. As expected, L-L seems to be the most difficult task for our models followed by H-L. This result is good since shows that our model does not overfit low frequency ingredients.
Since one of the main contributions of the paper is the Relation Encoder Model, We performed ablation study and evaluated version of model without it (W/O RELENC model). W/O RELENC has similar performance to NMF on the one-to-one Artificial Food Pairing task but fails on two-to-one and three-to-one coder module plays an important role in our models. For more detailed analysis, we compare Roc- curves for our models by ingredient frequency- -based pair (Fpgu(F 4., which consists of High Frequency-to-Hii Frequency(H-H) High Frequency-to-Lov High Frequency(L-H) and Low Frequency-tor to-woo Frequency(L-L), e.g. H-L describes pairs that pre-selected ingredient is used in various recipes frequently, while the paired ingredient is used in only few recipes.
sets in MAP@ 10. This clearly show that the Relation En-
Frequency(H-L), Low Frequency to-
Interpretablity We first analyzed attention weights in the Explicit model for some specific food pairs around chocolate (see Figure 5). The data shows that egg paired with chocolate because of correlations in food category. miso on the other hand has considerable flavor molecule related affinity to chocolate. This interpretation for eggs is consistent with the results reported by De Clereg al. (016). Next, we focus the attention weights for flavor molccules in trained Explicit model for a quantitative analysis. We compare the average Flavor Molecule attention weights with correlation coefficients which calculated full recipes. Details are in the caption of Table 3 In

---

Under review as a conference paper at ICLR 2021
(d) chocolate vs egg
Cusinee
Food Category
08
Tue Rostive Rate
0.7
Flavor Profie
0.6
0.
0.4
chocolate
HHH
H4
Cusine
Food Category
04 0.6
08
Flavor Profle
Figure 4: Comparison of Roc-curve by ingredient frequency-based pair type, which use the Explicit model on CulinaryDBB, one-to-one pairs. L. describes Low fre- quency, H describes High frequency.
Figure 5: Visualizations of atten- tion weights of our Explicit model on CulidnaryDB.
Table 3: Comparison of food ingredients sorted descending order scores. Left: ranked based on the average of Flavor Molecule attention weights in Explicit model. Right: ranked based on correlation coefficients between the number food ingredient co-occurrence in recipes and the shared flavor molecules number between food ingredients. Both scores calculated occurring ingredient pairs in recipes. Note that while the range of scores for our attentions weights is 1] and Freq the number of ingredient
[0, 1], the range of scores for correlation coefficients occurrences in whole recipes.
Our attention weights
Correlation coefficients
Rank
Name
Score
Freq
Name
Score
Freq
soy yogurt
1,000
2
true frog
0.730
4
2
wheaten bread
1,000
4
florida pompano
0.577
3
3
sandalwood
1.000
2
shellfish
0.510
7
common
1,000
palm
0.488
3
blackberry brandy
1.000
abalone
0.773
95
orange oil
0.403
8
wine
0.618
299
multigrain hread
0.358
8
8
sesame
0.603
1394
potato bread
0.356
7
gelatin
0.583
238
waffle
0.352
13
10
sherry
0.578
475
fruit juice
0.344
the ranking of attention weights, up to the top are high score ingredients that not seem to be flavor molecule correlated. It would appear that since the number of ingredient occurrences is very low, the relational representations are not learned properly. On the other hand, the top and below looks like fairly good result. In the ranking of correlation coefficients, there some ingredients like breads and waffle that seems to have little to do with flavor molecules. consequence the model is able to find hidden ingredients structures within recipes different from simple correlation coefficients.
CONCLUSION
We proposed interpretable models for food ingredient recommendation systems that rank ingredients based on the co-occurrence of ingredients in cooking recipes and learned relational representations. In our experiments, we found that explicitly and implicitly integrated factors can improve predic- tions, and our trained relational representations detect interesting correlations between ingredients and factors such molecular structure. Future work will focus controlling ranking scores based on manual modifications of relational representations by domain experts (e.g. chefs and food pro- fessionals).
8

---

Under review as a conference paper at ICLR 2021
REFERENCES
Y. Ahn, S. Ahnert, Bagrow, and A. Barabási. Flavor network and the principles food pairing. Sci Rep 196, 2011. URL http::/://bbb..rr///0000////////
Ganesh Bagler. Culinarydb. 2017. URL http:t::/oo///a...iiiic.
in htt//uuline intcuiiaruulybbbbbbbooooaa
Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Ok- sana Yakhnenko. Translating embeddings for modeling multi-relational data. In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger (eds.), Advances
Neural Information Processing Systems 26, pp- 2787-2795.
Curran Associates, Inc., 2013. URL bttp://papers: nips cc/paper/ 5071 1ev1--raae -rranaaaatinn ietereedinttneeeemmdddinnnnn embeddings for mode ing caa--multii mwwtim relatiiinnnn onal data.
pdf
Tien-TTze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Arad- hye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, Rohan Anil, Zakaria Haque, Lichan Hong, Vihan Jain, Xiaobing Liu, and Hemal Shah. Wide & deep learning for recom- mender systems. DLRS 2016, pp. 7-10, New York, NY, USA, 2016. Association for Com- puting Machinery. ISBN 978 9781450347995 doi: 10. 00.1145/988440029884555
URL https:
//dbi. 1145/2988450 2988454
Marlies De Clereq, Michiel Stock, Bernard De Baets, and Willem Waegeman. Data-driven recipe completion using machine learning methods. Trends in Food Science & Technology, 49:1-13, 03 2016. doi: t..0016/..iffs.2015...
Neelansh Garg, Apuroop Sethupathy, Rudraksh Tuwani, Rakhi NK, Shubham Dokania, Arvind Iyer, Ayushi Gupta, Shubhra Agrawal, Navjot Singh, Shubham Shukla, Kriti Kathuria, Rahul Badhwar, Rakesh Kanji, Anupam Jain, Avneet Kaur, Rashmi Nagpal, and Ganesh Bagler. FlavorDB: database of flavor molecules. Nucleic Acids Research, 46(D1):D210-D12116 10 2017. ISSN 0305-1048, doi: 10.1093//a///kxx55. URL URLLattps://Ce /ww. org 10t.:0.11199///aa/a.aaaa
Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. Neural collab- orative filtering. Proceedings of the 26th International Conference on World Wide Web, WWW '17, pp. 173-182, Republic and Canton of Geneva, CHE, 2017. International World Wide Web Conferences Steering Committee. ISBN 9781450349130. doi: 10. 10.1145530339912.005556 URL 3052569
https://doi... org/ 1141/3038183
A. Hermans, Lucas Beyer, and B. Leibe. In defense of the triplet loss for person re-identification. ArXiv, abs/1703.07737, 2017.
Jin Huang, Wayne Xin Zhao, Hongjian Dou, Ji-Rong Wen, and Edward Y. Chang. Improving sequential recommendation with knowledge-enhanced memory networks. In: The 41st Interna- tional ACM SIGIR Conference on Reseurch & Development Information Retrieval, pp. 505- 514.ACM, 2018.
Wang-Cheng Kang and Julian McAuley. Self-attentive sequential recommendation, 2018.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URI. htI.ht
//axxiv.org/ab//1220
6980
Alexander Miller, Adam Fisch, Jesse Dodge, Amir-Hossein Karimi, Antoine Bordes, and Jason
Weston. Key-value memory networks for directly reading documents. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 1400-1409, Austin, Texas, November 2016. Association for Computational Linguistics. doi: 10. 10.18653/v1/D16 1147. c. web. . org/ httpg//ahoollooo ogy 6-1147
URL wtt.hhttps////
Donghyeon Park, Keonwoo Kim, Yonggyu Park, Jungwoon Shin, and Jaewoo Kang. Kitchenette: Predicting and ranking food ingredient pairings using siamese neural network. In Proceedings of the Twenty-Eight International Joint Conference Artificial Intelligence, IJCAI-19, pp. 5930- 5936. International Joint Conferences on Artificial Intelligence Organization, 2019. doi: 10. 201.2.019991 992
htt/3//icai.2019//333 URL htt.https:://attt http://o....rr110.......3. http::/s//a...rr///0////////////

---

Under review as a conference paper at ICLR 2021
Massimo Quadrana, Alexandros Karatzoglou, Balázs Hidasi, and P. Cremonesi. Personalizing session-bars based recommendations with hierarchical recurrent neural networks. Proceedings of the Eleventh ACM Conference Recommender Systems, 2017.
Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, and Rob Fergus. End-to-end memory networks, 2015.
Yi Tay, Luu Anh Tuan, and Siu Cheung Hui. Latent relational metric learning via memory- tention for collaborative ranking. In Proceedings of the 2018 World Wide Web Conference (WWW "18). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland, pp. 729-772--77 739. 2018.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. arXiv preprint 2017.
Baocheng Wang and Wentao Cai. Knowledge-enhancee graph neural networks for sequential rec- ommendation. Information, 11(8), 2020. ISSN 2078-2489. doi: 10.3390/infol 1080388. URL
pestps://www.rr.
com ccom/200 7078 203222299111////2
F. Wang, J. Cheng, W. Liu, and H. Liu. Additive margin softmax for face verification. IEEE Signal Processing Letters, 25(7):926 930, 2018.
Liwei Wu, Shuqing Li, Cho-Jui Hsich, and James Sharpnack. Sse-pt: Sequential recommendation via personalized transformer. In Fourteenth ACM Conference on Recommender Systems, RecSys '20, pp. 328-337, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781450375832. doi: 00.1145/33833133341222
URL
33:331 3412258
Fuzheng Zhang, Nicholas Jing Yuan, Defu Lian, Xing Xie, and Wei-Ying Ma. Collaborative knowl- edge base embedding for recommender systems. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '16, pp. 353-362, New York, NY, USA, 2016. Association for Computing Machinery. ISBN 478145034232 doi: 01115//993967.22999677 URL httptttpp:///// org/10. 111111...111550 100.0.1112229988 2239673
A APPENDIX
A.I IMPLEMENTATION DETAILS AND HYPERPARAMETERS
We implemented our models in P on Linux machine. For tuning the hyperparameters, we select the model that performs best in validation set based on the precision metric. We report results on the test set. All models are optimized using the Adam optimizer (Kingma Ba 2015). and are trained until convergence, the maximum is 35 epochs. The dimensionality d and d is tuned amongst (64, 128, 200). the batch size Batch is tuned {16, 32, 64), the number of positive examples Pos is tuned 2,, 5, 10}. the number of negative examples tuned 2, 5, 10), the learning rate tuned (0.1, 0.01, 0.001]. the margin is tuned (0.1, 0.2, 0.25, 0.5). the temperature is tuned (0.5. 10). Since CulinaryDB has 4 attribute types and Flavornet has attribute types, we use the memory size NCulsnrryDB = and N Nitavornet = 3 respectively in our Implici//Expliit models.
For most datasets, we found the following hyperparameters work well: d = KB 128, Batch = 64, Pos 1 is for Recipe Completion Task, Pos = 2 is for Artificial Food Pairing Task, Neg 5. learning rate = 0.001, À 0.2, and - = 1. Ingredients with less than I occurrence in the train set learned as the UNK ingredient.
httss//pytorch..or/
10

---

Under review as a conference paper at ICLR 2021
INGREDIENT SET ENCODER A.2
oooooo
Ingredient mbodding
Σ
FF Nor
DO DOOO0O
Sum
Ingredient Embedding ng Layer
Figure 6: Ingredient Set Encoder
A.3 TRIPLETS FOR INGREDIENTS
E6 Epwetsden
Apple
calegery
aeple
Tavor molecule
(+>Deta-Cadinene Cadinone
Apple
fevor molecule
F-S skatechin
Dlsor molccute
color
red
aprico:
Tavor malecule
[n-Dela-Cadiien
categery
fruit
truit
Figure 7: Triplets for Ingredients
A.4 DATASET STATISTICS
Table 4: Dataset Statistics
Datasets
Recipe Num
Ing./Molecule Num
Attribute Types
45,772
cuisine category, food category, flavor molecules, flavor profile(e g. Bitter) cuisine category,
CulinaryDB
(train+valid: 41,231, test: 4,541)
658/1,788
35,001
Flavomet
(train+valid: 49,501, 5,500)
381/1,021
food category. flavor molecules
A.5 THE IMPLEMENTATION DETAILS OF NEUMF AND WIDE& DEEP
We implemented both the NeuMF model and Wide&Deep model from scratch in PyToret
We trained the NeuMF model without pre-training. The size of predictive factors is 8, then the architecture of the neural layers is 32 + 16 + 8, and the embedding size 16.
In the Wide&Deep model, the wide component consists of the cross-product transformation of pre- selected ingredient sets and candidate ingredients. For the deep part of the model, a 16-dimensional
hhupss//pyorchh.org/
11

---

Under review as a conference paper at ICLR 2021
embedding vector is learned for each categorical feature. We concatenate all the embeddings to- gether with the dense features, resulting in dense vector of approximately 80 dimensions.
A.6 EXAMPLE RANKING RESULTS ON THE CULINARYDB
Table Example Ranking Results on the CulinaryDB
Rank
Apple
Chocolate
Apple and Chocolate
canola
almond banana
canola
honey mint
orange
3
orange
almond
4
orange
cream
honey
5
cinnamon
raisin
raisin
6
white bread
cream cheese
mint
nutmeg
symup
cream
8
lime
mint
9
rice
canola oil
lemon juice
corn 10
cinnamon
12


Reviews:
[{"review_id": "wx_asc3GUk2", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Reject.\nSummary\nThis paper works on the problem of create new recipes. It uses recommendation approaches on it. The paper use two approaches for explainability and conducted experiments on two real-world datasets.\n\nStregthens\n1. The explainability in recommendation is an important research problem.\n2. The recommendation for food pairing is reasonable.\n3. This paper demostrate the proposed approach on two real-world datasets.\n\n\nWeakness\n1. This paper's writing can be greatly improved. It needs proofwriting. There are a lot of typos and also unfinished sentences. (See minors for details)\n2. The novelty of this paper is low. Most of the techniques are not proposed in this paper. For example the memory network in already used for explainability in recommendation system. [Huang 2018]. \n3. The compared approaches are very simple and outdated. There is no related work section of position this paper in the literature. There are many advanced recommendation approaches. The authors can do a better literature Survey. A few example below\n- He, Xiangnan, et al. \"Neural collaborative filtering.\" Proceedings of the 26th international conference on world wide web. 2017.\n- Cheng, Heng-Tze, et al. \"Wide & deep learning for recommender systems.\" Proceedings of the 1st workshop on deep learning for recommender systems. 2016.\n4. The paper mentions \"While both our Implicit and Explicit models outperform all counter parts on all metrics usually with a clear margin\". However, from figure 1, NMF is the best performing one on HR@10 on CulinaryDB.\n\n\nMinors:\nIn introduction line 2: \"food preference modelling\", -> modeling,\nAll the \"paring\" should be \"pairing\"\nRight above Section 2. Missing peorid at the end of the sentence.\n\"At the embedding layer,this\", add space after comma\n\"In this task we\" => In this task, we\n\"eventhough\" => even though\n\"This result is some good \" need to change.\n\nQuestions\n\"where the only concern is whether a user has interacted with an item and the system ...\" Why this is a concern? Is it an approach?\nFor section 3.5, it has a three steps for generating a mini batch.It mentions sample additional postiive examples as necessary. Why sampling addtional positive examples are needed?\n\"Note that although we also trained our models without the recipe restriction, \" What does it mean by without recipe restrictions?\nThe \"Artificial Food Pairing Task\" is a bit confusing. Does a pair mean two ingredients or mean one ingredient set and one ingredient?\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "nnMaIcPjDZ", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Overall, I vote for accepting. The authors propose a novel approach to support chefs with creating/experimenting with new recipes to overcome the challenging combinations of taste/texture etc. that can result from addition of new ingredients. Their idea of adding interpretability to their results from a Food Knowledge Base has a lot of appeal, especially to end-user’s (chef’s in this case). The author’s proposed solution is well supported by the robust and detailed evaluation results. The concerns detailed in the cons section is mainly related to the readability and clarity of the paper.\n\n######################################################################\n\nPros:\n(1) The model architecture for both implicit and explicit use cases in the paper has a strong appeal with solid foundation.\n(2) The objective function is well defined and the reasoning for changing the score function from the original paper’s LRML is well reasoned and convincing.\n(3) The proposed model’s efficacy is well supported by empirical experimentation on two large food datasets and good comparisons with established baseline methods.\n(4) The motivation for adding KB based on ingredient attributes is well reasoned and gives more ability to understand which ingredient attribute contributed to the prediction result.\n\nCons:\n(1) The readability of the paper can be improved. The authors have used a lot of prior work from different authors as a basis for many critical components of their architecture. But the explainability and the clear motivation behind using the related work is lacking. The authors could have given a brief summary of previous work they are using as critical components of their solution.\n(2) For the experiments trying to measure interpretability it requires domain knowledge and understanding of Food Science. For example, Table 3 results which are used to illustrate how the explicit model can help with interpretability requires understanding of the training dataset beyond general statistics. Why are the ingredients ranked on attention weights in Table 3 better related to flavor molecules? But it would probably really appeal to someone who has knowledge pertaining to them.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "88PQ9ekLtnT", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper tackles ingredient recommender systems problem. This paper proposes the Interpretable Relational Representation Model (IRRM) to achieve both usefulness and interpretableness. There are two variants of the model, first is to model latent relation between two ingredients, the second is to leverage external knowledge base and results from TransE to learn relational representations.\n\nThe problem setting is interesting: using ingredient recommender systems to help chefs to create better or more creative ingredient combinations. This topic relatively recessives little attention in recommender system community but it seems worth a while pursuing this direction. \n\nI like the way the paper models the problem but there are several technical components are too important to miss:\n\n1. The baseline methods chosen are too simple to be compared against. FREQ, PMI and TFIDF are simple rule-based methods and NMF is the only commonly used method in recommender system but matrix factorization is too weak a baseline given that the proposed method is a neural network approach. This is also related to second point.\n\n2. Connections of this problem to Sequential Recommendation is largely missing. It seems to me this problem is actually closed related to sequential or session-based recommendations. Because the problem can be formulated as given a list of ingredients the recipe has interacted with, what is the next ingredient that this particular recipe is most likely to engage next? Here ingredient is the item and recipe is the user in sequential recommendation, so all the methods that have been developed for sequential recommendation can be used for this problem. Worth noting methods in this category is 2018 SASRec paper (https://arxiv.org/abs/1808.09781) and 2020 SSE-PT paper (https://dl.acm.org/doi/abs/10.1145/3383313.3412258).\n\n3. Many ablation studies are missing. Going over the entire paper, I am not entirely sure what components are most important to the good performance of the model. Many neural network design choices seem too arbitrary to me, e.g. why we need to add p and q in Figure 1. A rigorous approach should be able to convince readers that each choice of model design is the best among all alternatives and without it, the performance would suffer.\n\n4. Regarding qualitative study, I was just wondering if it is possible to ask crowd-sourcing chefs to rate some newly created recipes for unseen recipes. This way the offline results would be convincing because we are not quite sure that better offline ranking results would necessarily lead to better online performance for this particular problem.", "extra_scores": {}, "extra_text": {}}, {"review_id": "5L9tBzINyVP", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper studies a promising task of interpretable food ingredients recommendation - there has been a growing interest in modeling recipes. The idea of leveraging KG to improve the interpretability/faithfulness of recipe-related ML tasks seems like a contribution to the community. In particular, the author proposes a method to learn pair specific relational representations for one-to-one (i.e. ingredient to ingredient) and many-to-one (ingredient-set to ingredient) food pairing tasks.\n\nPros:\n\nThe task itself is an interesting application; meanwhile, the task is non-trivial as the ingredient pairing is complicated and affected by various factors. \n\nIt uses recipes instead of interaction history to recommend complementary ingredients. \n\nIt proposes a method based on the memory network. In particular, it first embeds the preselected ingredient set vector and a candidate ingredient vector, then sums them and feeds into the memory network. The output of the memory network is called the relational vector, which is added with the input embedding and then put into a scoring function. The training follows the standard ranking problem which aims to optimize the triplet loss.\n\nKB triples of (ingredient, attribute, attribute value) are further represented as external KB embeddings to augment the memory network. \n\nConduct a qualitative analysis of the attention weights on the KB embedding and show that IRRM is able to capture some level of relation between ingredients.\n\n\nCons:\n\nThe experimental section seems to miss important baselines models. Most of the baseline models are non-neural network-based methods. Also, they are mostly based on interaction history. It would be convincing to adding several neural model baselines that use recipe text as training inputs. Otherwise, it's hard to see which component brings improvement. \n\nThe author proposes two new evaluation tasks to show the model's performance - both of them fall into a category of ingredients completion. The tasks seem a bit simple and less beneficial in a real scenario. There are more practical and challenging alternatives that could be used for better evaluation. For example, predict the complementary ingredients given the recipe name or recipe steps. \n\nThe improvement of IRRM on Hit@10 on the Recipe Completion Task seems marginal (i.e. it even underperforms NMF). Is there any reason? \n\nOne other concern is that - most contribution comes from the novel task. It may have a limited scope of the audience at ICLR as it's more like an industrial track application paper, though from the application perspective the paper may be impactful. Other top venues in the field of data mining and recommender systems seem like a better fit.\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: There is a broad consensus that this paper explores an interesting and novel problem space. Nonetheless, in their initial assessment, the reviewers pointed to a few limitations of the paper including lack of strong baselines, lack of an ablation study, and weaker results according to the HIT@10 metric. 

The authors provided an improved version of their paper as a response. The new paper added two baselines, is better written, and justifies some of the HIT@10 results (basically, the metric is biased for this task).

After discussion, the reviewers find that the contribution of the current manuscript falls short of the acceptance threshold.  In particular, the reviewers find that: 1) this contribution is for a specific domain of recommender systems, an area of interest, but perhaps only relevant to a subset of the ICLR community; 2) while more recent baselines helped, there has been lots of more recent work on collaborative filtering models for recommender systems over the last few years (the Wide&Deep baseline is from 2016); 3) since some of the usual recommender systems' metric does not seem appropriate here, why not suggest new ones (or propose a slightly different evaluation protocol); 4) the proposed model is useful, but somewhat incremental given prior work. All in all, while any of these limitations on their own might not have been sufficient to warrant rejection, I find that their combination does. 

Given the interest in this new task, I do strongly encourage the authors to pursue their work. I also find that the qualitative study propsoed by Reviewer 4 could add another interesting angle to this paper (I also imagine that it might not be that easy to carry out). 
Decision: Reject