Interpretable Relational Representations for Food Ingredient Recommendation Systems

1 INTRODUCTION . Data mining and machine learning methods play an increasingly prominent role in food preference modeling , food ingredient pairing discovery and new recipe generation . Solving these tasks is nontrivial , since the goodness of ingredient combinations depends on many factors like taste , smell , cuisine , texture , and culture . Ahn et al . ( 2011 ) detected that the number of shared flavor molecules between ingredients is one of important factors for food pairing . They found Western cuisines show a tendency to use ingredient pairs that share many flavor compounds , while East Asian cuisines tend to avoid compound sharing ingredients . Using this idea , Garg et al . ( 2017 ) developed a rule-based food pairing system which ranks ingredients based on the number of shares of flavor molecules . Recently , Park et al . ( 2019 ) suggested a neural network approach based on flavor molecules and co-occurrence of ingredients in recipes . These approaches focus on one-to-one food pairing . There is also research related to many-to-one pairing . De Clercq et al . ( 2016 ) proposed the Recipe Completion Task which tries to identify matching ingredients for a partial list of ingredients ( the recipe ) using a Matrix Factorization based recommender system . Although efforts have been made to detect good ingredient combinations , there is no current Machine Learning method in this field that allows to interpret why suggested pairs are good . Our work is targeted at interpretable recommendation systems for food pairing and recipe completion . Given a set of pre-selected ingredients ( cardinality 1 or more ) by a user , the recommender suggests top-N ingredients from a set of candidates . For example , suppose a user selects apple and chocolate as the pre-selected ingredients , our recommender suggests some good paired ingredients ( e.g . cinnamon ) and also identifies reasons ( e.g . cinnamon is good for apple and chocolate in terms of their flavor affinity ) . For this , we propose the Interpretable Relational Representations Model ( IRRM ) in two variants to address food pairing and recipe completion tasks . The model features a key-value memory network ( Sukhbaatar et al . ( 2015 ) , Miller et al . ( 2016 ) ) to represent relationships of ingredients . One variant of the model is trained to learn latent relational representations over a trainable memory network ( Implicit Model ) . The other model can learn explainable relational representations over the pretrained memory network integrating an external knowledge base ( Explicit Model ) . The relational representations are interpretable and can be queried as to the reasons why the ingredients have been suggested . The Explicit model can integrate any number of constraints which can be decided manually based on the characteristics of the desired recommender system . Our contributions are as follows : 1 . We model ingredient pairing as a general recommendation task with implicit feedback . 2 . We introduce the Interpretable Relational Representations Model and it ’ s two variants : Implicit and Explicit . Both of which can learn pair specific relational representations ( vectors ) for one-to-one ( i.e . ingredient to ingredient ) and many-to-one ( ingredient-set to ingredient ) food pairing tasks . The relational vectors are also interpretable . 3 . We propose a training procedure to learn one-to-one and many-to-one relationships effectively using recipes . 4 . We evaluate our proposed models in the Recipe Completion Task and the Artificial Food pairing Task on the CulinaryDB and the Flavornet datasets . Our proposed approaches demonstrate competitive results on all datasets , outperforming many other baselines . 5 . We perform qualitative analysis . The results presents our proposed Explicit model is capable of unraveling hidden ingredients structures within recipes . 2 RELATED WORK . There are two related streams of work in recommender systems that are important for this paper : the session-based setting and the knowledge-aware systems . In the session-based setting , user profile can be constructed from past user behavior . A natural solution to this problem is the item-to-item recommendation approach.A variety of methods exist for this problem . For example , Quadrana et al . ( 2017 ) models the item sequence using RNNs , Kang & McAuley ( 2018 ) uses Self-Attention layers , and Wu et al . ( 2020 ) uses Transformer layers . While these methods mainly focus on how to encode item click-sequence interactions , we target good ingredient pairing using only ingredient attributes and the relationship between a ingredient set and an ingredient based on co-occurrence in recipes . For this we develop a new architecture integrating set encoders and relational memory with novel loss and score functions . There are also increasingly methods for integrating knowledge into recommenders . Zhang et al . ( 2016 ) and Cheng et al . ( 2016 ) directly incorporate user and item features as user profile into neural network models . Huang et al . ( 2018 ) and Wang & Cai ( 2020 ) integrate them using a pre-trained knowledge graph . These methods try to represent user context using external knowledge base , therefore , usually these knowledge embeddings are integrated to user embeddings . In this work , we incorporate knowledge specifically to detect relationships between an ingredient set and an ingredient for interpretation to improve recommendation performance . 3 PROBLEM DEFINITION . We first introduce the notations used throughout this paper . We model recipe completion as a recommendation scenario with implicit feedback ( Huang et al. , 2018 , Tay et al. , 2018 . In such scenarios , a user has interacted with an item and the system infers the item that user will interact next based on the interaction records of the user . We apply this to the food domain by using recipes as interaction records . Let I denote a set of ingredients and { i1 , . . . , iM } denote a pre-selected ingredient set , where i ∈ I is the ingredient and M is the number of ingredients . We call { i1 , . . . , iM } pre-selected ingredient set in this paper . Next , let Icandidate denotes a set of candidate ingredients . Icandidate depends on each pre-selected ingredient set , that is , Icandidate = I − { i1 , . . . , iM } . In addition , we assume that a knowledge base ( KB ) of ingredients is also available and the KB contains factors which are related to why some ingredients are good combinations . A KB is defined as a set of triplets over a entity set E and a relationship set L. A KB triplet 〈ei , l , ea〉 composed of two entities ei , ea ∈ E and a relationship l ∈ L , where ei is an ingredient ( e.i . ei ∈ I ) and l is an attribute and ea is the attribute value . For instance , 〈apple , flavorMolecule , ( - ) -Epicatechin〉 denotes that apple contains the ( - ) -Epicatechin flavor molecule . Based on these preliminaries , we define the food ingredient recommendation task . Given a preselected ingredient set { i1 , . . . , iM } and candidate ingredients Icandidate , we would like to infer the top-N ingredients from Icandidate . 4 RECOMMENDATIONS WITH MEMORY NETWORKS . In this section , we introduce the IRRM architectures . We start with the Implicit model that consists of a trainable key-value memory network . We then augment the Implicit model using a key-value memory network which integrates pre-trained entity and relationship vectors with ingredient attributes in the KBs – we call this extention the Explicit model . The overall architecture is described in Figure 1 . The input of our architecture are a pre-selected ingredient set and a candidate ingredient icandidate ∈ Icandidate . The output is a score . In inference , our recommender uses these scores to rank Icandidate . 4.1 INGREDIENT EMBEDDING LAYER AND INGREDIENT SET ENCODER . Ingredients are represented as one-hot encoding vectors ( corresponding to a unique index key belonging to each ingredient ) . At the embedding layer , this one-hot encoded vector is converted into a low-dimensional real-valued dense vector representation which is multiplied with the embedding matrices Q ∈ Rd×|I| – which stores ingredient embeddings . d is the dimensionality of the ingredient embeddings while |I| is the total number of ingredients . icandidate is converted to q using this embedding layer . On the other hand , pre-selected ingredients { i1 , . . . , iM } are encoded by the Ingredient Set Encoder ( Figure 6 ) . At first , each ingredient ij is converted to a vector using the Ingredient Embedding Layer ( same as icandidate ) . As a result , ij ∈ Rd vectors are generated . The sum of these vectors is converted to the ingredient set vector p using a feed-forward network with a single hidden layer , followed by Layer Normalization . 4.2 RELATION ENCODER . Tay et al . ( 2018 ) introduced LRAM ( Latent Relational Attentive Memory ) , in order , to generate latent relational vectors between user-item interactions . We expand this module by adding a residual connection , followed by Layer Normalization . This idea is inspired by Vaswani et al . ( 2017 ) . Given the pair of a pre-selected ingredient set vector and a candidate ingredient vector , 〈p , q〉 , the Relation Encoder first applies s = p+ q to generate the joint embedding of p and q . The generated vector s ∈ Rd is of the same dimension of p and q . Note we also tried other transfer functions here such as element-wise multiplication or just using a multi-layered perceptron MLP ( p , q ) . However , we found that addition performs best . This joint embedding s is used as the input of the memory network . The attention vector a ∈ Rd is a vector of importance weights over keys which are represented as the key matrix K = [ k1 , . . . , kN ] T ∈ RN×d , where N is the number of key-value pairs in the memory network and kj ∈ Rd is a key vector . Each element of the attention vector a can be defined as aj = sTkj , where aj ∈ R. In order to normalize the attention vector a to a probability distribution , we use the Softmax function : Softmax ( aj ) = exp ( aj ) ∑N n=1 exp ( an ) . We generate the vector m = ∑N n=1 Softmax ( an ) vn as the summation of weighted value vectors which are represented as the value matrix V = [ v1 , . . . , vN ] T ∈ RN×d . Finally , in order to generate the relational vector – r , m is added with the joint embedding s and Layer Normalization is applied as follows r = LayerNorm ( s+m ) . 4.2.1 THE EXPLICIT MODEL . In order to improve interpretability and predictive performance , we incorporate ingredient attribute information from a given KB into the memory network . Inspired by recent works which integrate a memory network with external memories ( Huang et al . ( 2018 ) ) , we propose the Explicit Relational Encoder . Instead of the trainable key matrix K and value matrix V , we pre-train vectors over a given KB . We then freeze key and value matrix for training the explicit model . Given a pair of a preselected ingredient set { i1 , . . . , iM } and a candidate ingredient icandidate , { i1 , . . . , iM , icandidate } is converted into the entity vectors using the KB embeddings which provide the entity vectors e ∈ RdKB and the relationship vectors l ∈ RdKB . Note that in case of dKB 6= d , we convert the joint embedding s ∈ Rd into s′ ∈ RdKB and the relational vector r ∈ RdKB into r′ ∈ Rd with linear projections . We use the TransE ( Bordes et al . ( 2013 ) ) for the KB embeddings . The reason for this choice is that given triplet 〈ei , latt , eiatt〉 , TransE can learn entity vectors and relationship vectors to follow eiatt = ei+ latt . KB relationships usually correspond to attribute types of entities , so we use the notation latt as the attribute type and eiatt as its value . Hence , we set the key matrix as follows : K = [ latt1 , . . . , lattN ] T ( 1 ) where N depends on the number of attribute types which you want to integrate and K is constant through training . The value matrix is initialized as follows : vattj = ∑ i∈ { i1 , ... , iM , icandidate } eiattj = ∑ i∈ { i1 , ... , iM , icandidate } ( ei + lattj ) ( 2 ) V = [ vatt1 , . . . , vattN ] T ( 3 ) There can be many one-to-multiple relations in the KB . For instance , an apple has multiple flavor molecules . Therefore , the entity vector eatt should not be an ingredient specific vector and we use ei + latt instead of using eatt .

Summary: This paper tackles ingredient recommender systems problem. This paper proposes the Interpretable Relational Representation Model (IRRM) to achieve both usefulness and interpretableness. There are two variants of the model, first is to model latent relation between two ingredients, the second is to leverage external knowledge base and results from TransE to learn relational representations.