Interpretable Relational Representations for Food Ingredient Recommendation Systems

1 INTRODUCTION . Data mining and machine learning methods play an increasingly prominent role in food preference modeling , food ingredient pairing discovery and new recipe generation . Solving these tasks is nontrivial , since the goodness of ingredient combinations depends on many factors like taste , smell , cuisine , texture , and culture . Ahn et al . ( 2011 ) detected that the number of shared flavor molecules between ingredients is one of important factors for food pairing . They found Western cuisines show a tendency to use ingredient pairs that share many flavor compounds , while East Asian cuisines tend to avoid compound sharing ingredients . Using this idea , Garg et al . ( 2017 ) developed a rule-based food pairing system which ranks ingredients based on the number of shares of flavor molecules . Recently , Park et al . ( 2019 ) suggested a neural network approach based on flavor molecules and co-occurrence of ingredients in recipes . These approaches focus on one-to-one food pairing . There is also research related to many-to-one pairing . De Clercq et al . ( 2016 ) proposed the Recipe Completion Task which tries to identify matching ingredients for a partial list of ingredients ( the recipe ) using a Matrix Factorization based recommender system . Although efforts have been made to detect good ingredient combinations , there is no current Machine Learning method in this field that allows to interpret why suggested pairs are good . Our work is targeted at interpretable recommendation systems for food pairing and recipe completion . Given a set of pre-selected ingredients ( cardinality 1 or more ) by a user , the recommender suggests top-N ingredients from a set of candidates . For example , suppose a user selects apple and chocolate as the pre-selected ingredients , our recommender suggests some good paired ingredients ( e.g . cinnamon ) and also identifies reasons ( e.g . cinnamon is good for apple and chocolate in terms of their flavor affinity ) . For this , we propose the Interpretable Relational Representations Model ( IRRM ) in two variants to address food pairing and recipe completion tasks . The model features a key-value memory network ( Sukhbaatar et al . ( 2015 ) , Miller et al . ( 2016 ) ) to represent relationships of ingredients . One variant of the model is trained to learn latent relational representations over a trainable memory network ( Implicit Model ) . The other model can learn explainable relational representations over the pretrained memory network integrating an external knowledge base ( Explicit Model ) . The relational representations are interpretable and can be queried as to the reasons why the ingredients have been suggested . The Explicit model can integrate any number of constraints which can be decided manually based on the characteristics of the desired recommender system . Our contributions are as follows : 1 . We model ingredient pairing as a general recommendation task with implicit feedback . 2 . We introduce the Interpretable Relational Representations Model and it ’ s two variants : Implicit and Explicit . Both of which can learn pair specific relational representations ( vectors ) for one-to-one ( i.e . ingredient to ingredient ) and many-to-one ( ingredient-set to ingredient ) food pairing tasks . The relational vectors are also interpretable . 3 . We propose a training procedure to learn one-to-one and many-to-one relationships effectively using recipes . 4 . We evaluate our proposed models in the Recipe Completion Task and the Artificial Food pairing Task on the CulinaryDB and the Flavornet datasets . Our proposed approaches demonstrate competitive results on all datasets , outperforming many other baselines . 5 . We perform qualitative analysis . The results presents our proposed Explicit model is capable of unraveling hidden ingredients structures within recipes . 2 RELATED WORK . There are two related streams of work in recommender systems that are important for this paper : the session-based setting and the knowledge-aware systems . In the session-based setting , user profile can be constructed from past user behavior . A natural solution to this problem is the item-to-item recommendation approach.A variety of methods exist for this problem . For example , Quadrana et al . ( 2017 ) models the item sequence using RNNs , Kang & McAuley ( 2018 ) uses Self-Attention layers , and Wu et al . ( 2020 ) uses Transformer layers . While these methods mainly focus on how to encode item click-sequence interactions , we target good ingredient pairing using only ingredient attributes and the relationship between a ingredient set and an ingredient based on co-occurrence in recipes . For this we develop a new architecture integrating set encoders and relational memory with novel loss and score functions . There are also increasingly methods for integrating knowledge into recommenders . Zhang et al . ( 2016 ) and Cheng et al . ( 2016 ) directly incorporate user and item features as user profile into neural network models . Huang et al . ( 2018 ) and Wang & Cai ( 2020 ) integrate them using a pre-trained knowledge graph . These methods try to represent user context using external knowledge base , therefore , usually these knowledge embeddings are integrated to user embeddings . In this work , we incorporate knowledge specifically to detect relationships between an ingredient set and an ingredient for interpretation to improve recommendation performance . 3 PROBLEM DEFINITION . We first introduce the notations used throughout this paper . We model recipe completion as a recommendation scenario with implicit feedback ( Huang et al. , 2018 , Tay et al. , 2018 . In such scenarios , a user has interacted with an item and the system infers the item that user will interact next based on the interaction records of the user . We apply this to the food domain by using recipes as interaction records . Let I denote a set of ingredients and { i1 , . . . , iM } denote a pre-selected ingredient set , where i ∈ I is the ingredient and M is the number of ingredients . We call { i1 , . . . , iM } pre-selected ingredient set in this paper . Next , let Icandidate denotes a set of candidate ingredients . Icandidate depends on each pre-selected ingredient set , that is , Icandidate = I − { i1 , . . . , iM } . In addition , we assume that a knowledge base ( KB ) of ingredients is also available and the KB contains factors which are related to why some ingredients are good combinations . A KB is defined as a set of triplets over a entity set E and a relationship set L. A KB triplet 〈ei , l , ea〉 composed of two entities ei , ea ∈ E and a relationship l ∈ L , where ei is an ingredient ( e.i . ei ∈ I ) and l is an attribute and ea is the attribute value . For instance , 〈apple , flavorMolecule , ( - ) -Epicatechin〉 denotes that apple contains the ( - ) -Epicatechin flavor molecule . Based on these preliminaries , we define the food ingredient recommendation task . Given a preselected ingredient set { i1 , . . . , iM } and candidate ingredients Icandidate , we would like to infer the top-N ingredients from Icandidate . 4 RECOMMENDATIONS WITH MEMORY NETWORKS . In this section , we introduce the IRRM architectures . We start with the Implicit model that consists of a trainable key-value memory network . We then augment the Implicit model using a key-value memory network which integrates pre-trained entity and relationship vectors with ingredient attributes in the KBs – we call this extention the Explicit model . The overall architecture is described in Figure 1 . The input of our architecture are a pre-selected ingredient set and a candidate ingredient icandidate ∈ Icandidate . The output is a score . In inference , our recommender uses these scores to rank Icandidate . 4.1 INGREDIENT EMBEDDING LAYER AND INGREDIENT SET ENCODER . Ingredients are represented as one-hot encoding vectors ( corresponding to a unique index key belonging to each ingredient ) . At the embedding layer , this one-hot encoded vector is converted into a low-dimensional real-valued dense vector representation which is multiplied with the embedding matrices Q ∈ Rd×|I| – which stores ingredient embeddings . d is the dimensionality of the ingredient embeddings while |I| is the total number of ingredients . icandidate is converted to q using this embedding layer . On the other hand , pre-selected ingredients { i1 , . . . , iM } are encoded by the Ingredient Set Encoder ( Figure 6 ) . At first , each ingredient ij is converted to a vector using the Ingredient Embedding Layer ( same as icandidate ) . As a result , ij ∈ Rd vectors are generated . The sum of these vectors is converted to the ingredient set vector p using a feed-forward network with a single hidden layer , followed by Layer Normalization . 4.2 RELATION ENCODER . Tay et al . ( 2018 ) introduced LRAM ( Latent Relational Attentive Memory ) , in order , to generate latent relational vectors between user-item interactions . We expand this module by adding a residual connection , followed by Layer Normalization . This idea is inspired by Vaswani et al . ( 2017 ) . Given the pair of a pre-selected ingredient set vector and a candidate ingredient vector , 〈p , q〉 , the Relation Encoder first applies s = p+ q to generate the joint embedding of p and q . The generated vector s ∈ Rd is of the same dimension of p and q . Note we also tried other transfer functions here such as element-wise multiplication or just using a multi-layered perceptron MLP ( p , q ) . However , we found that addition performs best . This joint embedding s is used as the input of the memory network . The attention vector a ∈ Rd is a vector of importance weights over keys which are represented as the key matrix K = [ k1 , . . . , kN ] T ∈ RN×d , where N is the number of key-value pairs in the memory network and kj ∈ Rd is a key vector . Each element of the attention vector a can be defined as aj = sTkj , where aj ∈ R. In order to normalize the attention vector a to a probability distribution , we use the Softmax function : Softmax ( aj ) = exp ( aj ) ∑N n=1 exp ( an ) . We generate the vector m = ∑N n=1 Softmax ( an ) vn as the summation of weighted value vectors which are represented as the value matrix V = [ v1 , . . . , vN ] T ∈ RN×d . Finally , in order to generate the relational vector – r , m is added with the joint embedding s and Layer Normalization is applied as follows r = LayerNorm ( s+m ) . 4.2.1 THE EXPLICIT MODEL . In order to improve interpretability and predictive performance , we incorporate ingredient attribute information from a given KB into the memory network . Inspired by recent works which integrate a memory network with external memories ( Huang et al . ( 2018 ) ) , we propose the Explicit Relational Encoder . Instead of the trainable key matrix K and value matrix V , we pre-train vectors over a given KB . We then freeze key and value matrix for training the explicit model . Given a pair of a preselected ingredient set { i1 , . . . , iM } and a candidate ingredient icandidate , { i1 , . . . , iM , icandidate } is converted into the entity vectors using the KB embeddings which provide the entity vectors e ∈ RdKB and the relationship vectors l ∈ RdKB . Note that in case of dKB 6= d , we convert the joint embedding s ∈ Rd into s′ ∈ RdKB and the relational vector r ∈ RdKB into r′ ∈ Rd with linear projections . We use the TransE ( Bordes et al . ( 2013 ) ) for the KB embeddings . The reason for this choice is that given triplet 〈ei , latt , eiatt〉 , TransE can learn entity vectors and relationship vectors to follow eiatt = ei+ latt . KB relationships usually correspond to attribute types of entities , so we use the notation latt as the attribute type and eiatt as its value . Hence , we set the key matrix as follows : K = [ latt1 , . . . , lattN ] T ( 1 ) where N depends on the number of attribute types which you want to integrate and K is constant through training . The value matrix is initialized as follows : vattj = ∑ i∈ { i1 , ... , iM , icandidate } eiattj = ∑ i∈ { i1 , ... , iM , icandidate } ( ei + lattj ) ( 2 ) V = [ vatt1 , . . . , vattN ] T ( 3 ) There can be many one-to-multiple relations in the KB . For instance , an apple has multiple flavor molecules . Therefore , the entity vector eatt should not be an ingredient specific vector and we use ei + latt instead of using eatt .

Summary: The paper studies a promising task of interpretable food ingredients recommendation - there has been a growing interest in modeling recipes. The idea of leveraging KG to improve the interpretability/faithfulness of recipe-related ML tasks seems like a contribution to the community. In particular, the author proposes a method to learn pair specific relational representations for one-to-one (i.e. ingredient to ingredient) and many-to-one (ingredient-set to ingredient) food pairing tasks.