Food Knowledge Representation Learning with Adversarial Substitution (2022)

Abstract: Knowledge graph embedding (KGE) has been well-studied in general domains, but has not been examined for food computing. To fill this gap, we perform knowledge representation learning over a food knowledge graph (KG). We employ a pre-trained language model to encode entities and relations, thus emphasizing contextual information in food KGs. The model is trained on two tasks – predicting a masked entity from a given triple from the KG and predicting the plausibility of a triple. Analysis of food substitutions helps in dietary choices for enabling healthier eating behaviors. Previous work in food substitutions mainly focuses on semantic similarity while ignoring the context. It is also hard to evaluate the substitutions due to the lack of an adequate validation set, and further, the evaluation is subjective based on perceived purpose. To tackle this problem, we propose a collection of adversarial sample generation strategies for different food substitutions over our learnt KGE. We propose multiple strategies to generate high quality context-aware recipe and ingredient substitutions and also provide generalized ingredient substitutions to meet different user needs. The effectiveness and efficiency of the proposed knowledge graph learning method and the following attack strategies are verified by extensive evaluations on a large-scale food KG.

## introduction structured knowledge furnishes an in-depth understanding of the world. knowledge graph embedding (kge) maps entities and relations into vectors while retaining their semantics @xcite . kge has been well-studied and applied in general kgs with common ontological knowledge (i.e., wordnet @xcite , dbpedia @xcite , and freebase @xcite ). only a few works have targeted * diya li is the corresponding author. as for encoding models in kge, most deep learning-based methods like convolutional neural networks (cnn) @xcite , recurrent neural networks (rnn) @xcite and graph neural networks (gnn) @xcite allow a single static embedding for each entity or relation to describe its global meaning in a given kg. however, their intrinsic contextual nature is ignored, i.e., entities and relations may appear in different graph contexts and exhibit different properties. transformerbased models @xcite have boosted contextualized text representation learning. thus, to emphasize the contextual information in knowledge graphs, we employ transformers to encode entities and relations. specifically, we adopt bert @xcite to encode the triples in the food kg as paths. the model is trained with two typical tasks in pretrained language models and knowledge graph embedding: to predict a masked entity from a given path, and to predict the plausibility of a triple in the kg. large-scale food data offers rich knowledge that can help many issues related to healthy eating behaviors. among various food related research, the food substitution problem is gaining increasing attention owing to its applicability in tasks like food question answering @xcite and personalized dietary recommendation @xcite . in practice, there is a rising demand for people seeking food substitutions due to health concerns, ingredient shortage, or personal preferences @xcite . for instance, there are numerous posts on reddit asking for food alternatives like "substitutes for tomatoes in pizza". previous work discovers suitable substitution options based on semantic similarity via explicit substitution rules and additional context @xcite @xcite . they require many handcrafted features and there is no formal evaluation. efforts to apply machine learning methods to efficiently select substitutions have been limited due to the lack of public datasets with valid substitutions. moreover, evaluating the quality of ingredient substitutions is difficult since the validity of an ingredient substitution may be influenced by personal preference and perceived purpose of the substitution. massive food kgs have become good sources for suggesting substitutions, since they provide unified and standardized concepts and their relationships in structured form, which is very valuable for food related studies. however, kgs often suffer from sparseness if one only uses structure information in observed triple facts @xcite . we notice that the degree of nodes in food kgs are mostly small @xcite , and therefore contextual information will be ignored if we model food substitution directly on the kg. besides, we observe that the food substitutions should be distinct from context or be generalized according to different user query scenarios. for the first case, people often ask for ingredient substitutions with reference to a particular food or recipe. for example, "applesauce" can be a good substitute for "sugar" in "carrot cake", while "honey" is better for "sugar" in "brown sugar meatloaf ". thus, context is important in such scenarios. the second case refers to the huge number of queries on search engines asking for food substitutions for general purpose. for instance, "what can be substituted for heavy cream". to tackle the above issues, we conduct textual adversarial attack on our learnt kge model. we utilize a masked language model to generate high quality adversarial samples which finds substitutions that maximize the risk of making wrong assertions on kg triple plausibility prediction. we employ the generated adversarial samples as food substitutions. furthermore, to meet the different food substitution purposes, we design a collection of attack strategies to generate three types of food substitutions: context-aware recipe substitutions, context-aware ingredient substitutions and generalized ingredient substitutions. in order to generate context-aware recipe substitutions, we first find the vulnerable tokens in recipes, defined as those that trigger an error in a target prediction model. next, we apply a masked language model in a semanticpreserving way to generate substitutes, with flexibility to replace, add, or delete vulnerable tokens. the generation of context-aware ingredient substitutions is similar to recipe substitutions but only valid ingredients are selected as substitutions. the two types of substitutions are naturally aware of context since they are generated from a pre-trained language model, taking advantage of its superiority in contextualized information and rich linguistic knowledge. for the generalized ingredient substitutions, the adversarial attack is conducted among triples formed from all the ingredient's neighbors in the kg. a successful attack is achieved only when the adversarial sample fools most of its neighbors, preventing it to be contextualized to any specific neighbor. the contribution of our work is twofold: first, we address the sparseness problem in food kg and enrich its representation through the retraining of a pre-trained language model on two tasks -masked entity and triple plausibility prediction. second, we conduct the food substitution work over kgs to leverage the structured and large-scale knowledge. we propose a novel collection of attack strategies to create different types of food substitutions. we are the first to deeply generate food substitutions in an adversarial attack manner, thus avoiding the problem of substitutions ground truth shortage. both automatic and human evaluations show the high quality of our food substitutions. ## methodology in this section, we first encode a food kg into a pretrained language model (bert) to learn entity and relation representations. then, we conduct attacks on bert to generate different types of adversarial samples as food substitutions. ## conclusion and future work in this work, we proposed a novel framework to learn food kg embeddings via a pre-trained language model and generate high quality food substitutions by conducting attacks in the language model. specifically, we addressed the sparseness problem in food kg and enriched its contextualized representation via the retraining of bert model on two tasks. we then employed a masked language model to iteratively generate feasible food substitutions via adversarial attacks on kge. we further invented a collection of attack strategies to generate three types of food substitutions to meet different user needs: namely, contextualized recipe and ingredient substitutions for substitution queries with a given context, and generalized ingredient substitutions for general substitution purpose. for future work, we aim to take the health or nutrition information into consideration during adversarial sample generation, thus guiding healthier dietary choices for people.