Abstract: Knowledge bases (KBs) of real-world facts about entities and their relationships are useful resources for a variety of natural language processing tasks. However, because knowledge bases are typically incomplete, it is useful to be able to perform knowledge base completion or link prediction, i.e., predict whether a relationship not in the knowledge base is likely to be true. This article serves as a brief overview of embedding models of entities and relationships for knowledge base completion, summarizing up-to-date experimental results on standard benchmark datasets FB15k, WN18, FB15k-237, WN18RR, FB13 and WN11.

Introduction: Before introducing the KB completion task in details, let us return to the classic Word2Vec example of a “royal” relationship between “ $\mathsf {king}$ ” and “ $\mathsf {man}$ ”, and between “ $\mathsf {queen}$ ” and “ $\mathsf {woman}$ .” As illustrated in this example: $v_{king} - v_{man} \approx v_{queen} - v_{woman}$ , word vectors learned from a large corpus can model relational similarities or linguistic regularities between pairs of words as translations in the projected vector space BIBREF0 , BIBREF1 . Figure 1 shows another example of a relational similarity between word pairs of countries and capital cities: $
v_{Japan} - v_{Tokyo} &\approx & v_{Germany} - v_{Berlin}\\
v_{Germany} - v_{Berlin} &\approx & v_{Italy} - v_{Rome} \\
v_{Italy} - v_{Rome} &\approx & v_{Portugal} - v_{Lisbon}
$  Let us consider the country and capital pairs in Figure 1 to be pairs of entities rather than word types. That is, we now represent country and capital entities by low-dimensional and dense vectors. The relational similarity between word pairs is presumably to capture a “ $\mathsf {is\_capital\_of}$ ” relationship between country and capital entities. Also, we represent this relationship by a translation vector $v_{{is\_capital\_of}}$ in the entity vector space. Thus, we expect: $
v_{Tokyo} + v_{{is\_capital\_of}} - v_{Japan} &\approx & 0 \\
v_{Berlin} + v_{{is\_capital\_of}} - v_{Germany} &\approx & 0 \\
v_{Rome} + v_{{is\_capital\_of}} - v_{Italy} &\approx & 0 \\
v_{Lisbon} + v_{{is\_capital\_of}} - v_{Portugal} &\approx & 0
$  This intuition inspired the TransE model—a well-known embedding model for KB completion or link prediction in KBs BIBREF2 . Knowledge bases are collections of real-world triples, where each triple or fact $(h, r, t)$ in KBs represents some relation $r$ between a head entity $h$ and a tail entity $t$ . KBs can thus be formalized as directed multi-relational graphs, where nodes correspond to entities and edges linking the nodes encode various kinds of relationship BIBREF3 , BIBREF4 . Here entities are real-world things or objects such as persons, places, organizations, music tracks or movies. Each relation type defines a certain relationship between entities. For example, as illustrated in Figure 2 , the relation type “ $\mathsf {child\_of}$ ” relates person entities with each other, while the relation type “ $\mathsf {born\_in}$ ” relates person entities with place entities. Several KB examples include the domain-specific KB GeneOntology and popular generic KBs of WordNet BIBREF5 , YAGO BIBREF6 , Freebase BIBREF7 , NELL BIBREF8 and DBpedia BIBREF9 as well as commercial KBs such as Google's Knowledge Graph, Microsoft's Satori and Facebook's Open Graph. Nowadays, KBs are used in a number of commercial applications including search engines such as Google, Microsoft's Bing and Facebook's Graph search. They also are useful resources for many NLP tasks such as question answering BIBREF10 , BIBREF11 , word sense disambiguation BIBREF12 , BIBREF13 , semantic parsing BIBREF14 , BIBREF15 and co-reference resolution BIBREF16 , BIBREF17 . A main issue is that even very large KBs, such as Freebase and DBpedia, which contain billions of fact triples about the world, are still far from complete. In particular, in English DBpedia 2014, 60% of person entities miss a place of birth and 58% of the scientists do not have a fact about what they are known for BIBREF19 . In Freebase, 71% of 3 million person entities miss a place of birth, 75% do not have a nationality while 94% have no facts about their parents BIBREF20 . So, in terms of a specific application, question answering systems based on incomplete KBs would not provide a correct answer given a correctly interpreted question. For example, given the incomplete KB in Figure 2 , it would be impossible to answer the question “where was Jane born ?”, although the question is completely matched with existing entity and relation type information (i.e., “ $\mathsf {Jane}$ ” and “ $\mathsf {born\_in}$ ”) in KB. Consequently, much work has been devoted towards knowledge base completion to perform link prediction in KBs, which attempts to predict whether a relationship/triple not in the KB is likely to be true, i.e., to add new triples by leveraging existing triples in the KB BIBREF21 , BIBREF22 , BIBREF23 , BIBREF3 . For example, we would like to predict the missing tail entity in the incomplete triple $\mathsf {(Jane, born\_in, ?)}$ or predict whether the triple $\mathsf {(Jane, born\_in, Miami)}$ is correct or not. Embedding models for KB completion have been proven to give state-of-the-art link prediction performances, in which entities are represented by latent feature vectors while relation types are represented by latent feature vectors and/or matrices and/or third-order tensors BIBREF24 , BIBREF25 , BIBREF2 , BIBREF26 , BIBREF27 , BIBREF28 , BIBREF29 , BIBREF19 , BIBREF30 , BIBREF3 , BIBREF31 , BIBREF32 , BIBREF33 . This article briefly overviews the embedding models for KB completion, and then summarizes up-to-date experimental results on two standard evaluation tasks: i) the entity prediction task—which is also referred to as the link prediction task BIBREF2 —and ii) the triple classification task BIBREF34 .

Question: What datasets are used to evaluate the introduced method?
Answer: They used a dataset from Taobao which contained a collection of conversation records between customers and customer service staffs. It contains over five kinds of conversations,
including chit-chat, product and discount consultation, querying delivery progress and after-sales feedback.