A Review in Knowledge Extraction from Knowledge Bases (2023)

Abstract: Generative language models achieve the state of the art in many tasks within natural language processing (NLP). Although these models correctly capture syntactic information, they fail to interpret knowledge (semantics). Moreover, the lack of interpretability of these models promotes the use of other technologies as a replacement or complement to generative language models. This is the case with research focused on incorporating knowledge by resorting to knowledge bases mainly in the form of graphs. The generation of large knowledge graphs is carried out with unsupervised or semi-supervised techniques, which promotes the validation of this knowledge with the same type of techniques due to the size of the generated databases. In this review, we will explain the different techniques used to test and infer knowledge from graph structures with machine learning algorithms. The motivation of validating and inferring knowledge is to use correct knowledge in subsequent tasks with improved embeddings.

## introduction knowledge bases (kb) are widely used for storage information used in different machine learning tasks. knowledge bases are generally represented by knowledge graphs (kg), which store information that employ nodes (entities) and edges (relations) creating a network. this way of storing knowledge has been popularized in recent years due to it being a more expressive, versatile and scalable than traditional databases @xcite . the efficient use of knowledge stored in a kg with machine learning models is not a trivial task. traditional machine learning models, including deep neural networks use vectors as input, while the structure of kgs is more complex and can't be simplified in a vector, due to the need for representing nodes, edges, connectivity, global relations inside the graph and features of every element @xcite . methodologies used for extracting knowledge from kgs focus on creating latent vectors with the graph information (embeddings) or using neural networks specially designed for dealing with the graph structure. many kbs are developed using non supervised machine learning techniques, generating massive data in the process. those methods may cause errors when completing the kb due to false relations between nodes. large kbs also have problems with not useful information introduced for a specific task which can be considered as noise. ## knowledge graph structure kgs are made up of two sets of elements @xmath0, e. where v is the set of nodes (entities) and e is the set of edges (relations). where: a knowledge graph can be classified as homogeneous or heterogeneous if nodes are of the same class or different classes, cyclic or acyclic if its possible to reach the initial node traveling between edges or no and directed or undirected if nodes are connected in one direction only or both, respectively. there are variants from conventional graph, this is the case of hypergraphs which contains hyperedges linking more than two nodes or multigraphs which allow more than one edge between two nodes. graph knowledge can be represented in many ways due to the versatility of the graph structure. a representation of nodes and edges connecting those nodes is necessary. more complex graphs may consider many features inside each node, nodes containing subnodes with their own relations, features for each edge, different types of edges and global features associated with the entire graph @xcite . for representing the information contained in a kb, consider a head entity e h and a tail entity e t sharing a relation r. this is represented by a triple (e h , r, e t ). if the entities are directly connected with the relation we consider this a "1-hop" relation, otherwise it is called a "multi-hop" relation. multi-hop relations are more difficult to detect, mainly for entities with a distance of 3 hops or more. there are many tasks involving constructed kgs in nlp, most of the current research focus on entity linking, question answering (qa) and fact checking. the structure of triples is generally utilized for representing graph information in the lowest level. this is the general structure in query languages developed for graph databases as cypher for databases like neo4j and sparql for data in the resource description framework (rdf) format. ## machine learning techniques ml techniques used on kbs represent the information contained in nodes and edges in a structured format as embeddings, other models act directly over the graph structure, this is the case of graph neural networks (gnns). we have considered three different families of models for entity linking according to the techniques used to perform the task. there are translational models, which consider that the different relationships between elements can be represented as displacements in space, matrix factorization models represent the relationships between entities as tensors and perform decomposition operations on the tensors to represent each entity and relationship and finally deep neural models are used to obtain the main characteristics of each possible relationship and determine whether they are truthful or encode information from nearby entities. ## discussion and future directions the representation of knowledge bases as embedding vectors can be seen as a way to obtain contextualised embeddings of any knowledge base with a graph structure, such as ontologies. furthermore, contextualized embeddings can be used beyond tasks such as entity linking or knowledge base completion by representing the latent knowledge of the knowledge bases used in the form of vectors. contextualised embedding vectors commonly used in natural language processing (nlp) are usually obtained from a corpus with unsupervised techniques such as word2vec @xcite or glove @xcite , using long-short term memory neural networks (lstms) on the text as in the case of elmo @xcite or using language models with transformers-type architecture as bert @xcite . with the methods explained in this paper, contextual vectors can be created taking into account as context only the graph itself and the relations existing in it without taking into consideration any corpus. this becomes even more important taking into account the current research trends within nlp focused on combining knowledge from ontologies with the latent language of language models, creating synergies with the aim of improving the state of the art in different nlp tasks, achieving explainable models or reaching competitive results with lighter models @xcite . it is expected that the improvement of embeddings obtained from knowledge graphs will be useful to achieve a better integration between language models and knowledge graphs. ## conclusions both in the case of translational models and in tensor factorization, there is a tendency to represent increasingly complex spaces, to the point of combining different types of spaces into one (euclidean, hyperspherical and hyperbolic) or to represent increasingly complex vector spaces (complex space, quaternions, etc.). however, in some cases it is observed that the state of the art is surpassed without necessarily increasing the complexity of the space represented; this is the case of simple (which achieves results similar to complex) or tucker. alternative spaces to the euclidean with positive or negative curvature tend to better represent some properties of entities with a smaller number of features, such as circular relations in hyperspherical spaces and hierarchies in hyperbolic spaces, allowing the creation of embeddings at a lower computational cost. in the case of deep neural models, tests have also been carried out with positive and negative curvature spaces. in these cases, curvature is a parameter to be trained within the network. the current state of the art is led by models that combine different vector spaces (gie, m2gnn).