# Leveraging Multi-Modal Information for Cross-Lingual Entity Matching across Knowledge Graphs
## 1 Introduction  
Knowledge graphs [1] are attracting attention from both academics and industries due to their power to model structured information and professional knowledge. In recent years, many high-quality knowledge graphs have been built, such as Microsoft Concept Graph [2], NELL [3], Zhishi.me [4], etc. Knowledge graphs have been widely used in many vertical fields [5], e.g., finance, medical and e-commerce. Knowledge graphs and their related technologies have gradually become indispensable basic technologies in the era of artificial intelligence. However, a knowledge graph can be freely constructed by any organization or individual according to their own needs and languages. Therefore, the data in a knowledge graph can be multilingual and diverse, and there is a large amount of overlapping knowledge and complementary information across different knowledge graphs. To better utilize such various knowledge, knowledge graph matching has attracted the attention of more and more researchers. The purpose of knowledge graph matching is to integrate different knowledge graphs to form a global knowledge base and establish interoperability between the applications based on different knowledge graphs, especially in the fields of information retrieval, machine reading and knowledge base question answering [5]. The research of knowledge graph matching starts with traditional ontology matching [6], namely matching classes and properties in the schema level of knowledge graphs. More recently, with the fast development of knowledge graphs, the number of entities has increased rapidly. As a result, the entity matching between different knowledge graphs is becoming more and more important to overcome the heterogeneity problem. The process of entity matching decides whether two entities from different knowledge graphs are the same object or not. Owing to the growing number of multilingual knowledge graphsdue to information globalization, entity matching across multilingual knowledge graphs (namely, cross-lingual entity matching) is becoming an urgent sub-problem to be solved for multilingual intelligent applications. As part of the technologies on knowledge graph representation learning, cross-lingual entity matching has become much easier since we only need to train vector representations of entities from the knowledge graphs of different languages and then decide whether two entities are matched by computing vector similarities. Most of existing studies on cross-lingual entity matching with knowledge graph representation learning only use the structural information of the knowledge graph, i.e, the relation triples in the form of entity, relation, entity to learn the structural embedding of entities. For example, MTransE [7] uses TransE to map two knowledge graphs into their respective vector spaces and then learns the mapping relationship between knowledge graphs. IPTransE [8] and BootEA [9] use iterative methods to continuously discover new matched entities with updated embeddings. In addition to relation triples, some works also consider attribute triples, such as JAPE [10], etc. In addition, KDCoE [11] and HMAN [12] use the text descriptions of entities as the supplement of relation triples to realize entity matching. EVA [13] incorporates images as the complement to align entities across different knowledge graphs. Although the existing entity matching methods based on representation learning have made remarkable achievements, they still have the following two problems. Firstly, the existing entity matching methods cannot make full use of knowledge graph relevant muti-modal information. Most of them only use the relation triples in knowledge graphs but seldom use other modal information, such as attributes, texts, images, etc. Such information has its own characteristics and can provide useful information for entity representations from different perspectives. For example, text description can provide rich textual context information for entities, and images can provide concrete visual information for entities.