Efficient and Robust Knowledge Graph Construction (2022)

Abstract: Knowledge graph construction which aims to extract knowledge from the text corpus, has appealed to the NLP community researchers. Previous decades have witnessed the remarkable progress of knowledge graph construction on the basis of neural models; however, those models often cost massive computation or labeled data resources and suffer from unstable inference accounting for biased or adversarial samples. Recently, numerous approaches have been explored to mitigate the efficiency and robustness issues for knowledge graph construction, such as prompt learning and adversarial training. In this tutorial, we aim to bring interested NLP researchers up to speed on the recent and ongoing techniques for efficient and robust knowledge graph construction. Additionally, our goal is to provide a systematic and up-to-date overview of these methods and reveal new research opportunities to the audience.

## introduction motivation: knowledge graphs (kgs) regard the knowledge as fact triples in the form @xmath0, predicate, object>, which can benefit a wide range of natural language processing tasks including question answering @xcite @xcite , fact verification @xcite , data-to-text generation @xcite , commonsense reasoning @xcite and so on. knowledge graph construction tasks including named entity recognition (ner) @xcite , relation extraction (re) @xcite target to extract structural information from unstructured texts, have appealed to researchers in nlp community. while those researchers have largely separated approaches from tasks, they have encountered similar issues such as efficiency and robustness. intuitively, efficient and robust knowledge graph construction has been widely investigated due to its potential value of making models scenarioadaptable, data-efficient, and particularly convenient for real-world applications with cold-start issues. in this tutorial, we take a holistic view of knowledge graph construction, introducing the commonalities in the issues and solutions regarding efficiency and robustness. we will explore the approaches of named entity recognition and relation extraction with few-shot labeled data, limited computation resources and approaches to improve the model robustness. note that our tutorial is related to green deep learning @xcite that appeals to researchers to focus on carbon emission and energy usage during model training and inference, and relevant to robust nlp @xcite which focuses on addressing issues in current models' language understanding capabilities with adversarial attacks. meanwhile, trends within knowledge graph construction have shifted toward lowresource rather than considering massive labeled data and reliable & trustworthy knowledge graph construction. notably, it is worth considering the knowledge graph construction tasks as a whole to develop methodologies for efficiency and robustness issues. we will discuss these works and suggest avenues in the future. tutorial content: we will start this tutorial by defining tasks of knowledge graph construction, including named entity recognition relation extraction from sentences or documents. then, we will give introductions to the basic models, open datasets and tools used in knowledge graph construction covering both english and chinese @xcite . we plan to focus on methods that enable efficient knowledge graph construction, such as the distant supervision @xcite and data augmentation paradigms of creating training data @xcite , model enhancement methods like meta-learning @xcite , transfer learning @xcite and prompt learn-ing @xcite , parameter-efficient approaches @xcite , including adaptor-based tuning. we will then explore research focusing on robust knowledge graph construction for stable learning with adversarial attacks and selection or semantic biases. during the tutorial, we plan to deliver lessons learned from the diverse communities involved in knowledge graph construction research and will introduce insights from the industry when building a business knowledge graph in low-resource settings. section 3 has an outline of tutorial content. tutorial slides will be available at https://github.com/nlp-tutorials/ aacl-ijcnlp2022-kgc-tutorial . relevance to aacl: knowledge graphs benefit many crucial nlp tasks, and knowledge graph construction tasks such as relation extraction and named entity recognition are core tasks in information extraction. a 2018 naacl tutorial, "scalable construction and reasoning of massive knowledge bases" @xcite , introduced a summary of recent kb, and ie works. more recently, an acl tutorial, "multi-modal information extraction from text, semi-structured, and tabular data on the web" @xcite , provided an overview of information extraction (ie) from web data with two vital dimensions: the thrust to develop scalable approaches and the diversity in data modality. however, previous tutorials mainly focus on models with rich resources of labeled data and computation, and recent years have witnessed the fast development of efficient and robust knowledge graph construction. on the other hand, the nlp community has paid much attention to robust nlp, such as an emnlp 2021 tutorial, "robustness and adversarial examples in natural language processing" @xcite . different from this tutorial in general nlp, we target a small, focused domain of knowledge graph construction and introduce the detailed latest work in limited 3 hours. ## type of this tutorial this tutorial contains cutting-edge approaches in general knowledge graph construction approaches regarding efficiency and robustness issues. however, our coverage of this tutorial will contain introductory material of knowledge graph construction for widespread audiences of the nlp community. besides, we will introduce methods of chinese knowledge graph construction for asia audiences. ## prerequisites anyone with a background in natural language processing can access this tutorial. moreover, a basic understanding of neural networks, preferably with some knowledge of information extraction, knowledge graph, and pre-trained language models, is helpful. ningyu zhang is an associate professor at zhejiang university, leading the group about kg and nlp technologies. he is also a researcher at alibaba-zhejiang university joint research institute of frontier technologies (azft), co-pi of the alibaba open business knowledge graph 1 , which is devoted to benefiting e-commerce applications and to discovering socioeconomic values. he is a member of acl, a member of the youth working committee of the chinese information processing society of china, and the member of the language and knowledge computing professional committee of the chinese information processing society of china. he has published many papers in top international academic conferences and journals such as iclr, acl, ennlp, naacl, and ieee/acm transactions on audio speech and language. he has served as pcs for neurips, iclr, kdd, icml, aaai, ijcai and reviewer for arr, tkde, and tkdd. he has received the best paper award from the china conference on knowledge graph and semantic computing and the best paper nominations from the international joint conference on knowledge graphs. he has won first place in the trec precision medicine 2020 sponsored by the national institute of standards and technology (nist) and fourth place in the international semantic evaluation competition (semeval 2021 task4) sponsored by acl. he has given multiple talks on information extraction and knowledge graph. email: zhangningyu@zju.edu.cn homepage: https://person.zju.edu. cn/en/ningyu tao gui is an associate professor at the institute of modern languages and linguistics of fudan university. he is the key member of the fudannlp group 2 . he is a member of acl, a member of the youth working committee of the chinese information processing society of china, and the member of the language and knowledge computing professional committee of the chinese information processing society of china. he has published more than 40 papers in top international academic conferences and journals such as acl, ennlp, aaai, ijcai, sigir, and so on. he has served as area chair or pcs for sigir, aaai, ijcai, tpami, and arr. he has received the outstanding doctoral dissertation award of the chinese informa-tion processing society of china, the area chair favorite award of coling 2018, the outstanding paper award of nlpcc 2019, and a scholar of young talent promoting projects of cast. email: tgui@fudan.edu.cn homepage: https://guitaowufeng. github.io guoshun nan is a tenure-track professor in the school of cyber science and engineering, beijing university of posts and telecommunications (bupt). he is a key member of the national engineering research center of mobile network security and a member of the wireless technology innovation institute of bupt. before starting his academic career, he also worked in hewlett-packard company (china) for more than four years as an engineer. he is a member of acl. he has a broad interest in information extraction, model robustness, multimodal retrieval, cyber security and the next generation of wireless networks. he has published more than ten papers in top-tier conferences such as acl, cvpr, emnlp, sigir, ijcai, ckim and sigcomm. he served as a reviewer for acl, emnlp, aaai, ijcai, neurocomputing and ieee transaction on image processing. email: nanguo2021@bupt.edu.cn