Automatic Construction of Enterprise Knowledge Base (2021)

Abstract: In this paper, we present an automatic knowledge base construction system from large scale enterprise documents with minimal efforts of human intervention. In the design and deployment of such a knowledge mining system for enterprise, we faced several challenges including data distributional shift, performance evaluation, compliance requirements and other practical issues. We leveraged state-of-the-art deep learning models to extract information (named entities and definitions) at per document level, then further applied classical machine learning techniques to process global statistical information to improve the knowledge base. Experimental results are reported on actual enterprise documents. This system is currently serving as part of a Microsoft 365 service.

## introduction massive knowledge bases constructed from public web documents have been successful in enriching search engine results in bing and google for over a decade @xcite . there is growing interest in automatically constructing a similar knowledge base for each enterprise from their internal documents (e.g., web pages, reports, emails, presentation decks; textual contents in natural language form are all referred to as documents in this paper). such knowledge base can help an enterprise to better organize its domain knowledge, help employees (users) better find and explore knowledge, and to encourage knowledge sharing. mining knowledge from enterprise documents poses unique challenges. one challenge is that the system needs to be fully automated without per enterprise customization or existing (semi-) structured sources. knowledge base construction from web documents is often based on bootstrapping entities from human-curated sources such as wikipedia with customized extraction rules (dbpedia: @xcite @xcite , or the existence of a prior knowledge base (knowledge vault: @xcite . maintaining such wiki site and keep it fresh is costly for enterprise. another challenge is that most training data for natural language processing (nlp) models is from public documents. enterprise documents can have different writing style and vocabulary than the public documents. the data distributional shift @xcite is a challenge as (a) we need model to generalize better to enterprise domain and (b) we need test metrics to reflect the actual performance on enterprise documents to guide model development. on the other hand, enterprise domain brings new opportunities. for search engines, the knowledge base must be extremely accurate. this requirement limits the usage of nlp models to extract information from unstructured text as few models can achieve the required precision with meaningful coverage. in enterprise domain, we can relax the requirement on accuracy as enterprise users are expected to spend more time to absorb and discriminate information. in addition, users can curate and improve the automatically constructed knowledge base, which is not an option for search engine users. the relaxation on accuracy requirement makes it possible to perform knowledge mining on unstructured text by heavily relying on nlp techniques. in this paper, we present the first large-scale knowledge mining system for enterprise documents taking advantage of recent advances in nlp such as pretrained transformers @xcite as well as traditional nlp techniques. it is in production since february 2021 as part of a microsoft 365 feature (microsoft viva topics 1 ). for an enterprise that enables this feature, our system will build a knowledge base from its internal documents that already exist in microsoft cloud and will keep it fresh without the need of any customized intervention. at the core of our knowledge base are entities mined from documents that are of interest to the enterprise, such as product, organization and project. these entities are loosely referred to as topics to the end users (not to be confused with topic modeling in nlp). the knowledge base is a collection of "topic cards" with rich information: properties that help users understand the topic (such as alternative names, descriptions, and related documents), or enable users to connect with people who might be knowledgeable about the topic (related people) or explore related topics. the contributions of this work are as follows: ## system description the overall system architecture is depicted in figure 1. in this section, we discuss at length the knowledge mining system that works "offline". the system works in a semi-streaming mode: whenever there's a document update, the content of the document is sent to the ner and description mining components. the ner component extracts entities then updates information in the topic candidate store. the topic ranker periodically pulls the topic candidates store to select the top n topics. the topic card builder then builds topics cards with various attributes. note that this is a simplified view of the actual system. for example, there is another component that conflates information from other sources using techniques described in @xcite . ## use cases the detailed user guide and licensing information can be found on microsoft viva topics website 3 . here we briefly introduce two ways user can interact with the knowledge base. figure 3 shows the topic highlighting feature. topic mentions in documents get automatically linked to the knowledge base. user can hover over the topic mention to see figure 4: snapshot of a personalized topic center homepage. foot_2 the topic card and click the link in the topic cards to checkout more information. figure 4 shows an example topic center homepage. the view is personalized as the related topics for a user is presented for the user to confirm. users can also checkout all the topics from manage topics page. ## conclusion organizing resources inside an enterprise into one centralized location facilitates knowledge and expertise sharing and improves productivity. we present a system that automatically constructs a knowledge base from unstructured documents to help enterprises achieving this goal. the system is built upon a combination of deep learning models and classical techniques. we show the challenge of applying nlp models in enterprise domain. we also discuss how we improve the models and the whole system to meet our requirements with detailed experiment results. finally, we show two typical use cases. we hope our experience can benefit researchers and practitioners in this field.