# Natural Language Processing in Knowledge-Based Support for Operator Assistance
## 2 Literature Review
### State of the Art  
There are two main categories of NLP benchmarks: open-domain and closed-domain. Open-domain benchmarks refer to benchmarks that cover a wide range of topics and domains and are designed to evaluate the general language understanding abilities of NLP models, e.g., GLUE (General Language Understanding Evaluation) [25] and SuperGLUE [27]. For building the benchmark in the open domain, various sources can be used, such as Wikipedia or Google articles [24; 28], or the collection of multiple-choice Q/A tasks in different books, articles, and online forums [29]. In the open domain, some benchmarks utilize knowledge bases as a source of information, e.g., QALD [30], WebQuestion [31], and SimpleQuestion [32]. The goal of a knowledge-based benchmark is to evaluate a model's ability to reason and utilize structured knowledge. This type of benchmark typically involves tasks that require a model to answer questions or complete tasks based on a given knowledge base or graph [33]. The aim is to assess the model's ability to retrieve, understand, and utilize information from the knowledge base. Others, e.g., GLUE [25], SuperGLUE [27], and SQUAD [24], do not necessarily require knowledge bases. The goal of a non-knowledge-based benchmark is to evaluate a model's ability to understand language and perform general language tasks without relying on external knowledge sources. This type of benchmark typically involves tasks such as text classification, sentiment analysis, and language modeling. The aim is to assess the model's ability to generalize and make sense of natural language text without access to a specific knowledge base. Closed-domain benchmarks, on the other hand, are designed to evaluate NLP models in specific domains or tasks. These benchmarks typically have a narrower focus than open-domain benchmarks and are designed to evaluate the ability of NLP models to perform specific tasks in particular domains. There are some benchmarks available in the medical domain (MiPACQ clinical QA benchmark [34], MIMIC benchmark [35], 12b2/VA benchamrks [36]), and a bankruptcy dataset in the finance domain [37]. The purpose of creating benchmarks in the close domain of NLP is to foster innovation, improve the understanding of domain-specific language processing, and drive the development of intelligent and efficient solutions that cater to the requirements of specific industries, domains, or specialized fields. For the training data, in some research, authors have used the manually trained corpus in the open domain [38; 39] or the closed domain [40; 41] for their investigation. [PERSON] [42] proposed an approach in part-of-speech (POS) tagging for the open domain, considering their defined corpus with 77,860 tokens for training and 7544 for testing. In a similar study in the open domain, 14,369 tokens in the training set and 5000 tokens in the testing set were considered [43]. [PERSON] [44] offered a POS tagger corpus with 5,000,000 tokens in the open domain for training and 11,766 tokens in the test set for the Persian language. However, using the manually annotated corpus, the corpus size may not be enough for modeling and efficient evaluation [16; 45]. To address the challenge of corpus size, particularly in the closed domain, one solution can be to employ open-source NLP libraries to train their methodologies and evaluate their datasets. Open-source libraries in NLP can be traditional libraries, e.g., Stanford NLP Suite [46], Google SyntaxNet [47], NLTK [48], and SpaCy [49], or the deep learning-based models (transfer-based) libraries, e.g., BERT [50] and DistilBERT [51]. Many types of research analyzing NLP tasks consider both traditional and deep learning pre-trained libraries and compare them together in different aspects [52; 53]. The transformer-based technique performs efficiently in entity recognition, information extraction, and semantic analysis [54; 55; 56; 57].