# Natural Language Processing in Knowledge-Based Support for Operator Assistance
## 2 Literature Review
### State of the Art
\begin{table}
\begin{tabular}{p{56.9 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline \hline
**Reference** & **Domain** & **Sentence Picking Techniques** & **Number of Classes** & **Purpose** \\ \hline [28] & Open & Queried Google with the OR-linked keywords & One class related to requirements engineering & Oriented to enable replication of NLP experiments and generalization of results in requirements engineering. \\ \hline [29] & Open & Building three benchmarks considering rating numbers in AMAZON, extracting sentence pairs from Japanese datasets, and randomly picking from Wikipedia & - & Having a benchmark in the Japanese language considering cultural/social parameters without using the translation methods in order to evaluate NLU ability in the general domain \\ \hline [66] & Close & In three domains, medicine, technology, and finance, they pick the sentences in Wikipedia based on the defined categories & Depend on the domain in Wikipedia categories & Fill the gap of the lack of pre-trained domain-specific models for languages other than English \\ \hline [67] & Close & Build a benchmark based on one specific available corpus & 14 subcategories based on the definition of events and entity, with not equal amounts for each & Compare two approaches on a newly built benchmark \\ \hline [68] & Close & The benchmark is built on 6 existing tasks available in the state-of-the-art & 6 available subtasks in medical & To support downstream applications in computerized diagnostic decision support and improve the efficiency and accuracy of healthcare providers during patient care. \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Techniques and purposes for sentence selection in open and closed domains of NLP benchmark. open domain benchmark, where the goal is to test a model's ability to perform a wide range of tasks, it may be important to have a large number of sentences to ensure adequate coverage of different domains and topics. The GLUE benchmark [25] consists of a collection of nine different datasets each with its considered task. The number of sentences is between 780 for 755k to this benchmark. The SQuAD benchmark [24] consists of 100,000 question-answer pairs, and [28] build a benchmark in the open domain with 79 documents as 34,268 sentences with 865,551 tokens. However, in a closed domain benchmark, where the focus is on a specific task within a domain, a smaller number of sentences may be sufficient. It is important to note that while extensive datasets are vital for open-domain benchmarks to encompass diverse domains and topics, the process of creating and curating large datasets, especially in closed domains, can be inherently challenging and resource-intensive. This is exemplified by previous research studies that have successfully utilized limited sets of sentences and tokens for specific domains. For instance, a study examining the cooking domain in a question-answering context employed 2175 questions [69]. In the email domain for spam/thread detection, a dataset consisting of 350 paired questions was utilized for analysis [70]. Similarly, a study in the university field domain employed a corpus of 2903 sentences [71], while another study focusing on the oil industry utilized an 18,000 token dataset [72]. These examples highlight that even with a relatively smaller number of sentences or tokens, a closed domain benchmark can yield valuable insights and enable targeted analysis specific to the domain. Nonetheless, it is crucial for the selected sentences to encompass various aspects of the specific domain to ensure comprehensive coverage during evaluation. In addressing the challenge of incomplete sentences within the realm of NLP, various methodologies have been proposed to infer missing information and enhance comprehension effectively. One such approach, hole semantics [73], has garnered attention for its ability to interpret incomplete sentences by identifying and filling gaps in the semantic structure.