# An Evaluation Survey of Knowledge-Based Approaches in Telecommunication Applications
## 4 Knowledge-Driven Applications in Telecommunications
### Network Operations  
Large Language Models (LLMs), such as encoder-based models like BERT and causal/ Generative types like GPTs, have a significant impact on knowledge-intensive tasks in development and operations in the telecommunications domain. LLMs are large and deep neural networks whose underlying architecture is in the form of transformer, self-attention, convolutions, etc. The knowledge is implicitly captured in a high-dimensional vector space via the weights of the model. The models learn from vast volumes of textual data via a process called pre-training, where the input data plays a significant role in creating an efficient model. Furthermore, one of the key benefits of LLMs is the ease of interacting and querying the knowledge they contain. Unlike many other knowledge representation systems, LLMs do not necessitate the use of any specialized querying language or formalisms. Rather, the input/query usually involves a natural language expression in plain text. LLMs can be directly used for telecom purposes. However, for better performance in the applications that are heavily using the telecom terminologies in order to understand those terminologies and the relation among them, general LLMs need to be domain adapted, meaning the domain knowledge needs to be encapsulated into the models in the pre-training stage. The requirement of domain adaptation is creating a large telecom dataset [26]. The telecom domain, being on the frontiers of digital transformation, has the knowledge already documented in textual format in standard format, such as specifications of the 3 rd Generation Partnership Project (3 GPP) or company-wide documentation of product specifications handcrafted by subject matter experts or any technical reports in the field. Figure 3 shows two methods for building telecom-specific language models. In one method, existing pre-trained models in the general domain are used as a starting point, and the model is further pre-trained on a telecom dataset. In the second method, the models are trained from scratch using a mix of general and telecom domain datasets. The two approaches result in similar models that have a representation of general and in-domain knowledge. The key difference is that the second approach has a more tailored vocabulary and requires more training time and resources; as a result, it becomes more expensive [26]. The authors in [27] explored domain adaptation strategies for BERT-like models in the telecom domain. They investigated three approaches demonstrating that continued pre-training on telecom-domain text starting from a general-domain checkpoint, pre-training from scratch on telecom-domain text, or a combination of both yielded varying levels of performance and generalizability. Figure 2: Knowledge representation and reasoning categories. LLM use cases towards research and development can be leveraged in three directions: (i) _expert-guided_ use cases, (ii) _network-assisted_ use cases, and (iii) _customer-assisted_ use cases. The intended user of the first category can be an expert in the field, a NOC operator, a product developer, a support service, a researcher, a 3 GPP delegate, or any other individual involved in telecommunications seeking to address their information needs. In this first category, LLMs can be used for semantic search such as troubleshooting and test case recommendation. Troubleshooting is a very exhausting and time-consuming task that requires lots of expertise. One application of troubleshooting is answering the trouble reports (TRs) that correspond to some observed fault in a product. To this end, the knowledge source can be text descriptions from tickets. The authors in [28] present a BERT-based model designed for troubleshooting in the telecommunication industry. This approach utilizes a two-stage process to retrieve and re-rank candidate solutions for reported problems, outperforming non-BERT models like BM25. The efficacy of LLMs in enhancing accuracy in recommending solutions within a telecom-specific context is proven.