## 1 Introduction  
Recent advances in artificial intelligence and machine learning have produced a number of models and solutions that heavily rely on text. Most of the technologies and products we use today require vast amounts of data to be stored in different types of documents both during and after they are created. Large language models (LLMs) are possibly the most visible examples of such models to a wider audience. Machines can now comprehend natural languages thanks to architectures like BERT [1], RoBERTa [2], GPT [3],and RWKV [4]. These models allow for specialized processing, which is necessary for a range of natural language processing jobs. Word embedding dates back a few years, even though the creation of transformer architectures has signaled the start of a new phase in the evolution of language models. One of the earliest neural language models (NLMs) that resemble n-gram models was created by [PERSON] et al. [5]. [PERSON]. [6] then used NLMs for machine translation with success. NLMs became much more popular after [PERSON] [7], released RNNLM, an open source NLM toolkit. Machine translation, text creation, and text categorization [8] are just a few of the natural language applications that have made extensive use of NLMs based on recurrent neural networks (RNNs) and their variants, such as long short-term memory (LSTM) [9] and gated recurrent unit (GRU) [10]. With the addition of a self-attention mechanism on top of the transformers, BERT [1] marked the next important step in the development of language models. Due to its performance in pre-training tasks on large unlabelled corpora, BERT had an influence on the development of a number of models and architectures, including GPT-2 [11] and BART [12]. BERT architecture introduced the masked language modeling job as its focus. The objective of the masked language modeling job is to predict a masked token inside a sequence of tokens that the model processes bidirectionally. Therefore, when an in-depth contextual understanding of a sequence is required, masked language modeling is the appropriate option. Large language models acquire contextual representations in an unsupervised manner by minimizing a masked language modeling objective across a base corpus. The model training process in most cases includes two phases--pre-training and fine-tuning. Pre-training refers to the phase of unsupervised language model training. The fine-tuning procedure involves training a pre-trained model on a specific downstream task using labeled data for supervised tasks, following the replacement of the output head with a lightweight classifier. Masked language modeling (MLM) models are frequently used in recent language analysis approaches to assist in learning natural language features. However, domain-specific texts, such as formal terminology, present a unique training problem for these networks. Since the pretraining phase is usually based on the usage of a substantial amount of textual material, often retrieved from the internet, this corpus does not adequately represent domain-specific terms, such as legal terms. Subsequently, this type of text used in the language model pretraining phase frequently excludes domain-specific terms from the vocabulary building process. Thus, the model becomes partially unable to recommend phrases that are not part of its vocabulary, e.g., not directly represented within the tokenizer abilities, due to the nature of the training data and the previously described steps. The tokenizer, which transforms unprocessed text into a series of tokens the model can understand, is one of the essential parts supporting language models. Tokenization techniques are essential to LLM performance and comprehension [13] because they have a direct impact on how well they can capture context and semantics. In transformer-based models, tokenization efficiency and model efficiency are directly correlated [14]. Additionally, tokenizer performance may be significantly impacted by the choice of data used for training.