By understanding these uses, we can better analyze and understand the complex sentence structure of Tibetan. The functions of standard punctuation in modern Tibetan are shown in Table 1; this study focuses on Tibetan SBD based on the shad (\"[\"]). With the development of the global Tibetan language community and the popularization of digitization technology, the data resources of Tibetan text are gradually enriched, providing favorable conditions for developing Tibetan pre-training technology [12]. In recent years, some research has been attempted to pre-train Tibetan, and some results have been achieved. The core idea of the pre-trained language model is to use unlabeled text data to pre-train the model so that the model learns the intro, mosaic structure, and language pattern. Currently, models such as ELMo [13], GPT [14], and BERT [15] have become benchmark models in NLP. However, the development and application of pre-training techniques still face many challenges for low-resource languages like Tibetan [16]. \begin{table}
\begin{tabular}{p{142.3 pt} p{142.3 pt}} \hline \hline
**Punctuation Marks** & **Function** \\ \hline \hline Tsheg (“”) & The Tibetan tsheg is used to divide syllables. \\ \hline Shad (“[”]) & The Tibetan shad indicates the discourse’s pause, turning point, or end. \\ \hline Double Shad (“[”]) & The application of double shad is the same as the shad, but it emphasizes that a passage or sentence has ended. \\ \hline Quadruple Shad (“[][”]) & The quadruple shads are used in chapters, volumes, and book endings. It belongs to Tibetan stylistic symbols. \\ \hline Rinchenspung Shad (“[”]) & The left end of the sentence begins with a Rinchenspung shad when it is less than three letters. \\ \hline Snake-Shaped Shad (“[”]) & In a long Tibetan text, when the last syllable of a sentence is at the first position of a line, the syllable is separated by a snake-shaped shad. \\ \hline Double Omament (“[][”]) & The Tibetan double ornament is mainly used at the beginning of articles and books to indicate the start of the text. \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Modern Tibetan punctuation marks and their functions. In the pre-trained language model, the training unit is fundamental, and many tasks use the word as the minimum processing unit, and a word is a token. In Chinese text encoding, we can use \"character\" as the encoding unit, and in English processing, we can use \"letter\" or \"word\" as the primary encoding unit. The use of \"letters\" as the coding unit is prone to the problem of long coding length, which leads to a high computational cost of the model. When using \"word\" as the basic unit, the number of \"words\" is fixed, and words beyond the dictionary's scope cannot be processed, resulting in the instability of the word list. In deep learning model training, the word list is too extensive, which leads to computational overload; most models discard low-frequency words during training or fix the word list size [17]. Although such a setup reduces the computational load, it also encounters the Out of Vocabulary (OOV) problem. The simplest solution to the OOV problem is to label these words as Unknown (UNK) uniformly. However, too many words labeled \"UNK\" during training will affect the model's generalization ability. Based on this, people began to study different methods to solve this problem, namely, the subword cut algorithm. Subword refers to the subunit in a word, which is a method of dividing words or phrases into smaller units [18], with a granularity between words and letters. For example, \"subword\" can be divided into two subwords: \"sub\" and \"word\" [19]. Commonly used encoding methods are Byte Pair Encoding (BPE) [20], byte-level BPE(BBPE) [21], Wordpiece [22], and Sentencepiece [23].