# Inter-Sentence Segmentation of YouTube Subtitles Using Long-Short Term Memory (LSTM)
## 1 Introduction  
Speech to Text (STT) [1, 2] is a process in which a computer interprets a person's speech and then converts the contents into text. One of the most popular algorithms is HMM (Hidden Markov Model) [3], which constructs an acoustic model by statistically modeling voices spoken by various speakers [4] and constructs a language model using corpus [5]. Machine Translation (MT) is a sub-field of computational linguistics that investigates the use of software to translate text or speech from one language to another ([[https://en.wikipedia.org/wiki/Machine_translation](https://en.wikipedia.org/wiki/Machine_translation)]([https://en.wikipedia.org/wiki/Machine_translation](https://en.wikipedia.org/wiki/Machine_translation))). MT has been approached by rules [6], examples [7], and statistics [8]. Currently Neural Machine Translation (NMT) [9] has dramatically improved MT performance, and there are a lot of translation apps, such as iTranslate ([[https://www.itranslate.com/](https://www.itranslate.com/)]([https://www.itranslate.com/](https://www.itranslate.com/))) and Google Translate ([[https://translate.google.com/](https://translate.google.com/)]([https://translate.google.com/](https://translate.google.com/))), competing in the market. A recent video-sharing site, YouTube's automated captioning system [10] combines STT with MT technology. When uploading a video, YouTube extracts the voice data and writes subtitle files, and translates the file into the desired language. These techniques are of great help to users who cannot speak in the language spoken in the content. However, if the speaker does not deliberately distinguish between sentences, for example if they do not make a pause between sentences, multiple sentences are recognized as a single sentence. This problem significantly degrades the performance of the MT. What's even worse is that YouTube manages subtitles by time slots, not by utterances. The subtitles automatically generated in the actual YouTube are divided into units of scene, and these are to be translated. In this paper, we propose an automatic sentence segmentation method that automatically generates a period mark using deep neural networks to improve the accuracy of automatic translation of YouTube subtitles. In natural language processing, a period is a very important factor in determining the meaning of a sentence. Table 1 shows sentences with different meanings according to the different period positions. The example sentence is an STT-generated sentence with periods and capital letters are removed. This sentence is divided into completely different sentences, such as Case 1 and Case 2, depending on the position of the period, and the generated sentences have different meanings. If the original text is divided as in Case 1, here \"he\" is \"[PERSON]\". On the other hand, if divided as in Case 2, \"he\" would be \"[PERSON]\" and \"[PERSON]\" should eat \"[PERSON]\". Studies on past real-time sentence boundary detection and period position prediction [11; 12; 13] have attempted to combine words and other features (pause duration, pitch, etc.) into one framework. A research [14] verified that the detection process could improve translation quality. In addition, a research has been conducted to automatically detect sentence boundaries based on a combination of N-gram language models and decision trees [15]. Currently, Siri, Android, and Google API ([[https://cloud.google.com/speech-to-text/docs/automatic-punctuation](https://cloud.google.com/speech-to-text/docs/automatic-punctuation)]([https://cloud.google.com/speech-to-text/docs/automatic-punctuation](https://cloud.google.com/speech-to-text/docs/automatic-punctuation))) insert punctuation marks after recognition. However, it is more important for them to distinguish whether the punctuation of a sentence is a period or a question mark, rather than finding and recognizing appropriate position when recognizing multiple sentences at once.