Upload 4 files

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+train-hifigan-v2.ipynb filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,106 @@
----
-license: mit
----
+# MyOwn-TTS
+[![License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
+
+## Description
+
+MyOwn-TTS is a project aimed at creating a text-to-speech (TTS) system that reads sentences in your own voice. This repository includes pre-trained models that have been trained using my voice.
+
+## Table of Contents
+
+- [Installation](#installation)
+- [Usage](#usage)
+- [Contributing](#contributing)
+- [License](#license)
+
+## Installation
+
+This README focuses on guiding you through the process of synthesizing speech using pre-trained models, rather than detailing the model training process.
+
+1. Clone the repository:
+
+    ```bash
+    git clone https://github.com/IceCream0910/myown-tts.git
+    ```
+
+2. Modify the `run-server.bat` batch file in the `/server` directory to match your actual file paths.
+
+    For example, if your server folder is at `C:\myown-tts\server`, update the file as follows:
+
+    ```bat
+    @echo off
+    setlocal
+    cd /D "%~dp0"
+    set MECAB_KO_DIC_PATH=.\mecab\mecab-ko-dic -r .\mecab\mecabrc
+    set TTS_MODEL_FILE=C:\myown-tts\server\models\glowtts-v2\best_model.pth.tar
+    set TTS_MODEL_CONFIG=C:\myown-tts\server\models\glowtts-v2\config.json
+    set VOCODER_MODEL_FILE=C:\myown-tts\server\models\hifigan-v2\best_model.pth.tar
+    set VOCODER_MODEL_CONFIG=C:\myown-tts\server\models\hifigan-v2\config.json
+    server.exe
+    endlocal
+    ```
+
+3. Update the `glowtts-v2/config.json` and `hifigan-v2/config.json` files in the `/server/models/` directory with your actual file paths.
+
+    Ensure you double the backslash (`\\`) in the file paths, as shown below:
+
+    - For `glowtts-v2/config.json`:
+    ```json
+    "stats_path": "C:\\mydata\\tts-server\\models\\glowtts-v2\\scale_stats.npy"
+    ```
+
+    - For `hifigan-v2/config.json`:
+    ```json
+    "stats_path": "C:\\mydata\\tts-server\\models\\hifigan-v2\\scale_stats.npy"
+    ```
+
+## Usage
+
+To start the TTS server, execute `run-server.bat`. Once the server is running, you will see the message `INFO:werkzeug: * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)` in the command prompt, indicating that the speech synthesis feature is available through the TTS server. To stop the server, press CTRL+C in the command prompt.
+
+### API
+
+- Text preprocessing: `/tts-server/api/process-text`
+
+    Splits sentences and removes special characters to automatically stitch together and playback multi-line sentences as you type.
+
+- Text Inference: `/tts-server/api/infer-glowtts`
+
+    Synthesizes text to speech. Send the text to be synthesized in the `text` parameter of the URL.
+    
+    Example:
+    ```
+    http://localhost:5000/tts-server/api/infer-glowtts?text=hello
+    ```
+
+### Text Inference Demo Page
+
+Visit [http://localhost:5000/](http://localhost:5000/) for a demo.
+
+## Contributing
+
+1. Fork the repository (https://github.com/icecream0910/myown-tts/fork).
+2. Create a new branch: `git checkout -b feature/<featureName>`.
+3. Commit your changes: `git commit -am 'Add <featureName>'`.
+4. Push to the branch: `git push origin feature/<featureName>`.
+5. Submit a pull request.
+
+## License
+
+This project is licensed under the [MIT License](LICENSE).
+
+## References
+
+This implementation draws inspiration from the following repositories:
+
+- [SCE-TTS](https://github.com/sce-tts)
+- [g2pK](https://github.com/Kyubyong/g2pK)
+- [mimic-recording-studio](https://github.com/MycroftAI/mimic-recording-studio)
+- [coqui TTS](https://github.com/coqui-ai/TTS)
+
+The datasets below are distributed under the CC-BY 2.0 license, with the original text data provided by the Korea Information Society Development Institute's AI Hub, including Korean dialogue text data and Korean-English translation (parallel) corpus text data.
+
+- [Korean Corpus for Voice Recording](https://github.com/sce-tts/mimic-recording-studio/blob/master/backend/prompts/korean_corpus.csv)
+- [SleepingCE Speech Dataset](https://drive.google.com/file/d/1UpoBaZRTJXkTdsoemLBWV48QClm6hpTX/view?usp=sharing)
+- [Pre-trained Models for SleepingCE Speech Dataset (Glow-TTS)](https://drive.google.com/file/d/1DMKLdfZ_gzc_z0qDod6_G8fEXj0zCHvC/view?usp=sharing)
+- [Pre-trained Models for SleepingCE Speech Dataset (HiFi-GAN)](https://drive.google.com/file/d/1vRxp1RH-U7gSzWgyxnKY4h_7pB3tjPmU/view?usp=sharing)
+    - These models were fine-tuned from the model provided by [coqui-ai/TTS](https://github.com/coqui-ai/TTS), trained on the [VCTK dataset](https://datashare.ed.ac.uk/handle/10283/3443), available [here](https://github.com/coqui-ai/TTS/releases/download/v0.0.12/vocoder_model--en--vctk--hifigan_v2.zip).

infer-v2.ipynb

@@ -0,0 +1 @@
+{"cells":[{"cell_type":"markdown","metadata":{"id":"AvxXfBluyZvb"},"source":["## 1. 구글 드라이브 마운트\n","\n","음성합성을 위해 학습한 모델이 있는 구글 드라이브를 마운트합니다.  \n","마운트할 구글 드라이브 내에 다음 파일들이 존재하는지 꼭 확인해주세요.\n","\n","- `/Colab Notebooks/data/glowtts-v2/model_file.pth.tar`\n","- `/Colab Notebooks/data/glowtts-v2/config.json`\n","- `/Colab Notebooks/data/hifigan-v2/model_file.pth.tar`\n","- `/Colab Notebooks/data/hifigan-v2/config.json`\n","\n","\n","(존재하지 않는다면, [glowtts-v2.zip](https://drive.google.com/file/d/1DMKLdfZ_gzc_z0qDod6_G8fEXj0zCHvC/view?usp=sharing), [hifigan-v2.zip](https://drive.google.com/file/d/1vRxp1RH-U7gSzWgyxnKY4h_7pB3tjPmU/view?usp=sharing)을 내려받아 준비해주세요.)\n","\n","만약 아래에 `Enter your authorization code:`과 같은 메시지가 출력될 경우,  \n","같이 출력된 링크에 접속하여, 마운트할 구글 계정을 선택하신 후, 인증 코드를 복사하여 입력해주세요."]},{"cell_type":"code","execution_count":1,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":230434,"status":"ok","timestamp":1707413202236,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"4U2wrDOthrsF","outputId":"1a56442b-2686-468a-bb32-0ba7608ef988"},"outputs":[{"name":"stdout","output_type":"stream","text":["Mounted at /content/drive\n"]}],"source":["from google.colab import drive\n","drive.mount('/content/drive')"]},{"cell_type":"markdown","metadata":{"id":"8erClGSnzwge"},"source":["## 2. 필수 라이브러리 및 함수 불러오기\n","\n","실행에 필요한 라이브러리 및 함수를 불러옵니다.\n","\n","이 과정은 약 10분 정도 소요될 수 있습니다."]},{"cell_type":"code","execution_count":2,"metadata":{"executionInfo":{"elapsed":18,"status":"ok","timestamp":1707413202238,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"jYCym6hXge2_"},"outputs":[],"source":["import os\n","import sys\n","from pathlib import Path"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"background_save":true,"base_uri":"https://localhost:8080/"},"id":"JkWG-L13gReB"},"outputs":[{"name":"stdout","output_type":"stream","text":["/content\n","fatal: destination path 'TTS' already exists and is not an empty directory.\n","fatal: destination path 'g2pK' already exists and is not an empty directory.\n","/content/TTS\n","  \u001b[1;31merror\u001b[0m: \u001b[1msubprocess-exited-with-error\u001b[0m\n","  \n","  \u001b[31m×\u001b[0m \u001b[32mpip subprocess to install build dependencies\u001b[0m did not run successfully.\n","  \u001b[31m│\u001b[0m exit code: \u001b[1;36m1\u001b[0m\n","  \u001b[31m╰─\u003e\u001b[0m See above for output.\n","  \n","  \u001b[1;35mnote\u001b[0m: This error originates from a subprocess, and is likely not a problem with pip.\n","  Installing build dependencies ... \u001b[?25l\u001b[?25herror\n","\u001b[1;31merror\u001b[0m: \u001b[1msubprocess-exited-with-error\u001b[0m\n","\n","\u001b[31m×\u001b[0m \u001b[32mpip subprocess to install build dependencies\u001b[0m did not run successfully.\n","\u001b[31m│\u001b[0m exit code: \u001b[1;36m1\u001b[0m\n","\u001b[31m╰─\u003e\u001b[0m See above for output.\n","\n","\u001b[1;35mnote\u001b[0m: This error originates from a subprocess, and is likely not a problem with pip.\n","/content/g2pK\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m71.1/71.1 kB\u001b[0m \u001b[31m2.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25h  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n"]}],"source":["%cd /content\n","!git clone --depth 1 https://github.com/sce-tts/TTS.git -b sce-tts\n","!git clone --depth 1 https://github.com/sce-tts/g2pK.git\n","%cd /content/TTS\n","!pip install -q --no-cache-dir -e .\n","%cd /content/g2pK\n","!pip install -q --no-cache-dir \"pysbd\" \"konlpy\" \"jamo\" \"nltk\" \"python-mecab-ko\"\n","!pip install -q --no-cache-dir -e ."]},{"cell_type":"code","execution_count":4,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":2893,"status":"ok","timestamp":1707413252303,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"FUD8SfIxSY8j","outputId":"3b984ef6-c788-4694-84b1-2675d82ed0c3"},"outputs":[{"name":"stdout","output_type":"stream","text":["/content/g2pK\n"]},{"name":"stderr","output_type":"stream","text":["[nltk_data] Downloading package cmudict to /root/nltk_data...\n","[nltk_data]   Unzipping corpora/cmudict.zip.\n"]}],"source":["%cd /content/g2pK\n","import g2pk\n","g2p = g2pk.G2p()"]},{"cell_type":"code","execution_count":5,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":524},"executionInfo":{"elapsed":15,"status":"error","timestamp":1707413252303,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"Lt9bLLZ8I4GH","outputId":"d0eba754-a986-47ee-d154-7233c2ef24c6"},"outputs":[{"name":"stdout","output_type":"stream","text":["/content/TTS\n"]},{"ename":"ModuleNotFoundError","evalue":"No module named 'pysbd'","output_type":"error","traceback":["\u001b[0;31m---------------------------------------------------------------------------\u001b[0m","\u001b[0;31mModuleNotFoundError\u001b[0m                       Traceback (most recent call last)","\u001b[0;32m\u003cipython-input-5-73a1c4bbdabf\u003e\u001b[0m in \u001b[0;36m\u003ccell line: 7\u003e\u001b[0;34m()\u001b[0m\n\u001b[1;32m      5\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mIPython\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      6\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----\u003e 7\u001b[0;31m \u001b[0;32mfrom\u001b[0m \u001b[0mTTS\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mutils\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msynthesizer\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mSynthesizer\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      8\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      9\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mnormalize_text\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtext\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/content/TTS/TTS/utils/synthesizer.py\u001b[0m in \u001b[0;36m\u003cmodule\u003e\u001b[0;34m\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mnumpy\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----\u003e 5\u001b[0;31m \u001b[0;32mimport\u001b[0m \u001b[0mpysbd\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      6\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mtorch\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      7\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'pysbd'","","\u001b[0;31m---------------------------------------------------------------------------\u001b[0;32m\nNOTE: If your import is failing due to a missing package, you can\nmanually install dependencies using either !pip or !apt.\n\nTo view examples of installing some common dependencies, click the\n\"Open Examples\" button below.\n\u001b[0;31m---------------------------------------------------------------------------\u001b[0m\n"]}],"source":["%cd /content/TTS\n","import re\n","import sys\n","from unicodedata import normalize\n","import IPython\n","\n","from TTS.utils.synthesizer import Synthesizer\n","\n","def normalize_text(text):\n","    text = text.strip()\n","\n","    for c in \",;:\":\n","        text = text.replace(c, \".\")\n","    text = remove_duplicated_punctuations(text)\n","\n","    text = jamo_text(text)\n","\n","    text = g2p.idioms(text)\n","    text = g2pk.english.convert_eng(text, g2p.cmu)\n","    text = g2pk.utils.annotate(text, g2p.mecab)\n","    text = g2pk.numerals.convert_num(text)\n","    text = re.sub(\"/[PJEB]\", \"\", text)\n","\n","    text = alphabet_text(text)\n","\n","    # remove unreadable characters\n","    text = normalize(\"NFD\", text)\n","    text = \"\".join(c for c in text if c in symbols)\n","    text = normalize(\"NFC\", text)\n","\n","    text = text.strip()\n","    if len(text) == 0:\n","        return \"\"\n","\n","    # only single punctuation\n","    if text in '.!?':\n","        return punctuation_text(text)\n","\n","    # append punctuation if there is no punctuation at the end of the text\n","    if text[-1] not in '.!?':\n","        text += '.'\n","\n","    return text\n","\n","\n","def remove_duplicated_punctuations(text):\n","    text = re.sub(r\"[.?!]+\\?\", \"?\", text)\n","    text = re.sub(r\"[.?!]+!\", \"!\", text)\n","    text = re.sub(r\"[.?!]+\\.\", \".\", text)\n","    return text\n","\n","\n","def split_text(text):\n","    text = remove_duplicated_punctuations(text)\n","\n","    texts = []\n","    for subtext in re.findall(r'[^.!?\\n]*[.!?\\n]', text):\n","        texts.append(subtext.strip())\n","\n","    return texts\n","\n","\n","def alphabet_text(text):\n","    text = re.sub(r\"(a|A)\", \"에이\", text)\n","    text = re.sub(r\"(b|B)\", \"비\", text)\n","    text = re.sub(r\"(c|C)\", \"씨\", text)\n","    text = re.sub(r\"(d|D)\", \"디\", text)\n","    text = re.sub(r\"(e|E)\", \"이\", text)\n","    text = re.sub(r\"(f|F)\", \"에프\", text)\n","    text = re.sub(r\"(g|G)\", \"쥐\", text)\n","    text = re.sub(r\"(h|H)\", \"에이치\", text)\n","    text = re.sub(r\"(i|I)\", \"아이\", text)\n","    text = re.sub(r\"(j|J)\", \"제이\", text)\n","    text = re.sub(r\"(k|K)\", \"케이\", text)\n","    text = re.sub(r\"(l|L)\", \"엘\", text)\n","    text = re.sub(r\"(m|M)\", \"엠\", text)\n","    text = re.sub(r\"(n|N)\", \"엔\", text)\n","    text = re.sub(r\"(o|O)\", \"오\", text)\n","    text = re.sub(r\"(p|P)\", \"피\", text)\n","    text = re.sub(r\"(q|Q)\", \"큐\", text)\n","    text = re.sub(r\"(r|R)\", \"알\", text)\n","    text = re.sub(r\"(s|S)\", \"에스\", text)\n","    text = re.sub(r\"(t|T)\", \"티\", text)\n","    text = re.sub(r\"(u|U)\", \"유\", text)\n","    text = re.sub(r\"(v|V)\", \"브이\", text)\n","    text = re.sub(r\"(w|W)\", \"더블유\", text)\n","    text = re.sub(r\"(x|X)\", \"엑스\", text)\n","    text = re.sub(r\"(y|Y)\", \"와이\", text)\n","    text = re.sub(r\"(z|Z)\", \"지\", text)\n","\n","    return text\n","\n","\n","def punctuation_text(text):\n","    # 문장부호\n","    text = re.sub(r\"!\", \"느낌표\", text)\n","    text = re.sub(r\"\\?\", \"물음표\", text)\n","    text = re.sub(r\"\\.\", \"마침표\", text)\n","\n","    return text\n","\n","\n","def jamo_text(text):\n","    # 기본 자모음\n","    text = re.sub(r\"ㄱ\", \"기역\", text)\n","    text = re.sub(r\"ㄴ\", \"니은\", text)\n","    text = re.sub(r\"ㄷ\", \"디귿\", text)\n","    text = re.sub(r\"ㄹ\", \"리을\", text)\n","    text = re.sub(r\"ㅁ\", \"미음\", text)\n","    text = re.sub(r\"ㅂ\", \"비읍\", text)\n","    text = re.sub(r\"ㅅ\", \"시옷\", text)\n","    text = re.sub(r\"ㅇ\", \"이응\", text)\n","    text = re.sub(r\"ㅈ\", \"지읒\", text)\n","    text = re.sub(r\"ㅊ\", \"치읓\", text)\n","    text = re.sub(r\"ㅋ\", \"키읔\", text)\n","    text = re.sub(r\"ㅌ\", \"티읕\", text)\n","    text = re.sub(r\"ㅍ\", \"피읖\", text)\n","    text = re.sub(r\"ㅎ\", \"히읗\", text)\n","    text = re.sub(r\"ㄲ\", \"쌍기역\", text)\n","    text = re.sub(r\"ㄸ\", \"쌍디귿\", text)\n","    text = re.sub(r\"ㅃ\", \"쌍비읍\", text)\n","    text = re.sub(r\"ㅆ\", \"쌍시옷\", text)\n","    text = re.sub(r\"ㅉ\", \"쌍지읒\", text)\n","    text = re.sub(r\"ㄳ\", \"기역시옷\", text)\n","    text = re.sub(r\"ㄵ\", \"니은지읒\", text)\n","    text = re.sub(r\"ㄶ\", \"니은히읗\", text)\n","    text = re.sub(r\"ㄺ\", \"리을기역\", text)\n","    text = re.sub(r\"ㄻ\", \"리을미음\", text)\n","    text = re.sub(r\"ㄼ\", \"리을비읍\", text)\n","    text = re.sub(r\"ㄽ\", \"리을시옷\", text)\n","    text = re.sub(r\"ㄾ\", \"리을티읕\", text)\n","    text = re.sub(r\"ㄿ\", \"리을피읍\", text)\n","    text = re.sub(r\"ㅀ\", \"리을히읗\", text)\n","    text = re.sub(r\"ㅄ\", \"비읍시옷\", text)\n","    text = re.sub(r\"ㅏ\", \"아\", text)\n","    text = re.sub(r\"ㅑ\", \"야\", text)\n","    text = re.sub(r\"ㅓ\", \"어\", text)\n","    text = re.sub(r\"ㅕ\", \"여\", text)\n","    text = re.sub(r\"ㅗ\", \"오\", text)\n","    text = re.sub(r\"ㅛ\", \"요\", text)\n","    text = re.sub(r\"ㅜ\", \"우\", text)\n","    text = re.sub(r\"ㅠ\", \"유\", text)\n","    text = re.sub(r\"ㅡ\", \"으\", text)\n","    text = re.sub(r\"ㅣ\", \"이\", text)\n","    text = re.sub(r\"ㅐ\", \"애\", text)\n","    text = re.sub(r\"ㅒ\", \"얘\", text)\n","    text = re.sub(r\"ㅔ\", \"에\", text)\n","    text = re.sub(r\"ㅖ\", \"예\", text)\n","    text = re.sub(r\"ㅘ\", \"와\", text)\n","    text = re.sub(r\"ㅙ\", \"왜\", text)\n","    text = re.sub(r\"ㅚ\", \"외\", text)\n","    text = re.sub(r\"ㅝ\", \"워\", text)\n","    text = re.sub(r\"ㅞ\", \"웨\", text)\n","    text = re.sub(r\"ㅟ\", \"위\", text)\n","    text = re.sub(r\"ㅢ\", \"의\", text)\n","\n","    return text\n","\n","\n","def normalize_multiline_text(long_text):\n","    texts = split_text(long_text)\n","    normalized_texts = [normalize_text(text).strip() for text in texts]\n","    return [text for text in normalized_texts if len(text) \u003e 0]\n","\n","def synthesize(text):\n","    wavs = synthesizer.tts(text, None, None)\n","    return wavs"]},{"cell_type":"markdown","metadata":{"id":"SbPRQfl8z28u"},"source":["## 3. 학습한 모델 불러오기\n","\n","학습한 Glow-TTS와 HiFi-GAN 모델을 불러옵니다.\n","\n","만약 다른 체크포인트에서 불러오시려면 아래 코드에서 경로를 아래와 같이 적절하게 수정합니다.\n","\n","```python\n","synthesizer = Synthesizer(\n","    \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/glowtts-v2-May-31-2021_08+17AM-d897f2e/best_model.pth.tar\",\n","    \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/glowtts-v2-May-31-2021_08+17AM-d897f2e/config.json\",\n","    None,\n","    \"/content/drive/My Drive/Colab Notebooks/data/hifigan-v2/hifigan-v2-May-31-2021_08+26AM-d897f2e/checkpoint_300000.pth.tar\",\n","    \"/content/drive/My Drive/Colab Notebooks/data/hifigan-v2/hifigan-v2-May-31-2021_08+26AM-d897f2e/config.json\",\n","    None,\n","    None,\n","    False,\n",")\n","```"]},{"cell_type":"code","execution_count":null,"metadata":{"executionInfo":{"elapsed":10,"status":"aborted","timestamp":1707413252304,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"zwROk8zUHgUn"},"outputs":[],"source":["synthesizer = Synthesizer(\n","    \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/glowtts-v2-April-17-2022_04+46AM-3aa165a/best_model.pth.tar\",\n","    \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/glowtts-v2-April-17-2022_04+46AM-3aa165a/config.json\",\n","    None,\n","    \"/content/drive/My Drive/Colab Notebooks/data/hifigan-v2/hifigan-v2-April-16-2022_11+57AM-3aa165a/best_model.pth.tar\",\n","    \"/content/drive/My Drive/Colab Notebooks/data/hifigan-v2/hifigan-v2-April-16-2022_11+57AM-3aa165a/config.json\",\n","    None,\n","    None,\n","    False,\n",")\n","symbols = synthesizer.tts_config.characters.characters"]},{"cell_type":"markdown","metadata":{"id":"tmjT_BrV0XYD"},"source":["## 4. 음성 합성\n","\n","실제 음성 합성을 수행합니다."]},{"cell_type":"code","execution_count":null,"metadata":{"executionInfo":{"elapsed":11,"status":"aborted","timestamp":1707413252305,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"XSnF1D48F1tx"},"outputs":[],"source":["texts = \"\"\"\n","\n","\"\"\"\n","for text in normalize_multiline_text(texts):\n","    wav = synthesizer.tts(text, None, None)\n","    IPython.display.display(IPython.display.Audio(wav, rate=22050))"]}],"metadata":{"colab":{"collapsed_sections":["8erClGSnzwge"],"name":"","provenance":[{"file_id":"1YkxjzBz3V4eXoAaEgcFNEUg8ZyWV40x9","timestamp":1650109450650},{"file_id":"13pqat2mWsMha7Vn_-Q5_Ih8MDkvz3q5a","timestamp":1622375316346},{"file_id":"1IlZt42ETvNHthRFXfwNSSH-ftWthxzqr","timestamp":1596336131977},{"file_id":"1UinTd1Kp1ytwPQ4QWA610ZKOVfmPDdn5","timestamp":1596300568469}],"version":""},"kernelspec":{"display_name":"Python 3","name":"python3"}},"nbformat":4,"nbformat_minor":0}

train-glowtts-v2.ipynb

@@ -0,0 +1 @@
+{"cells":[{"cell_type":"markdown","metadata":{"id":"yIgz4PyCC9eY"},"source":["# Glow-TTS Training\n","Glow-TTS 학습 진행\n","음성의 말투와 음색을 결정하는 모델"]},{"cell_type":"markdown","metadata":{"id":"nMJBiJ6mECO1"},"source":["## 1. 런타임에 할당된 GPU 확인\n","\n","만약, `GPU: NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.` 라는 메시지가 출력된다면, 위쪽 메뉴에서 `런타임 -> 런타임 유형 변경`을 클릭하고 하드웨어 가속기를 `GPU`로 변경하여 저장한 후 다시 실행 필요"]},{"cell_type":"code","execution_count":1,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":742,"status":"ok","timestamp":1685149753708,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"pHat88bRD4_e","outputId":"ae42575e-4d76-4b02-d10f-8f7d855819b8"},"outputs":[{"name":"stdout","output_type":"stream","text":["GPU: Tesla T4\n"]}],"source":["import os\n","GPU_NAME = os.popen('nvidia-smi --query-gpu=name --format=csv,noheader').read().strip()\n","os.environ['GPU_NAME'] = GPU_NAME\n","print(f'GPU: {GPU_NAME}')"]},{"cell_type":"markdown","metadata":{"id":"CEgknkRoDKj9"},"source":["## 2. 구글 드라이브 마운트\n","\n","마운트할 구글 드라이브 내에 다음 파일이 존재해야함\n","\n","- `/Colab Notebooks/data/filelists.zip`"]},{"cell_type":"code","execution_count":2,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":24329,"status":"ok","timestamp":1685149778033,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"4U2wrDOthrsF","outputId":"2f3187bb-05e1-48f3-95f0-905f395f81e3"},"outputs":[{"name":"stdout","output_type":"stream","text":["Mounted at /content/drive\n"]}],"source":["from google.colab import drive\n","drive.mount('/content/drive')"]},{"cell_type":"markdown","metadata":{"id":"IxpzRw3SDvOL"},"source":["## 3. 필수 라이브러리 및 함수 불러오기\n"]},{"cell_type":"code","execution_count":3,"metadata":{"executionInfo":{"elapsed":6,"status":"ok","timestamp":1685149778033,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"jYCym6hXge2_"},"outputs":[],"source":["import sys\n","from pathlib import Path"]},{"cell_type":"code","execution_count":4,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":2765,"status":"ok","timestamp":1685149780793,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"JkWG-L13gReB","outputId":"c820f3d3-43d7-436a-bd25-1f8c1ee72148"},"outputs":[{"name":"stdout","output_type":"stream","text":["/content\n","Cloning into 'TTS'...\n","remote: Enumerating objects: 447, done.\u001b[K\n","remote: Counting objects: 100% (447/447), done.\u001b[K\n","remote: Compressing objects: 100% (413/413), done.\u001b[K\n","remote: Total 447 (delta 56), reused 222 (delta 22), pack-reused 0\u001b[K\n","Receiving objects: 100% (447/447), 13.77 MiB | 17.84 MiB/s, done.\n","Resolving deltas: 100% (56/56), done.\n","/content/TTS\n","/content/TTS/setup.py:15: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.\n","  if LooseVersion(sys.version) < LooseVersion(\"3.6\") or LooseVersion(sys.version) > LooseVersion(\"3.9\"):\n","Traceback (most recent call last):\n","  File \"/content/TTS/setup.py\", line 16, in <module>\n","    raise RuntimeError(\n","RuntimeError: TTS requires python >= 3.6 and <3.9 but your Python version is 3.10.11 (main, Apr  5 2023, 14:15:10) [GCC 9.4.0]\n"]}],"source":["%cd /content\n","!git clone --depth 1 https://github.com/sce-tts/TTS.git -b sce-tts\n","%cd /content/TTS\n","!python setup.py develop"]},{"cell_type":"markdown","metadata":{"id":"iiXsxJtZERyP"},"source":["## 4. 학습할 데이터셋 불러오기\n","\n","학습에 사용할 음성 데이터를 구글 드라이브에서 가져온다.\n","\n","mimic recording studio로 생성한 `filelists.zip` 필요"]},{"cell_type":"code","execution_count":5,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":5166,"status":"ok","timestamp":1685149785956,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"hExnC_2RhZ3m","outputId":"ee65017a-9473-4ec8-f676-0ee937202b6b"},"outputs":[{"name":"stdout","output_type":"stream","text":["/content/TTS\n"]}],"source":["%cd /content/TTS\n","!cp \"/content/drive/My Drive/Colab Notebooks/data/filelists.zip\" ./filelists.zip\n","!rm -rf ./filelists\n","!unzip -q filelists.zip -d ./filelists"]},{"cell_type":"markdown","metadata":{"id":"qD8zd4SMElbn"},"source":["## 5. 사전 학습 데이터 불러오기\n","\n","\n","> 사전 학습 데이터가 구글 드라이브에 존재하지 않을 경우, 다른 사람의 사전 학습 데이터를 내려받음.\n","\n"]},{"cell_type":"code","execution_count":6,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":32,"status":"ok","timestamp":1685149785960,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"MQxazTNayds-","outputId":"c3cf0aa3-b69e-479f-92b0-e51c93f541fa"},"outputs":[{"name":"stdout","output_type":"stream","text":["/content/TTS\n"]}],"source":["%cd /content/TTS\n","!mkdir -p \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2\"\n","if not Path(\"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/config.json\").exists():\n","    !gdown --id 1DMKLdfZ_gzc_z0qDod6_G8fEXj0zCHvC -O glowtts-v2.zip\n","    !unzip -q glowtts-v2.zip -d ./\n","    !cp -R ./glowtts-v2/* \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/\""]},{"cell_type":"code","execution_count":7,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":29,"status":"ok","timestamp":1685149785961,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"c73wfE5L7uK9","outputId":"859725c8-5287-4b3e-f6d1-bfab3772e250"},"outputs":[{"name":"stdout","output_type":"stream","text":["/content/TTS\n"]}],"source":["%cd /content/TTS\n","if not Path(\"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/scale_stats_new.npy\").exists():\n","    !python TTS/bin/compute_statistics.py \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/config.json\" \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/scale_stats_new.npy\" --data_path \"/content/TTS/filelists/wavs/\""]},{"cell_type":"code","execution_count":8,"metadata":{"executionInfo":{"elapsed":26,"status":"ok","timestamp":1685149785961,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"Q6TCF3Pu-MnV"},"outputs":[],"source":["with open(\"/content/TTS/test_sentences.txt\", mode=\"w\") as f:\n","    f.write(\"\"\"아래 문장들은 모델 학습을 위해 사용하지 않은 문장들입니다.\n","서울특별시 특허허가과 허가과장 허과장.\n","경찰청 철창살은 외철창살이고 검찰청 철창살은 쌍철창살이다.\n","지향을 지양으로 오기하는 일을 지양하는 언어 습관을 지향해야 한다.\n","그러니까 외계인이 우리 생각을 읽고 우리 생각을 우리가 다시 생각토록 해서 그 생각이 마치 우리가 생각한 것인 것처럼 속였다는 거냐?\"\"\")"]},{"cell_type":"markdown","metadata":{"id":"alQe2KpbE9di"},"source":["## 6. TensorBoard 실행\n","\n","학습 진행을 확인하기 위해 TensorBoard를 실행\n","\n","설정 버튼을 눌러 auto reload를 설정해 30초마다 자동 갱신 가능"]},{"cell_type":"code","execution_count":9,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":820},"executionInfo":{"elapsed":5674,"status":"ok","timestamp":1685149791608,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"ydwAZhCQilzJ","outputId":"a3296d49-99d3-4883-9939-ce14229a6367"},"outputs":[{"data":{"application/javascript":"\n        (async () => {\n            const url = new URL(await google.colab.kernel.proxyPort(6006, {'cache': true}));\n            url.searchParams.set('tensorboardColab', 'true');\n            const iframe = document.createElement('iframe');\n            iframe.src = url;\n            iframe.setAttribute('width', '100%');\n            iframe.setAttribute('height', '800');\n            iframe.setAttribute('frameborder', 0);\n            document.body.appendChild(iframe);\n        })();\n    ","text/plain":["<IPython.core.display.Javascript object>"]},"metadata":{},"output_type":"display_data"}],"source":["%load_ext tensorboard\n","%tensorboard --logdir=\"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2\""]},{"cell_type":"markdown","metadata":{"id":"32XUNFa-FQ-R"},"source":["## 7. Glow-TTS 학습 진행\n","\n","학습이 정상적으로 진행되면, 이 셀은 종료되지 않고 계속 실행되는 상태를 유지한다.\n","\n","이전에 학습을 진행하던 모델을 이어서 학습을 진행하시려면 다음과 같이 수정한 후 실행한다.\n","\n","- 아래 셀에서 2 ~ 3번째 줄의 코드를 주석을 해제한다다\n","- 3번째 줄의 경로를 이어서 학습을 진행할 모델의 경로로 변경한다다.  \n","(예시: `/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/glowtts-v2-May-31-2021_08+17AM-d897f2e`)\n","- 4번째 줄 아래의 코드를 제거한다.\n"]},{"cell_type":"code","execution_count":10,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":8529,"status":"ok","timestamp":1685149800131,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"9Yim0zgJk3cR","outputId":"831e6d6a-c0c3-4109-f266-031d0750c13a"},"outputs":[{"name":"stdout","output_type":"stream","text":["/content/TTS\n","Traceback (most recent call last):\n","  File \"/content/TTS/TTS/bin/train_glow_tts.py\", line 17, in <module>\n","    from TTS.tts.datasets.preprocess import load_meta_data\n","ModuleNotFoundError: No module named 'TTS'\n","Traceback (most recent call last):\n","  File \"/content/TTS/TTS/bin/train_glow_tts.py\", line 17, in <module>\n","    from TTS.tts.datasets.preprocess import load_meta_data\n","ModuleNotFoundError: No module named 'TTS'\n"]}],"source":["%cd /content/TTS\n","!(python TTS/bin/train_glow_tts.py \\\n","    -continue_path \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/glowtts-v2-April-17-2022_04+46AM-3aa165a/checkpoint_32000.pth.tar\")\n","!(python TTS/bin/train_glow_tts.py \\\n","    --config_path \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/glowtts-v2-April-17-2022_04+46AM-3aa165a/config.json\" \\\n","    --coqpit.datasets.0.path \"/content/TTS/filelists\"  \\\n","    --coqpit.audio.stats_path \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/scale_stats_new.npy\"  \\\n","    --coqpit.test_sentences_file \"/content/TTS/test_sentences.txt\"  \\\n","    --coqpit.output_path \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/\"  \\\n","    --coqpit.num_loader_workers 2  \\\n","    --coqpit.num_val_loader_workers 2  \\\n","   --restore_path \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/model_file.pth.tar\")"]}],"metadata":{"accelerator":"GPU","colab":{"provenance":[{"file_id":"1L5o8joH8LDV37eupNUpqqWrOcw1sGCit","timestamp":1650106813939},{"file_id":"1IlZt42ETvNHthRFXfwNSSH-ftWthxzqr","timestamp":1622371446894},{"file_id":"1UinTd1Kp1ytwPQ4QWA610ZKOVfmPDdn5","timestamp":1596300568469}]},"kernelspec":{"display_name":"Python 3","name":"python3"}},"nbformat":4,"nbformat_minor":0}

train-hifigan-v2.ipynb

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f6b5aa7fd7c485d9800f1e446b96a510a2a15998f419408339ccfbebd86aaf53
+size 11867103