icecream0910
commited on
Commit
β’
f5ced3a
1
Parent(s):
96e0a89
Upload 4 files
Browse files- .gitattributes +1 -0
- README.md +106 -3
- infer-v2.ipynb +1 -0
- train-glowtts-v2.ipynb +1 -0
- train-hifigan-v2.ipynb +3 -0
.gitattributes
CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
36 |
+
train-hifigan-v2.ipynb filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
@@ -1,3 +1,106 @@
|
|
1 |
-
|
2 |
-
license
|
3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# MyOwn-TTS
|
2 |
+
[![License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
|
3 |
+
|
4 |
+
## Description
|
5 |
+
|
6 |
+
MyOwn-TTS is a project aimed at creating a text-to-speech (TTS) system that reads sentences in your own voice. This repository includes pre-trained models that have been trained using my voice.
|
7 |
+
|
8 |
+
## Table of Contents
|
9 |
+
|
10 |
+
- [Installation](#installation)
|
11 |
+
- [Usage](#usage)
|
12 |
+
- [Contributing](#contributing)
|
13 |
+
- [License](#license)
|
14 |
+
|
15 |
+
## Installation
|
16 |
+
|
17 |
+
This README focuses on guiding you through the process of synthesizing speech using pre-trained models, rather than detailing the model training process.
|
18 |
+
|
19 |
+
1. Clone the repository:
|
20 |
+
|
21 |
+
```bash
|
22 |
+
git clone https://github.com/IceCream0910/myown-tts.git
|
23 |
+
```
|
24 |
+
|
25 |
+
2. Modify the `run-server.bat` batch file in the `/server` directory to match your actual file paths.
|
26 |
+
|
27 |
+
For example, if your server folder is at `C:\myown-tts\server`, update the file as follows:
|
28 |
+
|
29 |
+
```bat
|
30 |
+
@echo off
|
31 |
+
setlocal
|
32 |
+
cd /D "%~dp0"
|
33 |
+
set MECAB_KO_DIC_PATH=.\mecab\mecab-ko-dic -r .\mecab\mecabrc
|
34 |
+
set TTS_MODEL_FILE=C:\myown-tts\server\models\glowtts-v2\best_model.pth.tar
|
35 |
+
set TTS_MODEL_CONFIG=C:\myown-tts\server\models\glowtts-v2\config.json
|
36 |
+
set VOCODER_MODEL_FILE=C:\myown-tts\server\models\hifigan-v2\best_model.pth.tar
|
37 |
+
set VOCODER_MODEL_CONFIG=C:\myown-tts\server\models\hifigan-v2\config.json
|
38 |
+
server.exe
|
39 |
+
endlocal
|
40 |
+
```
|
41 |
+
|
42 |
+
3. Update the `glowtts-v2/config.json` and `hifigan-v2/config.json` files in the `/server/models/` directory with your actual file paths.
|
43 |
+
|
44 |
+
Ensure you double the backslash (`\\`) in the file paths, as shown below:
|
45 |
+
|
46 |
+
- For `glowtts-v2/config.json`:
|
47 |
+
```json
|
48 |
+
"stats_path": "C:\\mydata\\tts-server\\models\\glowtts-v2\\scale_stats.npy"
|
49 |
+
```
|
50 |
+
|
51 |
+
- For `hifigan-v2/config.json`:
|
52 |
+
```json
|
53 |
+
"stats_path": "C:\\mydata\\tts-server\\models\\hifigan-v2\\scale_stats.npy"
|
54 |
+
```
|
55 |
+
|
56 |
+
## Usage
|
57 |
+
|
58 |
+
To start the TTS server, execute `run-server.bat`. Once the server is running, you will see the message `INFO:werkzeug: * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)` in the command prompt, indicating that the speech synthesis feature is available through the TTS server. To stop the server, press CTRL+C in the command prompt.
|
59 |
+
|
60 |
+
### API
|
61 |
+
|
62 |
+
- Text preprocessing: `/tts-server/api/process-text`
|
63 |
+
|
64 |
+
Splits sentences and removes special characters to automatically stitch together and playback multi-line sentences as you type.
|
65 |
+
|
66 |
+
- Text Inference: `/tts-server/api/infer-glowtts`
|
67 |
+
|
68 |
+
Synthesizes text to speech. Send the text to be synthesized in the `text` parameter of the URL.
|
69 |
+
|
70 |
+
Example:
|
71 |
+
```
|
72 |
+
http://localhost:5000/tts-server/api/infer-glowtts?text=hello
|
73 |
+
```
|
74 |
+
|
75 |
+
### Text Inference Demo Page
|
76 |
+
|
77 |
+
Visit [http://localhost:5000/](http://localhost:5000/) for a demo.
|
78 |
+
|
79 |
+
## Contributing
|
80 |
+
|
81 |
+
1. Fork the repository (https://github.com/icecream0910/myown-tts/fork).
|
82 |
+
2. Create a new branch: `git checkout -b feature/<featureName>`.
|
83 |
+
3. Commit your changes: `git commit -am 'Add <featureName>'`.
|
84 |
+
4. Push to the branch: `git push origin feature/<featureName>`.
|
85 |
+
5. Submit a pull request.
|
86 |
+
|
87 |
+
## License
|
88 |
+
|
89 |
+
This project is licensed under the [MIT License](LICENSE).
|
90 |
+
|
91 |
+
## References
|
92 |
+
|
93 |
+
This implementation draws inspiration from the following repositories:
|
94 |
+
|
95 |
+
- [SCE-TTS](https://github.com/sce-tts)
|
96 |
+
- [g2pK](https://github.com/Kyubyong/g2pK)
|
97 |
+
- [mimic-recording-studio](https://github.com/MycroftAI/mimic-recording-studio)
|
98 |
+
- [coqui TTS](https://github.com/coqui-ai/TTS)
|
99 |
+
|
100 |
+
The datasets below are distributed under the CC-BY 2.0 license, with the original text data provided by the Korea Information Society Development Institute's AI Hub, including Korean dialogue text data and Korean-English translation (parallel) corpus text data.
|
101 |
+
|
102 |
+
- [Korean Corpus for Voice Recording](https://github.com/sce-tts/mimic-recording-studio/blob/master/backend/prompts/korean_corpus.csv)
|
103 |
+
- [SleepingCE Speech Dataset](https://drive.google.com/file/d/1UpoBaZRTJXkTdsoemLBWV48QClm6hpTX/view?usp=sharing)
|
104 |
+
- [Pre-trained Models for SleepingCE Speech Dataset (Glow-TTS)](https://drive.google.com/file/d/1DMKLdfZ_gzc_z0qDod6_G8fEXj0zCHvC/view?usp=sharing)
|
105 |
+
- [Pre-trained Models for SleepingCE Speech Dataset (HiFi-GAN)](https://drive.google.com/file/d/1vRxp1RH-U7gSzWgyxnKY4h_7pB3tjPmU/view?usp=sharing)
|
106 |
+
- These models were fine-tuned from the model provided by [coqui-ai/TTS](https://github.com/coqui-ai/TTS), trained on the [VCTK dataset](https://datashare.ed.ac.uk/handle/10283/3443), available [here](https://github.com/coqui-ai/TTS/releases/download/v0.0.12/vocoder_model--en--vctk--hifigan_v2.zip).
|
infer-v2.ipynb
ADDED
@@ -0,0 +1 @@
|
|
|
|
|
1 |
+
{"cells":[{"cell_type":"markdown","metadata":{"id":"AvxXfBluyZvb"},"source":["## 1. κ΅¬κΈ λλΌμ΄λΈ λ§μ΄νΈ\n","\n","μμ±ν©μ±μ μν΄ νμ΅ν λͺ¨λΈμ΄ μλ κ΅¬κΈ λλΌμ΄λΈλ₯Ό λ§μ΄νΈν©λλ€. \n","λ§μ΄νΈν κ΅¬κΈ λλΌμ΄λΈ λ΄μ λ€μ νμΌλ€μ΄ μ‘΄μ¬νλμ§ κΌ νμΈν΄μ£ΌμΈμ.\n","\n","- `/Colab Notebooks/data/glowtts-v2/model_file.pth.tar`\n","- `/Colab Notebooks/data/glowtts-v2/config.json`\n","- `/Colab Notebooks/data/hifigan-v2/model_file.pth.tar`\n","- `/Colab Notebooks/data/hifigan-v2/config.json`\n","\n","\n","(μ‘΄μ¬νμ§ μλλ€λ©΄, [glowtts-v2.zip](https://drive.google.com/file/d/1DMKLdfZ_gzc_z0qDod6_G8fEXj0zCHvC/view?usp=sharing), [hifigan-v2.zip](https://drive.google.com/file/d/1vRxp1RH-U7gSzWgyxnKY4h_7pB3tjPmU/view?usp=sharing)μ λ΄λ €λ°μ μ€λΉν΄μ£ΌμΈμ.)\n","\n","λ§μ½ μλμ `Enter your authorization code:`κ³Ό κ°μ λ©μμ§κ° μΆλ ₯λ κ²½μ°, \n","κ°μ΄ μΆλ ₯λ λ§ν¬μ μ μνμ¬, λ§μ΄νΈν κ΅¬κΈ κ³μ μ μ ννμ ν, μΈμ¦ μ½λλ₯Ό 볡μ¬νμ¬ μ
λ ₯ν΄μ£ΌμΈμ."]},{"cell_type":"code","execution_count":1,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":230434,"status":"ok","timestamp":1707413202236,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"4U2wrDOthrsF","outputId":"1a56442b-2686-468a-bb32-0ba7608ef988"},"outputs":[{"name":"stdout","output_type":"stream","text":["Mounted at /content/drive\n"]}],"source":["from google.colab import drive\n","drive.mount('/content/drive')"]},{"cell_type":"markdown","metadata":{"id":"8erClGSnzwge"},"source":["## 2. νμ λΌμ΄λΈλ¬λ¦¬ λ° ν¨μ λΆλ¬μ€κΈ°\n","\n","μ€νμ νμν λΌμ΄λΈλ¬λ¦¬ λ° ν¨μλ₯Ό λΆλ¬μ΅λλ€.\n","\n","μ΄ κ³Όμ μ μ½ 10λΆ μ λ μμλ μ μμ΅λλ€."]},{"cell_type":"code","execution_count":2,"metadata":{"executionInfo":{"elapsed":18,"status":"ok","timestamp":1707413202238,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"jYCym6hXge2_"},"outputs":[],"source":["import os\n","import sys\n","from pathlib import Path"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"background_save":true,"base_uri":"https://localhost:8080/"},"id":"JkWG-L13gReB"},"outputs":[{"name":"stdout","output_type":"stream","text":["/content\n","fatal: destination path 'TTS' already exists and is not an empty directory.\n","fatal: destination path 'g2pK' already exists and is not an empty directory.\n","/content/TTS\n"," \u001b[1;31merror\u001b[0m: \u001b[1msubprocess-exited-with-error\u001b[0m\n"," \n"," \u001b[31mΓ\u001b[0m \u001b[32mpip subprocess to install build dependencies\u001b[0m did not run successfully.\n"," \u001b[31mβ\u001b[0m exit code: \u001b[1;36m1\u001b[0m\n"," \u001b[31mβ°β\u003e\u001b[0m See above for output.\n"," \n"," \u001b[1;35mnote\u001b[0m: This error originates from a subprocess, and is likely not a problem with pip.\n"," Installing build dependencies ... \u001b[?25l\u001b[?25herror\n","\u001b[1;31merror\u001b[0m: \u001b[1msubprocess-exited-with-error\u001b[0m\n","\n","\u001b[31mΓ\u001b[0m \u001b[32mpip subprocess to install build dependencies\u001b[0m did not run successfully.\n","\u001b[31mβ\u001b[0m exit code: \u001b[1;36m1\u001b[0m\n","\u001b[31mβ°β\u003e\u001b[0m See above for output.\n","\n","\u001b[1;35mnote\u001b[0m: This error originates from a subprocess, and is likely not a problem with pip.\n","/content/g2pK\n","\u001b[2K \u001b[90mββββββββββββββββββββββββββββββββββββββββ\u001b[0m \u001b[32m71.1/71.1 kB\u001b[0m \u001b[31m2.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25h Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n"]}],"source":["%cd /content\n","!git clone --depth 1 https://github.com/sce-tts/TTS.git -b sce-tts\n","!git clone --depth 1 https://github.com/sce-tts/g2pK.git\n","%cd /content/TTS\n","!pip install -q --no-cache-dir -e .\n","%cd /content/g2pK\n","!pip install -q --no-cache-dir \"pysbd\" \"konlpy\" \"jamo\" \"nltk\" \"python-mecab-ko\"\n","!pip install -q --no-cache-dir -e ."]},{"cell_type":"code","execution_count":4,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":2893,"status":"ok","timestamp":1707413252303,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"FUD8SfIxSY8j","outputId":"3b984ef6-c788-4694-84b1-2675d82ed0c3"},"outputs":[{"name":"stdout","output_type":"stream","text":["/content/g2pK\n"]},{"name":"stderr","output_type":"stream","text":["[nltk_data] Downloading package cmudict to /root/nltk_data...\n","[nltk_data] Unzipping corpora/cmudict.zip.\n"]}],"source":["%cd /content/g2pK\n","import g2pk\n","g2p = g2pk.G2p()"]},{"cell_type":"code","execution_count":5,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":524},"executionInfo":{"elapsed":15,"status":"error","timestamp":1707413252303,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"Lt9bLLZ8I4GH","outputId":"d0eba754-a986-47ee-d154-7233c2ef24c6"},"outputs":[{"name":"stdout","output_type":"stream","text":["/content/TTS\n"]},{"ename":"ModuleNotFoundError","evalue":"No module named 'pysbd'","output_type":"error","traceback":["\u001b[0;31m---------------------------------------------------------------------------\u001b[0m","\u001b[0;31mModuleNotFoundError\u001b[0m Traceback (most recent call last)","\u001b[0;32m\u003cipython-input-5-73a1c4bbdabf\u003e\u001b[0m in \u001b[0;36m\u003ccell line: 7\u003e\u001b[0;34m()\u001b[0m\n\u001b[1;32m 5\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mIPython\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 6\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----\u003e 7\u001b[0;31m \u001b[0;32mfrom\u001b[0m \u001b[0mTTS\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mutils\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msynthesizer\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mSynthesizer\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 8\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 9\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mnormalize_text\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtext\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/content/TTS/TTS/utils/synthesizer.py\u001b[0m in \u001b[0;36m\u003cmodule\u003e\u001b[0;34m\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 4\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mnumpy\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----\u003e 5\u001b[0;31m \u001b[0;32mimport\u001b[0m \u001b[0mpysbd\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 6\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mtorch\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 7\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'pysbd'","","\u001b[0;31m---------------------------------------------------------------------------\u001b[0;32m\nNOTE: If your import is failing due to a missing package, you can\nmanually install dependencies using either !pip or !apt.\n\nTo view examples of installing some common dependencies, click the\n\"Open Examples\" button below.\n\u001b[0;31m---------------------------------------------------------------------------\u001b[0m\n"]}],"source":["%cd /content/TTS\n","import re\n","import sys\n","from unicodedata import normalize\n","import IPython\n","\n","from TTS.utils.synthesizer import Synthesizer\n","\n","def normalize_text(text):\n"," text = text.strip()\n","\n"," for c in \",;:\":\n"," text = text.replace(c, \".\")\n"," text = remove_duplicated_punctuations(text)\n","\n"," text = jamo_text(text)\n","\n"," text = g2p.idioms(text)\n"," text = g2pk.english.convert_eng(text, g2p.cmu)\n"," text = g2pk.utils.annotate(text, g2p.mecab)\n"," text = g2pk.numerals.convert_num(text)\n"," text = re.sub(\"/[PJEB]\", \"\", text)\n","\n"," text = alphabet_text(text)\n","\n"," # remove unreadable characters\n"," text = normalize(\"NFD\", text)\n"," text = \"\".join(c for c in text if c in symbols)\n"," text = normalize(\"NFC\", text)\n","\n"," text = text.strip()\n"," if len(text) == 0:\n"," return \"\"\n","\n"," # only single punctuation\n"," if text in '.!?':\n"," return punctuation_text(text)\n","\n"," # append punctuation if there is no punctuation at the end of the text\n"," if text[-1] not in '.!?':\n"," text += '.'\n","\n"," return text\n","\n","\n","def remove_duplicated_punctuations(text):\n"," text = re.sub(r\"[.?!]+\\?\", \"?\", text)\n"," text = re.sub(r\"[.?!]+!\", \"!\", text)\n"," text = re.sub(r\"[.?!]+\\.\", \".\", text)\n"," return text\n","\n","\n","def split_text(text):\n"," text = remove_duplicated_punctuations(text)\n","\n"," texts = []\n"," for subtext in re.findall(r'[^.!?\\n]*[.!?\\n]', text):\n"," texts.append(subtext.strip())\n","\n"," return texts\n","\n","\n","def alphabet_text(text):\n"," text = re.sub(r\"(a|A)\", \"μμ΄\", text)\n"," text = re.sub(r\"(b|B)\", \"λΉ\", text)\n"," text = re.sub(r\"(c|C)\", \"μ¨\", text)\n"," text = re.sub(r\"(d|D)\", \"λ\", text)\n"," text = re.sub(r\"(e|E)\", \"μ΄\", text)\n"," text = re.sub(r\"(f|F)\", \"μν\", text)\n"," text = re.sub(r\"(g|G)\", \"μ₯\", text)\n"," text = re.sub(r\"(h|H)\", \"μμ΄μΉ\", text)\n"," text = re.sub(r\"(i|I)\", \"μμ΄\", text)\n"," text = re.sub(r\"(j|J)\", \"μ μ΄\", text)\n"," text = re.sub(r\"(k|K)\", \"μΌμ΄\", text)\n"," text = re.sub(r\"(l|L)\", \"μ\", text)\n"," text = re.sub(r\"(m|M)\", \"μ \", text)\n"," text = re.sub(r\"(n|N)\", \"μ\", text)\n"," text = re.sub(r\"(o|O)\", \"μ€\", text)\n"," text = re.sub(r\"(p|P)\", \"νΌ\", text)\n"," text = re.sub(r\"(q|Q)\", \"ν\", text)\n"," text = re.sub(r\"(r|R)\", \"μ\", text)\n"," text = re.sub(r\"(s|S)\", \"μμ€\", text)\n"," text = re.sub(r\"(t|T)\", \"ν°\", text)\n"," text = re.sub(r\"(u|U)\", \"μ \", text)\n"," text = re.sub(r\"(v|V)\", \"λΈμ΄\", text)\n"," text = re.sub(r\"(w|W)\", \"λλΈμ \", text)\n"," text = re.sub(r\"(x|X)\", \"μμ€\", text)\n"," text = re.sub(r\"(y|Y)\", \"μμ΄\", text)\n"," text = re.sub(r\"(z|Z)\", \"μ§\", text)\n","\n"," return text\n","\n","\n","def punctuation_text(text):\n"," # λ¬Έμ₯λΆνΈ\n"," text = re.sub(r\"!\", \"λλν\", text)\n"," text = re.sub(r\"\\?\", \"λ¬Όμν\", text)\n"," text = re.sub(r\"\\.\", \"λ§μΉ¨ν\", text)\n","\n"," return text\n","\n","\n","def jamo_text(text):\n"," # κΈ°λ³Έ μλͺ¨μ\n"," text = re.sub(r\"γ±\", \"κΈ°μ\", text)\n"," text = re.sub(r\"γ΄\", \"λμ\", text)\n"," text = re.sub(r\"γ·\", \"λκ·Ώ\", text)\n"," text = re.sub(r\"γΉ\", \"리μ\", text)\n"," text = re.sub(r\"γ
\", \"λ―Έμ\", text)\n"," text = re.sub(r\"γ
\", \"λΉμ\", text)\n"," text = re.sub(r\"γ
\", \"μμ·\", text)\n"," text = re.sub(r\"γ
\", \"μ΄μ\", text)\n"," text = re.sub(r\"γ
\", \"μ§μ\", text)\n"," text = re.sub(r\"γ
\", \"μΉμ\", text)\n"," text = re.sub(r\"γ
\", \"ν€μ\", text)\n"," text = re.sub(r\"γ
\", \"ν°μ\", text)\n"," text = re.sub(r\"γ
\", \"νΌμ\", text)\n"," text = re.sub(r\"γ
\", \"νμ\", text)\n"," text = re.sub(r\"γ²\", \"μκΈ°μ\", text)\n"," text = re.sub(r\"γΈ\", \"μλκ·Ώ\", text)\n"," text = re.sub(r\"γ
\", \"μλΉμ\", text)\n"," text = re.sub(r\"γ
\", \"μμμ·\", text)\n"," text = re.sub(r\"γ
\", \"μμ§μ\", text)\n"," text = re.sub(r\"γ³\", \"κΈ°μμμ·\", text)\n"," text = re.sub(r\"γ΅\", \"λμμ§μ\", text)\n"," text = re.sub(r\"γΆ\", \"λμνμ\", text)\n"," text = re.sub(r\"γΊ\", \"리μκΈ°μ\", text)\n"," text = re.sub(r\"γ»\", \"리μλ―Έμ\", text)\n"," text = re.sub(r\"γΌ\", \"리μλΉμ\", text)\n"," text = re.sub(r\"γ½\", \"리μμμ·\", text)\n"," text = re.sub(r\"γΎ\", \"리μν°μ\", text)\n"," text = re.sub(r\"γΏ\", \"리μνΌμ\", text)\n"," text = re.sub(r\"γ
\", \"리μνμ\", text)\n"," text = re.sub(r\"γ
\", \"λΉμμμ·\", text)\n"," text = re.sub(r\"γ
\", \"μ\", text)\n"," text = re.sub(r\"γ
\", \"μΌ\", text)\n"," text = re.sub(r\"γ
\", \"μ΄\", text)\n"," text = re.sub(r\"γ
\", \"μ¬\", text)\n"," text = re.sub(r\"γ
\", \"μ€\", text)\n"," text = re.sub(r\"γ
\", \"μ\", text)\n"," text = re.sub(r\"γ
\", \"μ°\", text)\n"," text = re.sub(r\"γ
\", \"μ \", text)\n"," text = re.sub(r\"γ
‘\", \"μΌ\", text)\n"," text = re.sub(r\"γ
£\", \"μ΄\", text)\n"," text = re.sub(r\"γ
\", \"μ \", text)\n"," text = re.sub(r\"γ
\", \"μ\", text)\n"," text = re.sub(r\"γ
\", \"μ\", text)\n"," text = re.sub(r\"γ
\", \"μ\", text)\n"," text = re.sub(r\"γ
\", \"μ\", text)\n"," text = re.sub(r\"γ
\", \"μ\", text)\n"," text = re.sub(r\"γ
\", \"μΈ\", text)\n"," text = re.sub(r\"γ
\", \"μ\", text)\n"," text = re.sub(r\"γ
\", \"μ¨\", text)\n"," text = re.sub(r\"γ
\", \"μ\", text)\n"," text = re.sub(r\"γ
’\", \"μ\", text)\n","\n"," return text\n","\n","\n","def normalize_multiline_text(long_text):\n"," texts = split_text(long_text)\n"," normalized_texts = [normalize_text(text).strip() for text in texts]\n"," return [text for text in normalized_texts if len(text) \u003e 0]\n","\n","def synthesize(text):\n"," wavs = synthesizer.tts(text, None, None)\n"," return wavs"]},{"cell_type":"markdown","metadata":{"id":"SbPRQfl8z28u"},"source":["## 3. νμ΅ν λͺ¨λΈ λΆλ¬μ€κΈ°\n","\n","νμ΅ν Glow-TTSμ HiFi-GAN λͺ¨λΈμ λΆλ¬μ΅λλ€.\n","\n","λ§μ½ λ€λ₯Έ 체ν¬ν¬μΈνΈμμ λΆλ¬μ€μλ €λ©΄ μλ μ½λμμ κ²½λ‘λ₯Ό μλμ κ°μ΄ μ μ νκ² μμ ν©λλ€.\n","\n","```python\n","synthesizer = Synthesizer(\n"," \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/glowtts-v2-May-31-2021_08+17AM-d897f2e/best_model.pth.tar\",\n"," \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/glowtts-v2-May-31-2021_08+17AM-d897f2e/config.json\",\n"," None,\n"," \"/content/drive/My Drive/Colab Notebooks/data/hifigan-v2/hifigan-v2-May-31-2021_08+26AM-d897f2e/checkpoint_300000.pth.tar\",\n"," \"/content/drive/My Drive/Colab Notebooks/data/hifigan-v2/hifigan-v2-May-31-2021_08+26AM-d897f2e/config.json\",\n"," None,\n"," None,\n"," False,\n",")\n","```"]},{"cell_type":"code","execution_count":null,"metadata":{"executionInfo":{"elapsed":10,"status":"aborted","timestamp":1707413252304,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"zwROk8zUHgUn"},"outputs":[],"source":["synthesizer = Synthesizer(\n"," \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/glowtts-v2-April-17-2022_04+46AM-3aa165a/best_model.pth.tar\",\n"," \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/glowtts-v2-April-17-2022_04+46AM-3aa165a/config.json\",\n"," None,\n"," \"/content/drive/My Drive/Colab Notebooks/data/hifigan-v2/hifigan-v2-April-16-2022_11+57AM-3aa165a/best_model.pth.tar\",\n"," \"/content/drive/My Drive/Colab Notebooks/data/hifigan-v2/hifigan-v2-April-16-2022_11+57AM-3aa165a/config.json\",\n"," None,\n"," None,\n"," False,\n",")\n","symbols = synthesizer.tts_config.characters.characters"]},{"cell_type":"markdown","metadata":{"id":"tmjT_BrV0XYD"},"source":["## 4. μμ± ν©μ±\n","\n","μ€μ μμ± ν©μ±μ μνν©λλ€."]},{"cell_type":"code","execution_count":null,"metadata":{"executionInfo":{"elapsed":11,"status":"aborted","timestamp":1707413252305,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"XSnF1D48F1tx"},"outputs":[],"source":["texts = \"\"\"\n","\n","\"\"\"\n","for text in normalize_multiline_text(texts):\n"," wav = synthesizer.tts(text, None, None)\n"," IPython.display.display(IPython.display.Audio(wav, rate=22050))"]}],"metadata":{"colab":{"collapsed_sections":["8erClGSnzwge"],"name":"","provenance":[{"file_id":"1YkxjzBz3V4eXoAaEgcFNEUg8ZyWV40x9","timestamp":1650109450650},{"file_id":"13pqat2mWsMha7Vn_-Q5_Ih8MDkvz3q5a","timestamp":1622375316346},{"file_id":"1IlZt42ETvNHthRFXfwNSSH-ftWthxzqr","timestamp":1596336131977},{"file_id":"1UinTd1Kp1ytwPQ4QWA610ZKOVfmPDdn5","timestamp":1596300568469}],"version":""},"kernelspec":{"display_name":"Python 3","name":"python3"}},"nbformat":4,"nbformat_minor":0}
|
train-glowtts-v2.ipynb
ADDED
@@ -0,0 +1 @@
|
|
|
|
|
1 |
+
{"cells":[{"cell_type":"markdown","metadata":{"id":"yIgz4PyCC9eY"},"source":["# Glow-TTS Training\n","Glow-TTS νμ΅ μ§ν\n","μμ±μ λ§ν¬μ μμμ κ²°μ νλ λͺ¨λΈ"]},{"cell_type":"markdown","metadata":{"id":"nMJBiJ6mECO1"},"source":["## 1. λ°νμμ ν λΉλ GPU νμΈ\n","\n","λ§μ½, `GPU: NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.` λΌλ λ©μμ§κ° μΆλ ₯λλ€λ©΄, μμͺ½ λ©λ΄μμ `λ°νμ -> λ°νμ μ ν λ³κ²½`μ ν΄λ¦νκ³ νλμ¨μ΄ κ°μκΈ°λ₯Ό `GPU`λ‘ λ³κ²½νμ¬ μ μ₯ν ν λ€μ μ€ν νμ"]},{"cell_type":"code","execution_count":1,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":742,"status":"ok","timestamp":1685149753708,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"pHat88bRD4_e","outputId":"ae42575e-4d76-4b02-d10f-8f7d855819b8"},"outputs":[{"name":"stdout","output_type":"stream","text":["GPU: Tesla T4\n"]}],"source":["import os\n","GPU_NAME = os.popen('nvidia-smi --query-gpu=name --format=csv,noheader').read().strip()\n","os.environ['GPU_NAME'] = GPU_NAME\n","print(f'GPU: {GPU_NAME}')"]},{"cell_type":"markdown","metadata":{"id":"CEgknkRoDKj9"},"source":["## 2. κ΅¬κΈ λλΌμ΄λΈ λ§μ΄νΈ\n","\n","λ§μ΄νΈν κ΅¬κΈ λλΌμ΄λΈ λ΄μ λ€μ νμΌμ΄ μ‘΄μ¬ν΄μΌν¨\n","\n","- `/Colab Notebooks/data/filelists.zip`"]},{"cell_type":"code","execution_count":2,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":24329,"status":"ok","timestamp":1685149778033,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"4U2wrDOthrsF","outputId":"2f3187bb-05e1-48f3-95f0-905f395f81e3"},"outputs":[{"name":"stdout","output_type":"stream","text":["Mounted at /content/drive\n"]}],"source":["from google.colab import drive\n","drive.mount('/content/drive')"]},{"cell_type":"markdown","metadata":{"id":"IxpzRw3SDvOL"},"source":["## 3. νμ λΌμ΄λΈλ¬λ¦¬ λ° ν¨μ λΆλ¬μ€κΈ°\n"]},{"cell_type":"code","execution_count":3,"metadata":{"executionInfo":{"elapsed":6,"status":"ok","timestamp":1685149778033,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"jYCym6hXge2_"},"outputs":[],"source":["import sys\n","from pathlib import Path"]},{"cell_type":"code","execution_count":4,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":2765,"status":"ok","timestamp":1685149780793,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"JkWG-L13gReB","outputId":"c820f3d3-43d7-436a-bd25-1f8c1ee72148"},"outputs":[{"name":"stdout","output_type":"stream","text":["/content\n","Cloning into 'TTS'...\n","remote: Enumerating objects: 447, done.\u001b[K\n","remote: Counting objects: 100% (447/447), done.\u001b[K\n","remote: Compressing objects: 100% (413/413), done.\u001b[K\n","remote: Total 447 (delta 56), reused 222 (delta 22), pack-reused 0\u001b[K\n","Receiving objects: 100% (447/447), 13.77 MiB | 17.84 MiB/s, done.\n","Resolving deltas: 100% (56/56), done.\n","/content/TTS\n","/content/TTS/setup.py:15: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.\n"," if LooseVersion(sys.version) < LooseVersion(\"3.6\") or LooseVersion(sys.version) > LooseVersion(\"3.9\"):\n","Traceback (most recent call last):\n"," File \"/content/TTS/setup.py\", line 16, in <module>\n"," raise RuntimeError(\n","RuntimeError: TTS requires python >= 3.6 and <3.9 but your Python version is 3.10.11 (main, Apr 5 2023, 14:15:10) [GCC 9.4.0]\n"]}],"source":["%cd /content\n","!git clone --depth 1 https://github.com/sce-tts/TTS.git -b sce-tts\n","%cd /content/TTS\n","!python setup.py develop"]},{"cell_type":"markdown","metadata":{"id":"iiXsxJtZERyP"},"source":["## 4. νμ΅ν λ°μ΄ν°μ
λΆλ¬μ€κΈ°\n","\n","νμ΅μ μ¬μ©ν μμ± λ°μ΄ν°λ₯Ό κ΅¬κΈ λλΌμ΄λΈμμ κ°μ Έμ¨λ€.\n","\n","mimic recording studioλ‘ μμ±ν `filelists.zip` νμ"]},{"cell_type":"code","execution_count":5,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":5166,"status":"ok","timestamp":1685149785956,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"hExnC_2RhZ3m","outputId":"ee65017a-9473-4ec8-f676-0ee937202b6b"},"outputs":[{"name":"stdout","output_type":"stream","text":["/content/TTS\n"]}],"source":["%cd /content/TTS\n","!cp \"/content/drive/My Drive/Colab Notebooks/data/filelists.zip\" ./filelists.zip\n","!rm -rf ./filelists\n","!unzip -q filelists.zip -d ./filelists"]},{"cell_type":"markdown","metadata":{"id":"qD8zd4SMElbn"},"source":["## 5. μ¬μ νμ΅ λ°μ΄ν° λΆλ¬μ€κΈ°\n","\n","\n","> μ¬μ νμ΅ λ°μ΄ν°κ° κ΅¬κΈ λλΌμ΄λΈμ μ‘΄μ¬νμ§ μμ κ²½μ°, λ€λ₯Έ μ¬λμ μ¬μ νμ΅ λ°μ΄ν°λ₯Ό λ΄λ €λ°μ.\n","\n"]},{"cell_type":"code","execution_count":6,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":32,"status":"ok","timestamp":1685149785960,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"MQxazTNayds-","outputId":"c3cf0aa3-b69e-479f-92b0-e51c93f541fa"},"outputs":[{"name":"stdout","output_type":"stream","text":["/content/TTS\n"]}],"source":["%cd /content/TTS\n","!mkdir -p \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2\"\n","if not Path(\"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/config.json\").exists():\n"," !gdown --id 1DMKLdfZ_gzc_z0qDod6_G8fEXj0zCHvC -O glowtts-v2.zip\n"," !unzip -q glowtts-v2.zip -d ./\n"," !cp -R ./glowtts-v2/* \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/\""]},{"cell_type":"code","execution_count":7,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":29,"status":"ok","timestamp":1685149785961,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"c73wfE5L7uK9","outputId":"859725c8-5287-4b3e-f6d1-bfab3772e250"},"outputs":[{"name":"stdout","output_type":"stream","text":["/content/TTS\n"]}],"source":["%cd /content/TTS\n","if not Path(\"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/scale_stats_new.npy\").exists():\n"," !python TTS/bin/compute_statistics.py \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/config.json\" \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/scale_stats_new.npy\" --data_path \"/content/TTS/filelists/wavs/\""]},{"cell_type":"code","execution_count":8,"metadata":{"executionInfo":{"elapsed":26,"status":"ok","timestamp":1685149785961,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"Q6TCF3Pu-MnV"},"outputs":[],"source":["with open(\"/content/TTS/test_sentences.txt\", mode=\"w\") as f:\n"," f.write(\"\"\"μλ λ¬Έμ₯λ€μ λͺ¨λΈ νμ΅μ μν΄ μ¬μ©νμ§ μμ λ¬Έμ₯λ€μ
λλ€.\n","μμΈνΉλ³μ νΉννκ°κ³Ό νκ°κ³Όμ₯ νκ³Όμ₯.\n","κ²½μ°°μ² μ² μ°½μ΄μ μΈμ² μ°½μ΄μ΄κ³ κ²μ°°μ² μ² μ°½μ΄μ μμ² μ°½μ΄μ΄λ€.\n","μ§ν₯μ μ§μμΌλ‘ μ€κΈ°νλ μΌμ μ§μνλ μΈμ΄ μ΅κ΄μ μ§ν₯ν΄μΌ νλ€.\n","κ·Έλ¬λκΉ μΈκ³μΈμ΄ μ°λ¦¬ μκ°μ μ½κ³ μ°λ¦¬ μκ°μ μ°λ¦¬κ° λ€μ μκ°ν λ‘ ν΄μ κ·Έ μκ°μ΄ λ§μΉ μ°λ¦¬κ° μκ°ν κ²μΈ κ²μ²λΌ μμλ€λ κ±°λ?\"\"\")"]},{"cell_type":"markdown","metadata":{"id":"alQe2KpbE9di"},"source":["## 6. TensorBoard μ€ν\n","\n","νμ΅ μ§νμ νμΈνκΈ° μν΄ TensorBoardλ₯Ό μ€ν\n","\n","μ€μ λ²νΌμ λλ¬ auto reloadλ₯Ό μ€μ ν΄ 30μ΄λ§λ€ μλ κ°±μ κ°λ₯"]},{"cell_type":"code","execution_count":9,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":820},"executionInfo":{"elapsed":5674,"status":"ok","timestamp":1685149791608,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"ydwAZhCQilzJ","outputId":"a3296d49-99d3-4883-9939-ce14229a6367"},"outputs":[{"data":{"application/javascript":"\n (async () => {\n const url = new URL(await google.colab.kernel.proxyPort(6006, {'cache': true}));\n url.searchParams.set('tensorboardColab', 'true');\n const iframe = document.createElement('iframe');\n iframe.src = url;\n iframe.setAttribute('width', '100%');\n iframe.setAttribute('height', '800');\n iframe.setAttribute('frameborder', 0);\n document.body.appendChild(iframe);\n })();\n ","text/plain":["<IPython.core.display.Javascript object>"]},"metadata":{},"output_type":"display_data"}],"source":["%load_ext tensorboard\n","%tensorboard --logdir=\"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2\""]},{"cell_type":"markdown","metadata":{"id":"32XUNFa-FQ-R"},"source":["## 7. Glow-TTS νμ΅ μ§ν\n","\n","νμ΅μ΄ μ μμ μΌλ‘ μ§νλλ©΄, μ΄ μ
μ μ’
λ£λμ§ μκ³ κ³μ μ€νλλ μνλ₯Ό μ μ§νλ€.\n","\n","μ΄μ μ νμ΅μ μ§ννλ λͺ¨λΈμ μ΄μ΄μ νμ΅μ μ§ννμλ €λ©΄ λ€μκ³Ό κ°μ΄ μμ ν ν μ€ννλ€.\n","\n","- μλ μ
μμ 2 ~ 3λ²μ§Έ μ€μ μ½λλ₯Ό μ£Όμμ ν΄μ νλ€λ€\n","- 3λ²μ§Έ μ€μ κ²½λ‘λ₯Ό μ΄μ΄μ νμ΅μ μ§νν λͺ¨λΈμ κ²½λ‘λ‘ λ³κ²½νλ€λ€. \n","(μμ: `/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/glowtts-v2-May-31-2021_08+17AM-d897f2e`)\n","- 4λ²μ§Έ μ€ μλμ μ½λλ₯Ό μ κ±°νλ€.\n"]},{"cell_type":"code","execution_count":10,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":8529,"status":"ok","timestamp":1685149800131,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"9Yim0zgJk3cR","outputId":"831e6d6a-c0c3-4109-f266-031d0750c13a"},"outputs":[{"name":"stdout","output_type":"stream","text":["/content/TTS\n","Traceback (most recent call last):\n"," File \"/content/TTS/TTS/bin/train_glow_tts.py\", line 17, in <module>\n"," from TTS.tts.datasets.preprocess import load_meta_data\n","ModuleNotFoundError: No module named 'TTS'\n","Traceback (most recent call last):\n"," File \"/content/TTS/TTS/bin/train_glow_tts.py\", line 17, in <module>\n"," from TTS.tts.datasets.preprocess import load_meta_data\n","ModuleNotFoundError: No module named 'TTS'\n"]}],"source":["%cd /content/TTS\n","!(python TTS/bin/train_glow_tts.py \\\n"," -continue_path \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/glowtts-v2-April-17-2022_04+46AM-3aa165a/checkpoint_32000.pth.tar\")\n","!(python TTS/bin/train_glow_tts.py \\\n"," --config_path \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/glowtts-v2-April-17-2022_04+46AM-3aa165a/config.json\" \\\n"," --coqpit.datasets.0.path \"/content/TTS/filelists\" \\\n"," --coqpit.audio.stats_path \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/scale_stats_new.npy\" \\\n"," --coqpit.test_sentences_file \"/content/TTS/test_sentences.txt\" \\\n"," --coqpit.output_path \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/\" \\\n"," --coqpit.num_loader_workers 2 \\\n"," --coqpit.num_val_loader_workers 2 \\\n"," --restore_path \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/model_file.pth.tar\")"]}],"metadata":{"accelerator":"GPU","colab":{"provenance":[{"file_id":"1L5o8joH8LDV37eupNUpqqWrOcw1sGCit","timestamp":1650106813939},{"file_id":"1IlZt42ETvNHthRFXfwNSSH-ftWthxzqr","timestamp":1622371446894},{"file_id":"1UinTd1Kp1ytwPQ4QWA610ZKOVfmPDdn5","timestamp":1596300568469}]},"kernelspec":{"display_name":"Python 3","name":"python3"}},"nbformat":4,"nbformat_minor":0}
|
train-hifigan-v2.ipynb
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:f6b5aa7fd7c485d9800f1e446b96a510a2a15998f419408339ccfbebd86aaf53
|
3 |
+
size 11867103
|