train-glowtts-v2.ipynb

{"cells":[{"cell_type":"markdown","metadata":{"id":"yIgz4PyCC9eY"},"source":["# Glow-TTS Training\n","Glow-TTS 학습 진행\n","음성의 말투와 음색을 결정하는 모델"]},{"cell_type":"markdown","metadata":{"id":"nMJBiJ6mECO1"},"source":["## 1. 런타임에 할당된 GPU 확인\n","\n","만약, `GPU: NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.` 라는 메시지가 출력된다면, 위쪽 메뉴에서 `런타임 -> 런타임 유형 변경`을 클릭하고 하드웨어 가속기를 `GPU`로 변경하여 저장한 후 다시 실행 필요"]},{"cell_type":"code","execution_count":1,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":742,"status":"ok","timestamp":1685149753708,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"pHat88bRD4_e","outputId":"ae42575e-4d76-4b02-d10f-8f7d855819b8"},"outputs":[{"name":"stdout","output_type":"stream","text":["GPU: Tesla T4\n"]}],"source":["import os\n","GPU_NAME = os.popen('nvidia-smi --query-gpu=name --format=csv,noheader').read().strip()\n","os.environ['GPU_NAME'] = GPU_NAME\n","print(f'GPU: {GPU_NAME}')"]},{"cell_type":"markdown","metadata":{"id":"CEgknkRoDKj9"},"source":["## 2. 구글 드라이브 마운트\n","\n","마운트할 구글 드라이브 내에 다음 파일이 존재해야함\n","\n","- `/Colab Notebooks/data/filelists.zip`"]},{"cell_type":"code","execution_count":2,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":24329,"status":"ok","timestamp":1685149778033,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"4U2wrDOthrsF","outputId":"2f3187bb-05e1-48f3-95f0-905f395f81e3"},"outputs":[{"name":"stdout","output_type":"stream","text":["Mounted at /content/drive\n"]}],"source":["from google.colab import drive\n","drive.mount('/content/drive')"]},{"cell_type":"markdown","metadata":{"id":"IxpzRw3SDvOL"},"source":["## 3. 필수 라이브러리 및 함수 불러오기\n"]},{"cell_type":"code","execution_count":3,"metadata":{"executionInfo":{"elapsed":6,"status":"ok","timestamp":1685149778033,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"jYCym6hXge2_"},"outputs":[],"source":["import sys\n","from pathlib import Path"]},{"cell_type":"code","execution_count":4,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":2765,"status":"ok","timestamp":1685149780793,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"JkWG-L13gReB","outputId":"c820f3d3-43d7-436a-bd25-1f8c1ee72148"},"outputs":[{"name":"stdout","output_type":"stream","text":["/content\n","Cloning into 'TTS'...\n","remote: Enumerating objects: 447, done.\u001b[K\n","remote: Counting objects: 100% (447/447), done.\u001b[K\n","remote: Compressing objects: 100% (413/413), done.\u001b[K\n","remote: Total 447 (delta 56), reused 222 (delta 22), pack-reused 0\u001b[K\n","Receiving objects: 100% (447/447), 13.77 MiB | 17.84 MiB/s, done.\n","Resolving deltas: 100% (56/56), done.\n","/content/TTS\n","/content/TTS/setup.py:15: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.\n","  if LooseVersion(sys.version) < LooseVersion(\"3.6\") or LooseVersion(sys.version) > LooseVersion(\"3.9\"):\n","Traceback (most recent call last):\n","  File \"/content/TTS/setup.py\", line 16, in <module>\n","    raise RuntimeError(\n","RuntimeError: TTS requires python >= 3.6 and <3.9 but your Python version is 3.10.11 (main, Apr  5 2023, 14:15:10) [GCC 9.4.0]\n"]}],"source":["%cd /content\n","!git clone --depth 1 https://github.com/sce-tts/TTS.git -b sce-tts\n","%cd /content/TTS\n","!python setup.py develop"]},{"cell_type":"markdown","metadata":{"id":"iiXsxJtZERyP"},"source":["## 4. 학습할 데이터셋 불러오기\n","\n","학습에 사용할 음성 데이터를 구글 드라이브에서 가져온다.\n","\n","mimic recording studio로 생성한 `filelists.zip` 필요"]},{"cell_type":"code","execution_count":5,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":5166,"status":"ok","timestamp":1685149785956,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"hExnC_2RhZ3m","outputId":"ee65017a-9473-4ec8-f676-0ee937202b6b"},"outputs":[{"name":"stdout","output_type":"stream","text":["/content/TTS\n"]}],"source":["%cd /content/TTS\n","!cp \"/content/drive/My Drive/Colab Notebooks/data/filelists.zip\" ./filelists.zip\n","!rm -rf ./filelists\n","!unzip -q filelists.zip -d ./filelists"]},{"cell_type":"markdown","metadata":{"id":"qD8zd4SMElbn"},"source":["## 5. 사전 학습 데이터 불러오기\n","\n","\n","> 사전 학습 데이터가 구글 드라이브에 존재하지 않을 경우, 다른 사람의 사전 학습 데이터를 내려받음.\n","\n"]},{"cell_type":"code","execution_count":6,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":32,"status":"ok","timestamp":1685149785960,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"MQxazTNayds-","outputId":"c3cf0aa3-b69e-479f-92b0-e51c93f541fa"},"outputs":[{"name":"stdout","output_type":"stream","text":["/content/TTS\n"]}],"source":["%cd /content/TTS\n","!mkdir -p \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2\"\n","if not Path(\"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/config.json\").exists():\n","    !gdown --id 1DMKLdfZ_gzc_z0qDod6_G8fEXj0zCHvC -O glowtts-v2.zip\n","    !unzip -q glowtts-v2.zip -d ./\n","    !cp -R ./glowtts-v2/* \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/\""]},{"cell_type":"code","execution_count":7,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":29,"status":"ok","timestamp":1685149785961,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"c73wfE5L7uK9","outputId":"859725c8-5287-4b3e-f6d1-bfab3772e250"},"outputs":[{"name":"stdout","output_type":"stream","text":["/content/TTS\n"]}],"source":["%cd /content/TTS\n","if not Path(\"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/scale_stats_new.npy\").exists():\n","    !python TTS/bin/compute_statistics.py \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/config.json\" \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/scale_stats_new.npy\" --data_path \"/content/TTS/filelists/wavs/\""]},{"cell_type":"code","execution_count":8,"metadata":{"executionInfo":{"elapsed":26,"status":"ok","timestamp":1685149785961,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"Q6TCF3Pu-MnV"},"outputs":[],"source":["with open(\"/content/TTS/test_sentences.txt\", mode=\"w\") as f:\n","    f.write(\"\"\"아래 문장들은 모델 학습을 위해 사용하지 않은 문장들입니다.\n","서울특별시 특허허가과 허가과장 허과장.\n","경찰청 철창살은 외철창살이고 검찰청 철창살은 쌍철창살이다.\n","지향을 지양으로 오기하는 일을 지양하는 언어 습관을 지향해야 한다.\n","그러니까 외계인이 우리 생각을 읽고 우리 생각을 우리가 다시 생각토록 해서 그 생각이 마치 우리가 생각한 것인 것처럼 속였다는 거냐?\"\"\")"]},{"cell_type":"markdown","metadata":{"id":"alQe2KpbE9di"},"source":["## 6. TensorBoard 실행\n","\n","학습 진행을 확인하기 위해 TensorBoard를 실행\n","\n","설정 버튼을 눌러 auto reload를 설정해 30초마다 자동 갱신 가능"]},{"cell_type":"code","execution_count":9,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":820},"executionInfo":{"elapsed":5674,"status":"ok","timestamp":1685149791608,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"ydwAZhCQilzJ","outputId":"a3296d49-99d3-4883-9939-ce14229a6367"},"outputs":[{"data":{"application/javascript":"\n        (async () => {\n            const url = new URL(await google.colab.kernel.proxyPort(6006, {'cache': true}));\n            url.searchParams.set('tensorboardColab', 'true');\n            const iframe = document.createElement('iframe');\n            iframe.src = url;\n            iframe.setAttribute('width', '100%');\n            iframe.setAttribute('height', '800');\n            iframe.setAttribute('frameborder', 0);\n            document.body.appendChild(iframe);\n        })();\n    ","text/plain":["<IPython.core.display.Javascript object>"]},"metadata":{},"output_type":"display_data"}],"source":["%load_ext tensorboard\n","%tensorboard --logdir=\"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2\""]},{"cell_type":"markdown","metadata":{"id":"32XUNFa-FQ-R"},"source":["## 7. Glow-TTS 학습 진행\n","\n","학습이 정상적으로 진행되면, 이 셀은 종료되지 않고 계속 실행되는 상태를 유지한다.\n","\n","이전에 학습을 진행하던 모델을 이어서 학습을 진행하시려면 다음과 같이 수정한 후 실행한다.\n","\n","- 아래 셀에서 2 ~ 3번째 줄의 코드를 주석을 해제한다다\n","- 3번째 줄의 경로를 이어서 학습을 진행할 모델의 경로로 변경한다다.  \n","(예시: `/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/glowtts-v2-May-31-2021_08+17AM-d897f2e`)\n","- 4번째 줄 아래의 코드를 제거한다.\n"]},{"cell_type":"code","execution_count":10,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":8529,"status":"ok","timestamp":1685149800131,"user":{"displayName":"Cream Ice","userId":"02668969734157440879"},"user_tz":-540},"id":"9Yim0zgJk3cR","outputId":"831e6d6a-c0c3-4109-f266-031d0750c13a"},"outputs":[{"name":"stdout","output_type":"stream","text":["/content/TTS\n","Traceback (most recent call last):\n","  File \"/content/TTS/TTS/bin/train_glow_tts.py\", line 17, in <module>\n","    from TTS.tts.datasets.preprocess import load_meta_data\n","ModuleNotFoundError: No module named 'TTS'\n","Traceback (most recent call last):\n","  File \"/content/TTS/TTS/bin/train_glow_tts.py\", line 17, in <module>\n","    from TTS.tts.datasets.preprocess import load_meta_data\n","ModuleNotFoundError: No module named 'TTS'\n"]}],"source":["%cd /content/TTS\n","!(python TTS/bin/train_glow_tts.py \\\n","    -continue_path \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/glowtts-v2-April-17-2022_04+46AM-3aa165a/checkpoint_32000.pth.tar\")\n","!(python TTS/bin/train_glow_tts.py \\\n","    --config_path \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/glowtts-v2-April-17-2022_04+46AM-3aa165a/config.json\" \\\n","    --coqpit.datasets.0.path \"/content/TTS/filelists\"  \\\n","    --coqpit.audio.stats_path \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/scale_stats_new.npy\"  \\\n","    --coqpit.test_sentences_file \"/content/TTS/test_sentences.txt\"  \\\n","    --coqpit.output_path \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/\"  \\\n","    --coqpit.num_loader_workers 2  \\\n","    --coqpit.num_val_loader_workers 2  \\\n","   --restore_path \"/content/drive/My Drive/Colab Notebooks/data/glowtts-v2/model_file.pth.tar\")"]}],"metadata":{"accelerator":"GPU","colab":{"provenance":[{"file_id":"1L5o8joH8LDV37eupNUpqqWrOcw1sGCit","timestamp":1650106813939},{"file_id":"1IlZt42ETvNHthRFXfwNSSH-ftWthxzqr","timestamp":1622371446894},{"file_id":"1UinTd1Kp1ytwPQ4QWA610ZKOVfmPDdn5","timestamp":1596300568469}]},"kernelspec":{"display_name":"Python 3","name":"python3"}},"nbformat":4,"nbformat_minor":0}