Repository: voicepaw/so-vits-svc-fork

Issue:
Implement `num_workers` on trainer to boost performance
**Is your feature request related to a problem? Please describe.**
We currently don't use any `num_workers` for sub-processes as per torch documentation, resulting in warnings in the console that the validation and the training doesn't have any available.

Adding that variable to both and setting it to 4 for each results in a jump from about 1.40it/s to 2.50it/s, at least in my testing.

https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader
> num_workers ([int](https://docs.python.org/3/library/functions.html#int), optional) – how many subprocesses to use for data loading. 0 means that the data will be loaded in the main process. (default: 0)

**Describe the solution you'd like**
Implement support for `num_workers`, perhaps as extra fields for both the validator and the trainer in the config.

**Additional context**
**There is currently one downside:**
Doing so will, at least with the current pytorch version, throw multiple warnings in the console (and logs) mentioning the wrong usage of `TypedStorage` and how it's deprecated.
![image](https://user-images.githubusercontent.com/1345036/231594686-0b49b5b5-58ba-4143-8294-a20bc4c36d9f.png)
___
I have also noticed that with this change it is staying at a steady VRAM usage (with batch size 16 it stays at 14.8GB, with 20 it stays at around 16.8GB).

It drops to a lower usage whenever it's saving the checkpoints though, but that's fine.

I assume this is also part of where the improvements come from since it doesn't have to constantly reload through the main thread / process and can load the new stuff in the background.


Discussion:

Resolving patch:
diff --git a/src/so_vits_svc_fork/__main__.py b/src/so_vits_svc_fork/__main__.py
--- a/src/so_vits_svc_fork/__main__.py
+++ b/src/so_vits_svc_fork/__main__.py
@@ -2,6 +2,7 @@
 
 import os
 from logging import getLogger
+from multiprocessing import freeze_support
 from pathlib import Path
 from typing import Literal
 
@@ -819,4 +820,6 @@ def train_cluster(
     )
 
 
-cli()
+if __name__ == "__main__":
+    freeze_support()
+    cli()
diff --git a/src/so_vits_svc_fork/train.py b/src/so_vits_svc_fork/train.py
--- a/src/so_vits_svc_fork/train.py
+++ b/src/so_vits_svc_fork/train.py
@@ -2,6 +2,7 @@
 
 import warnings
 from logging import getLogger
+from multiprocessing import cpu_count
 from pathlib import Path
 from typing import Any
 
@@ -41,9 +42,10 @@ def __init__(self, hparams: Any):
         self.val_dataset = TextAudioDataset(self.__hparams, is_validation=True)
 
     def train_dataloader(self):
-        # since dataset just reads data from a file, set num_workers to 0
         return DataLoader(
             self.train_dataset,
+            # pin_memory=False,
+            num_workers=min(cpu_count(), 4),
             batch_size=self.__hparams.train.batch_size,
             collate_fn=self.collate_fn,
         )
@@ -51,6 +53,7 @@ def train_dataloader(self):
     def val_dataloader(self):
         return DataLoader(
             self.val_dataset,
+            # pin_memory=False,
             batch_size=1,
             collate_fn=self.collate_fn,
         )


Test patch: