kodoqmc commited on
Commit
422a7b7
1 Parent(s): f9cfb6b

Upload 9 files

Browse files
Files changed (10) hide show
  1. .gitattributes +2 -0
  2. LICENSE.txt +84 -0
  3. README.md +104 -0
  4. config.json +158 -0
  5. generated.wav +3 -0
  6. model.pth +3 -0
  7. reference.wav +3 -0
  8. speakers_xtts.pth +3 -0
  9. thesanti.jpg +0 -0
  10. vocab.json +0 -0
.gitattributes CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ generated.wav filter=lfs diff=lfs merge=lfs -text
37
+ reference.wav filter=lfs diff=lfs merge=lfs -text
LICENSE.txt ADDED
@@ -0,0 +1,84 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Coqui Public Model License 1.0.0
2
+ https://coqui.ai/cpml.txt
3
+
4
+
5
+ This license allows only non-commercial use of a machine learning model and its outputs.
6
+
7
+
8
+ ## Acceptance
9
+
10
+
11
+ In order to get any license under these terms, you must agree to them as both strict obligations and conditions to all your licenses.
12
+
13
+
14
+ ## Licenses
15
+
16
+
17
+ The licensor grants you a copyright license to do everything you might do with the model that would otherwise infringe the licensor's copyright in it, for any non-commercial purpose. The licensor grants you a patent license that covers patent claims the licensor can license, or becomes able to license, that you would infringe by using the model in the form provided by
18
+ the licensor, for any non-commercial purpose.
19
+
20
+
21
+ ## Non-commercial Purpose
22
+
23
+
24
+ Non-commercial purposes include any of the following uses of the model or its output, but only so far as you do not receive any direct or indirect payment arising from the use of the model or its output.
25
+
26
+
27
+ ### Personal use for research, experiment, and testing for the benefit of public knowledge, personal study, private entertainment, hobby projects, amateur pursuits, or religious
28
+ observance.
29
+
30
+
31
+ ### Use by commercial or for-profit entities for testing, evaluation, or non-commercial research and development. Use of the model to train other models for commercial use is not a non-commercial purpose.
32
+
33
+
34
+ ### Use by any charitable organization for charitable purposes, or for testing or evaluation. Use for revenue-generating activity, including projects directly funded by government grants, is not a non-commercial purpose.
35
+
36
+
37
+ ## Notices
38
+
39
+
40
+ You must ensure that anyone who gets a copy of any part of the model, or any modification of the model, or their output, from you also gets a copy of these terms or the URL for them above.
41
+
42
+
43
+ ## No Other Rights
44
+
45
+
46
+ These terms do not allow you to sublicense or transfer any of your licenses to anyone else, or prevent the licensor from granting licenses to anyone else. These terms do not imply
47
+ any other licenses.
48
+
49
+
50
+ ## Patent Defense
51
+
52
+
53
+ If you make any written claim that the model infringes or contributes to infringement of any patent, your licenses for the model granted under these terms ends immediately. If your company makes such a claim, your patent license ends immediately for work on behalf of your company.
54
+
55
+
56
+ ## Violations
57
+
58
+
59
+ The first time you are notified in writing that you have violated any of these terms, or done anything with the model or its output that is not covered by your licenses, your licenses can nonetheless continue if you come into full compliance with these terms, and take practical steps to correct past violations, within 30 days of receiving notice. Otherwise, all your licenses
60
+ end immediately.
61
+
62
+
63
+ ## No Liability
64
+
65
+
66
+ ***As far as the law allows, the model and its output come as is, without any warranty or condition, and the licensor will not be liable to you for any damages arising out of these terms or the use or nature of the model or its output, under any kind of legal claim. If this provision is not enforceable in your jurisdiction, your licenses are void.***
67
+
68
+
69
+ ## Definitions
70
+
71
+
72
+ The **licensor** is the individual or entity offering these terms, and the **model** is the model the licensor makes available under these terms, including any documentation or similar information about the model.
73
+
74
+
75
+ **You** refers to the individual or entity agreeing to these terms.
76
+
77
+
78
+ **Your company** is any legal entity, sole proprietorship, or other kind of organization that you work for, plus all organizations that have control over, are under the control of, or are under common control with that organization. **Control** means ownership of substantially all the assets of an entity, or the power to direct its management and policies by vote, contract, or otherwise. Control can be direct or indirect.
79
+
80
+
81
+ **Your licenses** are all the licenses granted to you under these terms.
82
+
83
+
84
+ **Use** means anything you do with the model or its output requiring one of your licenses.
README.md ADDED
@@ -0,0 +1,104 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ license_name: coqui-public-model-license
4
+ license_link: https://coqui.ai/cpml
5
+ library_name: coqui
6
+ pipeline_tag: text-to-speech
7
+ widget:
8
+ - text: "Once when I was six years old I saw a magnificent picture"
9
+ ---
10
+
11
+ # ⓍTTS_v2 - The San-Ti Fine-Tuned Model
12
+
13
+ This repository hosts a fine-tuned version of the ⓍTTS model, utilizing 4 minutes of unique voice lines from The San-Ti, The voice lines were sourced from the clip of 3 Body Problem on Youtube, can be found here:
14
+ [The San-Ti Explain how they Stop Science on Earth | 3 Body Problem | Netflix](https://www.youtube.com/watch?v=caxiX38DK68)
15
+
16
+ ![The San-Ti: Illustration](thesanti.jpg)
17
+
18
+ Listen to a sample of the ⓍTTS_v2 - The San-Ti Fine-Tuned Model:
19
+
20
+ <audio controls>
21
+ <source src="https://huggingface.co/kodoqmc/XTTS-v2_San-Ti/resolve/main/Generated.wav" type="audio/wav">
22
+ Your browser does not support the audio element.
23
+ </audio>
24
+
25
+ Here's a The San-Ti mp3 voice line clip from the training data:
26
+
27
+ <audio controls>
28
+ <source src="https://huggingface.co/kodoqmc/XTTS-v2_San-Ti/resolve/main/reference.wav" type="audio/wav">
29
+ Your browser does not support the audio element.
30
+ </audio>
31
+
32
+ ## Features
33
+ - 🎙️ **Voice Cloning**: Realistic voice cloning with just a short audio clip.
34
+ - 🌍 **Multi-Lingual Support**: Generates speech in 17 different languages while maintaining The San-Ti's voice.
35
+ - 😃 **Emotion & Style Transfer**: Captures the emotional tone and style of the original voice.
36
+ - 🔄 **Cross-Language Cloning**: Maintains the unique voice characteristics across different languages.
37
+ - 🎧 **High-Quality Audio**: Outputs at a 24kHz sampling rate for clear and high-fidelity audio.
38
+
39
+ ## Supported Languages
40
+ The model supports the following 17 languages: English (en), Spanish (es), French (fr), German (de), Italian (it), Portuguese (pt), Polish (pl), Turkish (tr), Russian (ru), Dutch (nl), Czech (cs), Arabic (ar), Chinese (zh-cn), Japanese (ja), Hungarian (hu), Korean (ko), and Hindi (hi).
41
+
42
+ ## Usage in Roll Cage
43
+ 🤖💬 Boost your AI experience with this Ollama add-on! Enjoy real-time audio 🎙️ and text 🔍 chats, LaTeX rendering 📜, agent automations ⚙️, workflows 🔄, text-to-image 📝➡️🖼️, image-to-text 🖼️➡️🔤, image-to-video 🖼️➡️🎥 transformations. Fine-tune text 📝, voice 🗣️, and image 🖼️ gens. Includes Windows macro controls 🖥️ and DuckDuckGo search.
44
+
45
+ [ollama_agent_roll_cage (OARC)](https://github.com/Leoleojames1/ollama_agent_roll_cage) is a completely local Python & CMD toolset add-on for the Ollama command line interface. The OARC toolset automates the creation of agents, giving the user more control over the likely output. It provides SYSTEM prompt templates for each ./Modelfile, allowing users to design and deploy custom agents quickly. Users can select which local model file is used in agent construction with the desired system prompt.
46
+
47
+ ## CoquiTTS and Resources
48
+ - 🐸💬 **CoquiTTS**: [Coqui TTS on GitHub](https://github.com/coqui-ai/TTS)
49
+ - 📚 **Documentation**: [ReadTheDocs](https://tts.readthedocs.io/en/latest/)
50
+ - 👩‍💻 **Questions**: [GitHub Discussions](https://github.com/coqui-ai/TTS/discussions)
51
+ - 🗯 **Community**: [Discord](https://discord.gg/5eXr5seRrv)
52
+
53
+ ## License
54
+ This model is licensed under the [Coqui Public Model License](https://coqui.ai/cpml). Read more about the origin story of CPML [here](https://coqui.ai/blog/tts/cpml).
55
+
56
+ ## Contact
57
+ Join our 🐸Community on [Discord](https://discord.gg/fBC58unbKE) and follow us on [Twitter](https://twitter.com/coqui_ai). For inquiries, email us at info@coqui.ai.
58
+
59
+ Using 🐸TTS API:
60
+
61
+ ```python
62
+ from TTS.api import TTS
63
+
64
+ tts = TTS(model_path="D:/AI/ollama_agent_roll_cage/AgentFiles/Ignored_TTS/XTTS-v2_PeterDrury/",
65
+ config_path="D:/AI/ollama_agent_roll_cage/AgentFiles/Ignored_TTS/XTTS-v2_PeterDrury/config.json", progress_bar=False, gpu=True).to(self.device)
66
+
67
+ # generate speech by cloning a voice using default settings
68
+ tts.tts_to_file(text="It took me quite a long time to develop a voice, and now that I have it I'm not going to be silent.",
69
+ file_path="output.wav",
70
+ speaker_wav="/path/to/target/speaker.wav",
71
+ language="en")
72
+
73
+ ```
74
+
75
+ Using 🐸TTS Command line:
76
+
77
+ ```console
78
+ tts --model_name tts_models/multilingual/multi-dataset/xtts_v2 \
79
+ --text "Bugün okula gitmek istemiyorum." \
80
+ --speaker_wav /path/to/target/speaker.wav \
81
+ --language_idx tr \
82
+ --use_cuda true
83
+ ```
84
+
85
+ Using the model directly:
86
+
87
+ ```python
88
+ from TTS.tts.configs.xtts_config import XttsConfig
89
+ from TTS.tts.models.xtts import Xtts
90
+
91
+ config = XttsConfig()
92
+ config.load_json("/path/to/xtts/config.json")
93
+ model = Xtts.init_from_config(config)
94
+ model.load_checkpoint(config, checkpoint_dir="/path/to/xtts/", eval=True)
95
+ model.cuda()
96
+
97
+ outputs = model.synthesize(
98
+ "It took me quite a long time to develop a voice and now that I have it I am not going to be silent.",
99
+ config,
100
+ speaker_wav="/data/TTS-public/_refclips/3.wav",
101
+ gpt_cond_len=3,
102
+ language="en",
103
+ )
104
+ ```
config.json ADDED
@@ -0,0 +1,158 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "output_path": "output",
3
+ "logger_uri": null,
4
+ "run_name": "run",
5
+ "project_name": null,
6
+ "run_description": "\ud83d\udc38Coqui trainer run.",
7
+ "print_step": 25,
8
+ "plot_step": 100,
9
+ "model_param_stats": false,
10
+ "wandb_entity": null,
11
+ "dashboard_logger": "tensorboard",
12
+ "save_on_interrupt": true,
13
+ "log_model_step": null,
14
+ "save_step": 10000,
15
+ "save_n_checkpoints": 5,
16
+ "save_checkpoints": true,
17
+ "save_all_best": false,
18
+ "save_best_after": 10000,
19
+ "target_loss": null,
20
+ "print_eval": false,
21
+ "test_delay_epochs": 0,
22
+ "run_eval": true,
23
+ "run_eval_steps": null,
24
+ "distributed_backend": "nccl",
25
+ "distributed_url": "tcp://localhost:54321",
26
+ "mixed_precision": false,
27
+ "precision": "fp16",
28
+ "epochs": 1000,
29
+ "batch_size": 32,
30
+ "eval_batch_size": 16,
31
+ "grad_clip": 0.0,
32
+ "scheduler_after_epoch": true,
33
+ "lr": 0.001,
34
+ "optimizer": "radam",
35
+ "optimizer_params": null,
36
+ "lr_scheduler": null,
37
+ "lr_scheduler_params": {},
38
+ "use_grad_scaler": false,
39
+ "allow_tf32": false,
40
+ "cudnn_enable": true,
41
+ "cudnn_deterministic": false,
42
+ "cudnn_benchmark": false,
43
+ "training_seed": 54321,
44
+ "model": "xtts",
45
+ "num_loader_workers": 0,
46
+ "num_eval_loader_workers": 0,
47
+ "use_noise_augment": false,
48
+ "audio": {
49
+ "sample_rate": 22050,
50
+ "output_sample_rate": 24000
51
+ },
52
+ "use_phonemes": false,
53
+ "phonemizer": null,
54
+ "phoneme_language": null,
55
+ "compute_input_seq_cache": false,
56
+ "text_cleaner": null,
57
+ "enable_eos_bos_chars": false,
58
+ "test_sentences_file": "",
59
+ "phoneme_cache_path": null,
60
+ "characters": null,
61
+ "add_blank": false,
62
+ "batch_group_size": 0,
63
+ "loss_masking": null,
64
+ "min_audio_len": 1,
65
+ "max_audio_len": Infinity,
66
+ "min_text_len": 1,
67
+ "max_text_len": Infinity,
68
+ "compute_f0": false,
69
+ "compute_energy": false,
70
+ "compute_linear_spec": false,
71
+ "precompute_num_workers": 0,
72
+ "start_by_longest": false,
73
+ "shuffle": false,
74
+ "drop_last": false,
75
+ "datasets": [
76
+ {
77
+ "formatter": "",
78
+ "dataset_name": "",
79
+ "path": "",
80
+ "meta_file_train": "",
81
+ "ignored_speakers": null,
82
+ "language": "",
83
+ "phonemizer": "",
84
+ "meta_file_val": "",
85
+ "meta_file_attn_mask": ""
86
+ }
87
+ ],
88
+ "test_sentences": [],
89
+ "eval_split_max_size": null,
90
+ "eval_split_size": 0.01,
91
+ "use_speaker_weighted_sampler": false,
92
+ "speaker_weighted_sampler_alpha": 1.0,
93
+ "use_language_weighted_sampler": false,
94
+ "language_weighted_sampler_alpha": 1.0,
95
+ "use_length_weighted_sampler": false,
96
+ "length_weighted_sampler_alpha": 1.0,
97
+ "model_args": {
98
+ "gpt_batch_size": 1,
99
+ "enable_redaction": false,
100
+ "kv_cache": true,
101
+ "gpt_checkpoint": null,
102
+ "clvp_checkpoint": null,
103
+ "decoder_checkpoint": null,
104
+ "num_chars": 255,
105
+ "tokenizer_file": "",
106
+ "gpt_max_audio_tokens": 605,
107
+ "gpt_max_text_tokens": 402,
108
+ "gpt_max_prompt_tokens": 70,
109
+ "gpt_layers": 30,
110
+ "gpt_n_model_channels": 1024,
111
+ "gpt_n_heads": 16,
112
+ "gpt_number_text_tokens": 6153,
113
+ "gpt_start_text_token": null,
114
+ "gpt_stop_text_token": null,
115
+ "gpt_num_audio_tokens": 1026,
116
+ "gpt_start_audio_token": 1024,
117
+ "gpt_stop_audio_token": 1025,
118
+ "gpt_code_stride_len": 1024,
119
+ "gpt_use_masking_gt_prompt_approach": true,
120
+ "gpt_use_perceiver_resampler": true,
121
+ "input_sample_rate": 22050,
122
+ "output_sample_rate": 24000,
123
+ "output_hop_length": 256,
124
+ "decoder_input_dim": 1024,
125
+ "d_vector_dim": 512,
126
+ "cond_d_vector_in_each_upsampling_layer": true,
127
+ "duration_const": 102400
128
+ },
129
+ "model_dir": null,
130
+ "languages": [
131
+ "en",
132
+ "es",
133
+ "fr",
134
+ "de",
135
+ "it",
136
+ "pt",
137
+ "pl",
138
+ "tr",
139
+ "ru",
140
+ "nl",
141
+ "cs",
142
+ "ar",
143
+ "zh-cn",
144
+ "hu",
145
+ "ko",
146
+ "ja"
147
+ ],
148
+ "temperature": 0.75,
149
+ "length_penalty": 1.0,
150
+ "repetition_penalty": 10.0,
151
+ "top_k": 50,
152
+ "top_p": 0.85,
153
+ "num_gpt_outputs": 1,
154
+ "gpt_cond_len": 30,
155
+ "gpt_cond_chunk_len": 4,
156
+ "max_ref_len": 30,
157
+ "sound_norm_refs": false
158
+ }
generated.wav ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6ade791020f88e73c93257b843f8266ae2f11c57acb7723e459fc92ec69a6d52
3
+ size 1220688
model.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:58fe73a40329705175ad884d16f2a88e68ddb6566db03a2749c42e71c3c11878
3
+ size 1863948438
reference.wav ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1d7531a47fa3fa5ab8df342296167397f94972c6470b361528aede140e91e782
3
+ size 1337192
speakers_xtts.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f0f6137c19a4eab0cbbe4c99b5babacf68b1746e50da90807708c10e645b943b
3
+ size 7754818
thesanti.jpg ADDED
vocab.json ADDED
The diff for this file is too large to render. See raw diff