first

CLAUDE.md

@@ -0,0 +1,96 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+KaniTTS-2 is a Gradio-based web application for text-to-speech generation using the KaniTTS model family from PyPI. It's designed to run on HuggingFace Spaces with GPU acceleration via the `@spaces.GPU` decorator.
+
+## Running the Application
+
+```bash
+python app.py
+```
+
+The app launches on `0.0.0.0:7860` with a Gradio interface.
+
+## Architecture
+
+### Initialization Flow
+
+The application follows a strict initialization sequence that must be maintained:
+
+1. **Dependency Setup** ([app.py:1-3](app.py#L1-L3)): `create_env.setup_dependencies()` runs first to install the dev version of transformers from GitHub. This uses a `/tmp/deps_installed` marker to prevent repeated installations.
+
+2. **Configuration Loading** ([app.py:12-13](app.py#L12-L13)): Loads `model_config.yaml` which defines multiple model checkpoints (e.g., "test-135000", "test-130000") with their HuggingFace paths and parameters.
+
+3. **Examples Loading** ([app.py:15-17](app.py#L15-L17)): Loads `examples.yaml` via the `Examples` adapter class, which transforms YAML examples into Gradio-compatible list-of-lists format.
+
+4. **Model Initialization** ([app.py:19-20](app.py#L19-L20)): `InitModels` loads all models upfront so the UI can switch between them without latency. Each model is a `KaniTTS` instance initialized directly with its config using unpacking (`**config`).
+
+### Key Components
+
+**InitModels** ([util.py:24-57](util.py#L24-L57))
+- Lazy initializer that constructs a map of `model_name -> KaniTTS`
+- Loads all models immediately in `__call__` for zero-switching latency
+- Each `KaniTTS` instance is initialized by unpacking its config directly: `KaniTTS(**config)`
+- No longer requires `NemoAudioPlayer` or HuggingFace token as these are handled internally by the PyPI package
+
+**Examples** ([util.py:59-97](util.py#L59-L97))
+- Adapter converting YAML examples to Gradio `gr.Examples` rows
+- Order must match UI inputs: `[text, model_dropdown, temp, top_p, rp]`
+- Centralizes format and defaults so UI input order changes only require updates here and in [app.py](app.py)
+
+**generate_speech_gpu** ([app.py:22-55](app.py#L22-L55))
+- Decorated with `@spaces.GPU` for HuggingFace Spaces GPU allocation
+- Calls the model directly: `audio, _ = model(text, temperature=t, top_p=top_p, repetition_penalty=rp)`
+- Returns only the audio tuple: `(sample_rate, audio)` - the text output is ignored
+
+### Configuration Files
+
+**model_config.yaml**
+- Defines multiple model checkpoints under the `models` key
+- Each model specifies: `model_name` (HF repo path), `device_map`, `use_bematts`, `audio_step`, `use_learnable_rope`
+- All config parameters are passed directly to `KaniTTS` constructor via unpacking
+
+**examples.yaml**
+- List of example prompts with their generation parameters
+- Each example requires: `text`, `model`, and optionally `temperature`, `top_p`, `repetition_penalty`
+- Missing parameters fall back to defaults in [util.py:90-92](util.py#L90-L92): temperature=1.0, top_p=0.95, repetition_penalty=1.1
+
+### Dependencies
+
+- **kani-tts==1.0.1**: Core TTS library from PyPI providing `KaniTTS` class with all inference logic
+- **gradio>=4.0.0**: UI framework
+- **transformers**: Installed from GitHub main branch via [create_env.py](create_env.py) for latest features
+
+### Environment Variables
+
+- `OMP_NUM_THREADS=4`: Set in [create_env.py:6](create_env.py#L6) to limit OpenMP threading
+
+## Important Implementation Notes
+
+### Model Inference
+- The KaniTTS model is called directly as a callable: `audio, text = model(text, temperature=..., top_p=..., repetition_penalty=...)`
+- Returns tuple of `(audio, text)` but only audio is used in the UI
+- No `max_tokens` parameter - the model handles sequence length internally
+- Sample rate is hardcoded to 22050 Hz ([app.py:48](app.py#L48))
+
+### Example Caching
+- Examples use `cache_examples=True` ([app.py:131](app.py#L131)) to pre-generate audio, speeding up demo interactions
+- If you modify generation logic, cached examples may need regeneration
+
+### GPU Allocation
+- The `@spaces.GPU` decorator is critical for HuggingFace Spaces deployment
+- Without it, the app runs on CPU which is significantly slower
+- Device selection falls back gracefully: `"cuda" if torch.cuda.is_available() else "cpu"` ([app.py:35](app.py#L35))
+
+### Input Order Dependency
+- The order of inputs in `gr.Examples` ([app.py:128](app.py#L128)) must exactly match the order in `Examples.__call__` ([util.py:94](util.py#L94))
+- Current order: `[text, model_dropdown, temp, top_p, rp]`
+- Changing this requires updates in both locations
+
+### Removed Features
+- **Speaker selection**: No longer supported in the new API
+- **Time reporting**: Generation timing is not tracked
+- **Max tokens slider**: Sequence length is handled automatically by the model

README.md

@@ -1,15 +1,14 @@
 ---
-title: KaniTTS 2
-emoji: 👀
-colorFrom: blue
-colorTo: blue
+title: KaniTTS-2
+emoji: 😻
+colorFrom: green
+colorTo: gray
 sdk: gradio
-sdk_version: 6.3.0
-python_version: '3.12'
+sdk_version: 5.46.0
 app_file: app.py
 pinned: false
 license: apache-2.0
-short_description: Pre Release space
+
 ---
 
 Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference

app.py

@@ -0,0 +1,124 @@
+from create_env import setup_dependencies
+
+setup_dependencies()
+
+import spaces
+import gradio as gr
+from util import InitModels, load_config, Examples
+import numpy as np
+import torch
+
+config = load_config("./model_config.yaml")
+models_configs = config.models
+
+examples_cfg = load_config("./examples.yaml")
+examples_maker = Examples(examples_cfg)
+examples = examples_maker()
+
+init_models = InitModels(models_configs)
+models = init_models()
+
+
+@spaces.GPU
+def generate_speech_gpu(text, model_choice, t, top_p, rp):
+    """
+    Generate speech from text using the selected model on GPU
+    """
+
+    if not text.strip():
+        return None
+
+    if not model_choice:
+        return None
+
+    try:
+        device = "cuda" if torch.cuda.is_available() else "cpu"
+        print(f"Using device: {device}")
+
+        selected_model = models[model_choice]
+
+        print(f"Generating speech with {model_choice}...")
+        audio, _ = selected_model(
+            text,
+            temperature=t,
+            top_p=top_p,
+            repetition_penalty=rp
+        )
+
+        sample_rate = 22050
+        print("Speech generation completed!")
+
+        return (sample_rate, audio)
+
+    except Exception as e:
+        print(f"Error during generation: {str(e)}")
+        return None
+
+# Create Gradio interface
+with gr.Blocks(title="😻 KaniTTS - Text to Speech", theme=gr.themes.Ocean()) as demo:
+    gr.Markdown("# 😻 KaniTTS: Fast and Expressive Speech Generation Model")
+    gr.Markdown("Select a model and enter text to generate emotional speech")
+    
+    with gr.Row():
+        with gr.Column(scale=1):
+            model_dropdown = gr.Dropdown(
+                choices=list(models_configs.keys()),
+                value=list(models_configs.keys())[0],
+                label="Selected Model"
+            )
+
+            text_input = gr.Textbox(
+                label="Text",
+                placeholder="Enter your text ...",
+                lines=3,
+                max_lines=10
+            )
+
+            with gr.Accordion("Settings", open=False):
+                temp = gr.Slider(
+                    minimum=0.1, maximum=1.5, value=0.6, step=0.05,
+                    label="Temp",
+                )
+                top_p = gr.Slider(
+                    minimum=0.1, maximum=1.0, value=0.95, step=0.05,
+                    label="Top P",
+                )
+                rp = gr.Slider(
+                    minimum=1.0, maximum=2.0, value=1.1, step=0.05,
+                    label="Repetition Penalty",
+                )
+            
+            generate_btn = gr.Button("Run", variant="primary", size="lg")
+
+            
+        with gr.Column(scale=1):
+            audio_output = gr.Audio(
+                label="Generated Audio",
+                type="numpy"
+            )
+    
+    # GPU generation event
+    generate_btn.click(
+        fn=generate_speech_gpu,
+        inputs=[text_input, model_dropdown, temp, top_p, rp],
+        outputs=[audio_output]
+    )
+    
+    with gr.Row():
+
+        examples = examples
+
+        gr.Examples(
+            examples=examples,
+            inputs=[text_input, model_dropdown, temp, top_p, rp],
+            fn=generate_speech_gpu,
+            outputs=[audio_output],
+            cache_examples=True,
+        )
+
+if __name__ == "__main__":
+    demo.launch(
+        server_name="0.0.0.0",
+        server_port=7860,
+        show_error=True
+    )

create_env.py

@@ -0,0 +1,21 @@
+import os
+import subprocess
+import sys
+
+def setup_dependencies():
+    os.environ["OMP_NUM_THREADS"] = "4"
+    try:
+        if os.path.exists('/tmp/deps_installed'):
+            return
+            
+        print("Installing transformers dev version...")
+        subprocess.check_call([
+            sys.executable, "-m", "pip", "install", "--force-reinstall", "--no-cache-dir",
+            "git+https://github.com/huggingface/transformers.git"
+        ])
+        
+        with open('/tmp/deps_installed', 'w') as f:
+            f.write('done')
+            
+    except Exception as e:
+        print(f"Dependencies setup error: {e}")

examples.yaml

@@ -0,0 +1,45 @@
+examples:
+  - text: >-
+      No, that does not make you a failure. No, sweetie, no. It just, uh, it just means that you're having a tough time...
+    model: "test-135000"
+    temperature: 1
+    top_p: 0.95
+    repetition_penalty: 1.1
+
+  - text: >-
+      Anyway, um, so, um, tell me, tell me all about her. I mean, what's she like? Is she really, you know, pretty?
+    model: "test-135000"
+    temperature: 1
+    top_p: 0.95
+    repetition_penalty: 1.1
+
+  - text: >-
+      Have some wine, the March Hare said in an encouraging tone. Alice looked all round the table, but there was nothing on it but tea. I don't see any wine, she remarked. There isn't any, said the March Hare. Then it wasn't very civil of you to offer it, said Alice angrily. It wasn't very civil of you to sit down without being invited, said the March Hare. I didn't know it was YOUR table, said Alice; it's laid for a great many more than three. Your hair wants cutting, said the Hatter.  He had been looking at Alice for some time with great curiosity, and this was his first speech. You should learn not to make personal remarks, Alice said with some severity; it's very rude.
+    model: "test-135000"
+    temperature: 1
+    top_p: 0.95
+    repetition_penalty: 1.1
+
+
+  - text: >-
+      Attention networks have proven to be an effective approach for embedding categorical inference within a deep neural network. However, for many tasks we may want to model richer structural dependencies without abandoning end-to-end training. In this work, we experiment with incorporating richer structural distributions, encoded using graphical models, within deep networks. We show that these structured attention networks are simple extensions of the basic attention procedure, and that they allow for extending attention beyond the standard softselection approach, such as attending to partial segmentations or to subtrees.
+    model: "test-135000"
+    temperature: 1
+    top_p: 0.95
+    repetition_penalty: 1.1
+
+  - text: >-
+      Кыргыз жери! Сенин ар бир ташыңда, ар бир тооңдо, ар бир сууңда менин жүрөгүмдүн бир бөлүгү бар. Сен менин ата-журтум, менин ыйык мекенимсиң.
+    model: "test-135000"
+    temperature: 1
+    top_p: 0.95
+    repetition_penalty: 1.1
+
+
+  - text: >-
+      ¿Será que todavía me recuerdas como antes?
+    model: "test-135000"
+    temperature: 1
+    top_p: 0.95
+    repetition_penalty: 1.1
+

model_config.yaml

@@ -0,0 +1,38 @@
+models:
+
+  "test-135000":
+    model_name: nineninesix/full-checkpoint-135000
+    device_map: null
+    use_bematts: true
+    audio_step: 1.0
+    use_learnable_rope: true
+
+  "test-130000":
+    model_name: nineninesix/full-checkpoint-130000
+    device_map: null
+    use_bematts: true
+    audio_step: 1.0
+    use_learnable_rope: true
+
+  "test-110000":
+    model_name: nineninesix/full-checkpoint-110000
+    device_map: null
+    use_bematts: true
+    audio_step: 1.0
+    use_learnable_rope: true
+
+  "test-95000":
+    model_name: nineninesix/full-checkpoint-95000
+    device_map: null
+    use_bematts: true
+    audio_step: 1.0
+    use_learnable_rope: true
+
+  "test-25000":
+    model_name: nineninesix/full-checkpoint-25000
+    device_map: null
+    use_bematts: true
+    audio_step: 1.0
+    use_learnable_rope: true
+
+

requirements.txt

@@ -0,0 +1,2 @@
+kani-tts==1.0.1
+gradio>=4.0.0

util.py

@@ -0,0 +1,97 @@
+from kani_tts import KaniTTS
+
+import os
+from omegaconf import OmegaConf
+
+
+def load_config(config_path: str):
+    """Load configuration from a YAML file using OmegaConf.
+
+    Args:
+        config_path (str): Path to the YAML configuration file.
+
+    Returns:
+        Any: The loaded OmegaConf DictConfig.
+    """
+    resolved_path = os.path.abspath(config_path)
+    if not os.path.exists(resolved_path):
+        raise FileNotFoundError(f"Config file not found: {resolved_path}")
+    config = OmegaConf.load(resolved_path)
+    return config
+
+
+
+class InitModels:
+
+    """
+    Lazy initializer that constructs a map of model name -> KaniTTS.
+
+    Parameters
+    ----------
+    models_configs : OmegaConf | DictConfig
+        The `models` section from `model_config.yaml` describing one or
+        more HF model checkpoints and their options (device_map, use_bematts, etc.).
+
+    Returns
+    -------
+    dict
+        When called, returns a dictionary `{model_name: KaniTTS}`.
+
+    Notes
+    -----
+    - All models are loaded immediately in `__call__` so the UI can list
+    them and switch between them without extra latency.
+    - Each KaniTTS instance is initialized with its config directly.
+    """
+
+    def __init__(self, models_configs: OmegaConf):
+        self.models_configs = models_configs
+
+    def __call__(self):
+        models = {}
+        for model_name, config in self.models_configs.items():
+            print(f"Loading {model_name}...")
+            models[model_name] = KaniTTS(**config)
+            print(f"{model_name} loaded!")
+        print("All models loaded!")
+        return models
+
+class Examples:
+
+    """
+    Adapter that converts YAML examples into Gradio `gr.Examples` rows.
+
+    Parameters
+    ----------
+    exam_cfg : OmegaConf | DictConfig
+        Parsed contents of `examples.yaml`. Expected structure:
+        `examples: [ {text, model, temperature?, top_p?, repetition_penalty?}, ... ]`.
+
+    Behavior
+    --------
+    - Produces a list-of-lists whose order must match the `inputs` order
+        used when constructing `gr.Examples` in `app.py`.
+    - Current order: `[text, model_dropdown, temp, top_p, rp]`.
+
+    Why this exists
+    ---------------
+    - Keeps format and defaults centralized, so changing the UI inputs
+        order only requires a single change here and in `app.py`.
+    """
+
+    def __init__(self, exam_cfg: OmegaConf):
+        self.exam_cfg = exam_cfg
+
+    def __call__(self) -> list[list]:
+        rows = []
+        for e in self.exam_cfg.examples:
+            text                = e.get("text")
+            model               = e.get("model")
+            temperature         = e.get("temperature", 1.0)
+            top_p               = e.get("top_p", 0.95)
+            repetition_penalty  = e.get("repetition_penalty", 1.1)
+            # Order must match gr.Examples inputs: [text, model_dropdown, temp, top_p, rp]
+            rows.append([text, model, temperature, top_p, repetition_penalty])
+
+        return rows
+