Spaces:

Ashiedu
/

Synesthesia

Runtime error

App Files Files Community

Synesthesia / github-issues /T-001.md

Ashiedu

Sync unified workbench

0490201 verified about 1 month ago

preview code

raw

history blame contribute delete

58.2 kB

A newer version of the Gradio SDK is available: 6.14.0

Upgrade

T-001: Bootstrap Synesthesia — Dioxus Fullstack, Tailwind v4, Kansas UI Port, Full ML Pipeline

Type: Task
Autonomy: agent:human-led — Do not merge without human review. Read every constraint before touching any file.
Stack: stack:rust stack:dioxus stack:tailwind stack:ml
Version: v0.1
Iteration: iter-1
Effort: L (multi-session — Jules may run for many hours across subtasks)
Blocks: All subsequent tasks — nothing else starts until all subtasks are green.

⚠️ CRITICAL — READ BEFORE ANY ACTION

The default Dioxus fullstack template (dx new) is already working. The project builds and serves. Your job is to add to it, not restructure it. Agents routinely break this template and leave it broken. You will not do that.

Before touching any file:

Run dx serve --package desktop --features desktop — it must succeed.

Run cargo check --workspace — it must succeed.

If either fails, stop and file a blocker comment — do not attempt fixes.

Make each subtask's changes, then re-run both checks before the next subtask.

If at any point dx serve stops working, revert your last change and file a blocker. Do not push forward through a broken build.

Workspace Ground Truth

These files are already correct. Do not modify them unless a subtask explicitly instructs it.

Root `Cargo.toml`

[workspace]
resolver = "2"
members = [
    "packages/ui",
    "packages/web",
    "packages/desktop",
    "packages/api",
]

[workspace.dependencies]
dioxus = { version = "0.7.3" }
ui  = { path = "packages/ui" }
api = { path = "packages/api" }

`packages/api/Cargo.toml` (current state)

[package]
name = "api"
version = "0.1.0"
edition = "2024"

[dependencies]
dioxus = { workspace = true, features = ["fullstack"] }

[features]
server = ["dioxus/server"]

`packages/ui/Cargo.toml` (current state)

[package]
name = "ui"
version = "0.1.0"
edition = "2024"

[dependencies]
dioxus = { workspace = true }
api = { workspace = true }

[features]
server = ["api/server"]

`packages/desktop/Cargo.toml` (current state)

[package]
name = "desktop"
version = "0.1.0"
edition = "2024"

[dependencies]
dioxus = { workspace = true, features = ["router", "fullstack"] }
ui = { workspace = true }

[features]
default = []
desktop = ["dioxus/desktop"]
server  = ["dioxus/server", "ui/server"]

`packages/web/Cargo.toml` (current state)

[package]
name = "web"
version = "0.1.0"
edition = "2024"

[dependencies]
dioxus = { workspace = true, features = ["router", "fullstack"] }
ui = { workspace = true }

[features]
default = []
web    = ["dioxus/web"]
server = ["dioxus/server", "ui/server"]

Package Roles

Package	Role	Builds for
`api`	Server functions, Burn inference stubs, HF hub pull	Server + WASM stub
`ui`	All shared Dioxus components — panels, layout, tokens	Desktop + Web + Server
`desktop`	Native window entry point	Desktop + Server (for LAN)
`web`	WASM entry point, served to iPhone/browser	WASM

File Structure After This Task

synesthesia/
├── Cargo.toml                     ← unchanged
├── Cargo.lock                     ← unchanged
├── Dioxus.toml                    ← NEW: dx CLI project config
├── MODELS.md                      ← NEW: model roadmap (T-001e)
├── AGENTS.md                      ← keep as-is
│
├── packages/
│   ├── api/
│   │   ├── Cargo.toml             ← MODIFIED: add hf-hub, burn stubs, once_cell (T-001d)
│   │   └── src/
│   │       ├── lib.rs             ← NEW: pub mod declarations
│   │       ├── quality.rs         ← NEW: Quality enum stub
│   │       └── models/
│   │           └── mod.rs         ← NEW: model stub registry
│   │
│   ├── ui/
│   │   ├── Cargo.toml             ← unchanged
│   │   └── src/
│   │       ├── lib.rs             ← MODIFIED: export all panel components
│   │       ├── layout.rs          ← NEW: root grid layout component
│   │       ├── header.rs          ← NEW: header bar (T-001c)
│   │       ├── track_mixer.rs     ← NEW: left panel stub (T-001c)
│   │       ├── center_canvas.rs   ← NEW: center panel (T-001c)
│   │       ├── pitch_intel.rs     ← NEW: right panel stub (T-001c)
│   │       └── bottom_bar.rs      ← NEW: bottom bar stub (T-001c)
│   │
│   ├── desktop/
│   │   ├── Cargo.toml             ← MODIFIED: add window-vibrancy, tokio (T-001a)
│   │   ├── assets/
│   │   │   ├── tailwind.css       ← NEW: Tailwind v4 input (T-001b)
│   │   │   └── main.css           ← keep existing if present; do not delete
│   │   └── src/
│   │       └── main.rs            ← MODIFIED: window config (T-001a)
│   │
│   └── web/
│       ├── Cargo.toml             ← unchanged
│       ├── assets/
│       │   └── tailwind.css       ← NEW: same Tailwind input as desktop (symlink or copy)
│       └── src/
│           └── main.rs            ← unchanged
│
└── .github/
    └── workflows/
        └── ci.yml                 ← NEW: cargo check + dx check stub

Subtask T-001a — Tailwind v4 + Window Config

Goal

Add Tailwind v4 to both desktop and web assets. Configure the native desktop window. Do not break dx serve.

1. Tailwind v4 Setup

Install the standalone Tailwind v4 CLI (no Node/npm required):

# Windows — download from https://github.com/tailwindlabs/tailwindcss/releases/latest
# Place tailwindcss.exe in your PATH or reference it directly in tasks

# Verify
tailwindcss --version   # must be v4.x

Create packages/desktop/assets/tailwind.css:

@import "tailwindcss";

/* Synesthesia design tokens as Tailwind v4 theme variables */
@theme {
  /* Backgrounds */
  --color-bg:        #070b10;
  --color-surface:   #0d1520;
  --color-surface-2: #111d2e;
  --color-border:    #1a2d45;

  /* Text */
  --color-text:        #c8ddf0;
  --color-text-dim:    #5a7a9a;
  --color-text-bright: #e8f4ff;

  /* Accent */
  --color-accent:     #00c8d4;
  --color-accent-dim: rgba(0, 200, 212, 0.12);
  --color-accent-2:   #0090a8;

  /* Status */
  --color-red:    #e05c5c;
  --color-green:  #4ade80;
  --color-yellow: #fbbf24;
  --color-orange: #fb923c;

  /* Stack badge colors */
  --color-rust:   #f0734a;
  --color-cpp:    #5b9bd5;
  --color-ts:     #3fb9e0;
  --color-python: #f5cb42;
  --color-ml:     #a78bfa;

  /* Typography */
  --font-mono: "JetBrains Mono", monospace;
  --font-ui:   "Space Grotesk", sans-serif;

  /* Font sizes */
  --text-xs:   11px;
  --text-sm:   12px;
  --text-base: 14px;
  --text-md:   15px;
  --text-lg:   18px;

  /* Spacing (4px grid) */
  --spacing-1: 4px;
  --spacing-2: 8px;
  --spacing-3: 12px;
  --spacing-4: 16px;
  --spacing-5: 20px;
  --spacing-6: 24px;
  --spacing-8: 32px;

  /* Radius */
  --radius-sm: 4px;
  --radius-md: 8px;
  --radius-lg: 12px;

  /* Transitions */
  --transition-fast: 120ms ease;
  --transition-base: 200ms ease;

  /* Layout */
  --panel-left-width:   240px;
  --panel-right-width:  280px;
  --header-height:      56px;
  --bottom-bar-height:  100px;
}

@layer base {
  *, *::before, *::after { box-sizing: border-box; }
  html, body { margin: 0; padding: 0; height: 100%; }
  body {
    background-color: var(--color-bg);
    color: var(--color-text);
    font-family: var(--font-ui);
    font-size: var(--text-base);
    -webkit-font-smoothing: antialiased;
  }
}

Copy tailwind.css to packages/web/assets/tailwind.css (identical content).

Build Tailwind output (run alongside dx serve):

# Desktop
tailwindcss -i packages/desktop/assets/tailwind.css -o packages/desktop/assets/output.css --watch

# Web
tailwindcss -i packages/web/assets/tailwind.css -o packages/web/assets/output.css --watch

2. desktop/Cargo.toml Additions

Add to the existing packages/desktop/Cargo.toml — do not change anything that already exists:

[dependencies]
# ... existing entries unchanged ...
window-vibrancy = "0.7.1"
tokio = { version = "1.43.0", features = ["full"] }

Add to [features]:

[features]
default = []
desktop = ["dioxus/desktop"]
server  = ["dioxus/server", "ui/server"]
# no changes needed — already correct

3. desktop/src/main.rs

Replace with the following. Critical: The stylesheet link uses asset!() pointing to output.css (the compiled Tailwind file), not tailwind.css (the input):

// packages/desktop/src/main.rs
#![allow(non_snake_case)]
use dioxus::prelude::*;
use dioxus::desktop::{Config, WindowBuilder, LogicalSize};
use ui::Layout;

fn main() {
    // Spawn Axum web server for LAN access (iPhone, browser)
    // Only active with --features server
    #[cfg(feature = "server")]
    std::thread::spawn(|| {
        tokio::runtime::Runtime::new()
            .unwrap()
            .block_on(start_server());
    });

    dioxus::LaunchBuilder::new()
        .with_cfg(
            Config::new()
                .with_window(
                    WindowBuilder::new()
                        .with_title("synesthesia")
                        .with_inner_size(LogicalSize::new(1280.0, 800.0))
                        .with_min_inner_size(LogicalSize::new(960.0, 600.0))
                        .with_transparent(true)
                        .with_decorations(false),
                )
                .with_background_color((7, 11, 16, 255))   // #070b10
                .with_on_window_ready(|window| {
                    // Apply Windows 11 acrylic vibrancy
                    #[cfg(target_os = "windows")]
                    {
                        use window_vibrancy::apply_acrylic;
                        let _ = apply_acrylic(&window, Some((7, 11, 16, 200)));
                    }
                    // wgpu surface init goes here in a later task
                })
                .with_devtools(cfg!(debug_assertions))
                .with_disable_context_menu(true),
        )
        .launch(App);
}

#[component]
fn App() -> Element {
    rsx! {
        document::Stylesheet { href: asset!("/assets/output.css") }
        Layout {}
    }
}

#[cfg(feature = "server")]
async fn start_server() {
    use dioxus::fullstack::prelude::ServeConfig;
    let router = axum::Router::new()
        .serve_static_assets("dist")
        .serve_dioxus_application(ServeConfig::new(), App);
    let listener = tokio::net::TcpListener::bind("0.0.0.0:3000").await.unwrap();
    println!("[synesthesia] web server → http://0.0.0.0:3000");
    axum::serve(listener, router).await.unwrap();
}

4. Dioxus.toml

Create at the repo root:

[application]
name             = "synesthesia"
default_platform = "desktop"
out_dir          = "dist"

[web.app]
base_path = "/"

[web.watcher]
reload_html = true
watch_path  = ["packages/ui/src", "packages/desktop/assets", "packages/web/assets"]

[desktop.window]
title      = "synesthesia"
width      = 1280
height     = 800
min_width  = 960
min_height = 600

Acceptance Criteria — T-001a

cargo check --workspace passes after changes
dx serve --package desktop --features desktop starts the window
Window background is #070b10 — no white flash
Window is 1280×800, min-resizable to 960×600
Tailwind output.css builds without errors
window-vibrancy compiles; acrylic applied on Windows 11 (log error, do not crash if unavailable)

Subtask T-001b — Design Token Verification

Goal

Confirm all design tokens from the Kansas prototype are represented in Tailwind @theme. No separate tokens.css file — Tailwind is the token system.

The token mapping from the Kansas CSS variables to Tailwind v4 @theme is already done in T-001a. This subtask verifies the mapping is correct and adds Tailwind utility aliases for the most common patterns.

Add to the bottom of tailwind.css (after @theme {}):

@layer utilities {
  /* Surface helpers */
  .bg-synth-bg        { background-color: var(--color-bg); }
  .bg-synth-surface   { background-color: var(--color-surface); }
  .bg-synth-surface-2 { background-color: var(--color-surface-2); }

  /* Text helpers */
  .text-synth         { color: var(--color-text); }
  .text-synth-dim     { color: var(--color-text-dim); }
  .text-synth-bright  { color: var(--color-text-bright); }
  .text-accent        { color: var(--color-accent); }

  /* Border helpers */
  .border-synth       { border-color: var(--color-border); }

  /* Font helpers */
  .font-mono-synth    { font-family: var(--font-mono); }
  .font-ui-synth      { font-family: var(--font-ui); }

  /* Panel geometry */
  .w-panel-left       { width: var(--panel-left-width); }
  .w-panel-right      { width: var(--panel-right-width); }
  .h-header           { height: var(--header-height); }
  .h-bottom-bar       { height: var(--bottom-bar-height); }

  /* Status dot */
  .status-dot {
    width: 8px; height: 8px;
    border-radius: 50%;
    display: inline-block;
  }
  .status-dot-green  { background-color: var(--color-green); }
  .status-dot-yellow { background-color: var(--color-yellow); }
  .status-dot-red    { background-color: var(--color-red); }
}

Acceptance Criteria — T-001b

All Kansas design token names exist in @theme {}
No hardcoded hex colors in any .rs component file
Tailwind builds with zero warnings

Subtask T-001c — Kansas UI Port to Dioxus RSX

Goal

Translate the Kansas prototype five-panel layout into Dioxus RSX components in packages/ui/src/. All panels are visual stubs — correct structure and tokens, no wired functionality. Functionality comes from T-002 onward.

Root Layout (`packages/ui/src/layout.rs`)

// packages/ui/src/layout.rs
use dioxus::prelude::*;
use crate::{Header, TrackMixer, CenterCanvas, PitchIntel, BottomBar};

#[component]
pub fn Layout() -> Element {
    rsx! {
        div {
            class: "grid h-screen w-screen overflow-hidden",
            style: "
                grid-template-rows: var(--header-height) 1fr var(--bottom-bar-height);
                grid-template-columns: var(--panel-left-width) 1fr var(--panel-right-width);
                grid-template-areas:
                    'header  header  header'
                    'left    center  right'
                    'bottom  bottom  bottom';
                background-color: var(--color-bg);
            ",
            Header {}
            TrackMixer {}
            CenterCanvas {}
            PitchIntel {}
            BottomBar {}
        }
    }
}

Header Bar (`packages/ui/src/header.rs`)

// packages/ui/src/header.rs
use dioxus::prelude::*;

#[component]
pub fn Header() -> Element {
    // WebGPU status — placeholder; wired in a later task
    let webgpu_active = use_signal(|| false);

    rsx! {
        header {
            class: "flex items-center justify-between px-4 border-b border-synth",
            style: "grid-area: header; background-color: var(--color-surface); height: var(--header-height);",

            // Left: logo + name + version
            div { class: "flex items-center gap-3",
                span { class: "text-accent font-mono-synth", "~" }
                span {
                    class: "font-mono-synth text-synth-bright",
                    style: "font-size: var(--text-xs); letter-spacing: 0.15em;",
                    "SYNESTHESIA"
                }
                span {
                    class: "font-mono-synth text-synth-dim",
                    style: "font-size: var(--text-xs);",
                    "[V3026.4]"
                }
            }

            // Right: WebGPU status + FREEZE + settings
            div { class: "flex items-center gap-4",

                // WebGPU status pill
                div {
                    class: "flex items-center gap-1.5 px-2 py-1 rounded",
                    style: "background-color: var(--color-surface-2); font-size: var(--text-xs);",
                    span {
                        class: "status-dot",
                        style: if webgpu_active() {
                            "background-color: var(--color-green);"
                        } else {
                            "background-color: var(--color-yellow);"
                        },
                    }
                    span {
                        class: "font-mono-synth",
                        style: "color: var(--color-text-dim);",
                        if webgpu_active() { "WebGPU Active" } else { "WebGPU Pending" }
                    }
                }

                // FREEZE button stub
                button {
                    class: "flex items-center gap-1.5 px-3 py-1 rounded border border-synth",
                    style: "background: transparent; font-size: var(--text-xs); color: var(--color-text-dim); font-family: var(--font-mono); cursor: pointer;",
                    onclick: move |_| { /* T-002+ */ },
                    "❄ FREEZE"
                }

                // Settings stub
                button {
                    style: "background: transparent; border: none; color: var(--color-text-dim); cursor: pointer; font-size: var(--text-lg);",
                    onclick: move |_| { /* T-002+ */ },
                    "⚙"
                }
            }
        }
    }
}

Track Mixer — Left Panel (`packages/ui/src/track_mixer.rs`)

// packages/ui/src/track_mixer.rs
use dioxus::prelude::*;

const CHANNELS: &[&str] = &["LEAD", "PAD", "BASS"];

#[component]
pub fn TrackMixer() -> Element {
    rsx! {
        aside {
            class: "flex flex-col overflow-hidden border-r border-synth",
            style: "grid-area: left; background-color: var(--color-bg); width: var(--panel-left-width);",

            // Panel header
            div {
                class: "flex items-center justify-between px-3 py-2 border-b border-synth",
                span {
                    class: "font-mono-synth text-synth-dim",
                    style: "font-size: var(--text-xs); letter-spacing: 0.1em;",
                    "TRACK MIXER"
                }
                span {
                    class: "font-mono-synth text-synth-dim",
                    style: "font-size: var(--text-xs);",
                    "[MODEL: MUSICVAE]"
                }
            }

            // Channel strips
            div { class: "flex flex-col gap-0 flex-1",
                for name in CHANNELS {
                    ChannelStrip { name: name.to_string() }
                }
            }
        }
    }
}

#[component]
fn ChannelStrip(name: String) -> Element {
    rsx! {
        div {
            class: "flex flex-col gap-2 p-3 border-b border-synth",

            // Channel name
            span {
                class: "font-mono-synth text-synth-bright",
                style: "font-size: var(--text-xs); font-weight: 700; letter-spacing: 0.1em;",
                "{name}"
            }

            // VOL fader stub
            div { class: "flex items-center gap-2",
                span {
                    class: "font-mono-synth text-synth-dim",
                    style: "font-size: var(--text-xs); min-width: 28px;",
                    "VOL"
                }
                input {
                    r#type: "range",
                    min: "0", max: "100", value: "80",
                    class: "flex-1",
                    style: "accent-color: var(--color-accent);",
                }
            }

            // PAN fader stub
            div { class: "flex items-center gap-2",
                span {
                    class: "font-mono-synth text-synth-dim",
                    style: "font-size: var(--text-xs); min-width: 28px;",
                    "PAN"
                }
                input {
                    r#type: "range",
                    min: "-50", max: "50", value: "0",
                    class: "flex-1",
                    style: "accent-color: var(--color-accent);",
                }
            }
        }
    }
}

Center Canvas (`packages/ui/src/center_canvas.rs`)

// packages/ui/src/center_canvas.rs
use dioxus::prelude::*;

#[component]
pub fn CenterCanvas() -> Element {
    rsx! {
        main {
            class: "relative overflow-hidden",
            style: "grid-area: center; background-color: var(--color-bg);",

            // WebGPU canvas — wired in a later task
            canvas {
                id: "webgpu-canvas",
                style: "width: 100%; height: 100%; display: block;",
            }

            // Top-left overlay
            div {
                class: "absolute top-3 left-3 px-2 py-1 rounded",
                style: "background-color: var(--color-surface); font-size: var(--text-xs); font-family: var(--font-mono); color: var(--color-text-dim);",
                "[ENGINE: WEBGPU] REAL-TIME SKELETAL POSE"
            }

            // Top-right overlay: REC indicator
            div {
                class: "absolute top-3 right-3",
                style: "font-size: var(--text-xs); font-family: var(--font-mono); color: var(--color-red);",
                "● REC 00:00:00"
            }
        }
    }
}

Pitch Intelligence — Right Panel (`packages/ui/src/pitch_intel.rs`)

// packages/ui/src/pitch_intel.rs
use dioxus::prelude::*;

#[component]
pub fn PitchIntel() -> Element {
    rsx! {
        aside {
            class: "flex flex-col overflow-hidden border-l border-synth",
            style: "grid-area: right; background-color: var(--color-surface); width: var(--panel-right-width);",

            div {
                class: "flex items-center px-3 py-2 border-b border-synth",
                span {
                    class: "font-mono-synth text-synth-dim",
                    style: "font-size: var(--text-xs); letter-spacing: 0.1em;",
                    "PITCH INTELLIGENCE"
                }
            }

            div { class: "flex flex-col gap-3 p-3",
                MetricBar { label: "AI AUTONOMY", value: 0.0 }
                MetricBar { label: "GPU COMPUTE", value: 0.0 }
            }
        }
    }
}

#[component]
fn MetricBar(label: String, value: f32) -> Element {
    rsx! {
        div { class: "flex flex-col gap-1",
            span {
                class: "font-mono-synth text-synth-dim",
                style: "font-size: var(--text-xs);",
                "{label}"
            }
            div {
                class: "w-full rounded overflow-hidden",
                style: "height: 4px; background-color: var(--color-surface-2);",
                div {
                    style: "height: 100%; width: {value * 100.0}%; background-color: var(--color-accent); transition: width var(--transition-base);",
                }
            }
        }
    }
}

Bottom Bar (`packages/ui/src/bottom_bar.rs`)

// packages/ui/src/bottom_bar.rs
use dioxus::prelude::*;

const MACROS: &[&str] = &["MACRO 1", "MACRO 2", "FILTER", "REVERB", "DELAY", "DRIVE"];

#[component]
pub fn BottomBar() -> Element {
    rsx! {
        footer {
            class: "flex items-center gap-6 px-4 border-t border-synth overflow-hidden",
            style: "grid-area: bottom; background-color: var(--color-surface); height: var(--bottom-bar-height);",

            // Transport left
            div { class: "flex items-center gap-3 shrink-0",
                // DCC ORIENT button stub
                div {
                    style: "width: 40px; height: 40px; border-radius: 50%; border: 1px solid var(--color-border); display: flex; align-items: center; justify-content: center; cursor: pointer;",
                    span {
                        class: "font-mono-synth text-synth-dim",
                        style: "font-size: 8px; letter-spacing: 0.05em;",
                        "DCC"
                    }
                }

                div { class: "flex flex-col gap-0.5",
                    span {
                        class: "font-mono-synth",
                        style: "font-size: var(--text-xs); color: var(--color-green);",
                        "● LIVE — NEURAL-CORE v4"
                    }
                    div { class: "flex items-center gap-3",
                        span {
                            class: "font-mono-synth text-synth-bright",
                            style: "font-size: var(--text-md); font-weight: 700;",
                            "128 BPM"
                        }
                        span {
                            class: "font-mono-synth text-synth-dim",
                            style: "font-size: var(--text-xs);",
                            "4/4  BAR 001  00:00:00"
                        }
                    }
                }
            }

            // Macro faders
            div { class: "flex items-end gap-4 flex-1 justify-center",
                for label in MACROS {
                    MacroFader { label: label.to_string() }
                }
            }
        }
    }
}

#[component]
fn MacroFader(label: String) -> Element {
    rsx! {
        div { class: "flex flex-col items-center gap-1",
            input {
                r#type: "range",
                min: "0", max: "100", value: "50",
                style: "
                    writing-mode: vertical-lr;
                    direction: rtl;
                    height: 60px;
                    width: 20px;
                    accent-color: var(--color-accent);
                ",
            }
            span {
                class: "font-mono-synth text-synth-dim",
                style: "font-size: 9px; letter-spacing: 0.08em;",
                "{label}"
            }
        }
    }
}

`packages/ui/src/lib.rs`

// packages/ui/src/lib.rs
pub mod layout;
pub mod header;
pub mod track_mixer;
pub mod center_canvas;
pub mod pitch_intel;
pub mod bottom_bar;

pub use layout::Layout;
pub use header::Header;
pub use track_mixer::TrackMixer;
pub use center_canvas::CenterCanvas;
pub use pitch_intel::PitchIntel;
pub use bottom_bar::BottomBar;

Acceptance Criteria — T-001c

cargo check --workspace passes with all new component files
dx serve --package desktop --features desktop renders the five-panel layout
Background is #070b10 — no white flash
Header shows: ~ SYNESTHESIA [V3026.4] left, WebGPU pill + FREEZE + ⚙ right
Left panel (240px): TRACK MIXER / [MODEL: MUSICVAE] header + LEAD/PAD/BASS channel strips with VOL/PAN sliders
Center panel: canvas fills remaining space, [ENGINE: WEBGPU] top-left overlay, ● REC top-right
Right panel (280px): PITCH INTELLIGENCE header + AI AUTONOMY + GPU COMPUTE metric bars
Bottom bar (100px): DCC circle + LIVE indicator + 128 BPM + 4/4 + MACRO 1-6 faders
Window resizes: layout holds at 960×600 minimum, no overflow
No hardcoded hex values in any .rs file — all via CSS variables

Subtask T-001d — API Dependencies (Full Runtime Stack)

Goal

Wire up the complete three-tier inference runtime in packages/api/Cargo.toml: Burn wgpu (primary, pure Rust), llama.cpp + Vulkan (LLMs), ORT DirectML (fallback until Burn op coverage catches up). All WASM-gated correctly. Add all 22 model stub modules matching the HF repo structure.

Updated `packages/api/Cargo.toml`

Replace entirely with:

[package]
name    = "api"
version = "0.1.0"
edition = "2024"

[dependencies]
dioxus     = { workspace = true, features = ["fullstack"] }
burn       = { version = "0.21.0-pre.2", default-features = false, features = ["train"] }
hf-hub     = { version = "0.5.0", default-features = false }
ndarray    = "0.17.2"
serde      = { version = "1.0.228", features = ["derive"] }
serde_json = "1.0.149"
anyhow     = "1.0.102"
once_cell  = "1.21.4"

# WASM: webgpu backend + wasm-compatible TLS + getrandom
[target.'cfg(target_arch = "wasm32")'.dependencies]
burn      = { version = "0.21.0-pre.2", default-features = false, features = ["train", "webgpu"] }
getrandom = { version = "0.4.2", features = ["wasm_js"] }
hf-hub    = { version = "0.5.0", default-features = false, features = ["rustls-tls"] }

# Native: full runtime stack
[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
burn          = { version = "0.21.0-pre.2", default-features = false, features = ["train", "wgpu"] }
hf-hub        = { version = "0.5.0", default-features = false, features = ["native-tls"] }
tokio         = { version = "1.43.0", features = ["full"] }
# llama.cpp + Vulkan — LLM inference (Gemma-3N, future LLMs)
# Zero-config: auto-downloads pre-built Vulkan binaries from llama.cpp releases.
# Same stack as LM Studio. Vulkan works on RX 6700 XT without ROCm.
llama-cpp-v3  = { version = "*" }
# ORT DirectML — temporary fallback for models burn-import cannot yet handle
# (LSTM-heavy: Performance RNN, MusicVAE, Melody/Drums/Improv/Polyphony RNN)
# Remove per-model as Burn op coverage matures. Tracked in MODELS.md.
ort           = { version = "2", features = ["directml"], optional = true }

[features]
server     = ["dioxus/server"]
ort-models = ["ort"]   # activate only for models blocked on Burn LSTM support

`packages/api/src/lib.rs`

// packages/api/src/lib.rs
pub mod quality;
pub mod models;

// Server-only modules — not compiled to WASM
#[cfg(not(target_arch = "wasm32"))]
pub mod backend;

`packages/api/src/quality.rs`

// packages/api/src/quality.rs

/// Model quality tier — controls which ONNX variant is loaded from HF.
/// Full = fp32 (reference), Half = fp16 (recommended), Lite = int8 (low latency).
#[derive(Debug, Clone, Copy, PartialEq, Eq, serde::Serialize, serde::Deserialize, Default)]
pub enum Quality {
    Full,
    #[default]
    Half,
    Lite,
}

impl Quality {
    pub fn onnx_suffix(self) -> &'static str {
        match self {
            Quality::Full => "fp32.onnx",
            Quality::Half => "fp16.onnx",
            Quality::Lite => "int8.onnx",
        }
    }
}

`packages/api/src/models/mod.rs`

All 22 models stubbed with ID, HF path, runtime tier, and Burn compatibility note. Follow this exact pattern for every module file — one pub struct with name(), hf_repo(), and hf_path() methods. No inference logic in T-001.

// packages/api/src/models/mod.rs

// ── Magenta RT ──────────────────────────────────────────────────────────────
/// MRT-001: Magenta RT LLM — 800M param autoregressive transformer
pub mod magenta_rt;
/// MRT-002/003: SpectroStream — 48kHz stereo audio codec (25Hz, 64 RVQ)
pub mod spectrostream;
/// MRT-004/005: MusicCoCa — text + audio → 768-dim music embeddings
pub mod musiccoca;

// ── Magenta Classic MIDI ─────────────────────────────────────────────────────
/// MC-001: Performance RNN — expressive MIDI performance generation
/// Runtime: ORT fallback (LSTM) → Burn when op coverage lands
pub mod perfrnn;
/// MC-002: Melody RNN — melody continuation (LSTM)
pub mod melody_rnn;
/// MC-003: Drums RNN — drum pattern generation (LSTM)
pub mod drums_rnn;
/// MC-004: Improv RNN — chord-conditioned melody (LSTM)
pub mod improv_rnn;
/// MC-005: Polyphony RNN — polyphonic generation (LSTM)
pub mod polyphony_rnn;
/// MC-006: MusicVAE — latent music VAE enc/dec (BiLSTM)
pub mod musicvae;
/// MC-007: GrooVAE — drum humanization VAE
pub mod groovae;
/// MC-008: MidiMe — personalize MusicVAE in-session
pub mod midime;
/// MC-009: Music Transformer — long-form piano generation (Attention)
pub mod music_transformer;
/// MC-010: Coconet — counterpoint by convolution
pub mod coconet;

// ── Magenta Classic Audio ────────────────────────────────────────────────────
/// MA-001: GANSynth — GAN audio synthesis (NSynth timbres)
/// Runtime: Burn wgpu (conv-heavy, good op coverage)
pub mod gansynth;
/// MA-002: NSynth — WaveNet neural audio synthesis
pub mod nsynth;
/// MA-003/004: DDSP — differentiable DSP enc/dec
pub mod ddsp;
/// MA-005: Piano Genie — 8-button → 88-key VQ-VAE
pub mod piano_genie;
/// MA-006: Onsets and Frames — polyphonic piano transcription
pub mod onsets_and_frames;
/// MA-007: SPICE — monophonic pitch extraction
pub mod spice;

// ── LLM / Vision ─────────────────────────────────────────────────────────────
/// LV-001: Gemma-3N e2b-it — vision + text → mood/energy/key JSON
/// Runtime: llama.cpp + Vulkan (GGUF). NOT Burn or ORT.
pub mod gemma3n;

Stub file template — repeat for every module above, updating the doc comment:

// packages/api/src/models/gansynth.rs

/// MA-001: GANSynth — GAN-based NSynth audio synthesis.
///
/// HF repo:  Ashiedu/Synesthesia
/// HF paths: audio/gansynth/fp32.onnx | fp16.onnx
/// Source:   google/magenta GANSynth (ICLR 2019)
/// Export:   tf2onnx from SavedModel checkpoint
/// Runtime:  Burn wgpu (primary) — conv-heavy, good op coverage
///
/// Inputs:
///   z:     [batch, 256]  — latent vector, sample from N(0,1)
///   pitch: [batch]       — MIDI pitch integer, range 24–84
/// Output:
///   audio: [batch, 64000] — raw PCM @ 16kHz, 4 seconds
pub struct GanSynth;

impl GanSynth {
    pub fn name()     -> &'static str { "gansynth" }
    pub fn hf_repo()  -> &'static str { "Ashiedu/Synesthesia" }
    pub fn hf_path(quality: crate::quality::Quality) -> String {
        format!("audio/gansynth/{}", quality.onnx_suffix())
    }
}

Key differences per model group:

Group	`hf_path` prefix	Runtime note in doc
Magenta RT	`magenta_rt/`	JAX checkpoint; ONNX export via jax2onnx
Magenta MIDI	`midi/<name>/`	ORT fallback (LSTM) → Burn when ready
Magenta Audio	`audio/<name>/`	Burn wgpu primary
Gemma-3N	`llm/gemma3n_e2b/`	llama.cpp + Vulkan; GGUF not ONNX

`packages/api/src/backend.rs` (native only)

// packages/api/src/backend.rs
// cfg guard is in lib.rs — this file only compiles on native targets

use burn::backend::wgpu::{Wgpu, WgpuDevice};

pub type Backend = Wgpu;

/// Returns the best available wgpu device.
/// On Windows with RX 6700 XT, selects AMD GPU via Vulkan/DX12.
pub fn best_device() -> WgpuDevice {
    WgpuDevice::BestAvailable
}

Acceptance Criteria — T-001d

cargo check --workspace passes for all targets
cargo check --package api --target wasm32-unknown-unknown passes
cargo tree -p api --target wasm32-unknown-unknown shows no tokio, burn/wgpu, llama-cpp-v3, or ort
All 22 model stub files exist with correct IDs, HF paths, and runtime notes
Quality enum compiles on both targets with hf_path() helper
llama-cpp-v3 present only under cfg(not(wasm32))
ort only activates under --features ort-models — not compiled by default

Subtask T-001e — HuggingFace Repo + MODELS.md

Goal

Push the canonical README to Ashiedu/Synesthesia, create manifest.json, create per-model directory README stubs, and write MODELS.md at the repo root. This makes the HF repo authoritative before any model files exist in it.

1. Push updated README to HuggingFace

The file HF_README.md (provided separately) is the canonical README for Ashiedu/Synesthesia. Push it as README.md to the HF repo:

from huggingface_hub import HfApi
api = HfApi(token=HF_TOKEN)  # HF_TOKEN in environment or Colab Secrets

api.upload_file(
    path_or_fileobj="HF_README.md",
    path_in_repo="README.md",
    repo_id="Ashiedu/Synesthesia",
    commit_message="T-001e: canonical README — full model inventory + runtime strategy",
)

If running locally (not Colab), use huggingface-cli login first.

2. manifest.json

Create at the repo root and push to HF at manifest.json:

{
  "version": "0.1.0",
  "repo": "Ashiedu/Synesthesia",
  "runtime_tiers": {
    "Full":  { "suffix": "_fp32.onnx", "description": "Reference quality" },
    "Half":  { "suffix": "_fp16.onnx", "description": "Default — RX 6700 XT" },
    "Lite":  { "suffix": "_int8.onnx", "description": "Lowest latency (MIDI models only)" },
    "GGUF_Q4": { "suffix": "_q4_k_m.gguf", "description": "Default LLM tier" },
    "GGUF_Q2": { "suffix": "_q2_k.gguf",   "description": "Lite LLM tier" },
    "GGUF_F16": { "suffix": "_f16.gguf",   "description": "Full LLM tier" }
  },
  "models": {
    "MRT-001": { "id": "MRT-001", "name": "Magenta RT LLM",     "path": "magenta_rt/llm/",          "format": "onnx",  "runtime": "burn-wgpu",   "burn_ready": false, "export": "jax2onnx" },
    "MRT-002": { "id": "MRT-002", "name": "SpectroStream Enc",  "path": "magenta_rt/spectrostream/", "format": "onnx",  "runtime": "ort-fallback","burn_ready": false, "export": "jax2onnx" },
    "MRT-003": { "id": "MRT-003", "name": "SpectroStream Dec",  "path": "magenta_rt/spectrostream/", "format": "onnx",  "runtime": "ort-fallback","burn_ready": false, "export": "jax2onnx" },
    "MRT-004": { "id": "MRT-004", "name": "MusicCoCa Text",     "path": "magenta_rt/musiccoca/",     "format": "onnx",  "runtime": "ort-fallback","burn_ready": false, "export": "jax2onnx" },
    "MRT-005": { "id": "MRT-005", "name": "MusicCoCa Audio",    "path": "magenta_rt/musiccoca/",     "format": "onnx",  "runtime": "ort-fallback","burn_ready": false, "export": "jax2onnx" },
    "MC-001":  { "id": "MC-001",  "name": "Performance RNN",    "path": "midi/perfrnn/",             "format": "onnx",  "runtime": "ort-fallback","burn_ready": false, "export": "tf2onnx",  "ui_role": "AI arpeggiator" },
    "MC-002":  { "id": "MC-002",  "name": "Melody RNN",         "path": "midi/melody_rnn/",          "format": "onnx",  "runtime": "ort-fallback","burn_ready": false, "export": "tf2onnx",  "ui_role": "Melody continuation" },
    "MC-003":  { "id": "MC-003",  "name": "Drums RNN",          "path": "midi/drums_rnn/",           "format": "onnx",  "runtime": "ort-fallback","burn_ready": false, "export": "tf2onnx",  "ui_role": "Beat generation" },
    "MC-004":  { "id": "MC-004",  "name": "Improv RNN",         "path": "midi/improv_rnn/",          "format": "onnx",  "runtime": "ort-fallback","burn_ready": false, "export": "tf2onnx",  "ui_role": "Live improv" },
    "MC-005":  { "id": "MC-005",  "name": "Polyphony RNN",      "path": "midi/polyphony_rnn/",       "format": "onnx",  "runtime": "ort-fallback","burn_ready": false, "export": "tf2onnx" },
    "MC-006":  { "id": "MC-006",  "name": "MusicVAE",           "path": "midi/musicvae/",            "format": "onnx",  "runtime": "ort-fallback","burn_ready": false, "export": "tf2onnx",  "ui_role": "Latent interpolation" },
    "MC-007":  { "id": "MC-007",  "name": "GrooVAE",            "path": "midi/groovae/",             "format": "onnx",  "runtime": "ort-fallback","burn_ready": false, "export": "tf2onnx",  "ui_role": "Drum humanization" },
    "MC-008":  { "id": "MC-008",  "name": "MidiMe",             "path": "midi/midime/",              "format": "onnx",  "runtime": "ort-fallback","burn_ready": false, "export": "tf2onnx" },
    "MC-009":  { "id": "MC-009",  "name": "Music Transformer",  "path": "midi/music_transformer/",   "format": "onnx",  "runtime": "ort-fallback","burn_ready": false, "export": "tf2onnx" },
    "MC-010":  { "id": "MC-010",  "name": "Coconet",            "path": "midi/coconet/",             "format": "onnx",  "runtime": "ort-fallback","burn_ready": false, "export": "tf2onnx" },
    "MA-001":  { "id": "MA-001",  "name": "GANSynth",           "path": "audio/gansynth/",           "format": "onnx",  "runtime": "burn-wgpu",   "burn_ready": true,  "export": "tf2onnx",  "ui_role": "Timbre synthesis" },
    "MA-002":  { "id": "MA-002",  "name": "NSynth",             "path": "audio/nsynth/",             "format": "onnx",  "runtime": "burn-wgpu",   "burn_ready": true,  "export": "tf2onnx" },
    "MA-003":  { "id": "MA-003",  "name": "DDSP Encoder",       "path": "audio/ddsp/",               "format": "onnx",  "runtime": "burn-wgpu",   "burn_ready": true,  "export": "tf2onnx" },
    "MA-004":  { "id": "MA-004",  "name": "DDSP Decoder",       "path": "audio/ddsp/",               "format": "onnx",  "runtime": "burn-wgpu",   "burn_ready": true,  "export": "tf2onnx" },
    "MA-005":  { "id": "MA-005",  "name": "Piano Genie",        "path": "audio/piano_genie/",        "format": "onnx",  "runtime": "burn-wgpu",   "burn_ready": true,  "export": "tf2onnx",  "ui_role": "Accessible performance" },
    "MA-006":  { "id": "MA-006",  "name": "Onsets and Frames",  "path": "audio/onsets_and_frames/",  "format": "onnx",  "runtime": "burn-wgpu",   "burn_ready": false, "export": "tf2onnx",  "ui_role": "Audio → MIDI" },
    "MA-007":  { "id": "MA-007",  "name": "SPICE",              "path": "audio/spice/",              "format": "onnx",  "runtime": "burn-wgpu",   "burn_ready": true,  "export": "tf2onnx",  "ui_role": "Pitch tracking" },
    "LV-001":  { "id": "LV-001",  "name": "Gemma-3N e2b-it",   "path": "llm/gemma3n_e2b/",          "format": "gguf",  "runtime": "llama-cpp",   "burn_ready": false, "export": "unsloth",  "ui_role": "Vision → mood/energy/key" }
  }
}

Push to HF:

api.upload_file(
    path_or_fileobj="manifest.json",
    path_in_repo="manifest.json",
    repo_id="Ashiedu/Synesthesia",
    commit_message="T-001e: manifest.json — authoritative model registry",
)

3. Per-model directory README stubs

For every model in manifest.json, create a README.md in its HF directory. Use this template, filling in the model-specific fields:

# <Model Name> — Synesthesia

**ID:** <MRT-001 etc>  
**HF path:** `Ashiedu/Synesthesia/<path>/`  
**Source:** <Magenta / Magenta RT / Google>  
**Task:** <one line>  
**Synesthesia role:** <UI role>  
**Export method:** `<tf2onnx | jax2onnx | unsloth>`  
**Runtime:** <Burn wgpu | ORT DirectML | llama.cpp Vulkan>  
**Burn compatible:** <yes / not yet — blocked on <op>>

## Quality Tiers Available
| Tier | File | Status |
|------|------|--------|
| Full | `fp32.onnx` | Planned |
| Half | `fp16.onnx` | Planned |
| Lite | `int8.onnx` | Planned (MIDI only) |

## Inputs / Outputs
| Tensor | Shape | Description |
|--------|-------|-------------|
| ...    | ...   | ...         |

## Export Command (Colab)
See T-001f Colab template. Model-specific command:
\`\`\`bash
<export command>
\`\`\`

## Fine-tuning
Train after the app is functional. Data source TBD from live usage.

Push all README stubs in a single batch:

from pathlib import Path

for model_id, model in manifest["models"].items():
    readme_content = generate_readme(model)   # fill template above
    api.upload_file(
        path_or_fileobj=readme_content.encode(),
        path_in_repo=f"{model['path']}README.md",
        repo_id="Ashiedu/Synesthesia",
        commit_message=f"T-001e: {model_id} {model['name']} README stub",
    )

4. MODELS.md at repo root

Create MODELS.md in the Synesthesia GitHub repo (not HF):

# Synesthesia — Model Roadmap

**HF Repo:** [Ashiedu/Synesthesia](https://hf.co/Ashiedu/Synesthesia)  
**Authoritative registry:** `manifest.json` in the HF repo  
**Pipeline docs:** `docs/ml-pipeline.md`

---

## Inference Runtime Strategy

### Tier 1 — llama.cpp + Vulkan
- **Crate:** `llama-cpp-v3` — zero-config Vulkan binaries, same stack as LM Studio
- **Models:** Gemma-3N e2b-it (GGUF). All future LLMs added here.
- **GPU:** RX 6700 XT via Vulkan — no ROCm, no CUDA needed on Windows 11

### Tier 2 — Burn wgpu (primary, pure Rust)
- **Crate:** `burn 0.21.0-pre.2`, wgpu backend → Vulkan/DX12
- **Models:** GANSynth, NSynth, DDSP, Piano Genie, SPICE (conv-heavy, good op coverage)
- **Migration:** Each ORT model moves here when `burn-onnx` succeeds in CI

### Tier 3 — ORT + DirectML (fallback, temporary)
- **Crate:** `ort = { features = ["directml"], optional = true }`, feature = `ort-models`
- **Models:** All LSTM/Attention models until Burn op coverage reaches them
- **Lifecycle:** Per-model exit tracked in the table below

---

## Model Inventory

### Magenta RT

| ID | Model | Task | Burn | ORT | llama.cpp |
|----|-------|------|------|-----|-----------|
| MRT-001 | Magenta RT LLM | Real-time stereo audio generation | 🔄 | ✅ | ❌ |
| MRT-002 | SpectroStream Enc | Audio → spectral tokens (48kHz) | 🔄 | ✅ | ❌ |
| MRT-003 | SpectroStream Dec | Spectral tokens → audio | 🔄 | ✅ | ❌ |
| MRT-004 | MusicCoCa Text | Text → 768-dim music embedding | 🔄 | ✅ | ❌ |
| MRT-005 | MusicCoCa Audio | Audio → 768-dim music embedding | 🔄 | ✅ | ❌ |

### Magenta Classic — MIDI

| ID | Model | Synesthesia Role | Burn | ORT | Export |
|----|-------|-----------------|------|-----|--------|
| MC-001 | Performance RNN | AI arpeggiator | 🔄 LSTM | ✅ | tf2onnx |
| MC-002 | Melody RNN | Melody continuation | 🔄 LSTM | ✅ | tf2onnx |
| MC-003 | Drums RNN | Beat generation | 🔄 LSTM | ✅ | tf2onnx |
| MC-004 | Improv RNN | Live improv over chords | 🔄 LSTM | ✅ | tf2onnx |
| MC-005 | Polyphony RNN | Harmonic voice gen | 🔄 LSTM | ✅ | tf2onnx |
| MC-006 | MusicVAE | Latent interpolation | 🔄 BiLSTM | ✅ | tf2onnx |
| MC-007 | GrooVAE | Drum humanization | 🔄 BiLSTM | ✅ | tf2onnx |
| MC-008 | MidiMe | User-adaptive latent space | 🔄 | ✅ | tf2onnx |
| MC-009 | Music Transformer | Long-form piano gen | 🔄 Attn | ✅ | tf2onnx |
| MC-010 | Coconet | Counterpoint / harmony fill | 🔄 Conv | ✅ | tf2onnx |

### Magenta Classic — Audio

| ID | Model | Synesthesia Role | Burn | ORT | Export |
|----|-------|-----------------|------|-----|--------|
| MA-001 | GANSynth | GANHarp timbre instrument | ✅ | ❌ | tf2onnx |
| MA-002 | NSynth | Neural sample synthesis | ✅ | ❌ | tf2onnx |
| MA-003 | DDSP Encoder | Audio → harmonic params | ✅ | ❌ | tf2onnx |
| MA-004 | DDSP Decoder | Harmonic params → audio | ✅ | ❌ | tf2onnx |
| MA-005 | Piano Genie | 8-button → 88-key piano | ✅ | ❌ | tf2onnx |
| MA-006 | Onsets & Frames | Audio → MIDI transcription | 🔄 Conv+LSTM | ✅ | tf2onnx |
| MA-007 | SPICE | Monophonic pitch tracking | ✅ | ❌ | tf2onnx |

### LLM / Vision

| ID | Model | Role | Runtime | Format |
|----|-------|------|---------|--------|
| LV-001 | Gemma-3N e2b-it | Camera → mood/energy/key JSON | llama.cpp + Vulkan | GGUF |

**Legend:** ✅ active · ❌ not used · 🔄 planned (blocked on op)

---

## CI — Burn Migration Tracking

Weekly CI job attempts `burn-onnx ModelGen` on each exported model.
On success: flip `burn_ready` in `manifest.json` and remove from `ort-models` feature.

```bash
# .github/workflows/burn-migration.yml
# Runs weekly: cargo run --bin burn-compat-check

Training Philosophy

Train after the app works. Identify real input distributions from live usage. Fine-tune on your own audio and MIDI once the signal chain is operational.

Tentative fine-tuning sequence:

Performance RNN → live MIDI from Track Mixer (MC-001)
Melody RNN → melody continuation from Track Mixer input (MC-002)
MusicVAE + GrooVAE → latent interpolation + drum humanization (MC-006/007)
GANSynth → timbre generation from Pitch Intelligence panel (MA-001)
DDSP → resynthesis of GANSynth outputs (MA-003/004)
Magenta RT → full audio, conditioned on your own catalog (MRT-001)
Gemma-3N → fine-tune on camera sessions for mood/energy mapping (LV-001)


### Acceptance Criteria — T-001e
- [ ] Updated README pushed to `Ashiedu/Synesthesia` on HuggingFace
- [ ] `manifest.json` pushed to HF with all 22 model entries
- [ ] Per-model README stub exists in every model subdirectory on HF
- [ ] `MODELS.md` exists at GitHub repo root with full inventory + CI tracking table
- [ ] Runtime strategy section present: llama.cpp / Burn / ORT tiers documented
- [ ] Training sequence present with "train after the app works" rule

---

## Subtask T-001f — Colab Export Template

### Goal
Create a generic Colab-ready export notebook template that covers all three
export paths (tf2onnx, jax2onnx, Unsloth GGUF) and the push-to-HF workflow.
Store as `docs/colab_export_template.ipynb` in the GitHub repo.
Gemini on Colab can execute this directly — paste the model's README as context.

### `docs/colab_export_template.ipynb` — Key Cells

Create a Jupyter notebook with these cells as markdown + code:

**Cell 1 — Setup**
```python
# Install dependencies
!pip install -q tf2onnx onnx onnxruntime onnxconverter-common \
               huggingface_hub unsloth transformers accelerate

# HF token — set in Colab Secrets (key: HF_TOKEN)
import os
from google.colab import userdata
HF_TOKEN = userdata.get("HF_TOKEN")

from huggingface_hub import HfApi, snapshot_download
api = HfApi(token=HF_TOKEN)
REPO_ID = "Ashiedu/Synesthesia"

Cell 2 — Pull current checkpoint from HF (if updating)

snapshot_download(REPO_ID, local_dir="./models", token=HF_TOKEN)

Cell 3a — Magenta Classic export (tf2onnx)

# Clone Magenta
!git clone --depth 1 https://github.com/magenta/magenta

# Load checkpoint and export — MODEL_NAME set per-model
import tensorflow as tf
import tf2onnx, onnx

MODEL_NAME   = "performance_rnn"   # change per model
CHECKPOINT   = f"./magenta_checkpoints/{MODEL_NAME}"
OUTPUT_ONNX  = f"{MODEL_NAME}.onnx"

# Load SavedModel or concrete function
model = tf.saved_model.load(CHECKPOINT)
spec  = (tf.TensorSpec((None, 128, 388), tf.float32, name="input"),)

model_proto, _ = tf2onnx.convert.from_function(
    model.signatures["serving_default"],
    input_signature=spec,
    opset=17,
    output_path=OUTPUT_ONNX,
)
print(f"Exported: {OUTPUT_ONNX}")

Cell 3b — Magenta RT export (jax2onnx)

# Clone Magenta RT
!git clone --depth 1 https://github.com/magenta/magenta-realtime
!pip install -q jax jaxlib jax2onnx

# MODEL_NAME: spectrostream_encoder | spectrostream_decoder | musiccoca_text | musiccoca_audio
MODEL_NAME = "spectrostream_encoder"
# Follow magenta-realtime export docs for each component

Cell 3c — Gemma-3N GGUF (Unsloth, free T4)

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    "google/gemma-3n-e2b-it",
    max_seq_length=512,
    load_in_4bit=True,
)

# Fine-tune if desired (LoRA), then export tiers
for quant in ["q4_k_m", "q2_k", "f16"]:
    out = f"gemma3n_e2b_{quant}"
    model.save_pretrained_gguf(out, tokenizer, quantization_method=quant)
    api.upload_folder(
        folder_path=out,
        path_in_repo=f"llm/gemma3n_e2b/",
        repo_id=REPO_ID,
        commit_message=f"LV-001 Gemma-3N {quant}",
    )

Cell 4 — Quantize ONNX to fp16 and int8

import onnx
import onnxconverter_common as occ
from onnxruntime.quantization import quantize_dynamic, QuantType

fp32_model = onnx.load(OUTPUT_ONNX)

# fp16
fp16_model = occ.convert_float_to_float16(fp32_model, keep_io_types=True)
fp16_path  = OUTPUT_ONNX.replace(".onnx", "_fp16.onnx")
onnx.save(fp16_model, fp16_path)

# int8 (MIDI models only — skip for audio models)
int8_path  = OUTPUT_ONNX.replace(".onnx", "_int8.onnx")
quantize_dynamic(OUTPUT_ONNX, int8_path, weight_type=QuantType.QInt8)

print(f"fp32: {os.path.getsize(OUTPUT_ONNX) / 1e6:.1f} MB")
print(f"fp16: {os.path.getsize(fp16_path)  / 1e6:.1f} MB")
print(f"int8: {os.path.getsize(int8_path)  / 1e6:.1f} MB")

Cell 5 — Validate with ORT before push

import onnxruntime as ort
import numpy as np

sess = ort.InferenceSession(fp16_path,
    providers=["CPUExecutionProvider"])   # CPU on Colab is fine for validation

# Run with dummy input matching the model's input shape
dummy = {inp.name: np.zeros(inp.shape, dtype=np.float32)
         for inp in sess.get_inputs()}
out = sess.run(None, dummy)
print("Validation passed:", [o.shape for o in out])

Cell 6 — Push all tiers to HF

MODEL_ID   = "MC-001"       # change per model
HF_PREFIX  = "midi/perfrnn" # change per model — matches manifest.json path

for local_path, repo_suffix in [
    (OUTPUT_ONNX,  "fp32.onnx"),
    (fp16_path,    "fp16.onnx"),
    (int8_path,    "int8.onnx"),
]:
    api.upload_file(
        path_or_fileobj=local_path,
        path_in_repo=f"{HF_PREFIX}/{repo_suffix}",
        repo_id=REPO_ID,
        commit_message=f"{MODEL_ID} {repo_suffix}",
    )
    print(f"Pushed {HF_PREFIX}/{repo_suffix}")

# Update manifest.json burn_ready if applicable

Gemini on Colab Workflow

Gemini does not need GitHub integration. The workflow:

Open a new Colab notebook
Paste the model's HF directory README as a Gemini context cell
Paste the relevant cells from colab_export_template.ipynb
Set MODEL_NAME, MODEL_ID, HF_PREFIX to match the model's README
Run — Gemini can drive execution and debug errors

Acceptance Criteria — T-001f

docs/colab_export_template.ipynb exists in the GitHub repo
Notebook has all 6 cells above with correct code
All three export paths covered: tf2onnx, jax2onnx, Unsloth GGUF
Validation cell present (ORT CPU check before push)
Gemini workflow documented in a markdown cell in the notebook

Global Acceptance Criteria

All must pass before closing T-001:

Build

cargo check --workspace — zero errors, zero warnings
dx serve --package desktop --features desktop — window opens, layout renders
dx serve --package web --features web — WASM builds, serves on localhost
cargo check --package api --target wasm32-unknown-unknown — passes cleanly
Tailwind output.css builds with zero errors

Visual

Five-panel layout matches Kansas prototype structure
All design tokens from Kansas tokens.css present in Tailwind @theme
No white flash on window open; background is #070b10
Window min-size 960×600 enforced
No hardcoded hex values in any .rs file

Codebase

All 22 model stub files exist with correct IDs, HF paths, runtime tier, and doc comments
llama-cpp-v3 only in native target, not in WASM
ort only activates under --features ort-models
MODELS.md at repo root — full 22-model inventory, CI tracking table, training sequence
docs/colab_export_template.ipynb exists with all 6 cells
Dioxus.toml at repo root

HuggingFace

Ashiedu/Synesthesia README updated — full model inventory, correct runtime strategy
manifest.json pushed to HF with all 22 model entries
Per-model README stub exists in every model subdirectory on HF

Do Not Do

Do not delete or rename any existing file from the working template
Do not change resolver = "2" in the workspace root
Do not change any existing dep versions — only add new ones
Do not add tokio to [dependencies] in api — native cfg block only
Do not add logic to stub components — structure + tokens only
Do not start T-002 until all T-001 criteria are green

Blocker Protocol

If any criterion cannot be met:

## Blocker — T-001[a/b/c/d/e/f] — [date]

**Subtask:** T-001x
**Criterion failing:** [verbatim]
**Root cause:** [what happened]
**Attempted fixes:** [what was tried]
**Decision needed:** [specific question for human]

Set status:blocked, remove status:in-progress. Do not guess or push forward.

Subtask Order for Jules

Jules should work through subtasks in this order. Complete and verify each before starting the next — do not parallelize across subtasks.

T-001a → T-001b → T-001c → T-001d → T-001e → T-001f
UI shell   Tokens   Layout   Deps     HF repo   Colab

T-001e (HF repo) requires Python + huggingface_hub and a valid HF_TOKEN in the environment. If not available locally, complete T-001a–d first and flag T-001e/f as requiring Colab or a machine with HF credentials.

GitHub CLI

# Create labels (safe to run if they already exist)
gh label create "stack:dioxus"    --color "#DEA584" --description "Dioxus UI framework" 2>/dev/null || true
gh label create "stack:tailwind"  --color "#38BDF8" --description "Tailwind CSS" 2>/dev/null || true
gh label create "stack:ml"        --color "#A78BFA" --description "ML / model pipeline" 2>/dev/null || true
gh label create "agent:human-led" --color "#F9A825" --description "Requires human review before merge" 2>/dev/null || true
gh label create "status:ready"    --color "#0075CA" --description "Ready to start" 2>/dev/null || true
gh label create "status:blocked"  --color "#E4E669" --description "Blocked, needs human input" 2>/dev/null || true
gh label create "day:1"           --color "#C5DEF5" --description "Day 1 tasks" 2>/dev/null || true

gh issue create \
  --title "T-001: Bootstrap Synesthesia — Dioxus fullstack, Tailwind v4, Kansas UI, full ML pipeline" \
  --label "type:task,stack:rust,stack:dioxus,stack:tailwind,stack:ml,agent:human-led,priority:critical,status:ready,day:1" \
  --body-file T-001.md

Blocks: T-002, T-003, T-004 and all subsequent tasks
Blocked By: none
Version: v0.1
Iteration: iter-1
Effort: L
Subtasks: T-001a · T-001b · T-001c · T-001d · T-001e · T-001f

T-001: Bootstrap Synesthesia — Dioxus Fullstack, Tailwind v4, Kansas UI Port, Full ML Pipeline

Workspace Ground Truth

Root Cargo.toml

packages/api/Cargo.toml (current state)

packages/ui/Cargo.toml (current state)

packages/desktop/Cargo.toml (current state)

packages/web/Cargo.toml (current state)

Package Roles

File Structure After This Task

Subtask T-001a — Tailwind v4 + Window Config

Goal

1. Tailwind v4 Setup

2. desktop/Cargo.toml Additions

3. desktop/src/main.rs

4. Dioxus.toml

Acceptance Criteria — T-001a

Subtask T-001b — Design Token Verification

Goal

Acceptance Criteria — T-001b

Subtask T-001c — Kansas UI Port to Dioxus RSX

Goal

Root Layout (packages/ui/src/layout.rs)

Header Bar (packages/ui/src/header.rs)

Track Mixer — Left Panel (packages/ui/src/track_mixer.rs)

Center Canvas (packages/ui/src/center_canvas.rs)

Pitch Intelligence — Right Panel (packages/ui/src/pitch_intel.rs)

Bottom Bar (packages/ui/src/bottom_bar.rs)

packages/ui/src/lib.rs

Acceptance Criteria — T-001c

Subtask T-001d — API Dependencies (Full Runtime Stack)

Goal

Updated packages/api/Cargo.toml

packages/api/src/lib.rs

packages/api/src/quality.rs

packages/api/src/models/mod.rs

packages/api/src/backend.rs (native only)

Acceptance Criteria — T-001d

Subtask T-001e — HuggingFace Repo + MODELS.md

Goal

1. Push updated README to HuggingFace

2. manifest.json

3. Per-model directory README stubs

4. MODELS.md at repo root

Training Philosophy

Gemini on Colab Workflow

Acceptance Criteria — T-001f

Global Acceptance Criteria

Build

Visual

Codebase

HuggingFace

Do Not Do

Blocker Protocol

Subtask Order for Jules

GitHub CLI

Root `Cargo.toml`

`packages/api/Cargo.toml` (current state)

`packages/ui/Cargo.toml` (current state)

`packages/desktop/Cargo.toml` (current state)

`packages/web/Cargo.toml` (current state)

Root Layout (`packages/ui/src/layout.rs`)

Header Bar (`packages/ui/src/header.rs`)

Track Mixer — Left Panel (`packages/ui/src/track_mixer.rs`)

Center Canvas (`packages/ui/src/center_canvas.rs`)

Pitch Intelligence — Right Panel (`packages/ui/src/pitch_intel.rs`)

Bottom Bar (`packages/ui/src/bottom_bar.rs`)

`packages/ui/src/lib.rs`

Updated `packages/api/Cargo.toml`

`packages/api/src/lib.rs`

`packages/api/src/quality.rs`

`packages/api/src/models/mod.rs`

`packages/api/src/backend.rs` (native only)