Skill: danbennettuk/voice-note-to-midi (version 0.1.0)

--- name: voice-note-to-midi description: Convert voice notes, humming, and melodic audio recordings to quantized MIDI files using ML-based pitch detection and intelligent post-processing author: Clawd tags: [audio, midi, music, transcription, machine-learning] --- # 🎵 Voice Note to MIDI Transform your voice memos, humming, and melodic recordings into clean, quantized MIDI files ready for your DAW. ## What It Does This skill provides a complete audio-to-MIDI conversion pipeline that: 1. **Stem Separation** - Uses HPSS (Harmonic-Percussive Source Separation) to isolate melodic content from drums, noise, and background sounds 2. **ML-Powered Pitch Detection** - Leverages Spotify's Basic Pitch model for accurate fundamental frequency extraction 3. **Key Detection** - Automatically detects the musical key of your recording using Krumhansl-Kessler key profiles 4. **Intelligent Quantization** - Snaps notes to a configurable timing grid with optional key-aware pitch correction 5. **Post-Processing** - Applies octave pruning, overlap-based harmonic removal, and legato note merging for clean output ### Pipeline Architecture ``` Audio Input (WAV/M4A/MP3) ↓ ┌─────────────────────────────────────┐ │ Step 1: Stem Separation (HPSS) │ │ - Isolate harmonic content │ │ - Remove drums/percussion │ │ - Noise gating │ └─────────────────────────────────────┘ ↓ ┌─────────────────────────────────────┐ │ Step 2: Pitch Detection │ │ - Basic Pitch ML model (Spotify) │ │ - Polyphonic note detection │ │ - Onset/offset estimation │ └─────────────────────────────────────┘ ↓ ┌─────────────────────────────────────┐ │ Step 3: Analysis │ │ - Pitch class distribution │ │ - Key detection │ │ - Dominant note identification │ └─────────────────────────────────────┘ ↓ ┌─────────────────────────────────────┐ │ Step 4: Quantization & Cleanup │ │ - Timing grid snap │ │ - Key-aware pitch correction │ │ - Octave pruning (harmonic removal) │ │ - Overlap-based pruning │ │ - Note merging (legato) │ │ - Velocity normalization │ └─────────────────────────────────────┘ ↓ MIDI Output (Standard MIDI File) ``` ## Setup ### Prerequisites - Python 3.11+ (Python 3.14+ recommended) - FFmpeg (for audio format support) - pip ### Installation **Quick Install (Recommended):** ```bash cd /path/to/voice-note-to-midi ./setup.sh ``` This automated script will: - Check Python 3.11+ is installed - Create the `~/melody-pipeline` directory - Set up the virtual environment - Install all dependencies (basic-pitch, librosa, music21, etc.) - Download and configure the hum2midi script - Add melody-pipeline to your PATH **Manual Install:** If you prefer manual setup: ```bash mkdir -p ~/melody-pipeline cd ~/melody-pipeline python3 -m venv venv-bp source venv-bp/bin/activate pip install basic-pitch librosa soundfile mido music21 chmod +x ~/melody-pipeline/hum2midi ``` 5. **Add to your PATH (optional):** ```bash echo 'export PATH="$HOME/melody-pipeline:$PATH"' >> ~/.bashrc source ~/.bashrc ``` ### Verify Installation ```bash cd ~/melody-pipeline ./hum2midi --help ``` ## Usage ### Basic Usage Convert a voice memo to MIDI: ```bash ./hum2midi my_humming.wav ``` This creates `my_humming.mid` with 16th-note quantization. ### Specify Output File ```bash ./hum2midi input.wav output.mid ``` ### Command-Line Options | Option | Description | Default | |--------|-------------|---------| | `--grid <value>` | Quantization grid: `1/4`, `1/8`, `1/16`, `1/32` | `1/16` | | `--min-note <ms>` | Minimum note duration in milliseconds | `50` | | `--no-quantize` | Skip quantization (output raw Basic Pitch MIDI) | disabled | | `--key-aware` | Enable key-aware pitch correction | disabled | | `--no-analysis` | Skip pitch analysis and key detection | disabled | ### Usage Examples #### Quantize to eighth notes ```bash ./hum2midi melody.wav --grid 1/8 ``` #### Key-aware quantization (recommended for tonal music) ```bash ./hum2midi song.wav --key-aware ``` #### Require longer minimum notes ```bash ./hum2midi humming.wav --min-note 100 ``` #### Skip analysis for faster processing ```bash ./hum2midi quick.wav --no-analysis ``` #### Combine options ```bash ./hum2midi recording.wav output.mid --grid 1/8 --key-aware --min-note 80 ``` ### Processing MIDI Input You can also process existing MIDI files through the quantization pipeline: ```bash ./hum2midi input.mid output.mid --grid 1/16 --key-aware ``` This skips the audio processing steps and goes directly to analysis and quantization. ## Sample Output ``` ═══════════════════════════════════════════════════════════════ hum2midi - Melody-to-MIDI Pipeline (Basic Pitch Edition) [Key-Aware Mode Enabled] ═══════════════════════════════════════════════════════════════ Input: my_humming.wav Output: my_humming.mid → Step 1: Stem Separation (HPSS) Isolating melodic content... Loaded: 5.23s @ 44100Hz ✓ Melody stem extracted → 5.23s → Step 2: Audio-to-MIDI Conversion (Basic Pitch) Running Spotify's Basic Pitch ML model on melody stem... ✓ Raw MIDI generated (Basic Pitch) → Step 3: Pitch Analysis & Key Detection Notes detected: 42 total, 7 unique Note range: C3 - G4 Pitch classes: C3, E3, G3, A3, C4, D4, G4 Dominant note: G3 (23.8% of notes) Detected key: G major → Step 4: Quantization & Cleanup Octave pruning: removed 3 harmonic notes above 67 (median+12) Overlap pruning: removed 2 harmonic notes at overlapping positions Note merging: merged 5 staccato chunks into legato notes (gap<=60 ticks) Grid: 240 ticks (1/16) Notes: 38 notes Key: G major Key-aware: 2 notes corrected to scale Tempo: 120 BPM ✓ Quantized MIDI saved ═══════════════════════════════════════════════════════════════ ✓ Done! Output: my_humming.mid ═══════════════════════════════════════════════════════════════ 📊 ANALYSIS SUMMARY ───────────────────────────────────────────────────────────── Detected Notes: C3, E3, G3, A3, C4, D4, G4 Detected Key: G major Quantization: Key-aware mode (notes snapped to scale) MIDI Info: 38 notes, 7 unique pitches, 120 BPM Pitches: C3, E3, G3, A3, C4, D4, G4 ``` ## Notes & Limitations ### Audio Quality Matters - **Clear, loud melody** produces the best results - **Background noise** can cause false note detection - **Reverb and effects** may confuse pitch detection - **Close-mic'd vocals** work significantly better than room recordings ### Musical Considerations - **Monophonic sources** work best (single melody line) - **Polyphonic audio** (chords, multiple instruments) will produce messy results - **Vibrato and pitch bends** may be quantized to stepped pitches - **Rapid note passages** may be missed or merged ### Technical Limitations - **Tempo is fixed** at 120 BPM in output (time positions are preserved, but tempo may need adjustment in your DAW) - **Note velocities** are normalized but may need manual adjustment - **Very short notes** (<50ms) may be filtered out by default - **Extreme pitch ranges** may cause octave detection issues ### Post-Processing Recommendations After generating MIDI, you may want to: 1. **Import into your DAW** and adjust tempo to match your original recording 2. **Quantize further** if stricter timing is needed 3. **Adjust note velocities** for dynamics 4. **Apply swing/groove** templates if the rigid grid sounds too mechanical 5. **Edit individual notes** that were misdetected (common with fast runs) ### Supported Audio Formats Input formats supported via FFmpeg: - WAV, AIFF, FLAC (uncompressed, best quality) - MP3, M4A, AAC (compressed, acceptable) - OGG, OPUS (open source formats) - Most other formats FFmpeg supports ## Troubleshooting ### No notes detected - Check that input file isn't silent or corrupted - Try increasing `--min-note` threshold - Verify audio has clear melodic content (not just noise) ### Too many notes / messy output - Enable octave pruning and overlap pruning (on by default) - Use `--key-aware` to constrain to musical scale - Check for background noise in source audio ### Wrong key detected - Key detection works best with at least 8-10 measures of music - Chromatic passages may confuse the detector - Manually review and adjust in your DAW if needed ### Notes in wrong octave - Basic Pitch sometimes detects harmonics instead of fundamentals - The pipeline includes pruning, but some may slip through - Use your DAW's transpose function for simple octave shifts ## References - [Basic Pitch](https://github.com/spotify/basic-pitch) - Spotify's polyphonic pitch detection model - [librosa HPSS](https://librosa.org/doc/latest/generated/librosa.decompose.hpss.html) - Harmonic-Percussive Source Separation - [Krumhansl-Kessler Key Profiles](https://rnhart.net/articles/key-finding/) - Key detection algorithm ## License This skill integrates Basic Pitch by Spotify, which is licensed under Apache 2.0. The pipeline script and documentation are provided under MIT license.

path: QUICKSTART.md
content: # Voice Note to MIDI - Quick Reference

## 🚀 Quick Start

```bash
cd ~/melody-pipeline
./hum2midi my_recording.wav
```

That's it! Creates `my_recording.mid` in the same directory.

## 📝 Common Use Cases

### Convert iPhone Voice Memo
```bash
# Copy from phone, convert if needed
./hum2mui "2025-01-29 14.32.m4a" --key-aware
```

### Quick Melody to DAW
```bash
# 8th note grid for more natural feel
./hum2midi melody.wav --grid 1/8
```

### Clean Up Existing MIDI
```bash
# Re-quantize a messy MIDI file
./hum2midi messy_input.mid cleaned_output.mid --grid 1/16
```

### Song Analysis
```bash
# See key and pitch analysis
./hum2midi song_snippet.wav
```

## ⚙️ Option Reference

| If you want... | Use this |
|----------------|----------|
| Tighter timing | `--grid 1/32` |
| Looser feel | `--grid 1/8` or `--grid 1/4` |
| Snap to scale | `--key-aware` |
| Remove short blips | `--min-note 100` |
| Faster processing | `--no-analysis` |
| Raw Basic Pitch output | `--no-quantize` |

## 🔧 Tips

- **Best results:** Hum/sing clearly, close to mic, minimal background noise
- **Tempo:** Output is always 120 BPM - adjust in your DAW
- **Key-aware mode:** Helps with tonal music, may hurt atonal/experimental melodies
- **Legato:** The pipeline merges staccato chunks automatically

## 🐛 Quick Troubleshooting

| Problem | Solution |
|---------|----------|
| Too many notes | Check for background noise; use `--key-aware` |
| Wrong octave | Use DAW transpose; check for harmonics in source |
| Timing off | Adjust quantization grid; DAW groove templates |
| No output | Check audio isn't silent; increase `--min-note` |

## 📂 File Locations

- Pipeline: `~/melody-pipeline/hum2midi`
- Virtual env: `~/melody-pipeline/venv-bp/`
- Temp stems: `~/melody-pipeline/.stems/` (auto-cleaned)

---

For detailed documentation, see SKILL.md


path: README.md
content: # 🎵 Voice Note to MIDI

Convert voice memos, humming, and melodic recordings to clean, quantized MIDI files.

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

Transform your 3 AM melody ideas into MIDI notes for your DAW — no keyboard required.

## What It Does

```
Voice Note (WAV/M4A/MP3)
    ↓
┌─────────────────────────────────────┐
│ 1. Stem Separation (HPSS)          │
│    Isolate melody from noise         │
└─────────────────────────────────────┘
    ↓
┌─────────────────────────────────────┐
│ 2. Basic Pitch ML (Spotify)        │
│    Detect fundamental frequencies    │
└─────────────────────────────────────┘
    ↓
┌─────────────────────────────────────┐
│ 3. Key Detection                    │
│    Identify musical key              │
└─────────────────────────────────────┘
    ↓
┌─────────────────────────────────────┐
│ 4. Quantization & Cleanup           │
│    • Snap to timing grid            │
│    • Key-aware pitch correction      │
│    • Harmonic pruning (octave/overlap)│
│    • Note merging (legato)          │
└─────────────────────────────────────┘
    ↓
MIDI File → Your DAW
```

## Quick Start

```bash
git clone https://github.com/DanBennettUK/voice-note-to-midi.git
cd voice-note-to-midi
./setup.sh
```

Then convert a voice memo:

```bash
hum2midi my_humming.wav          # Creates my_humming.mid
hum2midi voice.wav song.mid      # Custom output name
hum2midi hum.wav --key-aware     # Auto-detect key & quantize
```

## Features

- **ML-Powered Pitch Detection** — Spotify's Basic Pitch model
- **Key Detection** — Automatic musical key identification
- **Key-Aware Quantization** — Snap notes to detected scale
- **Harmonic Pruning** — Remove overtones, keep fundamentals
- **Legato Merging** — Combine note chunks into sustained tones
- **Configurable Grid** — 1/4, 1/8, 1/16, or 1/32 note quantization

## Usage

```bash
hum2midi <input.wav> [output.mid] [options]

Options:
  --grid <value>      Quantization: 1/4, 1/8, 1/16, 1/32 (default: 1/16)
  --min-note <ms>     Minimum note duration (default: 50ms)
  --key-aware         Enable key-aware pitch correction
  --no-quantize       Skip quantization (raw Basic Pitch output)
  --no-analysis       Skip pitch analysis
```

## Requirements

- Python 3.11+
- FFmpeg (optional but recommended)
- See `setup.sh` for full dependency installation

## Limitations

- **Monophonic only** — one note at a time (voice can't do chords)
- **Pitched audio required** — humming/singing works, whisper/"air notes" don't
- **Quality matters** — loud, clear melody = better results
- **Background noise** — can confuse pitch detection

## How It Works

The pipeline uses a multi-stage approach:

1. **HPSS** separates harmonic (melodic) content from percussive sounds
2. **Basic Pitch** neural network extracts pitch information
3. **Music21** analyzes pitch classes to detect the musical key
4. **Post-processing** cleans up harmonics and merges legato notes

## Documentation

- **Full Guide**: See [SKILL.md](SKILL.md)
- **Quick Reference**: See [QUICKSTART.md](QUICKSTART.md)
- **Setup Script**: [setup.sh](setup.sh)

## License

MIT — see [LICENSE](LICENSE)

Uses [Basic Pitch](https://github.com/spotify/basic-pitch) by Spotify, librosa, and music21.

---

Made with 🦊 for [Clawdbot](https://clawdhub.com)


path: setup.sh
content: #!/bin/bash
# Setup script for voice-note-to-midi skill
# This automates the installation of the melody-to-MIDI pipeline

set -e

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
INSTALL_DIR="${INSTALL_DIR:-$HOME/melody-pipeline}"
VENV_DIR="$INSTALL_DIR/venv-bp"

echo "=================================="
echo "Voice Note to MIDI - Setup Script"
echo "=================================="
echo ""
echo "This will install the melody-to-MIDI pipeline to:"
echo "  $INSTALL_DIR"
echo ""

# Check Python version
if ! command -v python3 &> /dev/null; then
    echo "❌ Error: Python 3 is required but not installed."
    exit 1
fi

PYTHON_VERSION=$(python3 --version 2>&1 | grep -oP '\d+\.\d+')
REQUIRED_VERSION="3.11"

if [ "$(printf '%s\n' "$REQUIRED_VERSION" "$PYTHON_VERSION" | sort -V | head -n1)" != "$REQUIRED_VERSION" ]; then 
    echo "❌ Error: Python 3.11+ is required (found $PYTHON_VERSION)"
    exit 1
fi

echo "✓ Python version: $PYTHON_VERSION"

# Check for pip
if ! command -v pip3 &> /dev/null; then
    echo "❌ Error: pip3 is required but not installed."
    exit 1
fi

echo "✓ pip3 available"

# Check for FFmpeg (optional but recommended)
if command -v ffmpeg &> /dev/null; then
    echo "✓ FFmpeg available (for extended audio format support)"
else
    echo "⚠ FFmpeg not found. Install for broader audio format support."
fi

echo ""
echo "Creating installation directory..."
mkdir -p "$INSTALL_DIR"
cd "$INSTALL_DIR"

# Create virtual environment
echo "Creating Python virtual environment..."
if [ -d "$VENV_DIR" ]; then
    echo "Virtual environment already exists. Update? [y/N]"
    read -r response
    if [[ "$response" =~ ^([yY][eE][sS]|[yY])$ ]]; then
        rm -rf "$VENV_DIR"
        python3 -m venv "$VENV_DIR"
    fi
else
    python3 -m venv "$VENV_DIR"
fi

echo "✓ Virtual environment ready"

# Install dependencies
echo ""
echo "Installing dependencies (this may take a few minutes)..."
source "$VENV_DIR/bin/activate"

pip install --upgrade pip
pip install basic-pitch librosa soundfile mido

# Install optional music21 for advanced key detection
echo ""
echo "Install music21 for enhanced key detection? [Y/n]"
read -r response
if [[ ! "$response" =~ ^([nN][oO]|[nN])$ ]]; then
    pip install music21
    echo "✓ music21 installed"
fi

echo "✓ Core dependencies installed"

# Download or create hum2midi script
echo ""
echo "Setting up hum2midi script..."

if [ -f "$SCRIPT_DIR/hum2midi" ]; then
    cp "$SCRIPT_DIR/hum2midi" "$INSTALL_DIR/hum2midi"
    chmod +x "$INSTALL_DIR/hum2midi"
    echo "✓ hum2midi script copied from skill directory"
elif [ -f "$INSTALL_DIR/hum2midi" ]; then
    echo "✓ hum2midi script already exists"
else
    echo "⚠ Please download the hum2midi script manually:"
    echo "  wget https://raw.githubusercontent.com/basic-pitch/basic-pitch/main/hum2midi -O $INSTALL_DIR/hum2midi"
    echo "  chmod +x $INSTALL_DIR/hum2midi"
fi

# Create .stems directory for temporary files
mkdir -p "$INSTALL_DIR/.stems"

# Optionally add to PATH
echo ""
echo "Add $INSTALL_DIR to your PATH? [y/N]"
read -r response
if [[ "$response" =~ ^([yY][eE][sS]|[yY])$ ]]; then
    SHELL_RC="$HOME/.bashrc"
    if [ -f "$HOME/.zshrc" ]; then
        SHELL_RC="$HOME/.zshrc"
    fi
    
    if ! grep -q "$INSTALL_DIR" "$SHELL_RC"; then
        echo "export PATH=\"$INSTALL_DIR:\$PATH\"" >> "$SHELL_RC"
        echo "✓ Added to $SHELL_RC"
        echo "  Run 'source $SHELL_RC' to apply changes"
    else
        echo "✓ Already in PATH"
    fi
fi

echo ""
echo "=================================="
echo "Installation complete!"
echo "=================================="
echo ""
echo "Test your installation:"
echo "  cd $INSTALL_DIR"
echo "  ./hum2midi --help"
echo ""
echo "Quick start:"
echo "  ./hum2midi your_voice_memo.wav"
echo ""


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: suspicious | SkillSpector: clean (LOW)