The fields you can use to run this model with an API. If you don't give a value for a field its default value will be used.

Field	Type	Default value	Description
audio_duration	
number
30
Duration of the generated audio in seconds
prompt	
string
pop music, catchy melody, upbeat
Text description of the music to generate
lyrics	
string
Lyrics for the music (optional)
infer_step	
integer
27
Min: 10

Max: 100

Number of inference steps
guidance_scale	
number
15
Min: 1

Max: 20

Guidance scale for classifier-free guidance
scheduler_type	
string
(enum)
euler
Options:

euler, heun

Scheduler type
cfg_type	
string
(enum)
apg
Options:

apg, cfg, cfg_star

CFG type
omega_scale	
number
10
Min: 1

Max: 20

Omega scale
seed	
integer
-1
Seed for random number generation
guidance_interval	
number
0.5
Max: 1

Guidance interval
guidance_interval_decay	
number
0
Max: 1

Guidance interval decay
min_guidance_scale	
number
3
Min: 1

Max: 15

Minimum guidance scale
use_erg_tag	
boolean
True
Use ERG for tags
use_erg_lyric	
boolean
True
Use ERG for lyrics
use_erg_diffusion	
boolean
True
Use ERG for diffusion

🚀 We introduce ACE-Step, a novel open-source foundation model for music generation that overcomes key limitations of existing approaches and achieves state-of-the-art performance through a holistic architectural design. Current methods face inherent trade-offs between generation speed, musical coherence, and controllability. For instance, LLM-based models (e.g., Yue, SongGen) excel at lyric alignment but suffer from slow inference and structural artifacts. Diffusion models (e.g., DiffRhythm), on the other hand, enable faster synthesis but often lack long-range structural coherence. 🎼

🌉 ACE-Step bridges this gap by integrating diffusion-based generation with Sana's Deep Compression AutoEncoder (DCAE) and a lightweight linear transformer. It further leverages MERT and m-hubert to align semantic representations (REPA) during training, enabling rapid convergence. As a result, our model synthesizes up to 4 minutes of music in just 20 seconds on an A100 GPU—15× faster than LLM-based baselines—while achieving superior musical coherence and lyric alignment across melody, harmony, and rhythm metrics. ⚡ Moreover, ACE-Step preserves fine-grained acoustic details, enabling advanced control mechanisms such as voice cloning, lyric editing, remixing, and track generation (e.g., lyric2vocal, singing2accompaniment). 🎚️

🔮 Rather than building yet another end-to-end text-to-music pipeline, our vision is to establish a foundation model for music AI: a fast, general-purpose, efficient yet flexible architecture that makes it easy to train sub-tasks on top of it. This paves the way for developing powerful tools that seamlessly integrate into the creative workflows of music artists, producers, and content creators. In short, we aim to build the Stable Diffusion moment for music. 🎸



framework


📋 Table of contents
🎯 Baseline Quality
General Songs
Experimental Inputs
Instrumentals
Multiple Languages
🎛️ Controlability
🔄 Variations Generation
🎨 Repainting
✏️ Lyric Edit
💡 Application
🎤 Lyric2Vocal
📝 Text2Sample
🎧 RapMachine
⏳ Coming Soon
🎹 StemGen
🎻 Singing2Accompaniment
framework
Expand All Lyrics
🎯 Baseline Quality
📝 Note:

🎵 Lyrics are random picked from AI music generation community or internet and not in our training set.

📏 Existing models either lack length control (LLMs) or are fixed-length (diffusion). We enable flexible length for practical music composition.

💻 B.T.W., the project page is vibe coded by Roocode. 😊

Prompt	Lyrics	ACE-Step generated
General Songs
Note:

- Supports all mainstream music styles with various description formats including short tags, descriptive text, or use-case scenarios.

electronic, rock, pop	
Lyrics
▼
▶
funk, pop, soul, melodic	
Lyrics
▼
▶
background music for parties, radio broadcasts, streaming platforms, female voice	
Lyrics
▼
▶
surf music	
Lyrics
▼
▶
DUBSTEP, OBSCURE, DUBSTEP, DUBSTEP, DUBSTEP, DARKNESS, DUBSTEP, DUBSTEP, DUBSTEP, FEAR, DUBSTEP, DUBSTEP, DUBSTEP, TERROR, DUBSTEP, DUBSTEP, DUBSTEP, DUBSTEP, DUBSTEP, DUBSTEP, DUBSTEP, DUBSTEP	
Lyrics
▼
▶
cyberpunk, Acid jazz, electro, em, soft electric drums	
Lyrics
▼
▶
dark, death rock, metal, hardcore, electric guitar, powerful, bass, drums, 110 bpm, G major	
Lyrics
▼
▶
Cuban music, salsa, son, Afro-Cuban, traditional Cuban	
Lyrics
▼
▶
alternative rock, pop, rock	
Lyrics
▼
▶
aggressive, Heavy Riffs, Blast Beats, Satanic Black Metal	
Lyrics
▼
▶
country rock, folk rock, southern rock, bluegrass, pop	
Lyrics
▼
▶
Dark Electro, Industrial Techno, Gothic Rave	
Lyrics
▼
▶
disco	
Lyrics
▼
▶
electronic rap	
Lyrics
▼
▶
808 bass, smooth flow, party atmosphere, theme, sub bassline, mandarin hip hop, smooth, bassline, fast, hip hop, rap	
Lyrics
▼
▶
rock, orchestral, bass, drums, electric guitar, piano, synthesizer, violin, viola, cello, fast, energetic, motivational, inspirational, empowering	
Lyrics
▼
▶
melancholic, world, sad	
Lyrics
▼
▶
Experimental Inputs
Note:

- Experimentally testing various inputs to assess its usability.

Acid House, Roland TB-303, Dub	
Lyrics
▼
▶
a cappella	
Lyrics
▼
▶
b-box, deep male voice, trap, hip-hop, super fast tempo	
Lyrics
▼
▶
drum & bass, 160bpm, ethereal dark liquid, deep bassline, female vocals	
Lyrics
▼
▶
female voice	
Lyrics
▼
▶
Classic opera, Mozart’s, Oratorio, Aria, female	
Lyrics
▼
▶
Instrumentals
Note:

- Supports various instrumental music generation across different genres and styles.

- Capable of producing realistic instrumental tracks with appropriate timbre and expression for each instrument.

- Can generate complex arrangements with multiple instruments while maintaining musical coherence.

Nightclubs, dance parties, workout playlists, radio broadcasts	
Lyrics
▼
▶
Minimal Techno	
Lyrics
▼
▶
phonk, russian dark accordion, 130 bpm, russian psaltery, russian harmonica, psychedelic, dark	
Lyrics
▼
▶
saxphone, jazz	
Lyrics
▼
▶
sonata, piano, Violin, B Flat Major, allegro	
Lyrics
▼
▶
tango finlandés, guitarra clásica	
Lyrics
▼
▶
Psychedelic trance	
Lyrics
▼
▶
volin, solo, fast tempo	
Lyrics
▼
▶
Multiple Languages
Note:

- 19 languages are supported. But due to data imbalance, less common languages may underperform. Top 10 well-performing languages are:

• English
• Chinese
• Russian
• Spanish
• Japanese
• German
• French
• Portuguese
• Italian
• Korean

pop, piano, rap, dark, atmospheric	
Chinese
▼
▶
electronic, house, electro house, synthesizer, drums, bass, percussion, fast, energetic, uplifting, exciting	
Chinese
▼
▶
gentle folk, soft R&B rhythms, heartwarming melody, soothing, acoustic guitars, soft percussion, delicate string arrangements	
Chinese
▼
▶
crystal-clear soprano voice, female voice, romantic melody, strings, pop, piano, french song, soft bass, cello, atmosphere	
French
▼
▶
R&B, Soul, Hip Hop, Dance, geman rock, Psychedelic rock	
German
▼
▶
hip-house, funk	
Chinese
▼
▶
folk song, protest song, Italian, historical, melancholic, anthemic	
Italian
▼
▶
anime, cute female vocals, kawaii pop, j-pop, childish, piano, guitar, synthesizer, fast, happy, cheerful, lighthearted	
Japanese
▼
▶
Electropop, Hip-Hop, 132 bpm, K-Pop, EDM	
Korean
▼
▶
mandopop	
Chinese
▼
▶
Yacht rock, female singer, catchy, lounge, funny, uplifting	
Portuguese
▼
▶
emotive soundscape for dramatic female vocals, sad, this traditional waltz draws from russian and romani folk influences, hungarian minor scales, orchestral strings, folk, russian folk, classic, and romani percussion create an epic	
Russian
▼
▶
Latin Music, melodic, subtle acoustic guitar, a warm male vocal	
Spanish
▼
▶
🎛️ Controlability
📝 Note:

The model supports various training-free applicaitons.

- retake: regenerate a variation of the same song.

- repaint: regenerate a specific part of the song.

- edit: modify the lyrics of the song.

Prompt	Lyrics	ACE-Step generated
Variations Generation
Note:

- This feature is implemented using training-free, inference-time optimization techniques.

- Our flow-matching model generates initial noise, then uses trigFlow's noise formula to add additional Gaussian noise.

- By controlling the mixing ratio between the original initial noise and the new Gaussian noise, we can adjust the degree of variation in the generated output.

The original style is funk/pop/soul/melodic. We can generate variations with adjustable similarity—higher 'variance' means more divergence from the original.	
Lyrics
▼
▶
variance=0.1	
Lyrics
▼
▶
variance=0.2	
Lyrics
▼
▶
variance=0.3	
Lyrics
▼
▶
variance=0.4	
Lyrics
▼
▶
variance=0.5	
Lyrics
▼
▶
variance=0.6	
Lyrics
▼
▶
Repaint
Note:

- Implemented by adding noise to the target audio input and applying mask constraints during the ODE process.

- When input conditions change from the original generation, only specific aspects can be modified while preserving the rest.

- Combined with Variations Generation techniques, it can also create localized variations in style, lyrics, or vocals.

We can repaint any specified area. To demonstrate the power and flexibility of repaint, here’s an example using a 0-30 second repaint orig style is "pop, rap, electronic, blues, hip-house, rhythm and blues"	
Lyrics
▼
▶
change singing gender	
Lyrics
▼
▶
change style	
Lyrics
▼
▶
change lyrics	
Lyrics
▼
▶
variance=0.7	
Lyrics
▼
▶
Edit
Note:

- We've innovatively applied flow-edit technology to enable localized lyric modifications while preserving melody, vocals, and accompaniment.

- Works with both generated content and uploaded audio, greatly enhancing creative possibilities.

- Current limitation: can only modify small segments of lyrics at once to avoid distortion, but multiple edits can be applied sequentially.

This is the input audio, the genre is "piano, pop, female voice" We can edit the part of lyrics but not change its melody, vocal timbre and bgm	
Lyrics
▼
▶
When I was young -> when you were kid	
Lyrics
▼
▶
When I was young -> When I was old	
Lyrics
▼
▶
I'd listen to the radio -> I'd listen to the spotify	
Lyrics
▼
▶
It made me smile -> It made me cry	
Lyrics
▼
▶
When I was young -> Quand j'étais jeune	
Lyrics
▼
▶
When I was young -> In meiner Jugend	
Lyrics
▼
▶
When I was young -> 子供の頃に	
Lyrics
▼
▶
When I was young -> 내가 어렸을 때	
Lyrics
▼
▶
When I was young -> 我小的时候	
Lyrics
▼
▶
💡 Application
📝 Note:

🎵 The model supports various fine-tuned applications

- Lyric2Vocal: lora finetune on pure vocals

- Text2Samples: lora finetune on musical samples & loops

- RapMachine: lora finetune on rap data


Prompt	Lyrics	ACE-Step generated
Lyric2Vocal (LoRA)
Note:

- Based on a LoRA fine-tuned on pure vocal data, allowing direct generation of vocal samples from lyrics.

- Offers numerous practical applications such as vocal demos, guide tracks, songwriting assistance, and vocal arrangement experimentation.

- Provides a quick way to test how lyrics might sound when sung, helping songwriters iterate faster.

clean vocals	
Lyrics
▼
▶
clean vocals	
Lyrics
▼
▶
clean vocals	
Lyrics
▼
▶
clean vocals	
Lyrics
▼
▶
clean vocals	
Lyrics
▼
▶
Text2Samples (LoRA)
Note:

- Similar to Lyric2Vocal, but fine-tuned on pure instrumental and sample data.

- Capable of generating conceptual music production samples from text descriptions.

- Useful for quickly creating instrument loops, sound effects, and musical elements for production.

Acoustic Guitar, 191.0 bpm	
Lyrics
▼
▶
110.0 bpm, electric, bass, loops, G# min keyscale	
Lyrics
▼
▶
loops, fills, acoustic, 90.0 bpm, lo-fi hip hop, drums	
Lyrics
▼
▶
160.0 bpm, grooves, drums	
Lyrics
▼
▶
140.0 bpm, Electronic Drum Kit	
Lyrics
▼
▶
A# maj keyscale, electric, guitar, loops, 103.0 bpm	
Lyrics
▼
▶
melody, erhu	
Lyrics
▼
▶
G# keyscale, 80.0 bpm, brass & woodwinds, flute, algoza	
Lyrics
▼
▶
guitar, 115.0 bpm, B min keyscale, loops, melody	
Lyrics
▼
▶
clean, metallic, hand pan, organic, 120.0 bpm, A min keyscale, loops	
Lyrics
▼
▶
fx, music, koto, melody, G min keyscale, strings, 90.0 bpm, loops	
Lyrics
▼
▶
Lead Electric Guitar, loops	
Lyrics
▼
▶
layered, A# min keyscale, lo-fi hip hop, loops, chords, synth, pads, lo-fi, 60.0 bpm	
Lyrics
▼
▶
F keyscale, loops, saxphone, 125.0 bpm	
Lyrics
▼
▶
trap edm, D# keyscale, loops, 140.0 bpm, synth	
Lyrics
▼
▶
melody, G min keyscale, acoustic, 105.0 bpm, violin, Arabic	
Lyrics
▼
▶
RapMachine (LoRA)
Note:

- Fine-tuned on pure rap data to create an AI system specialized in rap generation.

- Expected capabilities include AI rap battles and narrative expression through rap.

- Rap has exceptional storytelling and expressive capabilities, offering extraordinary application potential.

Orchestra, Symphony, Sonata, Opera, Concerto, Rap, Beat, DJ, MC, StreetCulture	
Lyrics
▼
▶
Rap, adult, male, spoken word, rapping, clear, warm, articulate, Lo-Fi Hip Hop, 100-120 BPM, Keyboard Chords, Male Rap, Lazy Rhythm, Melancholy, Rap	
Lyrics
▼
▶
东北话, spoken word, male, Tempo - Fast, Elements - Chorus Hook, Subgenre-Satirical Hip Hop, Rap, Chinese-language music, energetic, slightly nasal, Instrument - Live Bass Guitar, adult, Vocals - Syncopated Flow, Genre - Hip-Hop, rapping, bright	
Lyrics
▼
▶
articulate, spoken word, young adult, rap music, male, clear, energetic, warm, relaxed, breathy, night club, auto-tune, mumble rap, trap	
Lyrics
▼
▶
G-Funk, Hip Hop, Rap, Female Vocals, Melodic Rap, Summer, Laid-back Groove, Smooth Rhythm, Synthesizer Lead, Heavy Bassline, Groovy, West Coast Hip Hop	
Lyrics
▼
▶
Rap, adult, male, spoken word, singing, bright, energetic, clear	
Lyrics
▼
▶
singing, bright, slightly nasal, energetic, spoken word, young adult, male, rap music	
Lyrics
▼
▶
Chorus Hook, Melodic Rap, Ambient Synth Pads, adult, rap, Very Fast, Storytelling, Chinese Rap, male, spoken word, bright, energetic, Melodic Flow, clear	
Lyrics
▼
▶
J-Pop, Anime, kawaii future bass, Femal vocals, EDM, Boombap, Aggressive, Intense, Crisp Snare, Super Fast, Rap	
Lyrics
▼
▶
Hip Hop, Hi-hat Rolls, spoken word, Melodic Flow, articulate, Female Rap, 120 BPM, clear, warm, female, melodic Rap, adult, super fast	
Lyrics
▼
▶
Rap, Chinese Rap, J-Pop, Anime, kawaii pop, EDM, Aggressive, Intense, Crisp Snare, Super Fast, Clear	
Lyrics
▼
▶
Rap, J-Pop, Anime, kawaii pop, EDM, Aggressive, Intense, Crisp Snare, Super Fast, Clear	
Lyrics
▼
▶
lyrical rap, young adult, female, rap flow, spoken word, ad-libs, bright, energetic, eat, Fast, Engaging, Energetic	
Lyrics
▼
▶
articulate, spoken word, young adult, warm, rap music, male, clear, street, dark, rap flow, hardcore rap, fast	
Lyrics
▼
▶
四川话, spoken word, male, Tempo - Fast, Elements - Chorus Hook, Subgenre-Satirical Hip Hop, Rap, Chinese-language music, energetic, slightly nasal, Instrument - Live Bass Guitar, adult, Vocals - Syncopated Flow, Genre - Hip-Hop, rapping, bright	
Lyrics
▼
▶
articulate, spoken word, young adult, rap music, female, clear, energetic, warm	
Lyrics

based on all this info what is the best genre prompt to only get female voices?