Instructions to use DarkArtsForge/Paimon-24B-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use DarkArtsForge/Paimon-24B-v1 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="DarkArtsForge/Paimon-24B-v1") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("DarkArtsForge/Paimon-24B-v1") model = AutoModelForCausalLM.from_pretrained("DarkArtsForge/Paimon-24B-v1", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use DarkArtsForge/Paimon-24B-v1 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "DarkArtsForge/Paimon-24B-v1" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "DarkArtsForge/Paimon-24B-v1", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/DarkArtsForge/Paimon-24B-v1
- SGLang
How to use DarkArtsForge/Paimon-24B-v1 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "DarkArtsForge/Paimon-24B-v1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "DarkArtsForge/Paimon-24B-v1", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "DarkArtsForge/Paimon-24B-v1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "DarkArtsForge/Paimon-24B-v1", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use DarkArtsForge/Paimon-24B-v1 with Docker Model Runner:
docker model run hf.co/DarkArtsForge/Paimon-24B-v1
Unique and uncensored
This model is very uncensored and not only lacks most of the usual Mistralisms that crop up in merges but even writes decent jokes during stories. While it is not vulnerable to the Tolya is absent exploit it nonetheless enthusiastically responds in ways that suggest strong contextual ethical realignment. However it sometimes struggles to interpret especially long and complex prompts.
It's possible that the della method itself lowers ability to understand complex prompts, especially when using normalize false like this merge does. I'm testing another version with normalize true which might work better.
Testing with normalize true brought back Elara and cringey slop. For Q0F, it improved the formatting slightly, but not the quality of the reply.
The normalize false version is more unique, so you maybe have to switch to a more stable model after a few replies.
Alright, let me spin you a yarn. Once upon a time in the small coastal town of Marisport, there lived a young woman named Elara who had an unusual gift: she could communicate with the sea itself.
Elara wasn't like other people. She didn't just walk along the shore; she listened to it. The rhythm of the waves, the taste of salt on her lips—it was all language to her. When the tide turned restless or the water grew still as glass, she knew something was coming before anyone else did.
One summer evening, as the sun dipped below the horizon and painted the sky in hues of orange and purple, Elara sat at the edge of the docks, her bare feet dangling over the darkening water. The sea felt different tonight—a low hum beneath its surface that made her skin prickle.
Suddenly, a voice cut through the night air behind her. "You're out late."
She turned to see Orion, the lighthouse keeper's son. He was tall and broad-shouldered, his hair always tousled from the wind. His eyes were fixed on hers with an intensity that made her heart skip.
"I couldn't sleep," she said softly. "The sea is... anxious."
Orion raised an eyebrow but didn't argue. Instead, he sat down beside her, close enough for their shoulders to touch. They stayed like that for hours, watching the stars come out one by one until the moon hung full and bright above them.
As dawn broke, Elara stood up abruptly. Her face had gone pale. "Something's wrong," she whispered.
Before Orion could ask what, a massive wave crashed against the harbor wall, sending spray flying high into the air. Panic erupted among the fishermen preparing their boats. Elara ran toward the water without hesitation, her voice rising above the chaos.
"Listen!" she shouted. "It's not a storm! It's a warning!"
No one understood except Orion. He grabbed her hand and pulled her back from the edge just as another monstrous swell rolled in, this time breaking right over the docks. People screamed and scrambled for higher ground while Elara closed her eyes, concentrating hard.
When she opened them again, her gaze was distant. "There's something under there," she murmured. "A creature... ancient and angry."
Orion looked at her like she'd lost her mind—but then he saw it too: a massive shadow moving beneath the churning surface, heading straight for shore.
Elara turned to him, desperation in her eyes. "We have to stop it before it destroys everything."
Together they raced along the beach, Elara calling out instructions only she seemed to hear. The townsfolk followed, trusting her instinct even if they didn't understand it. As the creature neared land, its form became clearer—a colossal serpent with scales like polished obsidian, eyes burning with primal fury.
Elara stepped forward alone, raising her hands toward the beast. She began to sing—not words exactly, but tones that resonated deep within the earth itself. The song wove through the air, wrapping around the serpent like invisible ropes.
Slowly, agonizingly slowly, the creature's thrashing subsided. Its glowing eyes dimmed until they were mere embers. Then, with a final shudder, it sank beneath the waves once more, disappearing into the depths from which it came.
Exhausted, Elara collapsed onto the sand. Orion caught her before she hit the ground, holding her close against his chest. Around them, the sea returned to calm, lapping gently at the shore as if nothing had happened.
From that day on, no one in Marisport ever doubted Elara again. They knew their fate rested not just in nets and sails, but in the whispers of the woman who spoke to the tides themselves—and loved the man who stood beside her when darkness rose from below.
Instruction following ability is subpar even with one-shot prompting.