Instructions to use Vortex5/Glimmering-Citrus-31B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Vortex5/Glimmering-Citrus-31B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="Vortex5/Glimmering-Citrus-31B") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("Vortex5/Glimmering-Citrus-31B") model = AutoModelForMultimodalLM.from_pretrained("Vortex5/Glimmering-Citrus-31B", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Vortex5/Glimmering-Citrus-31B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Vortex5/Glimmering-Citrus-31B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Vortex5/Glimmering-Citrus-31B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/Vortex5/Glimmering-Citrus-31B
- SGLang
How to use Vortex5/Glimmering-Citrus-31B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Vortex5/Glimmering-Citrus-31B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Vortex5/Glimmering-Citrus-31B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Vortex5/Glimmering-Citrus-31B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Vortex5/Glimmering-Citrus-31B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use Vortex5/Glimmering-Citrus-31B with Docker Model Runner:
docker model run hf.co/Vortex5/Glimmering-Citrus-31B
Review (IQ3_S)
This merge exhibits an interesting vintage or classic feel, but still has some Gemma-isms (not X but Y, Tuesday, Elias). However, it uses creative emotional vocabulary, and leaves a better impression than the base model. It manages to weave an immersive atmosphere despite the purple prose, and perhaps with using SRP (system prompt) or deslopper LoRA it would help improve it even more.
Even at low quantization the output produced no errors. I would recommend this model if you have enough VRAM to run IQ3_S (16GB) without offloading, or Q5_K_M if you have a 3090.
The clock on the wall of the dusty antique shop didn’t just tell time; it whispered.
Elias, a man whose life was as measured and predictable as the ticking of his own pocket watch, had never noticed the whispers before. He was a collector of curiosities—old maps, tarnished silver spoons, faded postcards from places that no longer existed. His shop was a sanctuary of things forgotten, a place where history slept in velvet-lined boxes.
One rainy Tuesday, a woman named Elara entered his shop. She wore a coat the color of a bruised plum and carried an umbrella that dripped steadily onto the worn floorboards. Her eyes were the shade of a winter sea, cold and deep.
"I'm looking for something," she said, her voice like dry leaves skittering across pavement. "Something... lost."
Elias looked up from a ledger he was meticulously updating. "Lost is my specialty. What are you looking for?"
It also adapted well to the SRP test of my Oracle Seer card using the same input, fully embodying its cryptic style.
Tuesday
Tuesday comes up a lot in Mistral merges as well.