Instructions to use usermma/test with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use usermma/test with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="usermma/test", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("usermma/test", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use usermma/test with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf usermma/test:Q4_K_M # Run inference directly in the terminal: llama cli -hf usermma/test:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf usermma/test:Q4_K_M # Run inference directly in the terminal: llama cli -hf usermma/test:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf usermma/test:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf usermma/test:Q4_K_M
Use Docker
docker model run hf.co/usermma/test:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use usermma/test with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "usermma/test" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "usermma/test", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/usermma/test:Q4_K_M
- SGLang
How to use usermma/test with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "usermma/test" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "usermma/test", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "usermma/test" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "usermma/test", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Ollama
How to use usermma/test with Ollama:
ollama run hf.co/usermma/test:Q4_K_M
- Unsloth Desktop
- Pi
How to use usermma/test with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf usermma/test:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "usermma/test:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use usermma/test with Docker Model Runner:
docker model run hf.co/usermma/test:Q4_K_M
- Lemonade
How to use usermma/test with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull usermma/test:Q4_K_M
Run and chat with the model
lemonade run user.test-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use usermma/test with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf usermma/test:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default usermma/test:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use usermma/test with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf usermma/test:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "usermma/test:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Quark-150M-128K-Zero β GGUF
GGUF conversions of empero-ai/Quark-150M-128K-Zero,
a 165M-parameter research chat model (RL/GRPO from base, no SFT) built on the custom Quark architecture.
This repo ships the quantized weights plus a patched llama.cpp source tree that teaches llama.cpp
to recognize the quark architecture (needed to produce and load these files).
Files
| File | Quant | Size | BPW |
|---|---|---|---|
quark-150m-128k-zero-bf16.gguf |
BF16 | 318 MiB | 16.15 |
quark-150m-128k-zero-q8_0.gguf |
Q8_0 | 173 MiB | 8.72 |
quark-150m-128k-zero-q6_k.gguf |
Q6_K | 135 MiB | 6.80 |
quark-150m-128k-zero-q4_k_m.gguf |
Q4_K_M | 101 MiB | 5.07 |
quark-150m-128k-zero-q3_k_m.gguf |
Q3_K_M | 84 MiB | 4.18 |
quark-150m-128k-zero-q2_k.gguf |
Q2_K | 70 MiB | 3.50 |
Source snapshot of the patched inference/quantization runtime: llama-cpp-src.tar.gz.
β οΈ Architecture & runtime status β read this first
QuarkForCausalLM is a novel, custom architecture that upstream llama.cpp, Ollama and LM Studio do
not support out of the box. It is a hybrid decoder:
- Local layers (12): sliding-window attention (window 256) with RoPE, QK-norm and a learned per-head attention sink, blended per channel with a double-gated short causal convolution (kernel 4).
- Global layers (4, every 4th): full causal grouped-query attention (12 Q heads / 4 KV heads), RoPE, QK-norm.
- Multi-head gated value residuals anchored to layer 0, depth-scaled "sandwich" residuals, tied embeddings, 131,072-token context, 32,896-entry vocabulary.
What this means for these GGUF files:
- β
They are valid GGUF v3 containers, correctly shaped and quantized (produced with
llama-quantizeusing real k-quant mixtures, not naive re-packing). - β
They carry full metadata (arch =
quark, hyper-params, tokenizer, chat template) and can be inspected, quantized further, or consumed by tooling. - β οΈ Full inference is not yet implemented. The bundled
llama.cpppatch registers thequarkarchitecture so the model loads/quantizes, but the forward graph (conv blend, value residuals, sinks) is not wired up yet. So stockllama.cpp/ Ollama / LM Studio cannot generate text from these files today.
These artifacts are intended for research, model-distribution, and as a foundation for a future quark
inference implementation. If you need runnable inference, the next step is implementing the Quark graph in
llama.cpp (the arch is already registered in the bundled source).
What's in llama-cpp-src.tar.gz
A source snapshot of ggml-org/llama.cpp (shallow, tip commit) with a
minimal quark architecture registration patch applied β 4 files, +13 lines:
src/llama-arch.hβ addLLM_ARCH_QUARKsrc/llama-arch.cppβ map"quark"βLLM_ARCH_QUARKsrc/llama-model.cppβ factory case forllama_model_quarksrc/models/models.hβ minimalllama_model_quarkmodel class
Packaged like git clone --depth 1 (shallow history β only the tip commit, no full upstream history) and with
no build/ directory. Build it with CMake to get a llama-quantize (and friends) that understand quark.
How these were produced
- Downloaded the F32
model.safetensorsfrom the source model. - Converted to GGUF with a custom writer (arch
quark, GPT-2 byte-level BPE tokenizer, chat template preserved). - Quantized the F32 master to each target with
llama-quantizefrom the bundled patchedllama.cpp:BF16,Q8_0,Q6_K,Q4_K_M,Q3_K_M,Q2_K.
License
Apache-2.0 β same as the source model
empero-ai/Quark-150M-128K-Zero.
See LICENSE in the source repository and the llama.cpp license (MIT) for the bundled runtime source.
- Downloads last month
- 191
2-bit
3-bit
4-bit
6-bit
8-bit
16-bit
Model tree for usermma/test
Base model
empero-ai/Quark-150M-128K-Base