Text Generation
MLX
Safetensors
sentence-transformers
English
Chinese
qwen3
reranker
memory
agent
cross-encoder
conversational
4-bit precision
Instructions to use nisavid/MemReranker-4B-OptiQ-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use nisavid/MemReranker-4B-OptiQ-4bit with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("nisavid/MemReranker-4B-OptiQ-4bit") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - sentence-transformers
How to use nisavid/MemReranker-4B-OptiQ-4bit with sentence-transformers:
from sentence_transformers import CrossEncoder model = CrossEncoder("nisavid/MemReranker-4B-OptiQ-4bit") query = "Which planet is known as the Red Planet?" passages = [ "Venus is often called Earth's twin because of its similar size and proximity.", "Mars, known for its reddish appearance, is often referred to as the Red Planet.", "Jupiter, the largest planet in our solar system, has a prominent red spot.", "Saturn, famous for its rings, is sometimes mistaken for the Red Planet." ] scores = model.predict([(query, passage) for passage in passages]) print(scores) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- MLX LM
How to use nisavid/MemReranker-4B-OptiQ-4bit with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "nisavid/MemReranker-4B-OptiQ-4bit"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "nisavid/MemReranker-4B-OptiQ-4bit" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "nisavid/MemReranker-4B-OptiQ-4bit", "messages": [ {"role": "user", "content": "Hello"} ] }'
Built with mlx-optiq, the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon (no PyTorch, no cloud). Try the Lab 路 All OptiQ quants 路 Docs
- Downloads last month
- 21
Model size
0.8B params
Tensor type
BF16
路
U32 路
Hardware compatibility
Log In to add your hardware
4-bit
Model tree for nisavid/MemReranker-4B-OptiQ-4bit
Base model
Qwen/Qwen3-4B-Base Finetuned
Qwen/Qwen3-Reranker-4B Finetuned
IAAR-Shanghai/MemReranker-4B