Instructions to use build-small-hackathon/Nemotron-nano-4b-escape-room with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use build-small-hackathon/Nemotron-nano-4b-escape-room with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="build-small-hackathon/Nemotron-nano-4b-escape-room", filename="nemotron-room-lora-F16.gguf", )
llm.create_chat_completion( messages = [ { "role": "user", "content": "What is the capital of France?" } ] ) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use build-small-hackathon/Nemotron-nano-4b-escape-room with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf build-small-hackathon/Nemotron-nano-4b-escape-room:Q4_K_M # Run inference directly in the terminal: llama cli -hf build-small-hackathon/Nemotron-nano-4b-escape-room:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf build-small-hackathon/Nemotron-nano-4b-escape-room:Q4_K_M # Run inference directly in the terminal: llama cli -hf build-small-hackathon/Nemotron-nano-4b-escape-room:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf build-small-hackathon/Nemotron-nano-4b-escape-room:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf build-small-hackathon/Nemotron-nano-4b-escape-room:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf build-small-hackathon/Nemotron-nano-4b-escape-room:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf build-small-hackathon/Nemotron-nano-4b-escape-room:Q4_K_M
Use Docker
docker model run hf.co/build-small-hackathon/Nemotron-nano-4b-escape-room:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use build-small-hackathon/Nemotron-nano-4b-escape-room with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "build-small-hackathon/Nemotron-nano-4b-escape-room" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "build-small-hackathon/Nemotron-nano-4b-escape-room", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/build-small-hackathon/Nemotron-nano-4b-escape-room:Q4_K_M
- Ollama
How to use build-small-hackathon/Nemotron-nano-4b-escape-room with Ollama:
ollama run hf.co/build-small-hackathon/Nemotron-nano-4b-escape-room:Q4_K_M
- Unsloth Studio
How to use build-small-hackathon/Nemotron-nano-4b-escape-room with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for build-small-hackathon/Nemotron-nano-4b-escape-room to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for build-small-hackathon/Nemotron-nano-4b-escape-room to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for build-small-hackathon/Nemotron-nano-4b-escape-room to start chatting
- Pi
How to use build-small-hackathon/Nemotron-nano-4b-escape-room with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf build-small-hackathon/Nemotron-nano-4b-escape-room:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "build-small-hackathon/Nemotron-nano-4b-escape-room:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent new
How to use build-small-hackathon/Nemotron-nano-4b-escape-room with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf build-small-hackathon/Nemotron-nano-4b-escape-room:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default build-small-hackathon/Nemotron-nano-4b-escape-room:Q4_K_M
Run Hermes
hermes
- Atomic Chat new
- Docker Model Runner
How to use build-small-hackathon/Nemotron-nano-4b-escape-room with Docker Model Runner:
docker model run hf.co/build-small-hackathon/Nemotron-nano-4b-escape-room:Q4_K_M
- Lemonade
How to use build-small-hackathon/Nemotron-nano-4b-escape-room with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull build-small-hackathon/Nemotron-nano-4b-escape-room:Q4_K_M
Run and chat with the model
lemonade run user.Nemotron-nano-4b-escape-room-Q4_K_M
List all available models
lemonade list
Configuration Parsing Warning:Invalid JSON for config file config.json
Fine-tuned Nemotron-3 Nano 4B
This model is a fine-tuned version of NVIDIA Nemotron-3 Nano 4B, adapted for improved performance on a custom dataset for generating escape rooms. This fine-tune was realised for the build-small-hackathon using Unsloth and using compute credits provided by Modal.
Model Details
- Base model: https://huggingface.co/nvidia/Nemotron-3-Nano-4B
- Quantized model: https://huggingface.co/unsloth/NVIDIA-Nemotron-3-Nano-4B-GGUF
- Architecture: Decoder-only Transformer
- Parameters: ~4B
- Fine-tuning method: LoRA
- Framework: Unsloth
- Training environment: Modal
Intended Use
This model is intended for:
- Experimental / research use
- Generating room titles and descriptions / generating responses for trying to open doors/containers as json
It has not been optimized for production safety or alignment.
Limitations
- May produce incorrect or hallucinated outputs
- Not guaranteed to follow instructions perfectly
- Sensitive to prompt formatting
- May inherit biases from base model and training data
How to Use
llama.cpp
llama-cli.exe -m nemotron-room-lora-q4-k-m.gguf ^
-n 2048 --temp 0.7 --repeat-penalty 1.1 ^
--grammar-file json.gbnf ^
-sys "You are a creative dungeon-master AI that generates escape-room content. Always respond with a single valid JSON object and nothing else." ^
-p "{\"task\":\"generate_room\"} Respond with room_name, room_story, room_prompt, door_description, door_prompt, door_key_name, door_key_prompt, containers array, and keys array."
Output
{
"room_name": "The Crystal Lake of the Moon", "room_story": "A lake that reflects the moon so perfectly it seems like a second sky. The water is still and silver.", "room_prompt": "A serene lake under a full moon, reflecting the night sky perfectly, surrounded by misty mountains.", "door_description": "The entrance is carved into an ancient oak tree, with vines curling around the frame.", "door_prompt": "An old oak tree door with green vines and moss growing on the wood grain.", "door_key_name": "The Silver Moon Key", "door_key_prompt": "A small silver key shaped like a crescent moon, glowing softly.", "containers": [{"container_name": "The Hollow Oak Chest", "container_prompt": "An old wooden chest carved with animal faces, sitting on mossy ground."}, {"container_name": "The Stone Lantern", "container_prompt": "A large stone lantern containing a bundle of dried herbs and a small scroll."}, {"container_name": "The Leather Satchel", "container_prompt": "A worn leather satchel filled with various trinkets, including a compass and a coin."}, {"container_name": "The Glass Globe", "container_prompt": "A clear glass globe containing swirling blue and silver dust that catches the moonlight."}], "keys": [{"key_name": "The Rusty Key", "key_prompt": "A tarnished iron key with jagged teeth, rusted to a dull orange."}, {"key_name": "The Gold Coin", "key_prompt": "An ornate gold coin embedded with gemstones, resting on a piece of parchment."}]}
- Downloads last month
- 994
Model tree for build-small-hackathon/Nemotron-nano-4b-escape-room
Base model
nvidia/NVIDIA-Nemotron-Nano-12B-v2-Base