Instructions to use Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF",
	filename="Qwen3-1.7B-ResearchReasoning-JSON-RL-Q4_K_M.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF:Q4_K_M

Use Docker

docker model run hf.co/Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF:Q4_K_M

Ollama
How to use Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF with Ollama:
```
ollama run hf.co/Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF:Q4_K_M
```

Unsloth Studio

How to use Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF to start chatting

How to use Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF with Docker Model Runner:
```
docker model run hf.co/Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF:Q4_K_M
```

Lemonade

How to use Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Qwen3-1.7B-ResearchReasoning-JSON-RL-GGUF-Q4_K_M

List all available models

lemonade list

Qwen3-1.7B ResearchReasoning JSON RL — GGUF

GGUF quantizations of Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL, a Qwen3-1.7B fine-tune for evidence-grounded research reasoning, source-aware QA, claim verification, and strict JSON schema generation.

Adapter repository: https://huggingface.co/Ethosoft/Qwen3-1.7B-ResearchReasoning-JSON-RL
Base model: https://huggingface.co/Qwen/Qwen3-1.7B
Source code: https://github.com/Ahmet2001/QA-research-SLM

The PEFT adapter was merged into the base model before GGUF conversion. The project calls the final training stage RL-lite: verifier-guided rejection sampling followed by supervised fine-tuning, not full online policy-gradient RL.

Files

Quantization	Suggested use
`Q8_0`	Highest fidelity among the uploaded quantizations; largest file
`Q5_K_M`	Recommended balance for schema-sensitive local use
`Q4_K_M`	Smaller and faster; validate JSON carefully

GGUF structural validation

Each file was checked for a valid GGUF binary header, supported GGUF version, nonzero tensor count, and nonzero metadata count. The files were produced successfully with the llama.cpp converter and quantizer. This is a structural file validation, not a quant-specific inference or JSON benchmark.

Quantization	Size	GGUF version	Tensors	Metadata entries	Structural check
`Q8_0`	1.71 GiB	3	310	29	Passed
`Q5_K_M`	1.17 GiB	3	310	29	Passed
`Q4_K_M`	1.03 GiB	3	310	29	Passed

Raw results are available in structural_validation.json.

llama.cpp

llama-cli \
  -m Qwen3-1.7B-ResearchReasoning-JSON-RL-Q5_K_M.gguf \
  -cnv \
  -n 768

For programmatic use, supply a strong system prompt that requires valid JSON, lists all required fields, disables markdown, and restricts evidence entries to source IDs present in the prompt.

Recommended output schema

{
  "task_type": "document_qa",
  "research_plan": ["read the question", "inspect the sources", "answer with evidence"],
  "evidence_needed": ["supporting source text"],
  "selected_sources": ["src_001"],
  "claims": [
    {
      "claim": "The answer is supported by src_001.",
      "status": "supported",
      "confidence": 0.9,
      "evidence": ["src_001"],
      "importance": "high"
    }
  ],
  "conflicts": [],
  "uncertainties": [],
  "answer": "Evidence-grounded answer"
}

Benchmark diagnostics

The full diagnostic results were produced with the PEFT adapter before GGUF conversion. They are included here for model context, but should not be treated as quant-specific measurements.

Benchmark	Adapter result	Diagnostic size
PubMedQA PQA-L	52.50% label accuracy	80
HotpotQA Distractor	27.00 EM / 36.91 F1	200
SciFact claims-dev	40.00% label accuracy / 26.21 macro-F1	200

These are fixed-seed custom diagnostics using a strict JSON research prompt, not official leaderboard submissions. See benchmark_results.json and the adapter model card for full settings and base-model comparisons.

Quantization caution

This model is optimized for strict structured generation. Quantization can preserve semantic answers while still damaging commas, quotes, closing braces, field names, or source-ID arrays. Validate every response. For reliability-sensitive applications, prefer Q5_K_M or Q8_0 and use constrained decoding or application-level JSON repair.

VibeThinker inspiration and attribution

This project is conceptually inspired by the VibeThinker line of work on verifiable reasoning in compact models, but it was independently trained for research-oriented structured generation. It does not reuse VibeThinker weights, data, or code.

Sen Xu et al., Tiny Model, Big Logic: Diversity-Driven Optimization Elicits Large-Model Reasoning Ability in VibeThinker-1.5B, arXiv:2511.06221, 2025. https://arxiv.org/abs/2511.06221
Sen Xu et al., VibeThinker-3B: Exploring the Frontier of Verifiable Reasoning in Small Language Models, arXiv:2606.16140, 2026. https://arxiv.org/abs/2606.16140

License

Apache-2.0, following the Qwen3 base model license. Review the original dataset licenses before redistributing derived training data.

Quant-specific evaluation status

The GGUF files passed structural validation after conversion and quantization. A CPU-only llama.cpp generation attempt on the TRUBA node exceeded a five-minute startup timeout for the Q8_0 file, so this release does not claim completed quant-specific runtime, JSON-adherence, or public-benchmark results. The benchmark table above belongs to the PEFT adapter before GGUF conversion. Re-run the evaluation suite in your target llama.cpp, LM Studio, or Ollama environment before reliability-sensitive deployment.