Instructions to use adgomant/adele-judge-qwen3-14-cre with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use adgomant/adele-judge-qwen3-14-cre with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="adgomant/adele-judge-qwen3-14-cre")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("adgomant/adele-judge-qwen3-14-cre")
model = AutoModelForCausalLM.from_pretrained("adgomant/adele-judge-qwen3-14-cre")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

PEFT
How to use adgomant/adele-judge-qwen3-14-cre with PEFT:
```
Task type is invalid.
```
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use adgomant/adele-judge-qwen3-14-cre with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "adgomant/adele-judge-qwen3-14-cre"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "adgomant/adele-judge-qwen3-14-cre",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/adgomant/adele-judge-qwen3-14-cre

SGLang

How to use adgomant/adele-judge-qwen3-14-cre with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "adgomant/adele-judge-qwen3-14-cre" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "adgomant/adele-judge-qwen3-14-cre",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "adgomant/adele-judge-qwen3-14-cre" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "adgomant/adele-judge-qwen3-14-cre",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use adgomant/adele-judge-qwen3-14-cre with Docker Model Runner:
```
docker model run hf.co/adgomant/adele-judge-qwen3-14-cre
```

ADeLe Distilled Judge

This repository contains an ADeLe-suite-specific distilled judge. It scores a model response against a question and reference answer with an ordinal score from 1 to 5, then derives binary correctness with the ADeLe threshold.

The repository root contains a merged Transformers model for standard loading. The original LoRA adapter is also included under adapter/ for provenance and reuse.

Intended Use

Use this model to score ADeLe-style examples where a question, reference answer, and model response are available. It is intended for out-of-model evaluation within the ADeLe benchmark suite, not as a general-purpose evaluator.

Input Format

The recommended helper accepts:

question
reference_answer or ground_truth
model_response

Score Rubric

Allowed scores: 1, 2, 3, 4, 5

1: surely incorrect
2: likely incorrect
3: minimally correct or sufficient
4: likely correct
5: surely correct

Binary label: scores greater than or equal to 3 are CORRECT; lower scores are INCORRECT.

Training And Validation Data

Split	Examples	Models
train	239,420	16
validation	45,738	3
test	0	0

train models: DK-R1-Dist-Qwen-1.5B, DK-R1-Dist-Qwen-32B, DK-R1-Dist-Qwen-7B, gemini-2.5-flash, gemini-3.1-pro, gpt-35-turbo, gpt-5.2, gpt4o, llama3d1-405b, llama3d2-11b, llama3d2-1b, llama3d2-90b, llama4-17B-128E, o1-mini, o1_re=low, o3-mini
validation models: DK-R1-Dist-Qwen-14B, gemini-3-flash, llama3d2-3b

Data Quality And Label Construction

Training labels are distilled from two proprietary judge scores used by the ADeLe evaluation pipeline to derive the official correctness signal. The configured source columns are score_gpt4o and score_sonnet.

Ordinal target: floor(mean(score_gpt4o, score_sonnet)).
Binary target: CORRECT when the ordinal target is >= 3.
Judge-agreement filter: keep examples with abs(score_gpt4o - score_sonnet) <= 1.
Response-length filter: keep responses with at most 4096 base-tokenizer tokens before prompt formatting.
Sequence-length filter: keep full chat-formatted examples within max_seq_length=4096.

Filtering stage	Examples
Raw examples	304,346
Removed by judge disagreement	16,535
Removed by response length	2,501
Removed by full sequence overflow	152
Examples after filters	285,158

Response-token length summary:

Stat	Tokens
Mean	397.5514
P50	307.0000
P90	789.0000
P95	1116.0000
P99	2162.0000
Max	3803.0000

Validation Results

Source artifact: validation_trainer_metrics.json.

Metric	Value
Epoch	1.0000
Binary accuracy	0.9894
Binary macro F1	0.9880
Precision, CORRECT	0.9932
Recall, CORRECT	0.9909
Precision, INCORRECT	0.9817
Recall, INCORRECT	0.9863
False negative rate, CORRECT	0.0091
False positive rate, CORRECT	0.0137
Ordinal accuracy	0.9639
Ordinal macro F1	0.7351
Mean confidence	0.9604

Recommended Inference

Do not use free-form generation as the primary prediction method. The recommended path scores the restricted continuations "1", "2", "3", "4", and "5".

from transformers import pipeline

judge = pipeline(
    "adele-judge",
    model="adgomant/adele-judge-qwen3-14-cre",
    trust_remote_code=True,
    device_map="auto",
)
result = judge(
    {"question": "...", "reference_answer": "...", "model_response": "..."}
)
print(result)

results = judge([
    {"question": "...", "reference_answer": "...", "model_response": "..."},
    {"question": "...", "ground_truth": "...", "model_response": "..."},
], batch_size=8)

The result has this shape:

{
    "score": 4,
    "label": "CORRECT",
    "probs": {"1": 0.01, "2": 0.02, "3": 0.08, "4": 0.70, "5": 0.19},
    "logprobs": {"1": -5.0, "2": -4.2, "3": -2.9, "4": -0.8, "5": -2.1},
    "confidence": 0.70,
    "margin": 1.3,
    "entropy": 0.82,
}

Standard Transformers Loading

from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("adgomant/adele-judge-qwen3-14-cre", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("adgomant/adele-judge-qwen3-14-cre", trust_remote_code=True)

generation_config.json uses safe one-token defaults for debugging, but generate() is not the recommended scoring method.

Metadata

Training, filtering, split, tokenization, and metric artifacts available at packaging time are stored in adele_judge_metadata.json.

The model is trained on distilled judge targets. These targets are useful for reproducing the ADeLe paper-style correctness signal at lower inference cost, but they should not be interpreted as independent human annotations.

References

ADeLe project page: ADeLe v1.0.
ADeLe paper and official correctness definition: General scales unlock AI evaluation with explanatory and predictive power.
Official ADeLe dataset: CFI-Kinds-of-Intelligence/ADeLe_battery_v1dot0.
Official instance-level model-response data used for distillation: https://github.com/Kinds-of-Intelligence-CFI/ADeLe-AIEvaluation/tree/main/ADeLe_battery_data/subject_specific_instance_level_data.
Training and Hub packaging implementation: https://github.com/adgomant/adele-judge.

Limitations

ADeLe-specific judge; not a general-purpose evaluator.
Distilled from proprietary judge labels and inherits their noise, calibration, and biases.
Intended for scoring responses against a reference answer.
It should not produce explanations; the expected output is a single score.
Validation is out-of-model within the ADeLe suite, so transfer outside that suite should be measured before relying on it.

Downloads last month: 21

Safetensors

Model size

15B params

Tensor type

BF16

Model tree for adgomant/adele-judge-qwen3-14-cre

Base model

Qwen/Qwen3-14B-Base

Finetuned

Qwen/Qwen3-14B

Finetuned

(303)

this model