Instructions to use bactrianus/HotpotQA-Sentence-Retriever-Llama-3-8B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use bactrianus/HotpotQA-Sentence-Retriever-Llama-3-8B-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="bactrianus/HotpotQA-Sentence-Retriever-Llama-3-8B-Instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("bactrianus/HotpotQA-Sentence-Retriever-Llama-3-8B-Instruct")
model = AutoModelForCausalLM.from_pretrained("bactrianus/HotpotQA-Sentence-Retriever-Llama-3-8B-Instruct", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use bactrianus/HotpotQA-Sentence-Retriever-Llama-3-8B-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "bactrianus/HotpotQA-Sentence-Retriever-Llama-3-8B-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bactrianus/HotpotQA-Sentence-Retriever-Llama-3-8B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/bactrianus/HotpotQA-Sentence-Retriever-Llama-3-8B-Instruct

SGLang

How to use bactrianus/HotpotQA-Sentence-Retriever-Llama-3-8B-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "bactrianus/HotpotQA-Sentence-Retriever-Llama-3-8B-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bactrianus/HotpotQA-Sentence-Retriever-Llama-3-8B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "bactrianus/HotpotQA-Sentence-Retriever-Llama-3-8B-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bactrianus/HotpotQA-Sentence-Retriever-Llama-3-8B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use bactrianus/HotpotQA-Sentence-Retriever-Llama-3-8B-Instruct with Docker Model Runner:
```
docker model run hf.co/bactrianus/HotpotQA-Sentence-Retriever-Llama-3-8B-Instruct
```

Bactrainus HotpotQA Sentence Selector — Llama 3 8B Instruct

Artifact identity

Status: complete merged causal-language-model checkpoint
Base model: meta-llama/Meta-Llama-3-8B-Instruct
Audited Hub revision: 0ba9d0d84d7913f3d614061ebd71dca8690e9f93
Public artifact date: August 2024
Role: supporting-sentence selection within supplied paragraphs

The repository's historical “Retriever” name denotes generation of supporting-fact references within selected candidate paragraphs. It is not corpus retrieval.

This is a legacy Llama 3 artifact, separate from the revised manuscript's Llama 3.1 sentence selector.

Model summary

The model receives a multi-hop question and selected paragraphs, then emits the exact title and zero-based sentence index of each predicted supporting fact. Its output is designed to feed a reader through a compact facts interface.

Intended use

Supporting-fact selection for English HotpotQA-style inputs.
Sentence-level interface in a paragraph-to-sentence cascade.
Analysis of exact set prediction and error propagation.

Out-of-scope use

Searching outside the supplied paragraphs.
Treating selected sentences as verified evidence without checking them against the input.
General sentence ranking in unrelated domains.
Claiming direct reproduction of revised Llama 3.1 values.

Input and output contract

Input must preserve exact paragraph titles, sentence order, and zero-based indices. Output should be parsed into a deduplicated set of (title, sentence_index) pairs.

Every title and index must be validated against the current input instance. Unknown titles, negative indices, out-of-range indices, or malformed pairs should remain explicit errors rather than being silently repaired. The exact historical prompt is not embedded in the public configuration.

Loading

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_ID = "bactrianus/HotpotQA-Sentence-Retriever-Llama-3-8B-Instruct"
REVISION = "0ba9d0d84d7913f3d614061ebd71dca8690e9f93"

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, revision=REVISION)
model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    revision=REVISION,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model.eval()

Training data and lineage

The checkpoint derives from Meta Llama 3 8B Instruct and HotpotQA supporting-fact annotations. Its matching canonical training view is sentence-selector-sft, pinned at revision 7f3a1d4d21f22aad7262d8ffd6520f31186b284d. The view contains all 90,447 training source IDs and remains joinable to every other view through source_id. It is a clean deterministic view, not a claim of byte identity with all historical files.

from datasets import load_dataset

train = load_dataset(
    "bactrianus/bactrainus-hotpotqa",
    "sentence-selector-sft",
    split="train",
    revision="7f3a1d4d21f22aad7262d8ffd6520f31186b284d",
)

The revised Llama 3.1 settings in sentence_selector.yaml do not retrospectively define the training history of this legacy weight artifact.

Evaluation boundary

No evaluation artifacts or score claims are included. Use exact HotpotQA supporting-fact EM/F1 with a documented parser and pinned split. See the paper for the system-level analysis.

Limitations

Performance depends on the paragraph selector's recall.
Exact set prediction penalizes one missing or extra sentence.
Generated indices are vulnerable to off-by-one and formatting errors.
The model assumes HotpotQA-style title and sentence boundaries.
It was not validated as a general evidence-verification model.
Historical prompt and environment metadata are incomplete.

License and attribution

The weights remain subject to the Meta Llama 3 Community License and Acceptable Use Policy.

Meta Llama 3 is licensed under the Meta Llama 3 Community License, Copyright Meta Platforms, Inc. All Rights Reserved.

Built with Meta Llama 3.

HotpotQA-derived data is licensed under CC BY-SA 4.0. Bactrainus code is Apache-2.0 licensed.

Citation

@article{barati2025bactrainus,
  title   = {Bactrainus: Optimizing Large Language Models for Multi-hop Complex Question Answering Tasks},
  author  = {Barati, Iman and Ghafouri, Arash and Minaei-Bidgoli, Behrouz},
  journal = {arXiv preprint arXiv:2501.06286},
  year    = {2025},
  doi     = {10.48550/arXiv.2501.06286},
  url     = {https://arxiv.org/abs/2501.06286}
}

Downloads last month: -

Safetensors

Model size

8B params

Tensor type

BF16

F16

Model tree for bactrianus/HotpotQA-Sentence-Retriever-Llama-3-8B-Instruct

Base model

meta-llama/Meta-Llama-3-8B-Instruct

Finetuned

(1084)

this model

Dataset used to train bactrianus/HotpotQA-Sentence-Retriever-Llama-3-8B-Instruct

Paper for bactrianus/HotpotQA-Sentence-Retriever-Llama-3-8B-Instruct

Bactrainus: Optimizing Large Language Models for Multi-hop Complex Question Answering Tasks

Paper • 2501.06286 • Published Jan 10, 2025 • 1