Instructions to use bactrianus/HotpotQA-Paragraph-Retriever-Llama-3-8B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use bactrianus/HotpotQA-Paragraph-Retriever-Llama-3-8B-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="bactrianus/HotpotQA-Paragraph-Retriever-Llama-3-8B-Instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("bactrianus/HotpotQA-Paragraph-Retriever-Llama-3-8B-Instruct")
model = AutoModelForCausalLM.from_pretrained("bactrianus/HotpotQA-Paragraph-Retriever-Llama-3-8B-Instruct", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use bactrianus/HotpotQA-Paragraph-Retriever-Llama-3-8B-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "bactrianus/HotpotQA-Paragraph-Retriever-Llama-3-8B-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bactrianus/HotpotQA-Paragraph-Retriever-Llama-3-8B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/bactrianus/HotpotQA-Paragraph-Retriever-Llama-3-8B-Instruct

SGLang

How to use bactrianus/HotpotQA-Paragraph-Retriever-Llama-3-8B-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "bactrianus/HotpotQA-Paragraph-Retriever-Llama-3-8B-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bactrianus/HotpotQA-Paragraph-Retriever-Llama-3-8B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "bactrianus/HotpotQA-Paragraph-Retriever-Llama-3-8B-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bactrianus/HotpotQA-Paragraph-Retriever-Llama-3-8B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use bactrianus/HotpotQA-Paragraph-Retriever-Llama-3-8B-Instruct with Docker Model Runner:
```
docker model run hf.co/bactrianus/HotpotQA-Paragraph-Retriever-Llama-3-8B-Instruct
```

Bactrainus HotpotQA Paragraph Selector — Llama 3 8B Instruct

Artifact identity

Status: complete merged causal-language-model checkpoint
Base model: meta-llama/Meta-Llama-3-8B-Instruct
Audited Hub revision: 1bc650152552e5a0ee7ee4afdfae01abc3bdc76e
Public artifact date: August 2024
Role: relevant-paragraph selection from the supplied candidate set

The historical repository uses the word “Retriever,” but the model does not query an external corpus. It selects from the fixed candidate set supplied by HotpotQA.

This is a legacy Llama 3 checkpoint. The revised manuscript's paragraph selector is a distinct Llama 3.1 experiment.

Model summary

The paragraph selector receives a question and the candidate paragraphs supplied with that example, then generates the titles of the paragraphs predicted to contain the evidence needed for the answer. Its output is an intermediate interface for a sentence selector or reader.

Intended use

Fixed-candidate paragraph relevance selection for HotpotQA-style research.
Upstream component in a modular selector--reader pipeline.
Analysis of paragraph recall and paragraph-to-sentence error propagation.

Out-of-scope use

Open-domain retrieval, embedding search, or document indexing.
General reranking without task-specific validation.
Safety-critical or production search decisions.
Using revised Llama 3.1 selector results as measurements of this artifact.

Input and output contract

Input must preserve the question, candidate order, and exact paragraph titles. Output should be parsed as an ordered, deduplicated list of exact candidate titles.

Reject unknown titles. Fuzzy matching can hide model errors and change paragraph-level metrics. The public legacy configuration does not preserve a complete prompt template, so new serialization should be versioned as a new evaluation condition.

Loading

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_ID = "bactrianus/HotpotQA-Paragraph-Retriever-Llama-3-8B-Instruct"
REVISION = "1bc650152552e5a0ee7ee4afdfae01abc3bdc76e"

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, revision=REVISION)
model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    revision=REVISION,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model.eval()

Training data and lineage

The checkpoint derives from Meta Llama 3 8B Instruct and HotpotQA-derived paragraph-selection supervision. Its matching canonical training view is paragraph-selector-sft, pinned at revision 7f3a1d4d21f22aad7262d8ffd6520f31186b284d. The view contains all 90,447 training source IDs and remains joinable to every other view through source_id. It is not asserted to reproduce every historical byte of the 2024 training files.

from datasets import load_dataset

train = load_dataset(
    "bactrianus/bactrainus-hotpotqa",
    "paragraph-selector-sft",
    split="train",
    revision="7f3a1d4d21f22aad7262d8ffd6520f31186b284d",
)

The revised paragraph_selector.yaml is a Llama 3.1 configuration and must not be attached to these weights as a verified legacy run manifest.

Evaluation boundary

This card includes no predictions or results. Evaluate exact title sets against the same pinned HotpotQA split and parsing policy. See the paper for the broader selector--reader analysis.

Limitations

The model assumes exactly the benchmark-provided candidate boundary.
A selected paragraph can still contain many irrelevant sentences.
Title formatting and parser policy can affect exact-match scores.
Missing a relevant paragraph constrains all downstream stages.
Wikipedia-derived candidates inherit temporal and coverage bias.
Historical prompt and environment metadata are incomplete.

License and attribution

The weights remain subject to the Meta Llama 3 Community License and Acceptable Use Policy.

Meta Llama 3 is licensed under the Meta Llama 3 Community License, Copyright Meta Platforms, Inc. All Rights Reserved.

Built with Meta Llama 3.

HotpotQA-derived data is licensed under CC BY-SA 4.0. Bactrainus code is Apache-2.0 licensed.

Citation

@article{barati2025bactrainus,
  title   = {Bactrainus: Optimizing Large Language Models for Multi-hop Complex Question Answering Tasks},
  author  = {Barati, Iman and Ghafouri, Arash and Minaei-Bidgoli, Behrouz},
  journal = {arXiv preprint arXiv:2501.06286},
  year    = {2025},
  doi     = {10.48550/arXiv.2501.06286},
  url     = {https://arxiv.org/abs/2501.06286}
}

Downloads last month: 1

Safetensors

Model size

8B params

Tensor type

BF16

F16

Model tree for bactrianus/HotpotQA-Paragraph-Retriever-Llama-3-8B-Instruct

Base model

meta-llama/Meta-Llama-3-8B-Instruct

Finetuned

(1084)

this model

Dataset used to train bactrianus/HotpotQA-Paragraph-Retriever-Llama-3-8B-Instruct

Paper for bactrianus/HotpotQA-Paragraph-Retriever-Llama-3-8B-Instruct

Bactrainus: Optimizing Large Language Models for Multi-hop Complex Question Answering Tasks

Paper • 2501.06286 • Published Jan 10, 2025 • 1