Instructions to use bactrianus/HotpotQA-Reader-Llama-3-70B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use bactrianus/HotpotQA-Reader-Llama-3-70B-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="bactrianus/HotpotQA-Reader-Llama-3-70B-Instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("bactrianus/HotpotQA-Reader-Llama-3-70B-Instruct")
model = AutoModelForCausalLM.from_pretrained("bactrianus/HotpotQA-Reader-Llama-3-70B-Instruct", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use bactrianus/HotpotQA-Reader-Llama-3-70B-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "bactrianus/HotpotQA-Reader-Llama-3-70B-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bactrianus/HotpotQA-Reader-Llama-3-70B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/bactrianus/HotpotQA-Reader-Llama-3-70B-Instruct

SGLang

How to use bactrianus/HotpotQA-Reader-Llama-3-70B-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "bactrianus/HotpotQA-Reader-Llama-3-70B-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bactrianus/HotpotQA-Reader-Llama-3-70B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "bactrianus/HotpotQA-Reader-Llama-3-70B-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bactrianus/HotpotQA-Reader-Llama-3-70B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use bactrianus/HotpotQA-Reader-Llama-3-70B-Instruct with Docker Model Runner:
```
docker model run hf.co/bactrianus/HotpotQA-Reader-Llama-3-70B-Instruct
```

Bactrainus HotpotQA Reader — Llama 3 70B Instruct

Artifact identity

Status: complete merged causal-language-model checkpoint
Base model: meta-llama/Meta-Llama-3-70B-Instruct
Audited Hub revision: 8f76e68c65955843fffbf5c9b0e0ae1e446fbf1a
Public artifact date: August 2024
Role: large direct answer reader over supplied evidence

This is a historical Llama 3 artifact. The revised Bactrainus manuscript uses Llama 3.1 70B Instruct for its controlled large-reader experiments. The two checkpoint identities are not interchangeable.

Model summary

The model is the large historical Bactrainus reader. It generates a concise answer from a question and evidence selected by an upstream component. It does not select passages or search an external corpus.

Intended use

Research on capacity effects in evidence-conditioned multi-hop answer generation.
Reader-stage diagnostics with validated HotpotQA supporting facts.
Final-stage generation in a fixed-candidate modular pipeline.

Out-of-scope use

Open-domain retrieval or web search.
Unverified factual generation without supplied evidence.
Safety-critical or high-stakes applications.
Treating revised Llama 3.1 results as measurements of this Llama 3 checkpoint.

Input and output contract

Input should contain one question and a compact, title-preserving serialization of selected evidence. Output is free-form text intended to contain the answer. Parse and normalize the output explicitly.

The exact historical prompt serialization is not embedded in the public model configuration. Users seeking strict legacy replication must pin the model revision and preserve their complete prompt/runtime manifest.

Loading

The 70B checkpoint requires substantial accelerator memory. Pin the revision and select a supported sharding or quantization strategy:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_ID = "bactrianus/HotpotQA-Reader-Llama-3-70B-Instruct"
REVISION = "8f76e68c65955843fffbf5c9b0e0ae1e446fbf1a"

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, revision=REVISION)
model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    revision=REVISION,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model.eval()

The loading example does not prescribe a production serving topology.

Training data and lineage

The checkpoint is derived from Meta Llama 3 70B Instruct and adapted on HotpotQA-derived reader supervision. Its matching canonical training view is reader-sft, pinned at revision 7f3a1d4d21f22aad7262d8ffd6520f31186b284d. It contains all 90,447 training source IDs and remains joinable to every other view through source_id. Exact byte identity with the historical 2024 files is not asserted.

from datasets import load_dataset

train = load_dataset(
    "bactrianus/bactrainus-hotpotqa",
    "reader-sft",
    split="train",
    revision="7f3a1d4d21f22aad7262d8ffd6520f31186b284d",
)

The revised manuscript's reader_70b.yaml describes a later Llama 3.1 experiment and must not be presented as this historical checkpoint's verified training manifest.

Evaluation boundary

No evaluation predictions or results are included here. See the paper for reported experiments and their artifact boundary. A fresh evaluation of this exact pinned revision is a new measurement.

Limitations

The checkpoint is specialized for English HotpotQA-style evidence-conditioned answering.
It depends on upstream evidence recall and cannot reliably repair missing evidence.
Full paragraphs and distractors can reduce answer quality despite larger model capacity.
Serving cost, latency, throughput, and energy were not established by this model card.
Model generations can contain unsupported statements or formatting errors.
The historical release does not contain a complete environment/prompt manifest.

License and attribution

The weights remain subject to the Meta Llama 3 Community License and Acceptable Use Policy.

Meta Llama 3 is licensed under the Meta Llama 3 Community License, Copyright Meta Platforms, Inc. All Rights Reserved.

Built with Meta Llama 3.

HotpotQA-derived data is licensed under CC BY-SA 4.0. Bactrainus code is Apache-2.0 licensed.

Citation

@article{barati2025bactrainus,
  title   = {Bactrainus: Optimizing Large Language Models for Multi-hop Complex Question Answering Tasks},
  author  = {Barati, Iman and Ghafouri, Arash and Minaei-Bidgoli, Behrouz},
  journal = {arXiv preprint arXiv:2501.06286},
  year    = {2025},
  doi     = {10.48550/arXiv.2501.06286},
  url     = {https://arxiv.org/abs/2501.06286}
}

Downloads last month: 3

Safetensors

Model size

71B params

Tensor type

BF16

Model tree for bactrianus/HotpotQA-Reader-Llama-3-70B-Instruct

Base model

meta-llama/Meta-Llama-3-70B

Finetuned

meta-llama/Meta-Llama-3-70B-Instruct

Finetuned

(51)

this model

Dataset used to train bactrianus/HotpotQA-Reader-Llama-3-70B-Instruct

Paper for bactrianus/HotpotQA-Reader-Llama-3-70B-Instruct

Bactrainus: Optimizing Large Language Models for Multi-hop Complex Question Answering Tasks

Paper • 2501.06286 • Published Jan 10, 2025 • 1