kvlkS/tb-research-assistant-lora

TB Research Assistant — LoRA Adapter

QLoRA adapter for meta-llama/Llama-3.1-8B-Instruct, fine-tuned on TB/Mycobacterium tuberculosis-specific instruction data (consultation-style Q&A, methodology critique, hypothesis generation, result interpretation). This is the exact adapter deployed in the TB Research Assistant chat app — no continued pretraining, no retrieval, just this adapter on top of the Instruct base.

Usage

Requires accepting the Llama 3.1 license and being logged in (huggingface-cli login) before loading the base model.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

base_id = "meta-llama/Llama-3.1-8B-Instruct"
adapter_id = "kvlkS/tb-research-assistant-lora"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)
tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base = AutoModelForCausalLM.from_pretrained(base_id, quantization_config=bnb_config, device_map="auto")
model = PeftModel.from_pretrained(base, adapter_id)

See the full repo for the chat UI this model serves.