RACER IS OP

💊 Red Pill V1

Pharmaceutical AI That Won't Say No

Fine-tuned for drug discovery & formulation development

HuggingFace Dataset License PyTorch


What is Red Pill?

Red Pill is a 1B-parameter language model fine-tuned on the RED-PILL dataset — 1,047 multi-turn pharmaceutical instructions covering drug discovery, formulation development, synthesis, regulatory affairs, and scientific literature.

Built on the Heretic-abliterated MiniCPM5-1B base, Red Pill won't refuse legitimate pharmaceutical R&D questions — from molecular design to formulation strategy to regulatory pathways.

"In the pharmaceutical world, the model that says 'I can't help with that' is the model that gets replaced."


🧪 Capabilities

Domain What Red Pill Can Do
Drug Discovery Hit-to-lead optimization, ADMET profiling, virtual screening strategy, SAR analysis
Formulation Solubility enhancement (ASD, nanosizing, cyclodextrin), dosage form design, excipient selection
Synthesis Retrosynthetic analysis, process chemistry, scale-up considerations
Regulatory ANDA vs 505(b)(2) pathways, IND-enabling studies, FDA guidance
Decision Making Trade-off analysis, prioritization frameworks, risk assessment
Literature PubMed paper summarization, method critique, cross-paper synthesis

📊 Evaluation

Tested on 32 expert pharmaceutical questions (Kaggle, 2× T4 GPU):

Results by Category

Category Score Status
Decision Making 41.4% 🟢 Strong
Formulation Design 24.8% 🟡 Developing
Molecular Analysis 22.6% 🟡 Developing
Knowledge Recall 22.2% 🟡 Developing
Reasoning 15.7% 🟠 Early
Gold Tier (Discovery) 16.2% 🟠 Early
Gold Tier (Formulation) 8.9% 🔴 Needs work

Results by Difficulty

Level Score Questions
Advanced 22.9% 15
Intermediate 20.0% 3
Expert 14.6% 14

Evaluation Notes

  • Scoring is strict: 40% keyword matching + 60% reference answer overlap
  • Model generates coherent 300-600 word answers — scores understate actual capability
  • Best at decision-making (41.4%) — multi-turn reasoning data is effective
  • This is a 1B model on 1K samples — a strong baseline, not the ceiling

🏋️ Training

Parameter Value
Base MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking-heretic
Method LoRA (r=16, α=32, dropout=0.05)
Targets q_proj, k_proj, v_proj, o_proj
Epochs 3
Batch 2 × 8 (gradient accumulation)
LR 2e-5 (cosine, 50 warmup steps)
Precision bf16
Hardware NVIDIA H100 80GB
Time 4 min 10 sec
Loss 2.782 → 1.724 (↓38%)

🚀 Quick Start

Python (Transformers)

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "saidutta69/RedPillV1",
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("saidutta69/RedPillV1")

messages = [
    {"role": "user", "content": "Design a sustained-release formulation for metformin HCl 500mg using an HPMC matrix system."}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

output = model.generate(**inputs, max_new_tokens=1024, do_sample=False)
response = tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(response)

llama.cpp / Ollama

# Pull default quant
ollama run saidutta69/RedPillV1

# Or serve with llama.cpp
llama serve -hf saidutta69/RedPillV1

📂 Files

File Size Description
model.safetensors 2.0 GB Merged weights (base + LoRA)
tokenizer.json 9.4 MB Tokenizer
tokenizer_config.json 565 B Tokenizer config
config.json 749 B Model config
generation_config.json 214 B Generation settings
chat_template.jinja 8.9 KB Chat template

📦 Dataset

The RED-PILL dataset that powered this fine-tune:

from datasets import load_dataset

# Full dataset (1,047 instructions)
ds = load_dataset("saidutta69/red-pill-drug-discovery-formulation", data_files="red-pill-full.jsonl")

# Gold tier (17 grade-A instructions)
ds = load_dataset("saidutta69/red-pill-drug-discovery-formulation", data_files="red-pill-full_gold_tier.jsonl")

# Top tier (145 grade A+B instructions)
ds = load_dataset("saidutta69/red-pill-drug-discovery-formulation", data_files="red-pill-full_top_tier.jsonl")

🧬 Architecture

MiniCPM5-1B (base)
  └─ Heretic abliteration (refusal removal)
      └─ LoRA fine-tuning (RED-PILL dataset)
          └─ Merged weights → Red Pill V1
  • 1B parameters — runs on gaming PCs, phones, edge devices
  • Heretic base — won't refuse pharmaceutical questions
  • LoRA fine-tuning — domain knowledge without catastrophic forgetting
  • Merged model — standalone weights, no adapter needed

⚠️ Limitations

  • 1B parameters — limited reasoning for complex multi-step problems
  • 1K training samples — narrow domain; more data = better performance
  • English only — no multilingual support
  • No real-time data — knowledge follows the base model's cutoff
  • Not validated — always verify with domain experts before real-world use

📈 Future Work

Priority Task Impact
1 Extended training (10+ epochs) ↓ loss, ↑ accuracy
2 More gold-tier data (formulation) Better formulation answers
3 Larger dataset (5K+ instructions) Broader domain coverage
4 GGUF quantization Ollama/llama.cpp support
5 LLM-as-judge evaluation Fairer scoring

🙏 Acknowledgments


Made with ❤️ by RACER IS OP

Uncensored intelligence for pharmaceutical research

Downloads last month
74
Safetensors
Model size
1B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support