air5978/clinical-tinyllama

---

library_name: transformers license: apache-2.0 tags: - llm - causal-lm - tinyllama - clinical - summarization - peft - lora - healthcare - instruction-tuned base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0 pipeline_tag: text-generation datasets: - geekdom/clinical_data

Clinical TinyLlama 🏥🧠

Model Description

This model is a fine-tuned version of TinyLlama-1.1B-Chat-v1.0 trained on a synthetic clinical dataset for clinical note summarization.

It takes structured hospital notes (ICU admission, treatment, discharge condition) and generates concise medical summaries.

The model was fine-tuned using:

• LoRA (Low-Rank Adaptation)
• Hugging Face Trainer
• Clinical note → summary pairs

---

Model Type

• Architecture: Decoder-only Transformer (LLaMA-style)
• Base Model: TinyLlama 1.1B Chat
• Fine-tuning Method: LoRA (PEFT)

---

Intended Use

Direct Use

• Clinical note summarization
• Medical report compression
• Educational NLP demos

Downstream Use

• Healthcare documentation assistants
• EHR summarization tools
• Clinical NLP research

Out-of-Scope Use

• Medical diagnosis
• Real patient decision-making
• Legal/clinical decision support

---

Training Data

• Dataset: geekdom/clinical_data
• Format:
  • Input: Clinical hospital notes
  • Output: Human-written summaries

---

Training Details

• Framework: Hugging Face Transformers + TRL + PEFT
• Method: Supervised Fine-Tuning (SFT)
• Parameter-Efficient Fine-Tuning: LoRA
• Sequence length: 256 tokens
• Batch size: 1 (gradient accumulation used)
• Hardware: CPU / Apple MPS (depending on setup)

---

Limitations

• May generate overly verbose or slightly redundant summaries
• Not medically validated
• Performance depends heavily on prompt quality
• Not suitable for real clinical deployment

---

How to Use

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "air5978/clinical-tinyllama"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

prompt = """Summarize the following clinical note:
Hospital Course: ...
Discharge Condition: ..."""

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=120)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))