AgroguardAI/clm-agricultural-gpt2-lora

CLM Agricultural GPT-2 LoRA Adapter

A GPT-2 causal language model fine-tuned with LoRA (Low-Rank Adaptation) on agricultural domain text. This adapter is trained to generate coherent agricultural and farming-related text.

Model Details

• Base Model: GPT-2 (openai-community/gpt2)
• Fine-tuning Method: LoRA (PEFT)
• Task Type: Causal Language Modeling (CLM)
• Developer: AgroGuardAI
• Language: English
• License: MIT

LoRA Configuration

Parameter	Value
PEFT Type	LORA
Rank (r)	4
Alpha	8
Dropout	0.05
Target Modules	c_proj, c_attn
Fan-in/Fan-out	True
PEFT Version	0.19.1

Uses

Direct Use

This adapter can be loaded with the base GPT-2 model for agricultural text generation tasks including:

• Generating farming-related descriptions and narratives
• Agricultural knowledge synthesis
• Domain-specific text completion

Loading the Model

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model
base_model = AutoModelForCausalLM.from_pretrained("openai-community/gpt2")
tokenizer = AutoTokenizer.from_pretrained("AgroguardAI/clm-agricultural-gpt2-lora")
tokenizer.pad_token = tokenizer.eos_token

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "AgroguardAI/clm-agricultural-gpt2-lora")

# Generate text
inputs = tokenizer("In modern agriculture,", return_tensors="pt")
outputs = model.generate(**inputs, max_length=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training

Training Data

The model was fine-tuned on the AgroGuardAI Agricultural QA Dataset, which contains agricultural domain text covering farming practices, crop management, soil science, and agricultural technology.

Training Hyperparameters

• Training regime: fp32
• Training steps: 50
• LoRA rank: 4
• LoRA alpha: 8
• Dropout: 0.05
• Target modules: c_proj, c_attn

Bias, Risks, and Limitations

This model is specialized for the agricultural domain. It may not perform well on general-purpose text or topics outside its training data. Users should evaluate outputs for accuracy in production agricultural applications.

Citation

@misc{agroguardai-clm-gpt2-lora,
  author = {AgroGuardAI},
  title = {CLM Agricultural GPT-2 LoRA Adapter},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/AgroguardAI/clm-agricultural-gpt2-lora}}
}

Framework Versions

• Transformers: 4.x
• PEFT: 0.19.1
• Python: 3.14