danielangelo1/llama-climate-change-stance-ptbr-lora

Climate Change Stance Classifier — Portuguese (LoRA · Llama 3.1 8B)

Fine-tuned LoRA adapter for stance detection in Brazilian Portuguese climate change discourse

---

Overview · Label Mapping · Quick Start · Dataset · Training · Limitations · Citation · Contact

---

Overview

This repository provides a LoRA (PEFT) adapter fine-tuned on top of meta-llama/Llama-3.1-8B for three-class stance classification in Portuguese climate change social media comments.

Note: This repository contains only the LoRA adapter weights. The base model must be loaded separately from meta-llama/Llama-3.1-8B.

---

Label Mapping

Label	Class	Description
0	Denier	Explicitly expresses skepticism toward climate change; denies its occurrence; downplays its impacts; rejects anthropogenic responsibility; claims global warming is a "hoax," a "lie," a natural cycle, or a conspiracy; or articulates generalized denial of climate science.
1	Believer	Explicitly acknowledges climate change; agrees with the scientific consensus; expresses environmental concern; defends scientific evidence; or criticizes harmful practices such as deforestation or wildfires.
2	Inconclusive	Does not clearly belong to either of the above categories; contains ambiguous statements; expresses generic agreement or disagreement without clear stance; lacks sufficient information to infer position; or is irrelevant to the climate change debate.

---

Quick Start

Installation

pip install torch transformers peft accelerate

Inference

🔑 Access Token Required
The base model meta-llama/Llama-3.1-8B is a gated model. You must:

1. Request access at meta-llama/Llama-3.1-8B
2. Accept Meta's license agreement on Hugging Face
3. Generate a token at huggingface.co/settings/tokens and pass it via token= or run huggingface-cli login before loading the model

---

Training Details

Hyperparameter	Value
Method	QLoRA (Quantized LoRA — PEFT)
Base Model	meta-llama/Llama-3.1-8B
Quantization	4-bit NF4 with bfloat16 computation
LoRA rank (r)	64
LoRA alpha	16
LoRA dropout	— (not applied)
Target modules	q_proj, k_proj, v_proj
Max sequence length	192 tokens
Epochs	Up to 20 (early stopping, patience = 3)
Batch size	128
Learning rate	2 × 10⁻⁴
Loss function	Weighted cross-entropy (inverse class frequency)
Validation metric	Macro F1
Cross-validation	Stratified 5-fold
Precision	Mixed (FP16)
Hardware	1× NVIDIA A40 48GB · Intel Xeon Gold 6442Y 2.6GHz · 512GB RAM

---

Contact

Daniel Ângelo Rosa Morais
Universidade Federal de Ouro Preto (UFOP), Brazil
📧 daniel.morais@aluno.ufop.edu.br
📧 danielangelo1234@gmail.com

---