You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

MiniClimate

A lightweight climate-domain-adapted model based on MiniLM-L12-H384-uncased (~33M params), pretrained using DAPT + TAPT on climate-related YouTube comments.

What it does

Adapts MiniLM to climate-related text using Masked Language Modelling (MLM). Can be fine-tuned for downstream tasks like:

Stance detection
Environmental claim detection
Sentiment classification

Usage

from transformers import AutoModelForMaskedLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("Raj7722/MiniClimate")
model = AutoModelForMaskedLM.from_pretrained("Raj7722/MiniClimate")

This is a pretrained checkpoint. Fine-tune it with a classification head for your downstream task.

Results

Avg F1 across 4 downstream tasks (after task-specific fine-tuning): 75.22% (best F1-per-parameter efficiency vs. RoBERTa, ClimateBERT, SciBERT, DistilBERT).

Author

Rajdeep Bose — NSHM Knowledge Campus, Kolkata (M.Sc. Data Science & Analytics)

License

MIT

Downloads last month: -

Safetensors

Model size

33.4M params

Tensor type

F32

Model tree for Raj7722/MiniClimate

Base model

microsoft/MiniLM-L12-H384-uncased

Finetuned

(134)

this model