GIST-small-weborganizer-topic

A fine-tuned version of the bert architecture (BertForSequenceClassification) optimized for the text-classification task.

• Model type: bert
• Problem Type: single_label_classification
• Number of Labels: 24
• Vocabulary Size: 30522
• License: MIT

Use

To get started with this model in Python using the Hugging Face Transformers library, run the following code:

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "agentlans/GIST-small-weborganizer-topic"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

text = "Replace this with your input text."
inputs = tokenizer(text, return_tensors="pt")

with torch.no_grad():
    logits = model(**inputs).logits

predicted_class_id = logits.argmax().item()
predicted_class_name = model.config.id2label[predicted_class_id]

print(f"Predicted Class ID: {predicted_class_id}")
print(f"Predicted Class Name: {predicted_class_name}")

Intended Uses & Limitations

Intended Use

This model is designed for sequence classification tasks. Below are the specific class labels mapped to their corresponding IDs:

Label ID	Label Name
0	Adult Content
1	Art & Design
2	Crime & Law
3	Education & Jobs
4	Electronics & Hardare
5	Entertainment
6	Fashion & Beauty
7	Finance & Business
8	Food & Dining
9	Games
10	Health
11	History & Geography
12	Home & Hobbies
13	Industrial
14	Literature
15	Politics
16	Religion
17	Science, Math & Technology
18	Social Life
19	Software
20	Software Development
21	Sports & Fitness
22	Transportation
23	Travel & Tourism

Training Details

Hyperparameters

The following hyperparameters were used during fine-tuning:

• Learning Rate: 5e-05
• Train Batch Size: 8
• Eval Batch Size: 8
• Optimizer: OptimizerNames.ADAMW_TORCH_FUSED
• Number of Epochs: 3.0
• Mixed Precision: BF16

Show Advanced Training Configuration

Optimization & Regularization

• Gradient Accumulation Steps: 1
• Learning Rate Scheduler: SchedulerType.LINEAR
• Warmup Steps: 0
• Warmup Ratio: None
• Weight Decay: 0.0
• Max Gradient Norm: 1.0

Hardware & Reproducibility

• Number of GPUs: 1
• Seed: 42

Training Results & Evaluation

During fine-tuning, the model achieved the following results on the evaluation set:

Metric	Value
Train Loss	0.6471
Validation Loss	0.7848
Validation F1 Score	0.7646
Total FLOPs	3.9539e+15

For performance on the test set, click here.

Speed Performance

• Training Runtime: 513.1373 seconds
• Train Samples per Second: 467.711
• Evaluation Runtime: 5.5697 seconds
• Eval Samples per Second: 1795.418

Show Detailed Training Logs

Training Logs History

Step	Epoch	Learning Rate	Training Loss	Validation Loss	Validation F1
500	0.05	4.9168e-05	2.377	N/A	N/A
1000	0.1	4.8335e-05	1.5234	N/A	N/A
1500	0.15	4.7502e-05	1.2233	N/A	N/A
2000	0.2	4.6668e-05	1.0762	N/A	N/A
2500	0.25	4.5835e-05	1.0471	N/A	N/A
3000	0.3	4.5002e-05	0.9702	N/A	N/A
3500	0.35	4.4168e-05	0.9347	N/A	N/A
4000	0.4	4.3335e-05	0.935	N/A	N/A
4500	0.45	4.2502e-05	0.8752	N/A	N/A
5000	0.5	4.1668e-05	0.9333	N/A	N/A
5500	0.55	4.0835e-05	0.8817	N/A	N/A
6000	0.6	4.0002e-05	0.8634	N/A	N/A
6500	0.65	3.9168e-05	0.8654	N/A	N/A
7000	0.7	3.8335e-05	0.8731	N/A	N/A
7500	0.75	3.7502e-05	0.8417	N/A	N/A
8000	0.8	3.6668e-05	0.8155	N/A	N/A
8500	0.85	3.5835e-05	0.8126	N/A	N/A
9000	0.9	3.5002e-05	0.8291	N/A	N/A
9500	0.95	3.4168e-05	0.8215	N/A	N/A
10000	1.0	3.3335e-05	0.803	0.7996	0.7456
10500	1.05	3.2502e-05	0.5678	N/A	N/A
11000	1.1	3.1668e-05	0.587	N/A	N/A
11500	1.15	3.0835e-05	0.5868	N/A	N/A
12000	1.2	3.0002e-05	0.5535	N/A	N/A
12500	1.25	2.9168e-05	0.5698	N/A	N/A
13000	1.3	2.8335e-05	0.6105	N/A	N/A
13500	1.35	2.7502e-05	0.5476	N/A	N/A
14000	1.4	2.6668e-05	0.5714	N/A	N/A
14500	1.45	2.5835e-05	0.581	N/A	N/A
15000	1.5	2.5002e-05	0.5743	N/A	N/A
15500	1.55	2.4168e-05	0.572	N/A	N/A
16000	1.6	2.3335e-05	0.553	N/A	N/A
16500	1.65	2.2502e-05	0.5777	N/A	N/A
17000	1.7	2.1668e-05	0.5599	N/A	N/A
17500	1.75	2.0835e-05	0.5823	N/A	N/A
18000	1.8	2.0002e-05	0.5614	N/A	N/A
18500	1.85	1.9168e-05	0.5345	N/A	N/A
19000	1.9	1.8335e-05	0.5846	N/A	N/A
19500	1.95	1.7502e-05	0.5599	N/A	N/A
20000	2.0	1.6668e-05	0.5475	0.7848	0.7646
20500	2.05	1.5835e-05	0.3585	N/A	N/A
21000	2.1	1.5002e-05	0.372	N/A	N/A
21500	2.15	1.4168e-05	0.3328	N/A	N/A
22000	2.2	1.3335e-05	0.3766	N/A	N/A
22500	2.25	1.2502e-05	0.3755	N/A	N/A
23000	2.3	1.1668e-05	0.3482	N/A	N/A
23500	2.35	1.0835e-05	0.3897	N/A	N/A
24000	2.4	1.0002e-05	0.3567	N/A	N/A
24500	2.45	9.1683e-06	0.353	N/A	N/A
25000	2.5	8.3350e-06	0.3734	N/A	N/A
25500	2.55	7.5017e-06	0.3444	N/A	N/A
26000	2.6	6.6683e-06	0.3438	N/A	N/A
26500	2.65	5.8350e-06	0.3411	N/A	N/A
27000	2.7	5.0017e-06	0.371	N/A	N/A
27500	2.75	4.1683e-06	0.341	N/A	N/A
28000	2.8	3.3350e-06	0.3606	N/A	N/A
28500	2.85	2.5017e-06	0.3429	N/A	N/A
29000	2.9	1.6683e-06	0.3581	N/A	N/A
29500	2.95	8.3500e-07	0.3653	N/A	N/A
30000	3.0	1.6667e-09	0.3367	0.9586	0.7633

Framework Versions

• Transformers: 5.14.0.dev0
• PyTorch: 2.13.0+cu130