Edit model card

Environmental Impact (CODE CARBON DEFAULT)

Metric Value
Duration (in seconds) 180540.8281199932
Emissions (Co2eq in kg) 0.1889692277270039
CPU power (W) 42.5
GPU power (W) [No GPU]
RAM power (W) 37.5
CPU energy (kWh) 2.1313811261523066
GPU energy (kWh) [No GPU]
RAM energy (kWh) 1.8806192077472823
Consumed energy (kWh) 4.012000333899577
Country name Switzerland
Cloud provider nan
Cloud region nan
CPU count 4
CPU model Intel(R) Xeon(R) Platinum 8360Y CPU @ 2.40GHz
GPU count nan
GPU model nan

Environmental Impact (for one core)

Metric Value
CPU energy (kWh) 0.3475410941309869
Emissions (Co2eq in kg) 0.07071182434699733

Note

15 May 2024

My Config

Config Value
checkpoint albert-base-v2
model_name ThunBERT_bs32_lr5
sequence_length 400
num_epoch 6
learning_rate 5e-05
batch_size 32
weight_decay 0.0
warm_up_prop 0.0
drop_out_prob 0.1
packing_length 100
train_test_split 0.2
num_steps 20393

Training and Testing steps

Epoch Train Loss Test Loss
0.0 4.767287 3.817442
0.5 3.344291 3.201500
1.0 3.131225 3.044940
1.5 2.992692 2.967517
2.0 2.911544 2.897029
2.5 2.861460 2.844424
3.0 2.797808 2.814696
3.5 2.738420 2.776461
4.0 2.698689 2.730229
4.5 2.671872 2.693672
5.0 2.628263 2.662948
5.5 2.587204 2.639905
6.0 2.565847 2.620656
Downloads last month
56
Safetensors
Model size
11.8M params
Tensor type
F32
·