File size: 3,109 Bytes
174817e
f0ccef6
 
 
d1c684f
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
174817e
 
f0ccef6
174817e
f0ccef6
 
d1c684f
 
 
f0ccef6
d1c684f
 
f0ccef6
d1c684f
 
 
 
 
 
 
 
 
174817e
f0ccef6
174817e
f0ccef6
 
d1c684f
 
174817e
f0ccef6
174817e
f0ccef6
174817e
f0ccef6
174817e
f0ccef6
 
 
 
 
 
 
 
 
 
 
 
 
 
174817e
f0ccef6
174817e
 
 
 
 
 
f0ccef6
 
 
 
9f8a402
b759285
c038c22
dff30f7
1a9de1a
0341520
ab2cb1a
49dae73
1d32b94
8645812
35faa17
88229f1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
---
language: en
tags:
- fill-mask
kwargs:
  timestamp: '2024-07-12T16:45:44'
  project_name: fp_bs4_lr2e4_x2_emissions_tracker
  run_id: bd0c4a59-5b20-4dd0-86f9-f8b361cfa4e7
  duration: 168668.67397975922
  emissions: 0.1268902545670636
  emissions_rate: 7.523048090263094e-07
  cpu_power: 42.5
  gpu_power: 0.0
  ram_power: 15.0
  cpu_energy: 1.9912231882090328
  gpu_energy: 0
  ram_energy: 0.7027802207857362
  energy_consumed: 2.69400340899479
  country_name: Switzerland
  country_iso_code: CHE
  region: .nan
  cloud_provider: .nan
  cloud_region: .nan
  os: Linux-5.14.0-70.30.1.el9_0.x86_64-x86_64-with-glibc2.34
  python_version: 3.10.4
  codecarbon_version: 2.3.4
  cpu_count: 6
  cpu_model: Intel(R) Xeon(R) Platinum 8360Y CPU @ 2.40GHz
  gpu_count: .nan
  gpu_model: .nan
  longitude: .nan
  latitude: .nan
  ram_total_size: 40
  tracking_mode: machine
  on_cloud: N
  pue: 1.0
---

## Environmental Impact (CODE CARBON DEFAULT)

| Metric                   | Value                           |
|--------------------------|---------------------------------|
| Duration (in seconds)    | 168668.67397975922  |
| Emissions (Co2eq in kg)  | 0.1268902545670636 |
| CPU power (W)            | 42.5  |
| GPU power (W)            | [No GPU]  |
| RAM power (W)            | 15.0  |
| CPU energy (kWh)         | 1.9912231882090328  |
| GPU energy (kWh)         | [No GPU]  |
| RAM energy (kWh)         | 0.7027802207857362  |
| Consumed energy (kWh)    | 2.69400340899479  |
| Country name             | Switzerland  |
| Cloud provider           | nan  |
| Cloud region             | nan  |
| CPU count                | 6  |
| CPU model                | Intel(R) Xeon(R) Platinum 8360Y CPU @ 2.40GHz  |
| GPU count                | nan  |
| GPU model                | nan  |

## Environmental Impact (for one core)

| Metric                   | Value                           |
|--------------------------|---------------------------------|
| CPU energy (kWh)         | 0.32468719741103647  |
| Emissions (Co2eq in kg)  | 0.06606189730873901 |

## Note

5 juillet 2024 !

## My Config

| Config                   | Value           |
|--------------------------|-----------------|
| checkpoint               | albert-base-v2  |
| model_name               | fp_bs4_lr2e4_x2 |
| sequence_length          | 400  |
| num_epoch                | 6  |
| learning_rate            | 0.0002  |
| batch_size               | 4  |
| weight_decay             | 0.0  |
| warm_up_prop             | 0.0  |
| drop_out_prob            | 0.1 |
| packing_length           | 100 |
| train_test_split         | 0.2 |
| num_steps                | 165709 |

## Training and Testing steps






 
Epoch | Train Loss | Test Loss
---|---|---
| 0.0 | 13.298823 | 11.917336 |
| 0.5 | 7.196151 | 7.103504 |
| 1.0 | 7.056031 | 7.041916 |
| 1.5 | 7.253135 | 7.168228 |
| 2.0 | 7.131400 | 7.110774 |
| 2.5 | 7.090259 | 7.085124 |
| 3.0 | 7.093696 | 7.091162 |
| 3.5 | 7.076734 | 7.078840 |
| 4.0 | 7.070370 | 7.058439 |
| 4.5 | 7.049053 | 7.041694 |
| 5.0 | 7.040570 | 7.039958 |
| 5.5 | 7.039051 | 7.042496 |
| 6.0 | 7.030734 | 7.029607 |