File size: 2,264 Bytes
1c938f3 53ae39b 1c938f3 53ae39b 1c938f3 53ae39b 1c938f3 53ae39b 1c938f3 53ae39b 1c938f3 53ae39b 1c938f3 53ae39b 1c938f3 53ae39b 1c938f3 53ae39b 02e1a2b 1c938f3 53ae39b 1c938f3 53ae39b 02e1a2b 964a778 5577927 97cac61 d5066d0 1a4fef2 fdab030 447d0cf 5b145d1 5b7c3e7 fb07381 bf6251c |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 |
---
language: en
tags:
- fill-mask
---
## Environmental Impact (CODE CARBON DEFAULT)
| Metric | Value |
|--------------------------|---------------------------------|
| Duration (in seconds) | [More Information Needed] |
| Emissions (Co2eq in kg) | [More Information Needed] |
| CPU power (W) | [NO CPU] |
| GPU power (W) | [No GPU] |
| RAM power (W) | [More Information Needed] |
| CPU energy (kWh) | [No CPU] |
| GPU energy (kWh) | [No GPU] |
| RAM energy (kWh) | [More Information Needed] |
| Consumed energy (kWh) | [More Information Needed] |
| Country name | [More Information Needed] |
| Cloud provider | [No Cloud] |
| Cloud region | [No Cloud] |
| CPU count | [No CPU] |
| CPU model | [No CPU] |
| GPU count | [No GPU] |
| GPU model | [No GPU] |
## Environmental Impact (for one core)
| Metric | Value |
|--------------------------|---------------------------------|
| CPU energy (kWh) | [No CPU] |
| Emissions (Co2eq in kg) | [More Information Needed] |
## Note
23 May 2024
## My Config
| Config | Value |
|--------------------------|-----------------|
| checkpoint | albert-base-v2 |
| model_name | fp_bs64_lr5_x2 |
| sequence_length | 400 |
| num_epoch | 6 |
| learning_rate | 5e-05 |
| batch_size | 64 |
| weight_decay | 0.0 |
| warm_up_prop | 0.0 |
| drop_out_prob | 0.1 |
| packing_length | 100 |
| train_test_split | 0.2 |
| num_steps | 10349 |
## Training and Testing steps
Epoch | Train Loss | Test Loss
---|---|---
| 0.0 | 15.192704 | 11.464602 |
| 0.5 | 3.473664 | 3.040166 |
| 1.0 | 2.933131 | 2.870918 |
| 1.5 | 2.785708 | 2.762562 |
| 2.0 | 2.695953 | 2.685881 |
| 2.5 | 2.624580 | 2.636662 |
| 3.0 | 2.587490 | 2.600750 |
| 3.5 | 2.531073 | 2.541152 |
| 4.0 | 2.497039 | 2.520043 |
| 4.5 | 2.460271 | 2.480024 |
| 5.0 | 2.427286 | 2.458232 |
| 5.5 | 2.396717 | 2.434840 |
|