onybrunella commited on
Commit
b87dfd2
1 Parent(s): facd965

Training in progress epoch 0

Browse files
README.md CHANGED
@@ -2,18 +2,26 @@
2
  license: apache-2.0
3
  base_model: google/mobilebert-uncased
4
  tags:
5
- - generated_from_trainer
6
  model-index:
7
- - name: mobilebert-uncased-finetuned-squad-v2
8
  results: []
9
  ---
10
 
11
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
12
- should probably proofread and complete it, then remove this comment. -->
13
 
14
- # mobilebert-uncased-finetuned-squad-v2
15
 
16
  This model is a fine-tuned version of [google/mobilebert-uncased](https://huggingface.co/google/mobilebert-uncased) on an unknown dataset.
 
 
 
 
 
 
 
 
17
 
18
  ## Model description
19
 
@@ -32,25 +40,19 @@ More information needed
32
  ### Training hyperparameters
33
 
34
  The following hyperparameters were used during training:
35
- - learning_rate: 5e-05
36
- - train_batch_size: 64
37
- - eval_batch_size: 64
38
- - seed: 42
39
- - gradient_accumulation_steps: 4
40
- - total_train_batch_size: 256
41
- - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
42
- - lr_scheduler_type: linear
43
- - lr_scheduler_warmup_steps: 164.6925
44
- - num_epochs: 1
45
- - mixed_precision_training: Native AMP
46
 
47
  ### Training results
48
 
 
 
 
49
 
50
 
51
  ### Framework versions
52
 
53
  - Transformers 4.38.2
54
- - Pytorch 2.2.1+cu121
55
  - Datasets 2.18.0
56
  - Tokenizers 0.15.2
 
2
  license: apache-2.0
3
  base_model: google/mobilebert-uncased
4
  tags:
5
+ - generated_from_keras_callback
6
  model-index:
7
+ - name: onybrunella/mobilebert-uncased-finetuned-squad-v2
8
  results: []
9
  ---
10
 
11
+ <!-- This model card has been generated automatically according to the information Keras had access to. You should
12
+ probably proofread and complete it, then remove this comment. -->
13
 
14
+ # onybrunella/mobilebert-uncased-finetuned-squad-v2
15
 
16
  This model is a fine-tuned version of [google/mobilebert-uncased](https://huggingface.co/google/mobilebert-uncased) on an unknown dataset.
17
+ It achieves the following results on the evaluation set:
18
+ - Train Loss: 4650.7085
19
+ - Train End Logits Accuracy: 0.2902
20
+ - Train Start Logits Accuracy: 0.2428
21
+ - Validation Loss: 3.1353
22
+ - Validation End Logits Accuracy: 0.5043
23
+ - Validation Start Logits Accuracy: 0.4971
24
+ - Epoch: 0
25
 
26
  ## Model description
27
 
 
40
  ### Training hyperparameters
41
 
42
  The following hyperparameters were used during training:
43
+ - optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 8234, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
44
+ - training_precision: float32
 
 
 
 
 
 
 
 
 
45
 
46
  ### Training results
47
 
48
+ | Train Loss | Train End Logits Accuracy | Train Start Logits Accuracy | Validation Loss | Validation End Logits Accuracy | Validation Start Logits Accuracy | Epoch |
49
+ |:----------:|:-------------------------:|:---------------------------:|:---------------:|:------------------------------:|:--------------------------------:|:-----:|
50
+ | 4650.7085 | 0.2902 | 0.2428 | 3.1353 | 0.5043 | 0.4971 | 0 |
51
 
52
 
53
  ### Framework versions
54
 
55
  - Transformers 4.38.2
56
+ - TensorFlow 2.15.0
57
  - Datasets 2.18.0
58
  - Tokenizers 0.15.2
config.json CHANGED
@@ -22,7 +22,6 @@
22
  "num_feedforward_networks": 4,
23
  "num_hidden_layers": 24,
24
  "pad_token_id": 0,
25
- "torch_dtype": "float32",
26
  "transformers_version": "4.38.2",
27
  "trigram_input": true,
28
  "true_hidden_size": 128,
 
22
  "num_feedforward_networks": 4,
23
  "num_hidden_layers": 24,
24
  "pad_token_id": 0,
 
25
  "transformers_version": "4.38.2",
26
  "trigram_input": true,
27
  "true_hidden_size": 128,
logs/train/events.out.tfevents.1711749027.2d0c17002ee8.11121.0.v2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ed740ff2f9732c195e37e488992f80f813f7569c1d51ee462242e1a7747f9db9
3
+ size 11012094
logs/train/events.out.tfevents.1711750084.2d0c17002ee8.16204.0.v2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:22f7ece87513164f079a7f17a25b2b9a4d617448de9265cb9d7277efac3233ed
3
+ size 11012323
logs/validation/events.out.tfevents.1711753540.2d0c17002ee8.16204.1.v2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:20e8abdd808f0d2f98ef57eb2a3fc872e784413126da9b72233142fb30acf274
3
+ size 604
tf_model.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9005834110d9cd0da6845c73c20b08b300961c45bb5780809e821b98b24c0d34
3
+ size 99819976