Hugging Face's logo

lukasdrg
/
clinical_longformer_new_tokens

Fill-Mask
Transformers
PyTorch
TensorBoard
Safetensors
longformer
Generated from Trainer
Inference Endpoints
Model card Files Metrics Community
lukasdrg commited on
Commit
c461810
•
1 Parent(s): 6c30c12

End of training

Browse files
Files changed (9) hide show
  1. README.md +7 -17
  2. added_tokens.json +0 -0
  3. config.json +1 -1
  4. model.safetensors +3 -0
  5. runs/Nov03_17-27-05_ec4c1cbe8b31/events.out.tfevents.1699032431.ec4c1cbe8b31.220.0 +3 -0
  6. special_tokens_map.json +1 -1
  7. tokenizer.json +0 -0
  8. tokenizer_config.json +0 -0
  9. training_args.bin +2 -2
README.md CHANGED
@@ -15,7 +15,7 @@ should probably proofread and complete it, then remove this comment. -->
15
 
16
  This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the None dataset.
17
  It achieves the following results on the evaluation set:
18
- - Loss: 2.6416
19
 
20
  ## Model description
21
 
@@ -36,10 +36,10 @@ More information needed
36
  The following hyperparameters were used during training:
37
  - learning_rate: 2e-05
38
  - train_batch_size: 2
39
- - eval_batch_size: 4
40
  - seed: 42
41
- - gradient_accumulation_steps: 56
42
- - total_train_batch_size: 112
43
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
44
  - lr_scheduler_type: linear
45
  - lr_scheduler_warmup_steps: 1500
@@ -49,23 +49,13 @@ The following hyperparameters were used during training:
49
 
50
  | Training Loss | Epoch | Step | Validation Loss |
51
  |:-------------:|:-----:|:----:|:---------------:|
52
- | 4.3605 | 0.31 | 50 | 3.7983 |
53
- | 3.9502 | 0.62 | 100 | 3.4157 |
54
- | 3.4719 | 0.93 | 150 | 3.2378 |
55
- | 3.3937 | 1.24 | 200 | 3.0982 |
56
- | 3.3123 | 1.56 | 250 | 3.0227 |
57
- | 3.3164 | 1.87 | 300 | 2.9341 |
58
- | 2.9789 | 2.18 | 350 | 2.8408 |
59
- | 2.9593 | 2.49 | 400 | 2.8201 |
60
- | 2.8724 | 2.8 | 450 | 2.7561 |
61
- | 2.8753 | 3.11 | 500 | 2.7083 |
62
- | 2.8017 | 3.42 | 550 | 2.6593 |
63
- | 2.7496 | 3.73 | 600 | 2.6416 |
64
 
65
 
66
  ### Framework versions
67
 
68
- - Transformers 4.34.1
69
  - Pytorch 2.1.0+cu118
70
  - Datasets 2.14.6
71
  - Tokenizers 0.14.1
 
15
 
16
  This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the None dataset.
17
  It achieves the following results on the evaluation set:
18
+ - Loss: 4.5439
19
 
20
  ## Model description
21
 
 
36
  The following hyperparameters were used during training:
37
  - learning_rate: 2e-05
38
  - train_batch_size: 2
39
+ - eval_batch_size: 16
40
  - seed: 42
41
+ - gradient_accumulation_steps: 64
42
+ - total_train_batch_size: 128
43
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
44
  - lr_scheduler_type: linear
45
  - lr_scheduler_warmup_steps: 1500
 
49
 
50
  | Training Loss | Epoch | Step | Validation Loss |
51
  |:-------------:|:-----:|:----:|:---------------:|
52
+ | 2.5237 | 1.42 | 2 | 4.9660 |
53
+ | 2.6047 | 2.84 | 4 | 4.5439 |
 
 
 
 
 
 
 
 
 
 
54
 
55
 
56
  ### Framework versions
57
 
58
+ - Transformers 4.35.0
59
  - Pytorch 2.1.0+cu118
60
  - Datasets 2.14.6
61
  - Tokenizers 0.14.1
added_tokens.json CHANGED
The diff for this file is too large to render. See raw diff
 
config.json CHANGED
@@ -37,7 +37,7 @@
37
  "pad_token_id": 1,
38
  "sep_token_id": 2,
39
  "torch_dtype": "float32",
40
- "transformers_version": "4.34.1",
41
  "type_vocab_size": 1,
42
  "vocab_size": 60474
43
  }
 
37
  "pad_token_id": 1,
38
  "sep_token_id": 2,
39
  "torch_dtype": "float32",
40
+ "transformers_version": "4.35.0",
41
  "type_vocab_size": 1,
42
  "vocab_size": 60474
43
  }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2aa52e2f2845f15d778d84ff5f991f3752048bae656b8f9bff0f115157ed0975
3
+ size 626282192
runs/Nov03_17-27-05_ec4c1cbe8b31/events.out.tfevents.1699032431.ec4c1cbe8b31.220.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0718df9a92d32f3780076c44cc1510c0d9217cfedd4292288f0e6a5c1d50ee07
3
+ size 5600
special_tokens_map.json CHANGED
@@ -9,7 +9,7 @@
9
  "rstrip": false,
10
  "single_word": false
11
  },
12
- "pad_token": "</s>",
13
  "sep_token": "</s>",
14
  "unk_token": "<unk>"
15
  }
 
9
  "rstrip": false,
10
  "single_word": false
11
  },
12
+ "pad_token": "<pad>",
13
  "sep_token": "</s>",
14
  "unk_token": "<unk>"
15
  }
tokenizer.json CHANGED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json CHANGED
The diff for this file is too large to render. See raw diff
 
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b0f86460df581f0cf4d866cd5242d81490b367c4cdb1e6f5e92bea790fa94125
3
- size 4536
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:063e274ba203b8e66b8dcadbf0e2b51a6300f8a5ae9a4f3c887b57632113f4e3
3
+ size 4600