Hugging Face's logo

KatoHF
/
deberta-v3-large-classifier

Text Classification
Transformers
TensorBoard
Safetensors
deberta-v2
Generated from Trainer
Inference Endpoints
Model card Files Metrics Community
lewtun HF staff commited on
Commit
17cbd13
•
1 Parent(s): 7b64a2b

Training in progress, step 500

Browse files
Files changed (23) hide show
  1. README.md +68 -0
  2. added_tokens.json +3 -0
  3. config.json +35 -0
  4. model.safetensors +3 -0
  5. runs/Dec22_09-11-09_ip-26-0-161-78/events.out.tfevents.1703236275.ip-26-0-161-78.241801.0 +3 -0
  6. runs/Dec22_09-15-16_ip-26-0-161-78/events.out.tfevents.1703236539.ip-26-0-161-78.243407.0 +3 -0
  7. runs/Dec22_09-17-07_ip-26-0-161-78/events.out.tfevents.1703236650.ip-26-0-161-78.243947.0 +3 -0
  8. runs/Dec22_09-18-39_ip-26-0-161-78/events.out.tfevents.1703236742.ip-26-0-161-78.245284.0 +3 -0
  9. runs/Dec22_09-26-34_ip-26-0-161-78/events.out.tfevents.1703237221.ip-26-0-161-78.246666.0 +3 -0
  10. runs/Dec22_09-30-05_ip-26-0-161-78/events.out.tfevents.1703237429.ip-26-0-161-78.247437.0 +3 -0
  11. runs/Dec22_09-30-05_ip-26-0-161-78/events.out.tfevents.1703237886.ip-26-0-161-78.247437.1 +3 -0
  12. runs/Dec22_09-42-28_ip-26-0-161-78/events.out.tfevents.1703238176.ip-26-0-161-78.250085.0 +3 -0
  13. runs/Dec22_09-42-28_ip-26-0-161-78/events.out.tfevents.1703238620.ip-26-0-161-78.250085.1 +3 -0
  14. runs/Dec22_10-13-09_ip-26-0-173-202/events.out.tfevents.1703240016.ip-26-0-173-202.3051674.0 +3 -0
  15. runs/Dec22_10-15-47_ip-26-0-173-202/events.out.tfevents.1703240170.ip-26-0-173-202.3052975.0 +3 -0
  16. runs/Dec22_10-20-57_ip-26-0-173-202/events.out.tfevents.1703240484.ip-26-0-173-202.3053888.0 +3 -0
  17. runs/Dec22_10-26-41_ip-26-0-173-202/events.out.tfevents.1703240830.ip-26-0-173-202.3055231.0 +3 -0
  18. runs/Dec22_10-29-52_ip-26-0-173-202/events.out.tfevents.1703241020.ip-26-0-173-202.3055916.0 +3 -0
  19. special_tokens_map.json +15 -0
  20. spm.model +3 -0
  21. tokenizer.json +0 -0
  22. tokenizer_config.json +58 -0
  23. training_args.bin +3 -0
README.md ADDED
@@ -0,0 +1,68 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ base_model: microsoft/deberta-v3-large
4
+ tags:
5
+ - generated_from_trainer
6
+ metrics:
7
+ - accuracy
8
+ - f1
9
+ model-index:
10
+ - name: kato-dummy
11
+ results: []
12
+ ---
13
+
14
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
15
+ should probably proofread and complete it, then remove this comment. -->
16
+
17
+ # kato-dummy
18
+
19
+ This model is a fine-tuned version of [microsoft/deberta-v3-large](https://huggingface.co/microsoft/deberta-v3-large) on the None dataset.
20
+ It achieves the following results on the evaluation set:
21
+ - Loss: 0.5742
22
+ - Accuracy: 0.647
23
+ - F1: 0.6466
24
+
25
+ ## Model description
26
+
27
+ More information needed
28
+
29
+ ## Intended uses & limitations
30
+
31
+ More information needed
32
+
33
+ ## Training and evaluation data
34
+
35
+ More information needed
36
+
37
+ ## Training procedure
38
+
39
+ ### Training hyperparameters
40
+
41
+ The following hyperparameters were used during training:
42
+ - learning_rate: 2e-05
43
+ - train_batch_size: 4
44
+ - eval_batch_size: 4
45
+ - seed: 42
46
+ - distributed_type: multi-GPU
47
+ - num_devices: 8
48
+ - total_train_batch_size: 32
49
+ - total_eval_batch_size: 32
50
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
51
+ - lr_scheduler_type: cosine
52
+ - lr_scheduler_warmup_ratio: 0.1
53
+ - num_epochs: 2
54
+
55
+ ### Training results
56
+
57
+ | Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 |
58
+ |:-------------:|:-----:|:----:|:---------------:|:--------:|:------:|
59
+ | 0.6472 | 1.0 | 250 | 0.6111 | 0.5915 | 0.4867 |
60
+ | 0.5499 | 2.0 | 500 | 0.5742 | 0.647 | 0.6466 |
61
+
62
+
63
+ ### Framework versions
64
+
65
+ - Transformers 4.36.2
66
+ - Pytorch 2.1.2+cu121
67
+ - Datasets 2.14.6
68
+ - Tokenizers 0.15.0
added_tokens.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {
2
+ "[MASK]": 128000
3
+ }
config.json ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "microsoft/deberta-v3-large",
3
+ "architectures": [
4
+ "DebertaV2ForSequenceClassification"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "hidden_act": "gelu",
8
+ "hidden_dropout_prob": 0.1,
9
+ "hidden_size": 1024,
10
+ "initializer_range": 0.02,
11
+ "intermediate_size": 4096,
12
+ "layer_norm_eps": 1e-07,
13
+ "max_position_embeddings": 512,
14
+ "max_relative_positions": -1,
15
+ "model_type": "deberta-v2",
16
+ "norm_rel_ebd": "layer_norm",
17
+ "num_attention_heads": 16,
18
+ "num_hidden_layers": 24,
19
+ "pad_token_id": 0,
20
+ "pooler_dropout": 0,
21
+ "pooler_hidden_act": "gelu",
22
+ "pooler_hidden_size": 1024,
23
+ "pos_att_type": [
24
+ "p2c",
25
+ "c2p"
26
+ ],
27
+ "position_biased_input": false,
28
+ "position_buckets": 256,
29
+ "relative_attention": true,
30
+ "share_att_key": true,
31
+ "torch_dtype": "bfloat16",
32
+ "transformers_version": "4.36.2",
33
+ "type_vocab_size": 0,
34
+ "vocab_size": 128100
35
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:de0c7615fc37a24282042af711ebc27a1948d0de910ba7498f34122308c9ee04
3
+ size 870176748
runs/Dec22_09-11-09_ip-26-0-161-78/events.out.tfevents.1703236275.ip-26-0-161-78.241801.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c07d3bd6a58e67ef0e4f32861fa0e3d7e9ab2ebf367c2c561efa4460776aa7af
3
+ size 4490
runs/Dec22_09-15-16_ip-26-0-161-78/events.out.tfevents.1703236539.ip-26-0-161-78.243407.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d0a0c13fcc5af5ce4fca1f3db143b14a84ff4b9645ff719405c4efd4e235470d
3
+ size 4489
runs/Dec22_09-17-07_ip-26-0-161-78/events.out.tfevents.1703236650.ip-26-0-161-78.243947.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3962fc550f9a6b2c09e637f7943161bf8dddbdddc81a1cd3b0747566ec0069fa
3
+ size 4489
runs/Dec22_09-18-39_ip-26-0-161-78/events.out.tfevents.1703236742.ip-26-0-161-78.245284.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d2b5ed6f1410a2ea9ecc44dfecc213ea4f055dadd37a1f39708f57404550a9b2
3
+ size 4489
runs/Dec22_09-26-34_ip-26-0-161-78/events.out.tfevents.1703237221.ip-26-0-161-78.246666.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:406d6c736b62009b74ae143e4783dba327ac955fb59dc3f6d1057bb032b6cd99
3
+ size 6807
runs/Dec22_09-30-05_ip-26-0-161-78/events.out.tfevents.1703237429.ip-26-0-161-78.247437.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a9cfefc7cd0e87dfcdf05e13c623497a832672434fca932c4d3c33bdb70701b4
3
+ size 13394
runs/Dec22_09-30-05_ip-26-0-161-78/events.out.tfevents.1703237886.ip-26-0-161-78.247437.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2e80a995c382e0254c3f8c589db7db5a5f97e70f1a2b34593a4c0ba27f10739f
3
+ size 457
runs/Dec22_09-42-28_ip-26-0-161-78/events.out.tfevents.1703238176.ip-26-0-161-78.250085.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:771dd79c1b81cf49d08ecffc54880f407389ae2aec8d0dceab53f0b94cb9d1f0
3
+ size 13394
runs/Dec22_09-42-28_ip-26-0-161-78/events.out.tfevents.1703238620.ip-26-0-161-78.250085.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:751526ecf44b43518368b1cb8df056c7615e82c7f5bba7e07f5867dc21bcea2f
3
+ size 457
runs/Dec22_10-13-09_ip-26-0-173-202/events.out.tfevents.1703240016.ip-26-0-173-202.3051674.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:88e3c358e4b413afcfdb073928f45cb8fbf437588b6c29b255e1022c1a46fb4f
3
+ size 4522
runs/Dec22_10-15-47_ip-26-0-173-202/events.out.tfevents.1703240170.ip-26-0-173-202.3052975.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:39cf3b7d20caec4265e073b6339d04b5cca346b0bf62d76cc94613b91f387362
3
+ size 4675
runs/Dec22_10-20-57_ip-26-0-173-202/events.out.tfevents.1703240484.ip-26-0-173-202.3053888.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4a342ca7960570f399b7d771728c283344559bc49db13e37f65e6e2b1b84e9a0
3
+ size 4523
runs/Dec22_10-26-41_ip-26-0-173-202/events.out.tfevents.1703240830.ip-26-0-173-202.3055231.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:60130d69071b2e6dcda3b5d771b6608c4a53726e3788b417f01e40907b2e6e8b
3
+ size 4522
runs/Dec22_10-29-52_ip-26-0-173-202/events.out.tfevents.1703241020.ip-26-0-173-202.3055916.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:32f3018b78c2d98475813e052c38a5b55306eb69a9c1ebfe7278da82860c90e2
3
+ size 5304
special_tokens_map.json ADDED
@@ -0,0 +1,15 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": "[CLS]",
3
+ "cls_token": "[CLS]",
4
+ "eos_token": "[SEP]",
5
+ "mask_token": "[MASK]",
6
+ "pad_token": "[PAD]",
7
+ "sep_token": "[SEP]",
8
+ "unk_token": {
9
+ "content": "[UNK]",
10
+ "lstrip": false,
11
+ "normalized": true,
12
+ "rstrip": false,
13
+ "single_word": false
14
+ }
15
+ }
spm.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c679fbf93643d19aab7ee10c0b99e460bdbc02fedf34b92b05af343b4af586fd
3
+ size 2464616
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "[PAD]",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "1": {
12
+ "content": "[CLS]",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "2": {
20
+ "content": "[SEP]",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "3": {
28
+ "content": "[UNK]",
29
+ "lstrip": false,
30
+ "normalized": true,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "128000": {
36
+ "content": "[MASK]",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ }
43
+ },
44
+ "bos_token": "[CLS]",
45
+ "clean_up_tokenization_spaces": true,
46
+ "cls_token": "[CLS]",
47
+ "do_lower_case": false,
48
+ "eos_token": "[SEP]",
49
+ "mask_token": "[MASK]",
50
+ "model_max_length": 1000000000000000019884624838656,
51
+ "pad_token": "[PAD]",
52
+ "sep_token": "[SEP]",
53
+ "sp_model_kwargs": {},
54
+ "split_by_punct": false,
55
+ "tokenizer_class": "DebertaV2Tokenizer",
56
+ "unk_token": "[UNK]",
57
+ "vocab_type": "spm"
58
+ }
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5010f484fb35ef6cfb2e8d34a5b4891a7cca06be86bc07ef7342b064ca53c57b
3
+ size 5688