Hugging Face's logo

Mozart-coder
/
dna_bert_3_1000seq-finetuned

Fill-Mask
Transformers
PyTorch
TensorBoard
bert
generated_from_trainer
Inference Endpoints
Model card Files Metrics Community
1
Mozart-coder commited on
Commit
dd76509
•
1 Parent(s): 3180f4d

update model card README.md

Browse files
Files changed (1) hide show
  1. README.md +155 -0
README.md ADDED
@@ -0,0 +1,155 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - generated_from_trainer
4
+ model-index:
5
+ - name: dna_bert_3_1000seq-finetuned
6
+ results: []
7
+ ---
8
+
9
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
10
+ should probably proofread and complete it, then remove this comment. -->
11
+
12
+ # dna_bert_3_1000seq-finetuned
13
+
14
+ This model is a fine-tuned version of [armheb/DNA_bert_3](https://huggingface.co/armheb/DNA_bert_3) on an unknown dataset.
15
+ It achieves the following results on the evaluation set:
16
+ - Loss: 0.4684
17
+
18
+ ## Model description
19
+
20
+ More information needed
21
+
22
+ ## Intended uses & limitations
23
+
24
+ More information needed
25
+
26
+ ## Training and evaluation data
27
+
28
+ More information needed
29
+
30
+ ## Training procedure
31
+
32
+ ### Training hyperparameters
33
+
34
+ The following hyperparameters were used during training:
35
+ - learning_rate: 2e-05
36
+ - train_batch_size: 16
37
+ - eval_batch_size: 16
38
+ - seed: 42
39
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
40
+ - lr_scheduler_type: linear
41
+ - num_epochs: 100
42
+ - mixed_precision_training: Native AMP
43
+
44
+ ### Training results
45
+
46
+ | Training Loss | Epoch | Step | Validation Loss |
47
+ |:-------------:|:-----:|:----:|:---------------:|
48
+ | 0.8607 | 1.0 | 62 | 0.6257 |
49
+ | 0.6177 | 2.0 | 124 | 0.6120 |
50
+ | 0.6098 | 3.0 | 186 | 0.6062 |
51
+ | 0.604 | 4.0 | 248 | 0.6052 |
52
+ | 0.5999 | 5.0 | 310 | 0.6040 |
53
+ | 0.5982 | 6.0 | 372 | 0.5996 |
54
+ | 0.5985 | 7.0 | 434 | 0.5985 |
55
+ | 0.5956 | 8.0 | 496 | 0.5968 |
56
+ | 0.5936 | 9.0 | 558 | 0.5950 |
57
+ | 0.5908 | 10.0 | 620 | 0.5941 |
58
+ | 0.5904 | 11.0 | 682 | 0.5932 |
59
+ | 0.59 | 12.0 | 744 | 0.5917 |
60
+ | 0.5877 | 13.0 | 806 | 0.5921 |
61
+ | 0.5847 | 14.0 | 868 | 0.5903 |
62
+ | 0.5831 | 15.0 | 930 | 0.5887 |
63
+ | 0.5852 | 16.0 | 992 | 0.5878 |
64
+ | 0.5805 | 17.0 | 1054 | 0.5872 |
65
+ | 0.5795 | 18.0 | 1116 | 0.5853 |
66
+ | 0.5754 | 19.0 | 1178 | 0.5869 |
67
+ | 0.5757 | 20.0 | 1240 | 0.5839 |
68
+ | 0.5722 | 21.0 | 1302 | 0.5831 |
69
+ | 0.5693 | 22.0 | 1364 | 0.5811 |
70
+ | 0.5667 | 23.0 | 1426 | 0.5802 |
71
+ | 0.5652 | 24.0 | 1488 | 0.5775 |
72
+ | 0.5608 | 25.0 | 1550 | 0.5788 |
73
+ | 0.5591 | 26.0 | 1612 | 0.5724 |
74
+ | 0.5538 | 27.0 | 1674 | 0.5736 |
75
+ | 0.552 | 28.0 | 1736 | 0.5689 |
76
+ | 0.5483 | 29.0 | 1798 | 0.5689 |
77
+ | 0.5442 | 30.0 | 1860 | 0.5671 |
78
+ | 0.5405 | 31.0 | 1922 | 0.5658 |
79
+ | 0.537 | 32.0 | 1984 | 0.5605 |
80
+ | 0.5349 | 33.0 | 2046 | 0.5575 |
81
+ | 0.5275 | 34.0 | 2108 | 0.5569 |
82
+ | 0.5227 | 35.0 | 2170 | 0.5537 |
83
+ | 0.52 | 36.0 | 2232 | 0.5509 |
84
+ | 0.5173 | 37.0 | 2294 | 0.5504 |
85
+ | 0.5123 | 38.0 | 2356 | 0.5435 |
86
+ | 0.5088 | 39.0 | 2418 | 0.5472 |
87
+ | 0.5037 | 40.0 | 2480 | 0.5383 |
88
+ | 0.501 | 41.0 | 2542 | 0.5379 |
89
+ | 0.4931 | 42.0 | 2604 | 0.5365 |
90
+ | 0.4923 | 43.0 | 2666 | 0.5328 |
91
+ | 0.4879 | 44.0 | 2728 | 0.5301 |
92
+ | 0.482 | 45.0 | 2790 | 0.5295 |
93
+ | 0.4805 | 46.0 | 2852 | 0.5261 |
94
+ | 0.4772 | 47.0 | 2914 | 0.5221 |
95
+ | 0.4738 | 48.0 | 2976 | 0.5234 |
96
+ | 0.4674 | 49.0 | 3038 | 0.5210 |
97
+ | 0.4646 | 50.0 | 3100 | 0.5169 |
98
+ | 0.4621 | 51.0 | 3162 | 0.5142 |
99
+ | 0.4574 | 52.0 | 3224 | 0.5129 |
100
+ | 0.4552 | 53.0 | 3286 | 0.5127 |
101
+ | 0.4539 | 54.0 | 3348 | 0.5124 |
102
+ | 0.4506 | 55.0 | 3410 | 0.5076 |
103
+ | 0.4457 | 56.0 | 3472 | 0.5082 |
104
+ | 0.4454 | 57.0 | 3534 | 0.5027 |
105
+ | 0.4398 | 58.0 | 3596 | 0.5019 |
106
+ | 0.4386 | 59.0 | 3658 | 0.4998 |
107
+ | 0.4332 | 60.0 | 3720 | 0.4970 |
108
+ | 0.4277 | 61.0 | 3782 | 0.4995 |
109
+ | 0.4273 | 62.0 | 3844 | 0.4962 |
110
+ | 0.4235 | 63.0 | 3906 | 0.4909 |
111
+ | 0.4201 | 64.0 | 3968 | 0.4913 |
112
+ | 0.4198 | 65.0 | 4030 | 0.4899 |
113
+ | 0.4182 | 66.0 | 4092 | 0.4919 |
114
+ | 0.4157 | 67.0 | 4154 | 0.4902 |
115
+ | 0.4104 | 68.0 | 4216 | 0.4881 |
116
+ | 0.4095 | 69.0 | 4278 | 0.4881 |
117
+ | 0.4077 | 70.0 | 4340 | 0.4861 |
118
+ | 0.4064 | 71.0 | 4402 | 0.4868 |
119
+ | 0.4041 | 72.0 | 4464 | 0.4826 |
120
+ | 0.4029 | 73.0 | 4526 | 0.4833 |
121
+ | 0.3976 | 74.0 | 4588 | 0.4819 |
122
+ | 0.3997 | 75.0 | 4650 | 0.4809 |
123
+ | 0.3974 | 76.0 | 4712 | 0.4801 |
124
+ | 0.3953 | 77.0 | 4774 | 0.4783 |
125
+ | 0.3938 | 78.0 | 4836 | 0.4775 |
126
+ | 0.3934 | 79.0 | 4898 | 0.4762 |
127
+ | 0.3923 | 80.0 | 4960 | 0.4742 |
128
+ | 0.3893 | 81.0 | 5022 | 0.4742 |
129
+ | 0.3909 | 82.0 | 5084 | 0.4740 |
130
+ | 0.3856 | 83.0 | 5146 | 0.4739 |
131
+ | 0.3904 | 84.0 | 5208 | 0.4740 |
132
+ | 0.3883 | 85.0 | 5270 | 0.4701 |
133
+ | 0.3865 | 86.0 | 5332 | 0.4727 |
134
+ | 0.3809 | 87.0 | 5394 | 0.4736 |
135
+ | 0.3853 | 88.0 | 5456 | 0.4704 |
136
+ | 0.3821 | 89.0 | 5518 | 0.4704 |
137
+ | 0.3809 | 90.0 | 5580 | 0.4701 |
138
+ | 0.3814 | 91.0 | 5642 | 0.4698 |
139
+ | 0.3795 | 92.0 | 5704 | 0.4702 |
140
+ | 0.3804 | 93.0 | 5766 | 0.4692 |
141
+ | 0.377 | 94.0 | 5828 | 0.4683 |
142
+ | 0.3812 | 95.0 | 5890 | 0.4692 |
143
+ | 0.3806 | 96.0 | 5952 | 0.4683 |
144
+ | 0.3745 | 97.0 | 6014 | 0.4690 |
145
+ | 0.3825 | 98.0 | 6076 | 0.4684 |
146
+ | 0.374 | 99.0 | 6138 | 0.4687 |
147
+ | 0.3795 | 100.0 | 6200 | 0.4684 |
148
+
149
+
150
+ ### Framework versions
151
+
152
+ - Transformers 4.21.1
153
+ - Pytorch 1.12.0+cu113
154
+ - Datasets 2.4.0
155
+ - Tokenizers 0.12.1