Hugging Face's logo

floriangardin
/
model

Text Generation
Transformers
PyTorch
TensorBoard
Safetensors
gpt2
Generated from Trainer
text-generation-inference
Inference Endpoints
Model card Files Metrics Community
floriangardin commited on
Commit
4099e02
•
1 Parent(s): c340281

musiclang/musiclang-v2-xl

Browse files
Files changed (5) hide show
  1. README.md +44 -52
  2. config.json +9 -8
  3. generation_config.json +3 -3
  4. model.safetensors +3 -0
  5. training_args.bin +2 -2
README.md CHANGED
@@ -1,4 +1,5 @@
1
  ---
 
2
  tags:
3
  - generated_from_trainer
4
  model-index:
@@ -11,9 +12,9 @@ should probably proofread and complete it, then remove this comment. -->
11
 
12
  # model
13
 
14
- This model is a fine-tuned version of [floriangardin/musiclang_medium](https://huggingface.co/floriangardin/musiclang_medium) on the None dataset.
15
  It achieves the following results on the evaluation set:
16
- - Loss: 0.5640
17
 
18
  ## Model description
19
 
@@ -32,64 +33,55 @@ More information needed
32
  ### Training hyperparameters
33
 
34
  The following hyperparameters were used during training:
35
- - learning_rate: 5e-05
36
- - train_batch_size: 64
37
- - eval_batch_size: 64
38
  - seed: 42
39
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
40
- - lr_scheduler_type: linear
41
- - lr_scheduler_warmup_steps: 15
42
- - num_epochs: 6
 
43
 
44
  ### Training results
45
 
46
  | Training Loss | Epoch | Step | Validation Loss |
47
  |:-------------:|:-----:|:-----:|:---------------:|
48
- | No log | 0.15 | 400 | 0.6429 |
49
- | 0.6819 | 0.3 | 800 | 0.6389 |
50
- | 0.6744 | 0.44 | 1200 | 0.6335 |
51
- | 0.6664 | 0.59 | 1600 | 0.6319 |
52
- | 0.659 | 0.74 | 2000 | 0.6246 |
53
- | 0.659 | 0.89 | 2400 | 0.6203 |
54
- | 0.6519 | 1.04 | 2800 | 0.6178 |
55
- | 0.6446 | 1.19 | 3200 | 0.6136 |
56
- | 0.6403 | 1.33 | 3600 | 0.6103 |
57
- | 0.6363 | 1.48 | 4000 | 0.6052 |
58
- | 0.6363 | 1.63 | 4400 | 0.6051 |
59
- | 0.6302 | 1.78 | 4800 | 0.6011 |
60
- | 0.6257 | 1.93 | 5200 | 0.5985 |
61
- | 0.6229 | 2.07 | 5600 | 0.5971 |
62
- | 0.6185 | 2.22 | 6000 | 0.5948 |
63
- | 0.6185 | 2.37 | 6400 | 0.5938 |
64
- | 0.6155 | 2.52 | 6800 | 0.5911 |
65
- | 0.6123 | 2.67 | 7200 | 0.5883 |
66
- | 0.6096 | 2.82 | 7600 | 0.5867 |
67
- | 0.6079 | 2.96 | 8000 | 0.5856 |
68
- | 0.6079 | 3.11 | 8400 | 0.5835 |
69
- | 0.6026 | 3.26 | 8800 | 0.5814 |
70
- | 0.5998 | 3.41 | 9200 | 0.5804 |
71
- | 0.5993 | 3.56 | 9600 | 0.5779 |
72
- | 0.5978 | 3.71 | 10000 | 0.5770 |
73
- | 0.5978 | 3.85 | 10400 | 0.5761 |
74
- | 0.5958 | 4.0 | 10800 | 0.5746 |
75
- | 0.5937 | 4.15 | 11200 | 0.5737 |
76
- | 0.5909 | 4.3 | 11600 | 0.5733 |
77
- | 0.5884 | 4.45 | 12000 | 0.5714 |
78
- | 0.5884 | 4.59 | 12400 | 0.5704 |
79
- | 0.588 | 4.74 | 12800 | 0.5690 |
80
- | 0.5875 | 4.89 | 13200 | 0.5685 |
81
- | 0.5848 | 5.04 | 13600 | 0.5679 |
82
- | 0.5827 | 5.19 | 14000 | 0.5668 |
83
- | 0.5827 | 5.34 | 14400 | 0.5663 |
84
- | 0.5839 | 5.48 | 14800 | 0.5658 |
85
- | 0.5806 | 5.63 | 15200 | 0.5650 |
86
- | 0.5803 | 5.78 | 15600 | 0.5644 |
87
- | 0.5796 | 5.93 | 16000 | 0.5640 |
88
 
89
 
90
  ### Framework versions
91
 
92
- - Transformers 4.29.2
93
- - Pytorch 2.0.1+cu118
94
- - Datasets 2.12.0
95
- - Tokenizers 0.13.3
 
1
  ---
2
+ base_model: musiclang/musiclang-v2-xl
3
  tags:
4
  - generated_from_trainer
5
  model-index:
 
12
 
13
  # model
14
 
15
+ This model is a fine-tuned version of [musiclang/musiclang-v2-xl](https://huggingface.co/musiclang/musiclang-v2-xl) on the None dataset.
16
  It achieves the following results on the evaluation set:
17
+ - Loss: 0.2930
18
 
19
  ## Model description
20
 
 
33
  ### Training hyperparameters
34
 
35
  The following hyperparameters were used during training:
36
+ - learning_rate: 0.0001
37
+ - train_batch_size: 24
38
+ - eval_batch_size: 24
39
  - seed: 42
40
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
41
+ - lr_scheduler_type: cosine_with_restarts
42
+ - lr_scheduler_warmup_steps: 500
43
+ - training_steps: 0
44
+ - mixed_precision_training: Native AMP
45
 
46
  ### Training results
47
 
48
  | Training Loss | Epoch | Step | Validation Loss |
49
  |:-------------:|:-----:|:-----:|:---------------:|
50
+ | 0.4098 | 0.03 | 2000 | 0.3828 |
51
+ | 0.3901 | 0.07 | 4000 | 0.3684 |
52
+ | 0.3737 | 0.1 | 6000 | 0.3569 |
53
+ | 0.3652 | 0.13 | 8000 | 0.3489 |
54
+ | 0.356 | 0.17 | 10000 | 0.3393 |
55
+ | 0.35 | 0.2 | 12000 | 0.3342 |
56
+ | 0.3443 | 0.23 | 14000 | 0.3282 |
57
+ | 0.3371 | 0.26 | 16000 | 0.3235 |
58
+ | 0.3361 | 0.3 | 18000 | 0.3201 |
59
+ | 0.3301 | 0.33 | 20000 | 0.3160 |
60
+ | 0.3253 | 0.36 | 22000 | 0.3131 |
61
+ | 0.327 | 0.4 | 24000 | 0.3109 |
62
+ | 0.3225 | 0.43 | 26000 | 0.3089 |
63
+ | 0.3156 | 0.46 | 28000 | 0.3066 |
64
+ | 0.3147 | 0.5 | 30000 | 0.3045 |
65
+ | 0.3182 | 0.53 | 32000 | 0.3026 |
66
+ | 0.3129 | 0.56 | 34000 | 0.3017 |
67
+ | 0.3132 | 0.59 | 36000 | 0.3008 |
68
+ | 0.3109 | 0.63 | 38000 | 0.2987 |
69
+ | 0.3092 | 0.66 | 40000 | 0.2972 |
70
+ | 0.3091 | 0.69 | 42000 | 0.2963 |
71
+ | 0.3034 | 0.73 | 44000 | 0.2960 |
72
+ | 0.3061 | 0.76 | 46000 | 0.2956 |
73
+ | 0.3044 | 0.79 | 48000 | 0.2946 |
74
+ | 0.3036 | 0.83 | 50000 | 0.2940 |
75
+ | 0.3003 | 0.86 | 52000 | 0.2939 |
76
+ | 0.303 | 0.89 | 54000 | 0.2934 |
77
+ | 0.3007 | 0.93 | 56000 | 0.2932 |
78
+ | 0.3009 | 0.96 | 58000 | 0.2930 |
79
+ | 0.3 | 0.99 | 60000 | 0.2930 |
 
 
 
 
 
 
 
 
 
 
80
 
81
 
82
  ### Framework versions
83
 
84
+ - Transformers 4.37.2
85
+ - Pytorch 2.2.0+cu121
86
+ - Datasets 2.17.0
87
+ - Tokenizers 0.15.1
config.json CHANGED
@@ -1,21 +1,22 @@
1
  {
2
- "_name_or_path": "floriangardin/musiclang_medium",
3
  "activation_function": "gelu_new",
4
  "architectures": [
5
  "GPT2LMHeadModel"
6
  ],
7
  "attn_pdrop": 0.1,
8
- "bos_token_id": 79,
9
  "embd_pdrop": 0.1,
10
- "eos_token_id": 79,
11
  "initializer_range": 0.02,
12
  "layer_norm_epsilon": 1e-05,
13
  "model_type": "gpt2",
14
  "n_embd": 300,
15
- "n_head": 12,
16
  "n_inner": null,
17
- "n_layer": 12,
18
- "n_positions": 1024,
 
19
  "reorder_and_upcast_attn": false,
20
  "resid_pdrop": 0.1,
21
  "scale_attn_by_inverse_layer_idx": false,
@@ -26,7 +27,7 @@
26
  "summary_type": "cls_index",
27
  "summary_use_proj": true,
28
  "torch_dtype": "float32",
29
- "transformers_version": "4.29.2",
30
  "use_cache": true,
31
- "vocab_size": 281
32
  }
 
1
  {
2
+ "_name_or_path": "musiclang/musiclang-v2-xl",
3
  "activation_function": "gelu_new",
4
  "architectures": [
5
  "GPT2LMHeadModel"
6
  ],
7
  "attn_pdrop": 0.1,
8
+ "bos_token_id": 5,
9
  "embd_pdrop": 0.1,
10
+ "eos_token_id": 374,
11
  "initializer_range": 0.02,
12
  "layer_norm_epsilon": 1e-05,
13
  "model_type": "gpt2",
14
  "n_embd": 300,
15
+ "n_head": 10,
16
  "n_inner": null,
17
+ "n_layer": 10,
18
+ "n_positions": 4096,
19
+ "padding_token_id": 3,
20
  "reorder_and_upcast_attn": false,
21
  "resid_pdrop": 0.1,
22
  "scale_attn_by_inverse_layer_idx": false,
 
27
  "summary_type": "cls_index",
28
  "summary_use_proj": true,
29
  "torch_dtype": "float32",
30
+ "transformers_version": "4.37.2",
31
  "use_cache": true,
32
+ "vocab_size": 374
33
  }
generation_config.json CHANGED
@@ -1,6 +1,6 @@
1
  {
2
  "_from_model_config": true,
3
- "bos_token_id": 79,
4
- "eos_token_id": 79,
5
- "transformers_version": "4.29.2"
6
  }
 
1
  {
2
  "_from_model_config": true,
3
+ "bos_token_id": 5,
4
+ "eos_token_id": 374,
5
+ "transformers_version": "4.37.2"
6
  }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0edaec61be7b684821241f0871d5e8a7b48d865f5ad1aee2495c9dc9f54290a8
3
+ size 48734696
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0df02bc9ca652060ebcb1749fa422f319a2ec8cc541de2f5763933254d03693d
3
- size 3899
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ded85577c88c0610c6cc2a534d279f3cac2c97cc898df360f85bd6ae1a843d78
3
+ size 4664