phi-1_5-finetuned-SQL

This model is a fine-tuned version of microsoft/phi-1_5 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5914

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 48000

Training results

Training Loss	Epoch	Step	Validation Loss
2.3757	0.04	100	2.0747
2.0269	0.08	200	1.9990
1.9535	0.12	300	1.9450
1.9136	0.16	400	1.9067
1.892	0.2	500	1.8757
1.8753	0.24	600	1.8574
1.8507	0.28	700	1.8359
1.8759	0.32	800	1.8167
1.8166	0.36	900	1.8054
1.8224	0.4	1000	1.7818
1.7852	0.44	1100	1.7814
1.8164	0.48	1200	1.7664
1.7632	0.52	1300	1.7598
1.8485	0.56	1400	1.7439
1.7712	0.6	1500	1.7303
1.7632	0.64	1600	1.7277
1.7378	0.68	1700	1.7135
1.7581	0.72	1800	1.7075
1.7261	0.76	1900	1.6933
1.7243	0.8	2000	1.6891
1.7311	0.84	2100	1.6837
1.7554	0.88	2200	1.6808
1.7026	0.92	2300	1.6646
1.7193	0.96	2400	1.6664
1.6861	1.0	2500	1.6577
1.68	1.04	2600	1.6470
1.5931	1.08	2700	1.6425
1.6655	1.12	2800	1.6352
1.629	1.16	2900	1.6298
1.6567	1.2	3000	1.6236
1.6225	1.24	3100	1.6242
1.6249	1.28	3200	1.6150
1.6263	1.32	3300	1.6077
1.6055	1.36	3400	1.6034
1.6338	1.4	3500	1.5996
1.6032	1.44	3600	1.5947
1.6447	1.48	3700	1.5882
1.6063	1.52	3800	1.5877
1.5933	1.56	3900	1.5850
1.6267	1.6	4000	1.5814
1.6151	1.64	4100	1.5709
1.6047	1.68	4200	1.5683
1.5811	1.72	4300	1.5661
1.5877	1.76	4400	1.5648
1.6321	1.8	4500	1.5645
1.5969	1.84	4600	1.5584
1.5971	1.88	4700	1.5565
1.622	1.92	4800	1.5547
1.6265	1.96	4900	1.5496
1.6145	2.0	5000	1.5466
1.526	2.04	5100	1.5427
1.5793	2.08	5200	1.5390
1.5714	2.12	5300	1.5375
1.5228	2.16	5400	1.5360
1.5383	2.2	5500	1.5343
1.5117	2.24	5600	1.5322
1.5427	2.28	5700	1.5316
1.4959	2.32	5800	1.5306
1.5456	2.36	5900	1.5299
1.5175	2.4	6000	1.5295
1.5823	2.44	6100	1.5498
1.5615	2.48	6200	1.5447
1.5326	2.52	6300	1.5463
1.567	2.56	6400	1.5450
1.5243	2.6	6500	1.5456
1.5214	2.64	6600	1.5383
1.6086	2.68	6700	1.5393
1.5391	2.72	6800	1.5285
1.5224	2.76	6900	1.5318
1.5567	2.8	7000	1.5292
1.5525	2.84	7100	1.5207
1.5399	2.88	7200	1.5135
1.5399	2.92	7300	1.5104
1.5765	2.96	7400	1.5085
1.556	3.0	7500	1.5042
1.4977	3.04	7600	1.4997
1.4818	3.08	7700	1.4930
1.4912	3.12	7800	1.4908
1.517	3.16	7900	1.4933
1.4971	3.2	8000	1.4857
1.4827	3.24	8100	1.4805
1.5096	3.28	8200	1.4804
1.4788	3.32	8300	1.4756
1.457	3.36	8400	1.4728
1.4819	3.4	8500	1.4717
1.5241	3.44	8600	1.4678
1.5081	3.48	8700	1.4676
1.5173	3.52	8800	1.4657
1.4765	3.56	8900	1.4643
1.4691	3.6	9000	1.4603
1.5034	3.64	9100	1.4577
1.4997	3.68	9200	1.4552
1.4849	3.72	9300	1.4504
1.5144	3.76	9400	1.4518
1.4972	3.8	9500	1.4469
1.4695	3.84	9600	1.4474
1.5088	3.88	9700	1.4468
1.4772	3.92	9800	1.4418
1.5207	3.96	9900	1.4390
1.5088	4.0	10000	1.4378
1.4915	4.04	10100	1.4324
1.4356	4.08	10200	1.4305
1.4388	4.12	10300	1.4268
1.4004	4.16	10400	1.4251
1.3909	4.2	10500	1.4225
1.4284	4.24	10600	1.4218
1.4422	4.28	10700	1.4213
1.4301	4.32	10800	1.4198
1.4309	4.36	10900	1.4174
1.415	4.4	11000	1.4147
1.4697	4.44	11100	1.4136
1.4241	4.48	11200	1.4123
1.4416	4.52	11300	1.4100
1.4229	4.56	11400	1.4094
1.4498	4.6	11500	1.4091
1.4023	4.64	11600	1.4083
1.4197	4.68	11700	1.4075
1.4165	4.72	11800	1.4070
1.4103	4.76	11900	1.4067
1.4214	4.8	12000	1.4066
1.4223	9.68	12100	1.4162
1.4471	9.76	12200	1.4210
1.4165	9.84	12300	1.4154
1.4088	9.92	12400	1.4105
1.4057	10.0	12500	1.4100
1.3778	10.08	12600	1.4034
1.4081	10.16	12700	1.4055
1.4127	10.24	12800	1.4001
1.4282	10.32	12900	1.3924
1.4069	10.4	13000	1.3909
1.4097	10.48	13100	1.3885
1.4173	10.56	13200	1.3824
1.4282	10.64	13300	1.3798
1.4266	10.72	13400	1.3778
1.4205	10.8	13500	1.3760
1.4347	10.88	13600	1.3730
1.4088	10.96	13700	1.3659
1.3859	11.04	13800	1.3605
1.3711	11.12	13900	1.3572
1.3896	11.2	14000	1.3550
1.343	11.28	14100	1.3510
1.3866	11.36	14200	1.3485
1.3603	11.44	14300	1.3468
1.3881	11.52	14400	1.3448
1.3841	11.6	14500	1.3422
1.358	11.68	14600	1.3379
1.3704	11.76	14700	1.3352
1.3656	11.84	14800	1.3350
1.367	11.92	14900	1.3299
1.3765	12.0	15000	1.3302
1.32	12.08	15100	1.3240
1.343	12.16	15200	1.3186
1.3254	12.24	15300	1.3159
1.3433	12.32	15400	1.3134
1.3347	12.4	15500	1.3113
1.3304	12.48	15600	1.3110
1.3235	12.56	15700	1.3106
1.3099	12.64	15800	1.3056
1.3176	12.72	15900	1.3027
1.3613	12.8	16000	1.3057
1.3238	12.88	16100	1.3006
1.354	12.96	16200	1.3003
1.3324	13.04	16300	1.2967
1.322	13.12	16400	1.2945
1.3029	13.2	16500	1.2898
1.317	13.28	16600	1.2892
1.2982	13.36	16700	1.2882
1.3092	13.44	16800	1.2878
1.3161	13.52	16900	1.2866
1.2895	13.6	17000	1.2844
1.28	13.68	17100	1.2834
1.2849	13.76	17200	1.2822
1.3136	13.84	17300	1.2828
1.2938	13.92	17400	1.2810
1.2994	14.0	17500	1.2803
1.3158	14.08	17600	1.2788
1.2783	14.16	17700	1.2779
1.2811	14.24	17800	1.2774
1.2824	14.32	17900	1.2771
1.2881	14.4	18000	1.2770
1.2971	14.48	18100	1.2880
1.2878	14.56	18200	1.2883
1.3081	14.64	18300	1.2812
1.2949	14.72	18400	1.2812
1.3153	14.8	18500	1.2827
1.3316	14.88	18600	1.2777
1.3225	14.96	18700	1.2789
1.3022	15.04	18800	1.2719
1.2773	15.12	18900	1.2685
1.2787	15.2	19000	1.2674
1.2876	15.28	19100	1.2644
1.2801	15.36	19200	1.2630
1.3197	15.44	19300	1.2615
1.2968	15.52	19400	1.2572
1.2992	15.6	19500	1.2581
1.2739	15.68	19600	1.2511
1.2925	15.76	19700	1.2485
1.2831	15.84	19800	1.2456
1.3055	15.92	19900	1.2415
1.2883	16.0	20000	1.2432
1.2378	16.08	20100	1.2358
1.2618	16.16	20200	1.2354
1.2475	16.24	20300	1.2294
1.2534	16.32	20400	1.2267
1.2362	16.4	20500	1.2249
1.2442	16.48	20600	1.2245
1.2727	16.56	20700	1.2209
1.2645	16.64	20800	1.2192
1.2535	16.72	20900	1.2158
1.2673	16.8	21000	1.2131
1.2693	16.88	21100	1.2133
1.2419	16.96	21200	1.2104
1.2165	17.04	21300	1.2064
1.2184	17.12	21400	1.2047
1.2195	17.2	21500	1.2036
1.2126	17.28	21600	1.2024
1.2048	17.36	21700	1.1989
1.2158	17.44	21800	1.1991
1.2372	17.52	21900	1.1966
1.2502	17.6	22000	1.1964
1.23	17.68	22100	1.1924
1.1967	17.76	22200	1.1913
1.2021	17.84	22300	1.1896
1.2323	17.92	22400	1.1904
1.2276	18.0	22500	1.1872
1.2072	18.08	22600	1.1851
1.157	18.16	22700	1.1828
1.1805	18.24	22800	1.1827
1.1812	18.32	22900	1.1812
1.1993	18.4	23000	1.1800
1.1887	18.48	23100	1.1803
1.194	18.56	23200	1.1779
1.2097	18.64	23300	1.1777
1.2049	18.72	23400	1.1769
1.2002	18.8	23500	1.1758
1.2178	18.88	23600	1.1755
1.1969	18.96	23700	1.1745
1.198	19.04	23800	1.1741
1.1919	19.12	23900	1.1736
1.149	19.2	24000	1.1735
1.2083	19.28	24100	1.2311
1.2362	19.36	24200	1.2287
1.2758	19.44	24300	1.2308
1.2554	19.52	24400	1.2333
1.2907	19.6	24500	1.2203
1.2535	19.68	24600	1.2216
1.2817	19.76	24700	1.2221
1.2834	19.84	24800	1.2164
1.2752	19.92	24900	1.2123
1.2982	20.0	25000	1.2207
1.2229	20.08	25100	1.1983
1.2081	20.16	25200	1.1894
1.2322	20.24	25300	1.1889
1.248	20.32	25400	1.1880
1.2237	20.4	25500	1.1826
1.237	20.48	25600	1.1731
1.23	20.56	25700	1.1791
1.2618	20.64	25800	1.1745
1.2452	20.72	25900	1.1707
1.2475	20.8	26000	1.1642
1.257	20.88	26100	1.1740
1.2378	20.96	26200	1.1652
1.2055	21.04	26300	1.1479
1.1479	21.12	26400	1.1450
1.1799	21.2	26500	1.1454
1.1724	21.28	26600	1.1372
1.1852	21.36	26700	1.1409
1.1842	21.44	26800	1.1322
1.1843	21.52	26900	1.1292
1.1875	21.6	27000	1.1245
1.1904	21.68	27100	1.1212
1.1814	21.76	27200	1.1171
1.1906	21.84	27300	1.1105
1.2078	21.92	27400	1.1055
1.2157	22.0	27500	1.1058
1.1111	22.08	27600	1.0881
1.109	22.16	27700	1.0827
1.1118	22.24	27800	1.0780
1.1279	22.32	27900	1.0749
1.1435	22.4	28000	1.0727
1.1161	22.48	28100	1.0713
1.1295	22.56	28200	1.0717
1.1439	22.64	28300	1.0660
1.1343	22.72	28400	1.0661
1.1564	22.8	28500	1.0557
1.1542	22.88	28600	1.0540
1.1234	22.96	28700	1.0543
1.1001	23.04	28800	1.0453
1.045	23.12	28900	1.0357
1.0757	23.2	29000	1.0308
1.083	23.28	29100	1.0259
1.0547	23.36	29200	1.0241
1.091	23.44	29300	1.0265
1.074	23.52	29400	1.0207
1.1001	23.6	29500	1.0191
1.0884	23.68	29600	1.0205
1.0943	23.76	29700	1.0172
1.0869	23.84	29800	1.0121
1.0925	23.92	29900	1.0094
1.0999	24.0	30000	1.0003
1.0	24.08	30100	0.9898
1.0128	24.16	30200	0.9874
1.0056	24.24	30300	0.9833
1.0303	24.32	30400	0.9807
1.0201	24.4	30500	0.9731
1.0371	24.48	30600	0.9743
1.0439	24.56	30700	0.9666
1.0424	24.64	30800	0.9670
1.0281	24.72	30900	0.9662
1.0449	24.8	31000	0.9595
1.0556	24.88	31100	0.9540
1.0589	24.96	31200	0.9552
1.0032	25.04	31300	0.9438
0.9534	25.12	31400	0.9400
0.9932	25.2	31500	0.9360
0.9863	25.28	31600	0.9354
0.9759	25.36	31700	0.9275
0.9761	25.44	31800	0.9310
0.9719	25.52	31900	0.9299
0.9702	25.6	32000	0.9269
1.0005	25.68	32100	0.9217
0.9975	25.76	32200	0.9161
0.9935	25.84	32300	0.9134
1.0178	25.92	32400	0.9145
1.011	26.0	32500	0.9098
0.9145	26.08	32600	0.8993
0.931	26.16	32700	0.8957
0.9326	26.24	32800	0.8905
0.9421	26.32	32900	0.8898
0.949	26.4	33000	0.8879
0.9224	26.48	33100	0.8838
0.952	26.56	33200	0.8818
0.9431	26.64	33300	0.8741
0.9463	26.72	33400	0.8747
0.9456	26.8	33500	0.8742
0.9533	26.88	33600	0.8734
0.9643	26.96	33700	0.8643
0.9037	27.04	33800	0.8546
0.8834	27.12	33900	0.8552
0.9008	27.2	34000	0.8519
0.8851	27.28	34100	0.8498
0.8812	27.36	34200	0.8485
0.9006	27.44	34300	0.8435
0.8893	27.52	34400	0.8413
0.8949	27.6	34500	0.8372
0.908	27.68	34600	0.8349
0.9121	27.76	34700	0.8312
0.9066	27.84	34800	0.8285
0.9146	27.92	34900	0.8291
0.9217	28.0	35000	0.8280
0.8282	28.08	35100	0.8158
0.8346	28.16	35200	0.8131
0.8503	28.24	35300	0.8133
0.8431	28.32	35400	0.8090
0.8479	28.4	35500	0.8087
0.8604	28.48	35600	0.8062
0.8559	28.56	35700	0.8028
0.8644	28.64	35800	0.7994
0.8761	28.72	35900	0.7983
0.8821	28.8	36000	0.7926
0.8712	28.88	36100	0.7918
0.8725	28.96	36200	0.7903
0.834	29.04	36300	0.7816
0.8119	29.12	36400	0.7739
0.8063	29.2	36500	0.7716
0.8097	29.28	36600	0.7719
0.8177	29.36	36700	0.7727
0.8098	29.44	36800	0.7683
0.8103	29.52	36900	0.7682
0.8251	29.6	37000	0.7634
0.8382	29.68	37100	0.7635
0.8193	29.76	37200	0.7609
0.85	29.84	37300	0.7631
0.8371	29.92	37400	0.7546
0.8304	30.0	37500	0.7508
0.7676	30.08	37600	0.7474
0.7782	30.16	37700	0.7466
0.7754	30.24	37800	0.7450
0.7774	30.32	37900	0.7406
0.7728	30.4	38000	0.7390
0.7812	30.48	38100	0.7361
0.79	30.56	38200	0.7339
0.8072	30.64	38300	0.7323
0.8051	30.72	38400	0.7308
0.7895	30.8	38500	0.7268
0.7932	30.88	38600	0.7251
0.7939	30.96	38700	0.7218
0.7643	31.04	38800	0.7168
0.7378	31.12	38900	0.7143
0.7498	31.2	39000	0.7128
0.7448	31.28	39100	0.7109
0.749	31.36	39200	0.7092
0.7558	31.44	39300	0.7080
0.7622	31.52	39400	0.7040
0.7572	31.6	39500	0.7047
0.7578	31.68	39600	0.6997
0.7567	31.76	39700	0.6968
0.758	31.84	39800	0.6938
0.7645	31.92	39900	0.6935
0.7728	32.0	40000	0.6932
0.7008	32.08	40100	0.6888
0.7172	32.16	40200	0.6898
0.6954	32.24	40300	0.6858
0.7251	32.32	40400	0.6838
0.7229	32.4	40500	0.6804
0.7263	32.48	40600	0.6781
0.7221	32.56	40700	0.6767
0.723	32.64	40800	0.6760
0.7396	32.72	40900	0.6747
0.7349	32.8	41000	0.6710
0.7427	32.88	41100	0.6713
0.7479	32.96	41200	0.6655
0.7212	33.04	41300	0.6650
0.6975	33.12	41400	0.6626
0.686	33.2	41500	0.6599
0.6874	33.28	41600	0.6584
0.695	33.36	41700	0.6569
0.6854	33.44	41800	0.6561
0.6917	33.52	41900	0.6540
0.6994	33.6	42000	0.6527
0.6939	33.68	42100	0.6540
0.7118	33.76	42200	0.6487
0.715	33.84	42300	0.6487
0.7164	33.92	42400	0.6452
0.7116	34.0	42500	0.6450
0.6701	34.08	42600	0.6421
0.66	34.16	42700	0.6412
0.6709	34.24	42800	0.6381
0.6708	34.32	42900	0.6382
0.6874	34.4	43000	0.6376
0.6838	34.48	43100	0.6350
0.6721	34.56	43200	0.6340
0.6782	34.64	43300	0.6326
0.6831	34.72	43400	0.6300
0.6897	34.8	43500	0.6304
0.679	34.88	43600	0.6281
0.6678	34.96	43700	0.6260
0.6705	35.04	43800	0.6251
0.6443	35.12	43900	0.6226
0.6479	35.2	44000	0.6224
0.6434	35.28	44100	0.6199
0.6461	35.36	44200	0.6196
0.6516	35.44	44300	0.6181
0.6516	35.52	44400	0.6190
0.6667	35.6	44500	0.6171
0.6583	35.68	44600	0.6153
0.664	35.76	44700	0.6143
0.6548	35.84	44800	0.6135
0.6713	35.92	44900	0.6118
0.6681	36.0	45000	0.6110
0.6315	36.08	45100	0.6079
0.6451	36.16	45200	0.6084
0.6396	36.24	45300	0.6082
0.6291	36.32	45400	0.6072
0.6391	36.4	45500	0.6060
0.6381	36.48	45600	0.6052
0.6417	36.56	45700	0.6041
0.6347	36.64	45800	0.6036
0.6436	36.72	45900	0.6022
0.6352	36.8	46000	0.6012
0.6515	36.88	46100	0.6005
0.63	36.96	46200	0.5992
0.6317	37.04	46300	0.5979
0.6313	37.12	46400	0.5977
0.6226	37.2	46500	0.5971
0.6155	37.28	46600	0.5967
0.6248	37.36	46700	0.5961
0.6329	37.44	46800	0.5958
0.6249	37.52	46900	0.5953
0.6264	37.6	47000	0.5946
0.6271	37.68	47100	0.5941
0.6281	37.76	47200	0.5936
0.6222	37.84	47300	0.5931
0.6133	37.92	47400	0.5925
0.6298	38.0	47500	0.5920
0.6123	38.08	47600	0.5918
0.6073	38.16	47700	0.5918
0.6129	38.24	47800	0.5915
0.6336	38.32	47900	0.5914
0.6094	38.4	48000	0.5914

Framework versions

• Transformers 4.34.1
• Pytorch 2.1.0+cu118
• Datasets 2.14.6
• Tokenizers 0.14.1