bert-base-spanish-wwm-uncased-finetuned-clinical

This model is a fine-tuned version of dccuchile/bert-base-spanish-wwm-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.7962
• F1: 0.1081

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 80

Training results

Training Loss	Epoch	Step	Validation Loss	F1
1.1202	1.0	2007	1.0018	0.0062
1.0153	2.0	4014	0.9376	0.0166
0.9779	3.0	6021	0.9026	0.0342
0.9598	4.0	8028	0.8879	0.0337
0.9454	5.0	10035	0.8699	0.0598
0.9334	6.0	12042	0.8546	0.0682
0.9263	7.0	14049	0.8533	0.0551
0.9279	8.0	16056	0.8538	0.0715
0.9184	9.0	18063	0.8512	0.0652
0.9151	10.0	20070	0.8313	0.0789
0.9092	11.0	22077	0.8299	0.0838
0.9083	12.0	24084	0.8331	0.0718
0.9057	13.0	26091	0.8319	0.0719
0.9018	14.0	28098	0.8133	0.0969
0.9068	15.0	30105	0.8234	0.0816
0.9034	16.0	32112	0.8151	0.0899
0.9008	17.0	34119	0.8145	0.0967
0.8977	18.0	36126	0.8168	0.0891
0.898	19.0	38133	0.8167	0.0818
0.8956	20.0	40140	0.8076	0.1030
0.8983	21.0	42147	0.8129	0.0867
0.896	22.0	44154	0.8118	0.0892
0.8962	23.0	46161	0.8066	0.1017
0.8917	24.0	48168	0.8154	0.0908
0.8923	25.0	50175	0.8154	0.0897
0.8976	26.0	52182	0.8089	0.0910
0.8926	27.0	54189	0.8069	0.0947
0.8911	28.0	56196	0.8170	0.0882
0.8901	29.0	58203	0.7991	0.1112
0.8934	30.0	60210	0.7996	0.1112
0.8903	31.0	62217	0.8049	0.0950
0.8924	32.0	64224	0.8116	0.0951
0.8887	33.0	66231	0.7982	0.1075
0.8922	34.0	68238	0.8013	0.1025
0.8871	35.0	70245	0.8064	0.0979
0.8913	36.0	72252	0.8108	0.0909
0.8924	37.0	74259	0.8081	0.0889
0.8848	38.0	76266	0.7923	0.1228
0.8892	39.0	78273	0.8025	0.0959
0.8886	40.0	80280	0.7954	0.1148
0.8938	41.0	82287	0.8017	0.1058
0.8897	42.0	84294	0.7946	0.1146
0.8906	43.0	86301	0.7983	0.1102
0.889	44.0	88308	0.8068	0.0950
0.8872	45.0	90315	0.7999	0.1089
0.8902	46.0	92322	0.7992	0.0999
0.8912	47.0	94329	0.7981	0.1048
0.886	48.0	96336	0.8024	0.0991
0.8848	49.0	98343	0.8026	0.0984
0.8866	50.0	100350	0.7965	0.1135
0.8848	51.0	102357	0.8054	0.0926
0.8863	52.0	104364	0.8068	0.0917
0.8866	53.0	106371	0.7993	0.0964
0.8823	54.0	108378	0.7929	0.1126
0.8911	55.0	110385	0.7938	0.1132
0.8911	56.0	112392	0.7932	0.1144
0.8866	57.0	114399	0.8018	0.0957
0.8841	58.0	116406	0.7976	0.1015
0.8874	59.0	118413	0.8035	0.0966
0.887	60.0	120420	0.7954	0.1112
0.888	61.0	122427	0.7927	0.1164
0.8845	62.0	124434	0.7982	0.1012
0.8848	63.0	126441	0.7978	0.1034
0.8857	64.0	128448	0.8036	0.0969
0.8827	65.0	130455	0.7958	0.1036
0.8878	66.0	132462	0.7983	0.1030
0.885	67.0	134469	0.7956	0.1055
0.8859	68.0	136476	0.7964	0.1058
0.8872	69.0	138483	0.7989	0.1005
0.8841	70.0	140490	0.7949	0.1138
0.8846	71.0	142497	0.7960	0.1062
0.8867	72.0	144504	0.7965	0.1058
0.8856	73.0	146511	0.7980	0.1007
0.8852	74.0	148518	0.7971	0.1012
0.8841	75.0	150525	0.7975	0.1049
0.8865	76.0	152532	0.7981	0.1010
0.8887	77.0	154539	0.7945	0.1095
0.8853	78.0	156546	0.7965	0.1053
0.8843	79.0	158553	0.7966	0.1062
0.8858	80.0	160560	0.7962	0.1081

Framework versions

• Transformers 4.20.1
• Pytorch 1.9.0+cu102
• Datasets 2.3.2
• Tokenizers 0.12.1