metadata
license: apache-2.0
tags:
- generated_from_trainer
metrics:
- f1
- accuracy
model-index:
- name: roberta-finetuned-CPV_Spanish
results: []
roberta-finetuned-CPV_Spanish
This model is a fine-tuned version of PlanTL-GOB-ES/roberta-base-bne on a dataset derived from Spanish Public Procurement documents from 2019. The whole fine-tuning process is available in the following Kaggle notebook.
It achieves the following results on the evaluation set:
- Loss: 0.0465
- F1: 0.7918
- Roc Auc: 0.8860
- Accuracy: 0.7376
- Coverage Error: 10.2744
- Label Ranking Average Precision Score: 0.7973
Intended uses & limitations
This model only predicts the first two digits of the CPV codes. The list of divisions CPV codes is the following:
Division | English | Spanish | |||
---|---|---|---|---|---|
30 | Agricultural, farming, fishing, forestry and related products | Productos de la agricultura, ganadería, pesca, silvicultura y productos afines | |||
90 | Petroleum products, fuel, electricity and other sources of energy | Derivados del petróleo, combustibles, electricidad y otras fuentes de energía | |||
14 | Mining, basic metals and related products | Productos de la minería, de metales de base y productos afines | |||
15 | Food, beverages, tobacco and related products | Alimentos, bebidas, tabaco y productos afines | |||
16 | Agricultural machinery | Maquinaria agrícola | |||
18 | Clothing, footwear, luggage articles and accessories | Prendas de vestir, calzado, artículos de viaje y accesorios | |||
19 | Leather and textile fabrics, plastic and rubber materials | Piel y textiles, materiales de plástico y caucho | |||
22 | Printed matter and related products | Impresos y productos relacionados | |||
24 | Chemical products | Productos químicos | |||
30 | Office and computing machinery, equipment and supplies except furniture and software packages | Máquinas, equipo y artículos de oficina y de informática, excepto mobiliario y paquetes de software | |||
31 | Electrical machinery, apparatus, equipment and consumables; lighting | Máquinas, aparatos, equipo y productos consumibles eléctricos; iluminación | |||
32 | Radio, television, communication, telecommunication and related equipment | Equipos de radio, televisión, comunicaciones y telecomunicaciones y equipos conexos | |||
33 | Medical equipments, pharmaceuticals and personal care products | Equipamiento y artículos médicos, farmacéuticos y de higiene personal | |||
34 | Transport equipment and auxiliary products to transportation | Equipos de transporte y productos auxiliares | |||
35 | Security, fire | Equipo de seguridad, extinción de incendios, policía y defensa | |||
37 | Musical instruments, sport goods, games, toys, handicraft, art materials and accessories | Instrumentos musicales, artículos deportivos, juegos, juguetes, artículos de artesanía, materiales artísticos y accesorios | |||
38 | Laboratory, optical and precision equipments (excl. glasses) | Equipo de laboratorio, óptico y de precisión (excepto gafas) | |||
39 | Furniture (incl. office furniture), furnishings, domestic appliances (excl. lighting) and cleaning products | Mobiliario (incluido el de oficina), complementos de mobiliario, aparatos electrodomésticos (excluida la iluminación) y productos de limpieza | |||
41 | Collected and purified water | Agua recogida y depurada | |||
42 | Industrial machinery | Maquinaria industrial | |||
43 | Machinery for mining, quarrying, construction equipment | Maquinaria para la minería y la explotación de canteras y equipo de construcción | |||
44 | Construction structures and materials; auxiliary products to construction (except electric apparatus) | Estructuras y materiales de construcción; productos auxiliares para la construcción (excepto aparatos eléctricos) | |||
45 | Construction work | Trabajos de construcción | |||
48 | Software package and information systems | Paquetes de software y sistemas de información | |||
50 | Repair and maintenance services | Servicios de reparación y mantenimiento | |||
51 | Installation services (except software) | Servicios de instalación (excepto software) | |||
55 | Hotel, restaurant and retail trade services | Servicios comerciales al por menor de hostelería y restauración | |||
60 | Transport services (excl. Waste transport) | Servicios de transporte (excluido el transporte de residuos) | |||
63 | Supporting and auxiliary transport services; travel agencies services | Servicios de transporte complementarios y auxiliares; servicios de agencias de viajes | |||
64 | Postal and telecommunications services | Servicios de correos y telecomunicaciones | |||
65 | Public utilities | Servicios públicos | |||
66 | Financial and insurance services | Servicios financieros y de seguros | |||
70 | Real estate services | Servicios inmobiliarios | |||
71 | Architectural, construction, engineering and inspection services | Servicios de arquitectura, construcción, ingeniería e inspección | |||
72 | IT services: consulting, software development, Internet and support | Servicios TI: consultoría, desarrollo de software, Internet y apoyo | |||
73 | Research and development services and related consultancy services | Servicios de investigación y desarrollo y servicios de consultoría conexos | |||
75 | Administration, defence and social security services | Servicios de administración pública, defensa y servicios de seguridad social | |||
76 | Services related to the oil and gas industry | Servicios relacionados con la industria del gas y del petróleo | |||
77 | Agricultural, forestry, horticultural, aquacultural and apicultural services | Servicios agrícolas, forestales, hortícolas, acuícolas y apícolas | |||
79 | Business services: law, marketing, consulting, recruitment, printing and security | Servicios a empresas: legislación, mercadotecnia, asesoría, selección de personal, imprenta y seguridad | |||
80 | Education and training services | Servicios de enseñanza y formación | |||
85 | Health and social work services | Servicios de salud y asistencia social | |||
90 | Sewage, refuse, cleaning and environmental services | Servicios de alcantarillado, basura, limpieza y medio ambiente | |||
92 | Recreational, cultural and sporting services | Servicios de esparcimiento, culturales y deportivos | |||
98 | Other community, social and personal services | Otros servicios comunitarios, sociales o personales |
Training and evaluation data
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10
Training results
Training Loss | Epoch | Step | Validation Loss | F1 | Roc Auc | Accuracy | Coverage Error | Label Ranking Average Precision Score |
---|---|---|---|---|---|---|---|---|
0.0354 | 1.0 | 9054 | 0.0362 | 0.7560 | 0.8375 | 0.6963 | 14.0835 | 0.7357 |
0.0311 | 2.0 | 18108 | 0.0331 | 0.7756 | 0.8535 | 0.7207 | 12.7880 | 0.7633 |
0.0235 | 3.0 | 27162 | 0.0333 | 0.7823 | 0.8705 | 0.7283 | 11.5179 | 0.7811 |
0.0157 | 4.0 | 36216 | 0.0348 | 0.7821 | 0.8699 | 0.7274 | 11.5836 | 0.7798 |
0.011 | 5.0 | 45270 | 0.0377 | 0.7799 | 0.8787 | 0.7239 | 10.9173 | 0.7841 |
0.008 | 6.0 | 54324 | 0.0395 | 0.7854 | 0.8787 | 0.7309 | 10.9042 | 0.7879 |
0.0042 | 7.0 | 63378 | 0.0421 | 0.7872 | 0.8823 | 0.7300 | 10.5687 | 0.7903 |
0.0025 | 8.0 | 72432 | 0.0439 | 0.7884 | 0.8867 | 0.7305 | 10.2220 | 0.7934 |
0.0015 | 9.0 | 81486 | 0.0456 | 0.7889 | 0.8872 | 0.7316 | 10.1781 | 0.7945 |
0.001 | 10.0 | 90540 | 0.0465 | 0.7918 | 0.8860 | 0.7376 | 10.2744 | 0.7973 |
Framework versions
- Transformers 4.16.2
- Pytorch 1.9.1
- Datasets 1.18.4
- Tokenizers 0.11.6