FeverCodeChallenge

This model is a fine-tuned version of microsoft/deberta-v3-small on a a simplified version of Amazon 2018, only containing products and their descriptions.

Model description

SequenceClassification to predict amazon product's main category (22 categories):

{0: 'All Electronics',
 1: 'Amazon Fashion',
 2: 'Amazon Home',
 3: 'Arts, Crafts & Sewing',
 4: 'Automotive',
 5: 'Books',
 6: 'Camera & Photo',
 7: 'Cell Phones & Accessories',
 8: 'Computers',
 9: 'Digital Music',
 10: 'Grocery',
 11: 'Health & Personal Care',
 12: 'Home Audio & Theater',
 13: 'Industrial & Scientific',
 14: 'Movies & TV',
 15: 'Musical Instruments',
 16: 'Office Products',
 17: 'Pet Supplies',
 18: 'Sports & Outdoors',
 19: 'Tools & Home Improvement',
 20: 'Toys & Games',
 21: 'Video Games'}

Data

Example of a product in the dataset

{
 "also_buy": ["B071WSK6R8", "B006K8N5WQ", "B01ASDJLX0", "B00658TPYI"],
 "also_view": [],
 "asin": "B00N31IGPO",
 "brand": "Speed Dealer Customs",
 "category": ["Automotive", "Replacement Parts", "Shocks, Struts & Suspension", "Tie Rod Ends & Parts", "Tie Rod Ends"],
 "description": ["Universal heim joint tie rod weld in tube adapter bung. Made in the USA by Speed Dealer Customs. Tube adapter measurements are as in the title, please contact us about any questions you 
may have."],
 "feature": ["Completely CNC machined 1045 Steel", "Single RH Tube Adapter", "Thread: 3/4-16", "O.D.: 1-1/4", "Fits 1-1/4\" tube with .120\" wall thickness"],
 "image": [],
 "price": "",
 "title": "3/4-16 RH Weld In Threaded Heim Joint Tube Adapter Bung for 1-1/4" Dia by .120 Wall Tube",
 "main_cat": "Automotive"
}

Fields used

• [Used for the split] also_buy/also_view: IDs of related products
• description: description of the product
• feature: bullet point format features of the product
• title: name of the product
• [label] main_cat: main category of the product

Split of the data

	# Samples
Train	317662
Validation	53890
Test	54716

Evaluation results

TEST	precision	recall	f1-score	support
0	0.56	0.42	0.48	5327
1	0.81	0.86	0.83	1595
2	0.75	0.76	0.76	2224
3	0.80	0.82	0.81	1190
4	0.93	0.92	0.93	2632
5	0.99	0.97	0.98	4775
6	0.74	0.80	0.77	1024
7	0.71	0.64	0.67	1111
8	0.79	0.80	0.80	9765
9	0.94	0.93	0.94	840
10	0.94	0.98	0.96	1639
11	0.62	0.52	0.56	539
12	0.57	0.74	0.64	3802
13	0.79	0.84	0.81	2476
14	0.83	0.94	0.88	813
15	0.88	0.87	0.87	3004
16	0.76	0.61	0.68	2031
17	0.88	0.88	0.88	577
18	0.73	0.71	0.72	1813
19	0.79	0.85	0.82	3840
20	0.89	0.91	0.90	3253
21	0.69	0.75	0.72	446
accuracy			0.79	54716
macro avg	0.79	0.80	0.79	54716
weighted avg	0.79	0.79	0.79	54716

VALIDATION	precision	recall	f1-score	support
0	0.55	0.32	0.40	1034
1	0.79	0.85	0.82	1747
2	0.75	0.78	0.76	2273
3	0.84	0.88	0.86	2982
4	0.93	0.92	0.93	2236
5	0.97	0.98	0.97	2893
6	0.88	0.76	0.81	1335
7	0.77	0.74	0.75	837
8	0.76	0.73	0.74	790
9	0.95	0.96	0.95	839
10	0.96	0.98	0.97	13182
11	0.50	0.30	0.37	907
12	0.55	0.74	0.64	965
13	0.83	0.86	0.85	2780
14	0.93	0.94	0.93	1245
15	0.89	0.92	0.91	930
16	0.87	0.85	0.86	3226
17	0.96	0.97	0.96	2633
18	0.75	0.71	0.73	2518
19	0.74	0.81	0.77	2303
20	0.92	0.91	0.92	6032
21	0.72	0.89	0.79	203
accuracy			0.87	53890
macro avg	0.81	0.81	0.81	53890
weighted avg	0.87	0.87	0.87	53890

Training results

train_runtime	train_samples_per_second	train_steps_per_second	eval_loss	epoch
48601.2302	13.072	1.634	0.5335464077893132	2