joaovux/ncm-chapter-bertimbau

NCM chapter classifier (BERTimbau)

Reads the description of a good bought by a Brazilian public body and predicts its NCM chapter (Nomenclatura Comum do Mercosul, the first two digits of the code): one of 14 chapters, or outros for the rarer ones. Fine-tuned from neuralmind/bert-base-portuguese-cased on joaovux/pncp-ncm-chapters. Code, evaluation and the comparison with other models: classificador-ncm.

from transformers import pipeline

classify = pipeline("text-classification", model="joaovux/ncm-chapter-bertimbau", top_k=3)
classify("Seringa descartável 10 ml, bico luer lock, estéril")

Results

On the test set: 1,492 descriptions of catalogue items first seen on the last day collected, none of them, in any wording, in training.

Model	Macro-F1	Accuracy
bertimbau	0.914	0.949
tfidf-logreg	0.903	0.935
gemini-3.5-flash-lite-zero-shot	0.611	0.757
majority	0.036	0.337

Against tfidf-logreg, a paired bootstrap over the test items gives macro-F1 +1.0 points (95% interval -1.7 to +3.3, within noise) and accuracy +1.4 points (95% interval +0.2 to +2.7, a real edge). Details in the report.

Label	Chapter	Test F1	Test examples
30	Produtos farmacêuticos	0.97	132
38	Produtos diversos das indústrias químicas	0.89	15
39	Plásticos e suas obras	0.92	14
48	Papel e cartão; obras de pasta de celulose, de papel ou de cartão	0.92	7
61	Vestuário e seus acessórios, de malha	0.80	4
82	Ferramentas, artefatos de cutelaria e talheres, de metais comuns	0.87	49
83	Obras diversas de metais comuns	0.89	43
84	Reatores nucleares, caldeiras, máquinas, aparelhos e instrumentos mecânicos	0.95	311
85	Máquinas, aparelhos e materiais elétricos; aparelhos de som e de imagem	0.94	282
87	Veículos automóveis, tratores, ciclos e outros veículos terrestres	0.98	92
90	Instrumentos e aparelhos de óptica, fotografia, medida, médico-cirúrgicos	0.97	503
94	Móveis; mobiliário médico-cirúrgico; colchões; iluminação; construções pré-fabricadas	0.87	15
96	Obras diversas	1.00	3
outros	Outros capítulos (poucos exemplos no treino)	0.82	22

Training

3 epochs, learning rate 3e-05, batch 32, sequences cut at 128 tokens, class-weighted loss, best epoch chosen by validation macro-F1. 3.6 minutes on one Tesla T4.

In the browser

onnx/model_quantized.onnx is an int8 export (dynamic quantization, one scale per output channel, chosen on validation over one per tensor) for transformers.js: 105 MB against 416 MB in fp32, and it agrees with the full-precision model on 99.3% of the test items: macro-F1 0.912 and accuracy 0.948.

Limitations

• Catalogue text only. Every training description is a CATMAT catalogue entry. Most goods are bought without one, described in the agency's own words, and the model will do worse on them than the test set says.
• Chapter level only, and only the 14 chapters with enough examples; everything else is outros.
• The catalogue's NCM mapping is taken as truth, including where it follows its own convention (printer ink under printers, chapter 84).