Fernandosr85/fatobr-cvm-entity-scoped-adapter

FatoBR Evidence Guard — CVM Entity-Scoped Filing Adapter

LoRA adapter for entity-scoped classification of Brazilian regulatory filings, fine-tuned on Llama-4-Scout-17B-16E-Instruct via Adaption's AutoScientist platform.

Given a filing (a CVM Fato Relevante or Comunicado ao Mercado) and a named target entity, the adapter emits four independent labels — EVENT, STATUS, SCOPE, SIGNAL — each drawn from a fixed vocabulary. The SCOPE field is the point of the task: it forces the model to classify with respect to the tagged entity, distinguishing an event about the target from an event about another company named in the same document.

---

Task

This is closed-label classification, not open generation. Each field has a fixed set of values; anything outside them is a label hallucination.

Field	Values
EVENT	EARNINGS, DIVIDEND, BUYBACK, MERGER_ACQUISITION, CAPITAL_RAISE, DEBT, REGULATORY, LEGAL, OPERATIONAL, MANAGEMENT, RESTRUCTURING, OTHER
STATUS	CONFIRMED, COMPLETED, PENDING_APPROVAL, UNDER_NEGOTIATION, RUMOR, DENIED, CANCELLED
SCOPE	TARGET_ENTITY, OTHER_ENTITY, SECTOR, MARKET_WIDE
SIGNAL	POSITIVE, NEGATIVE, NEUTRAL, MIXED, UNCERTAIN

Output format is four lines, one value each, no explanation:

EVENT=<value>
STATUS=<value>
SCOPE=<value>
SIGNAL=<value>

---

Usage

This is a LoRA adapter. Load it on top of the base model with PEFT. The prompt at inference must match the training template — a differently-formatted prompt degrades results on the same task.

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "meta-llama/Llama-4-Scout-17B-16E-Instruct"
tokenizer = AutoTokenizer.from_pretrained(base)
base_model = AutoModelForCausalLM.from_pretrained(
    base, torch_dtype=torch.float16, device_map="auto",
)
model = PeftModel.from_pretrained(
    base_model, "Fernandosr85/fatobr-cvm-entity-scoped-adapter",
)

prompt = """Classify the following Brazilian regulatory filing with respect to the target entity below.
Target entity: BRF S.A.

Text:
A BRF S.A. ("BRF" ou "Companhia") comunica ao mercado que recebeu notificacao da
BlackRock, Inc. informando alteracao de participacao acionaria, reduzindo sua
posicao para aproximadamente 3,4% das acoes ordinarias de emissao da Companhia.

Answer with EVENT, STATUS, SCOPE, and SIGNAL, one value each, in this exact format:
EVENT=<value>
STATUS=<value>
SCOPE=<value>
SIGNAL=<value>"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=64, do_sample=False)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

# Expected:
# EVENT=REGULATORY
# STATUS=CONFIRMED
# SCOPE=TARGET_ENTITY
# SIGNAL=NEUTRAL

Inference note. The 109B base does not fit on free-tier (T4/P100) GPUs; a larger-VRAM GPU is required to load it.

---

Evaluation results

Training win rates (Adaption internal)

Model	Win Rate (in-domain)	Win Rate (Corporate & Business)
Base (Llama-4-Scout-17B-16E-Instruct)	27%	43%
Adapted	73%	57%

A +46 point improvement in-domain. Unlike a typical task-specific adapter, the Corporate & Business reference category also improved (+14) rather than regressing — consistent with a closed-vocabulary task, where the win-rate judge has little room to reward style over correctness. No catastrophic forgetting was observed on the reference category.

Fase 1 diagnostic (simple prompt vs. scaffolded label)

There is no independent human-labeled reference in Portuguese for this task, so the table below is not an external-ground-truth baseline. It measures whether a simple prompt (label lists only, no per-value definitions) reproduces what a carefully-scaffolded prompt produces:

Field	Agreement, eval set	Agreement, OTHER_ENTITY slice (n=12)
EVENT	83.3%	90.9%
STATUS	76.2%	90.9%
SCOPE	95.2%	36.4%
SIGNAL	71.4%	45.5%

The eval-set SCOPE figure is dominated by TARGET_ENTITY rows and says little. On the genuine OTHER_ENTITY slice, SCOPE agreement drops to 36.4% — the unscaffolded model defaults to TARGET_ENTITY and attributes another party's event to the tagged entity roughly two times in three. Learning that entity-scope distinction is what this adapter is trained to do.

A holdout evaluation of this adapter against fatobr_sft_eval.jsonl (per-field accuracy, with the OTHER_ENTITY slice reported separately) is the comparable number and is pending; it requires loading the 109B base on adequate GPU. Fill this in before treating the adapter's accuracy as established.

Train/eval metrics

Metric	Value
Loss curve	decrease then plateau
Train vs validation	tracked closely, no overfit divergence
LR scheduler	cosine (warmup 0.05)
Gradient norm	initial spike, stable thereafter

---

Model details

Field	Value
Base model	meta-llama/Llama-4-Scout-17B-16E-Instruct (109B, 16-expert MoE)
Training method	Supervised Fine-Tuning (SFT) + LoRA
LoRA rank (r)	16
LoRA alpha	32
LoRA dropout	0
Trainable modules	k/o/q/v_proj + shared_expert (gate/up/down) + feed_forward (gate/up/down)
Epochs	1
Batch size	max
Learning rate	5e-5 (cosine, 0.5 cycles)
Warmup ratio	0.05
Weight decay	0.05
Max grad norm	1.0
Train on inputs	false

---

Training dataset

Fernandosr85/adaption-brazilian-regulatory-filings

~290 prompt/completion pairs, built from the CVM IPE dataset with a deliberate methodology:

Step	Detail
Source year	2023 (2022+ avoids the legacy OLE2 format in older filings)
Categories	Fato Relevante and Comunicado ao Mercado only
Text extraction	Per-page; pages below a character floor (chart/table-only) dropped
Excluded	Investor decks / earnings calls (multi-event, don't map to one label set)
Identity	Anchored on CNPJ (stable across renames), name read from the filing text
Second entity	Filings naming a genuine second entity labeled once per entity, to exercise SCOPE; service providers (auditors, advisory banks, rating agencies, the exchange/regulator) filtered out
Completions	Closed-taxonomy, not platform-regenerated
Split	Grouped by filer CNPJ, stratified by SIGNAL, best of N folds

Note on corporate renames. The CVM index reports a company's current legal name for a given CNPJ, even on documents filed years before a rename. Five real cases in this corpus (3R→Brava Energia, CTEEP→ISA Energia Brasil, Aliansce Sonae→Allos, Omega→Serena Energia, Via→Grupo Casas Bahia) required anchoring identity on CNPJ and reading the name from the filing text rather than trusting the index name.

---

Known limitations

• Completion regeneration. Adaptive Data regenerates completions on every run; a freeform Blueprint cannot enforce membership in a closed label set. This adapter was trained on the original (verbatim) completions, not the platform-enhanced ones, which invented values outside the taxonomy (observed EVENT/SCOPE fabrications such as Ownership Change, Divestiture, Entity-Specific in the Enhanced column). Downstream users of the platform on closed-label tasks should map the original completion column.
• Pilot scale. 290 filings; the genuine OTHER_ENTITY slice is small (12 rows), so every figure on that axis is directional, not conclusive.
• Model-generated reference labels. The training labels are produced by a language model against the closed taxonomy, traceable to a per-row justification, but not human-adjudicated.
• Holdout accuracy pending. The Adaption win rate is the platform's internal metric; a per-field holdout evaluation of this adapter is not yet run (see the diagnostic note above).

---

Model repositories

Platform	Link
Kaggle Model	fernandosr85/fatobr-evidence-guard-adapter
HuggingFace Dataset	Fernandosr85/adaption-brazilian-regulatory-filings
Kaggle Diagnostic	fatobr-cvm-entity-scoped-filings

---

Credits

• Fine-tuning platform: Adaption — AutoScientist & Adaptive Data
• Challenge: AutoScientist Challenge 2026 — Market-Analysis & News category
• Data source: CVM Dados Abertos — Cias Abertas: Documentos Periódicos e Eventuais (IPE) (public)
• Base model: meta-llama/Llama-4-Scout-17B-16E-Instruct (Llama 4 Community License)
• Author: Fernando Rodrigues · Kaggle: fernandosr85 · HuggingFace: Fernandosr85

---

Disclaimer

Experimental research artifact submitted to the AutoScientist Challenge 2026 (Market-Analysis & News category). Classification outputs are automated and require review before any operational use. Derived from public CVM data; the exact terms-of-use text was not independently confirmed at the time of writing and should be verified before redistribution.