SEC Restatement Arms
Nine fitted models over the 2021 window of
sec-restatement-8k-402:
three feature sets (numeric XBRL tag presence, words bag-of-words, embed sentence
embeddings) by three learners (random forest, histogram gradient boosting, L2 logistic
regression).
Each pickle carries the estimator and its feature map, because a model without its vocabulary is not reproducible.
These ship without a decision threshold, on purpose
The paper these come from measures what happens when you pick one badly. Selecting a threshold on the rows a model was fitted on inflates test cost by up to 5.5x, is governed by how completely the model memorises its training set rather than by model quality, and is therefore worst for the strongest model. Calibration cannot repair it: a monotone map cannot change which rows a threshold selects.
Shipping a fitted threshold would hand you that bug. Choose yours out of fold, or on a held-out block if your training set is small. Five extra fits, no held-out data, recovers a median 95% of the loss.
Do not expect these to beat a trivial baseline
Under an asymmetric cost with a miss worth ten false alarms, none of them beat flagging every
filing. That is the paper's finding, not a defect in the weights. Evaluate at a stated review
capacity with precision@k instead, which needs no cost ratio.
Code, results and the paper: https://github.com/hmmrlabs/cost-functions