Train your own on typestrict.ai |
Model page |
Docs |
MCP for AI assistants |
All TS models
License: Apache 2.0 |
Authors: typestrict, trained with typestrict.ai
TS-PRGuard
A decision model trained on typestrict.ai that answers 5 questions in one pass, each with the probability of every option and a confidence, on a CPU.
Beats Jev on 4 of 5 questions, the market reference model, on held-out examples never used in training, by the refund calculation (a model of several questions has to win on two thirds of them). Anyone can recompute the verdict from the audit file.
- Typed decisions: each question is a
choice(one option out of many), anoul(is this statement true) or ascore(a value on an ordered rubric), with the probability of every option and a confidence. - Small and fast: 150M parameters, an 8-bit ONNX file that runs on a CPU, in a browser or on a phone.
- Measured, not claimed: every number below comes from examples held out before training, and the audit file recomputes the verdict.
📊 Results
Macro-F1 of this model and of Jev on the same held-out examples, and the gain the refund calculation measures (a weighted score of four metrics, with its 95% bootstrap interval). A question is won when the gain holds up under resampling (the interval stays above zero) and no metric collapses.
| Question | Held-out examples | This model | Jev | Gain over Jev (95% CI) | Result |
|---|---|---|---|---|---|
| Does the code change do what the pull request description says? | 436 | 0.928 | 0.807 | +0.109 [+0.078; +0.141] | Beats Jev |
| How hard is this change to review? | 221 | 0.660 | 0.383 | +0.196 [+0.131; +0.248] | Beats Jev |
| Does this change break compatibility for the users of this code? | 2,000 | 0.870 | 0.910 | −0.037 [−0.048; −0.026] | Jev ahead |
| Do the tests in this change cover the changed code? | 2,000 | 0.921 | 0.879 | +0.036 [+0.025; +0.047] | Beats Jev |
| Does this code change introduce a security vulnerability? | 597 | 0.904 | 0.755 | +0.139 [+0.108; +0.168] | Beats Jev |
| Question | Held-out examples | This model | Jev | Gain over Jev (95% CI) | Result |
|---|---|---|---|---|---|
| Does the code change do what the pull request description says? | 436 | 0.928 | 0.807 | +0.109 [+0.078; +0.141] | Beats Jev |
| How hard is this change to review? | 221 | 0.660 | 0.383 | +0.196 [+0.131; +0.248] | Beats Jev |
| Does this change break compatibility for the users of this code? | 2,000 | 0.870 | 0.910 | −0.037 [−0.048; −0.026] | Jev ahead |
| Do the tests in this change cover the changed code? | 2,000 | 0.921 | 0.879 | +0.036 [+0.025; +0.047] | Beats Jev |
| Does this code change introduce a security vulnerability? | 597 | 0.904 | 0.755 | +0.139 [+0.108; +0.168] | Beats Jev |
❓ The questions
| Question | Type | Options |
|---|---|---|
| Does the code change do what the pull request description says? | noul | No, the diff does something different from the description. Yes, the diff implements what the description says. |
| How hard is this change to review? | choice | Trivial: a few lines in one place Small: a short, focused change Medium: several files or some new branching logic Large: many lines or files, or a lot of new branching Very large: hundreds of lines across many files with much new logic |
| Does this change break compatibility for the users of this code? | noul | No, existing users keep working without changes. Yes, existing users must change their code or configuration. |
| Do the tests in this change cover the changed code? | noul | No, the change needs tests and the ones included do not cover it. Yes, the tests in the change exercise the changed code. |
| Does this code change introduce a security vulnerability? | noul | No, the change does not introduce a security vulnerability. Yes, the change introduces a security vulnerability. |
| Question | Type | Options |
|---|---|---|
| Does the code change do what the pull request description says? | noul | No, the diff does something different from the description. Yes, the diff implements what the description says. |
| How hard is this change to review? | choice | Trivial: a few lines in one place Small: a short, focused change Medium: several files or some new branching logic Large: many lines or files, or a lot of new branching Very large: hundreds of lines across many files with much new logic |
| Does this change break compatibility for the users of this code? | noul | No, existing users keep working without changes. Yes, existing users must change their code or configuration. |
| Do the tests in this change cover the changed code? | noul | No, the change needs tests and the ones included do not cover it. Yes, the tests in the change exercise the changed code. |
| Does this code change introduce a security vulnerability? | noul | No, the change does not introduce a security vulnerability. Yes, the change introduces a security vulnerability. |
🏃 Use it
decision_ai.json describes the package: the files, the inputs and outputs of the ONNX model, and each question with its options (tasks).
pip install huggingface_hub
huggingface-cli download typestrict/TS-PRGuard --local-dir ts-prguard
- Flutter (phone, browser): the decision_ai package reads this package as it is.
- Python, Node, any runtime: ONNX Runtime with the inputs listed in
decision_ai.json. - Try it first: the playground runs it in your browser.
📚 Training data
124,194 labeled examples; 10,389 of them held out from training to compare with Jev. The questions, the examples and the training run are on the model page.
Sources and their licenses, as given by the owner:
Built from the TS-PRGuard dataset (labels under CC BY 4.0; every diff under its repository's license, kept with its repo, ref and license), plus more test-coverage examples from SWE-rebench-V2 and more pull requests labeled as breaking. Commercial use is allowed by every source.
| Source | License | Questions |
|---|---|---|
| Git histories of permissively licensed repositories (MIT, Apache-2.0, BSD and similar) | Each repository's plexity, tests, vulnerability | |
| bigcode/commitpack | MIT(code under each repository's license) | description, complexity |
Pull requests labeled as brter pull requests labeled c:API break |
Each repository's license (Flutter: BSD-3-Clause) | breaking change |
| nebius/SWE-rebench-V2 | CC BY 4.0 | tests |
| GitHub Advisory Database | CC BY 4.0 | vulnerability |
| [MoreFixes](https://zenodo. BY 4.0 | vulnerability |
🚀 Train your own on typestrict.ai
This model started from TS-Ettin-150M and learned its questions from its owner's data on typestrict.ai. Yours can too: send your data, review what the agent proposes, and get a model measured against Jev on your held-out examples. If it does not beat Jev, the credit comes back.
From your AI assistant (ChatGPT, Claude, Cursor, Codex, Copilot and others), with the remote MCP server:
claude mcp add --transport http typestrict https://api.typestrict.ai/mcp
From a terminal or CI, with the typestrict CLI:
curl -fsSL https://typestrict.ai/install.sh | sh && typestrict login
🗒️ Model details
| Property | TS-PRGuard |
|---|---|
| Base model | TS-Ettin-150M (Apache-2.0) |
| Parameters | 150M |
| Languages | EN |
| Questions | 5 (noul, choice) |
| Files | decision_ai.json, tokenizer.json, onnx/model_8bit.onnx (8-bit ONNX) |
| Runs on | CPU, in a browser or on a phone (ONNX Runtime) |
| Version | v1 on typestrict.ai |
| License | Apache-2.0 |
| Property | TS-PRGuard |
|---|---|
| Base model | TS-Ettin-150M (Apache-2.0) |
| Parameters | 150M |
| Languages | EN |
| Questions | 5 (noul, choice) |
| Files | decision_ai.json, tokenizer.json, onnx/model_8bit.onnx (8-bit ONNX) |
| Runs on | CPU, in a browser or on a phone (ONNX Runtime) |
| Version | v1 on typestrict.ai |
| License | Apache-2.0 |
Links
- Model page: typestrict.ai/models/01a19e01-da22-729e-8d6c-4869efc4691e, with the data, the training run, the audit file and the playground
- typestrict.ai: typestrict.ai, the docs at docs.typestrict.ai
- Open source: CLI and MCP server, decision_ai for Flutter
Model tree for typestrict/TS-PRGuard
Base model
jhu-clsp/ettin-encoder-150m