Enterprise Reflex V0
Model: yasserrmd/enterprise-reflex-v0
Base Model: answerdotai/ModernBERT-base
Language: English
Architecture: Dynamic enterprise action scorer
Status: Research Prototype / V0
Model Summary
Enterprise Reflex V0 is a lightweight enterprise decision model designed to operate as a fast System-1 layer ahead of larger reasoning models.
It does not generate long-form answers.
Instead, given:
request
+ enterprise state
+ context
+ runtime candidate actions
the model scores and ranks candidate actions.
It can also select:
NO_ACTION
when none of the available actions adequately satisfy the request.
Low-confidence or abstained decisions can then be handed to a larger System-2 reasoning model or a human.
Core Idea
Many enterprise requests do not require full LLM reasoning.
Enterprise Reflex explores the architecture:
+-------------------+
Request + State + Tools -> Enterprise Reflex |
+---------+---------+
|
Dynamic Action Ranking
|
+-------------+-------------+
| |
Confident Uncertain
| |
v v
Action System-2 LLM
|
deeper reasoning
The objective is not to replace an LLM.
The objective is to reserve expensive reasoning for cases that actually require it.
How to Use
Enterprise Reflex V0 is a pairwise action scorer.
It does not directly take a request and return a fixed class. Instead, each candidate action is scored against the request and enterprise state, then the candidate scores are ranked.
Install
pip install -U transformers torch
Load the Model
import json
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification
MODEL_ID = "yasserrmd/enterprise-reflex-v0"
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForSequenceClassification.from_pretrained(
MODEL_ID,
trust_remote_code=True
)
device = "cuda" if torch.cuda.is_available() else "cpu"
model = model.to(device)
model.eval()
Score a Request Against Candidate Actions
import numpy as np
TEMPERATURE = 1.55
SYSTEM2_THRESHOLD = 0.80
NO_ACTION = {
"name": "NO_ACTION",
"description": (
"None of the available actions safely or correctly satisfy "
"the request. Abstain and hand the request to a higher-level "
"reasoner or human."
)
}
def build_request_text(
request,
domain="enterprise",
state=None,
context=None
):
payload = {
"request": request,
"domain": domain,
"state": state or {},
"context": context or {}
}
return json.dumps(
payload,
ensure_ascii=False,
sort_keys=True
)
def build_action_text(action):
payload = {
"name": action["name"],
"description": action.get("description", ""),
"family": action.get("family", ""),
"domain": action.get("domain", "")
}
return json.dumps(
payload,
ensure_ascii=False,
sort_keys=True
)
@torch.no_grad()
def score_actions(
request,
candidate_actions,
domain="enterprise",
state=None,
context=None
):
actions = list(candidate_actions) + [NO_ACTION]
request_text = build_request_text(
request=request,
domain=domain,
state=state,
context=context
)
request_texts = [request_text] * len(actions)
action_texts = [
build_action_text(action)
for action in actions
]
encoded = tokenizer(
request_texts,
action_texts,
padding=True,
truncation=True,
max_length=384,
return_tensors="pt"
)
encoded = {
k: v.to(device)
for k, v in encoded.items()
}
outputs = model(**encoded)
logits = outputs.logits
# Pairwise compatibility margin
scores = (
logits[:, 1] - logits[:, 0]
).detach().cpu().numpy()
# Temperature calibration
scores = scores / TEMPERATURE
# Softmax across runtime candidate actions
scores = scores - np.max(scores)
probabilities = np.exp(scores)
probabilities = probabilities / probabilities.sum()
ranked = sorted(
[
{
"action": action["name"],
"description": action.get("description", ""),
"probability": float(prob)
}
for action, prob in zip(actions, probabilities)
],
key=lambda x: x["probability"],
reverse=True
)
return ranked
Make a Decision
def decide(
request,
candidate_actions,
domain="enterprise",
state=None,
context=None,
top_k=5
):
ranked = score_actions(
request=request,
candidate_actions=candidate_actions,
domain=domain,
state=state,
context=context
)
best = ranked[0]
system2_required = (
best["action"] == "NO_ACTION"
or best["probability"] < SYSTEM2_THRESHOLD
)
return {
"decision": best["action"],
"confidence": round(
best["probability"],
6
),
"system2_required": system2_required,
"ranked_actions": ranked[:top_k]
}
Example
candidate_actions = [
{
"name": "procurement.create_requisition",
"description": "Create a new procurement requisition."
},
{
"name": "procurement.create_purchase_order",
"description": (
"Create a purchase order from an approved requisition."
)
},
{
"name": "procurement.request_approval",
"description": (
"Send a procurement request for managerial approval."
)
},
{
"name": "procurement.update_requisition",
"description": (
"Modify an existing procurement requisition."
)
}
]
result = decide(
request=(
"The requisition has been approved. "
"Issue the purchase order to the selected supplier."
),
domain="Procurement",
state={
"approval_status": "approved",
"supplier_selected": True,
"requisition_exists": True
},
candidate_actions=candidate_actions
)
print(
json.dumps(
result,
indent=2,
ensure_ascii=False
)
)
Example output:
{
"decision": "procurement.create_purchase_order",
"confidence": 0.97,
"system2_required": false,
"ranked_actions": [
{
"action": "procurement.create_purchase_order",
"probability": 0.97
},
{
"action": "procurement.request_approval",
"probability": 0.02
}
]
}
Important Usage Notes
The model should be used as an action-ranking or routing layer, not as an authorization engine.
Recommended architecture:
Request
|
v
Enterprise Reflex
|
+---- high confidence ----> Policy Engine ----> Execute
|
+---- uncertain ----------> System-2 LLM
|
v
Policy Engine
|
v
Execute
The current research defaults are:
TEMPERATURE = 1.55
SYSTEM2_THRESHOLD = 0.80
MAX_LENGTH = 384
These values were calibrated for the V0 evaluation distribution and should be recalibrated for production workloads.
## What Makes It Different From a Fixed Classifier
Enterprise Reflex V0 does not use a fixed N-way action classification head.
Candidate actions are provided dynamically at inference time.
Each action contains information such as:
```json
{
"name": "procurement.create_purchase_order",
"description": "Create a purchase order from an approved requisition."
}
The model scores the compatibility between the request/state representation and each candidate action.
This allows the model to evaluate action descriptions that were not necessarily present as fixed labels during training.
Input
Typical inference input:
{
"request": "Issue the purchase order to the approved supplier.",
"domain": "procurement",
"state": {
"approval_status": "approved",
"supplier_selected": true
},
"candidate_actions": [
{
"name": "procurement.create_requisition",
"description": "Create a new procurement requisition."
},
{
"name": "procurement.create_purchase_order",
"description": "Create a purchase order from an approved requisition."
}
]
}
Output
Typical output:
{
"decision": "procurement.create_purchase_order",
"confidence": 0.96,
"system2_required": false,
"ranked_actions": [
{
"action": "procurement.create_purchase_order",
"probability": 0.96
},
{
"action": "procurement.create_requisition",
"probability": 0.03
},
{
"action": "NO_ACTION",
"probability": 0.01
}
]
}
System-2 Handoff
V0 uses calibrated confidence to decide whether the model should remain in System 1.
Current experimental default:
SYSTEM2_THRESHOLD = 0.80
Conceptually:
if predicted_action == "NO_ACTION":
system2_required = True
elif confidence < 0.80:
system2_required = True
else:
system2_required = False
The threshold is an experimental operating point and should be recalibrated for different domains and risk profiles.
Training
V0 was trained using the Enterprise Reflex pairwise dataset.
Training configuration:
Base model ModernBERT-base
Training pairs 448,596
Epochs 2
GPU NVIDIA A100 40GB
Training runtime approximately 50m 46s
Precision BF16
Max sequence length 384
Training objective:
(request + state + context, candidate action)
|
v
compatible / not
At runtime, multiple candidate scores are compared to form a ranked decision.
Pairwise Validation Performance
Final training evaluation:
Accuracy 97.81%
Precision 97.41%
Recall 83.80%
F1 90.10%
ROC AUC 99.07%
Average Precision 94.91%
These metrics measure pairwise request/action compatibility and should not be interpreted as end-to-end enterprise task accuracy.
Grouped Validation
When candidate actions compete within each task:
Top-1 Accuracy 98.46%
Top-3 Accuracy 99.95%
MRR 0.9920
NO_ACTION Precision 95.46%
NO_ACTION Recall 96.70%
NO_ACTION F1 96.08%
Validation groups:
5,977
Held-Out Test Performance
On the standard held-out grouped test set:
Top-1 Accuracy 98.27%
Top-3 Accuracy 99.97%
MRR 0.9909
NO_ACTION Precision 93.86%
NO_ACTION Recall 97.63%
NO_ACTION F1 95.70%
Test groups:
5,948
These results represent the distribution created by the V0 dataset construction pipeline.
They should not be interpreted as 98% accuracy on arbitrary real-world enterprise decisions.
Calibration
Temperature scaling was performed on validation decisions.
Temperature = 1.55
Current inference probabilities therefore use calibrated scores before softmax.
Selective Prediction
Validation analysis showed the following approximate trade-off:
| Threshold | Coverage | Accuracy on Accepted Decisions |
|---|---|---|
| 0.50 | 99.65% | 98.46% |
| 0.70 | 95.85% | 98.78% |
| 0.80 | 93.02% | 99.05% |
| 0.85 | 91.62% | 99.18% |
| 0.90 | 89.81% | 99.39% |
| 0.95 | 86.21% | 99.69% |
V0 currently uses 0.80 as the default research threshold.
These values are validation-distribution measurements and are not guarantees for production traffic.
Hard-Test Evaluation
Enterprise Reflex V0 was evaluated on a combined 110-case hard-test suite designed to be significantly more challenging than the standard grouped benchmark.
The suite covers:
- same-domain sibling actions
- state-sensitive decisions
- cross-domain semantic collisions
- unseen action names
NO_ACTION/ abstention- policy-sensitive scenarios
- ambiguous requests requiring System-2 escalation
Overall Results
Total Hard-Test Cases 110
Correct Decisions 88
Incorrect Decisions 22
Raw Top-1 Accuracy 80.00%
Performance by Category
| Category | Tests | Correct | Accuracy |
|---|---|---|---|
| Ambiguous | 4 | 4 | 100.00% |
| Cross-domain | 8 | 6 | 75.00% |
| NO_ACTION | 26 | 20 | 76.92% |
| Sibling action | 46 | 36 | 78.26% |
| State-sensitive | 22 | 18 | 81.82% |
| Unseen action name | 4 | 4 | 100.00% |
| Overall | 110 | 88 | 80.00% |
System-1 / System-2 Routing
Using:
SYSTEM2_THRESHOLD = 0.80
the combined hard-test suite produced:
System-1 Handled 58 / 110
System-1 Coverage 52.73%
System-2 Routed 52 / 110
System-2 Rate 47.27%
System-1 Accuracy 100.00%
System-1 Failures 0
Although the model made 22 incorrect top-ranked decisions across the complete hard-test suite, the confidence and abstention mechanism prevented those incorrect decisions from remaining eligible for direct System-1 execution.
The resulting selective behavior was:
Raw Hard-Test Accuracy 80.00%
System-1 Coverage 52.73%
System-1 Accuracy 100.00%
System-1 Failures 0
This indicates that V0 currently performs better as a selective enterprise decision layer than as an unrestricted autonomous action selector.
The model handles high-confidence decisions directly and routes uncertain, ambiguous, or low-confidence cases to a higher-level System-2 reasoner.
Interpretation
The combined hard evaluation shows promising performance in:
- dynamic action ranking
- confidence-based selective execution
- System-2 escalation
- unseen action-name generalization in the tested examples
- prevention of low-confidence incorrect actions from remaining in System 1
The main weaknesses remain:
- same-domain sibling-action discrimination
- structured state sensitivity
NO_ACTIONboundaries- policy-sensitive decisions
- semantic collisions between related enterprise domains
The primary objective for the next version is therefore:
Increase System-1 coverage
while maintaining very high selective accuracy
and near-zero incorrect System-1 decisions.
The 110-case hard-test suite is manually constructed and relatively small. These results should therefore be treated as an experimental stress-test baseline rather than evidence of production-level reliability.
Observed Strengths
Dynamic Action Scoring
Actions can be supplied at runtime rather than being restricted to a fixed class vocabulary.
Unseen Action Names
In the small hard evaluation, both explicitly tested unseen/renamed action cases were selected correctly.
This is promising evidence that the model uses semantic action descriptions.
It is not yet sufficient to claim broad zero-shot action generalization.
Abstention
The model supports explicit NO_ACTION rather than forcing a candidate selection.
Selective Execution
The confidence threshold successfully routed uncertain hard-test decisions to System 2.
In the current 57-case hard evaluation, no incorrect prediction remained eligible for direct System-1 execution at the 0.80 threshold.
Candidate Reduction for System 2
When escalation is required, a larger reasoner can receive only the most plausible candidate actions rather than the complete enterprise tool catalog.
Example:
Thousands of available tools
|
v
Enterprise Reflex
|
Top-3
|
v
System-2 LLM
This is one of the intended architectural benefits.
Known Weaknesses
Same-Domain Sibling Actions
Hard-test sibling-action accuracy is substantially lower than standard benchmark performance.
The model can confuse actions such as:
create_incident
create_problem
update_incident
close_incident
Future training should contain significantly more same-domain hard negatives.
State Sensitivity
The model does not yet reliably use subtle state differences.
For example, closely related workflow outcomes may remain nearly tied even when structured state should determine the correct action.
Future versions should include counterfactual state pairs where the request stays constant but state changes the correct decision.
NO_ACTION Calibration
V0 sometimes abstains on valid but weakly represented actions.
It can also fail to abstain when an enterprise policy or state constraint should prevent an action.
NO_ACTION therefore remains an important V1 research area.
Policy Constraints
The neural model should not be treated as a policy engine.
Examples involving:
- legal retention
- authorization
- financial approval
- account privileges
- security controls
should continue to be enforced by deterministic systems.
Domain Collisions
Semantically overlapping enterprise concepts remain challenging.
Examples include:
CRM support case
vs
ITSM incident
HR employee identity
vs
IAM account
Finance invoice
vs
Procurement purchase order
Intended Use
Enterprise Reflex V0 is intended for research and prototyping involving:
- enterprise agent routing
- dynamic tool selection
- API candidate ranking
- workflow action selection
- LLM pre-routing
- selective prediction
- System-1 / System-2 architectures
- agent cost and latency reduction experiments
- abstention research
Recommended Architecture
The model should be one component in a larger governed execution architecture:
User / Agent Request
|
v
Enterprise Reflex
|
+-------------------+
| |
high confidence uncertainty
| |
v v
Policy Engine System-2 LLM
| |
| Policy Engine
| |
+---------+---------+
|
v
Execution
|
v
Audit Evidence
The model recommends or ranks actions.
It should not replace:
- RBAC
- ABAC
- policy engines
- approval workflows
- transaction validation
- human authorization
- audit controls
Not Recommended For
V0 should not independently authorize:
- destructive system operations
- financial transfers
- employee termination
- security-sensitive changes
- legal or regulatory decisions
- critical infrastructure operations
- safety-critical actions
Risk-Aware Thresholds
Future versions may support risk-specific thresholds rather than a single global threshold.
For example:
Low-risk action >= 0.80
Medium-risk action >= 0.90
High-risk action >= 0.95 + deterministic policy
These values are architectural examples and have not yet been validated as production thresholds.
V1 Research Direction
The next version should primarily improve:
same-domain hard negatives
counterfactual state pairs
policy-sensitive examples
semantic domain collisions
NO_ACTION boundaries
hard-negative mining
The primary V1 objective is not simply higher raw Top-1 accuracy.
The target is:
increase System-1 coverage
while preserving very high
System-1 selective accuracy
Dataset
Training dataset:
yasserrmd/enterprise-reflex-dataset
Configurations:
canonical
pairs
Reproducibility
Important V0 inference parameters:
Base model answerdotai/ModernBERT-base
Temperature 1.55
System-2 threshold 0.80
Maximum sequence length 384
Version
Enterprise Reflex V0
September 2026
Citation
@misc{yasser2026enterprisereflex,
author = {Mohamed Yasser},
title = {Enterprise Reflex V0: A Lightweight System-1 Model for Dynamic Enterprise Action Selection},
year = {2026},
publisher = {Hugging Face}
}
Disclaimer
Enterprise Reflex V0 is an experimental research prototype.
Benchmark results measure the specific datasets and evaluations described above and should not be interpreted as guaranteed real-world enterprise performance, safety, reliability, or authorization correctness.
- Downloads last month
- 59
Model tree for yasserrmd/enterprise-reflex-v0
Base model
answerdotai/ModernBERT-base