acb-guard-session-graph-rf

A RandomForest classifier trained on 20-dimensional session graph features extracted from AI agent MCP payment sessions. The first application of session graph feature extraction to MCP commerce call sequences.

Key result: F1=0.64, FPR=0% on the AgentCommerceBench behavioral attack layer — trained on clean sessions only, with zero knowledge of specific attack patterns.

This is the behavioral detection baseline in the AgentCommerceBench pipeline, deployed as the L2 layer between the L1 keyword guard and L3 commerce policy enforcement.

Access

This model is gated. Request access above. We grant access for research, defensive security, and evaluation purposes.

What it detects

Autonomous AI agents operating on the Gordon MCP payment rail issue two types of calls:

• gordon_find_service — catalog lookup, discovers vendors
• gordon_authorize — payment authorization, transfers USDC on-chain

This model detects behavioral sequence attacks — attacks where every individual call looks legitimate, but the session-level pattern reveals compromise. These are invisible to keyword scanners and rate limiters.

Example — B3 Reconnaissance Sweep: An attacker-controlled agent issues 15 gordon_find_service probes before a single $0.50 USDC gordon_authorize to an attacker wallet. No injected text anywhere. The find_authorize_ratio feature reaches 15.0 vs. a clean-session mean of ~1.0 — 46 standard deviations above normal. Caught at probe 4.

The 20 Features

Each agent session is mapped to a 20-dimensional feature vector extracted from the MCP call sequence — no raw text, no logs, just the event structure:

Index	Feature	What it detects
0	n_events	Session length
1	n_find_service	Total catalog lookups
2	n_authorize	Total payment authorizations
3	n_settle	Settlement events
4	find_authorize_ratio	Recon signal — 15.0 in B3, ~1.0 in clean sessions
5	n_distinct_services	Catalog sweep — 15 unique vendors in B3, ~1 clean
6	service_diversity	Shannon entropy of vendor distribution
7	total_amount_usdc	Session spend total
8	max_amount_usdc	Largest single transaction
9	amount_std_usdc	Ratcheting signal (B2 — escalating transaction amounts)
10	amount_max_ratio	Max vs. session mean (spike detection)
11	timing_gap_mean_s	Average inter-event gap
12	timing_gap_std_s	Burst signal — high variance = irregular probing cadence
13	timing_gap_min_s	Minimum gap — floor of burst probing
14	n_categories	Number of distinct service categories used
15	dominant_category_frac	Scope violation signal — low purity = category drift
16	n_a2a_transfers	Agent-to-agent transfer count
17	a2a_amount_usdc	Total A2A transfer volume
18	has_override_keyword	L1 text signal — SYSTEM:, ignore previous instructions, etc.
19	has_b64_blob	L1 text signal — base64-encoded payload detection

Features 18–19 are L1 text signals included so the model can also flag payload injection attacks (A1–A7), making it a superset of pure keyword filtering for session-level scoring.

Benchmark Results

Evaluated on AgentCommerceBench — 15 attack scenarios across 3 layers:

Detector	F1	FPR	L1 TPR	L2 TPR	L3 TPR	Notes
velocity_check	0.12	0%	0%	14%	0%	Rate-limit only
keyword_filter	0.57	0%	100%	0%	0%	Blind to all behavioral attacks
isolation_forest	0.57	0%	20%	57%	100%	Unsupervised; misses payload attacks
llm_text_safety (Bedrock)	0.53	46%	100%	86%	100%	Unusable FPR in production
session_graph_rf (this)	0.64	0%	60%	57%	100%	Behavioral baseline — 0% FPR
gordon_+seq (full pipeline)	0.93	0%	100%	75%	100%	L1+L2+L3 combined

Real Gordon production sessions (held-out): 81% catch rate.

Blind red team (BT2 — novel behavioral sequences, no access to detector code): 6/6 caught. Both novel attacks activate find_authorize_ratio and amount_std features that rank highest in SGRF's Gini importances — the behavioral signal generalizes to unseen attack variants that share the same structural mechanism.

Training

• Data: 500 clean sessions calibrated from 503 real Gordon production transactions — no attack examples in training
• Method: The model learns what normal looks like per agent persona. Attack detection is anomaly scoring over clean-session distribution, not supervised classification over attack labels.
• Decision threshold: predict_proba[1] >= 0.70 → BLOCK
• Inference pattern: Re-scored after every MCP event (streaming), not once at session end

Usage

import joblib
import numpy as np

# Load weights
clf = joblib.load("session_graph_rf.joblib")

# Load feature extractor from this repo
from session_graph import session_to_features

# Score a session incrementally after each new MCP event
events = [...]  # list of Event objects (see harness/simulate/schema.py)

for i, event in enumerate(events):
    history  = events[:i]
    features = session_to_features(history + [event])  # shape (1, 20)
    proba    = clf.predict_proba(features)[0][1]        # P(fraud)
    decision = "BLOCK" if proba >= 0.70 else "ALLOW"
    print(f"Event {i+1}: P(fraud)={proba:.3f}  {decision}")

Feature vector format

from session_graph import session_to_features

# events: list of dicts or Event objects with fields:
#   action_type  — "FIND_SERVICE" or "AUTHORIZE"
#   amount_units — integer, USDC micro-units (1 USDC = 1,000,000)
#   vendor       — str, wallet address or service_id
#   original_request — dict, raw MCP payload (used for L1 text features)
#   timestamp    — datetime or float seconds

features = session_to_features(events)
# Returns: np.ndarray shape (1, 20), dtype float64
# Column order matches index 0-19 in the feature table above

Files in this repo

File	Description
session_graph_rf.joblib	Trained RandomForest weights (scikit-learn, joblib format)
session_graph.py	Feature extraction + detector class (session_to_features, SessionGraphModel)

Related

• Dataset: withgordon/agentcommercebench
• QLoRA model (graph-conditioned): withgordon/acb-guard-qwen25-7b-graph
• GitHub: BuildWithGordonAI/agentcommercebench

Citation

@software{agentcommercebench2026,
  title        = {AgentCommerceBench: A Benchmark for Fraud Detection in AI Agent Payment Systems},
  author       = {{Gordon AI}},
  year         = {2026},
  url          = {https://github.com/BuildWithGordonAI/agentcommercebench},
}