YellowCab

YellowCab is a compact multimodal model for forecasting a taxi's observed maneuver approximately 20 seconds ahead. It combines three chronological road images with twelve past-and-current telemetry signals and returns calibrated probabilities for:

• continue
• slow
• stop
• turn_left
• turn_right

The model has 5.29 million parameters. It is intended for fleet-video indexing, maneuver analytics, multimodal research, and prototyping—not vehicle control.

What is technically useful about it?

YellowCab tests a practical hypothesis: recent visual context and vehicle motion history are more useful together than either signal alone.

On the route-held-out silver benchmark, the causal temporal-fusion checkpoint reaches 0.3916 macro F1, compared with 0.3414 for the strongest tested past-only telemetry baseline and 0.3050 for the tested current-frame vision baseline. This supports uses such as:

• assigning likely maneuver tags to fleet footage;
• retrieving candidate stop and turn events for human review;
• prioritizing uncertain sequences for data collection or annotation;
• studying disagreement between visual and motion signals; and
• prototyping calibrated, abstaining multimodal classifiers.

These results do not establish a global rank. YellowCab uses a private, task-specific benchmark and has not been evaluated under a directly comparable public autonomous-driving leaderboard protocol.

Architecture

• Frozen ImageNet-pretrained EfficientNet-B0 visual encoder
• Three frames at nominal offsets t-20 s, t-10 s, and t
• Layer normalization followed by a one-layer unidirectional GRU
• Telemetry MLP over twelve normalized causal features
• Fusion classifier over five maneuver classes
• Temperature scaling and optional confidence-based abstention

The repository contains the exact architecture, configuration, integrity-checked Safetensors checkpoint, and custom Hugging Face endpoint handler.

Evaluation

Protocol

• 74,680 examples across 36 routes
• 55,711 training examples from 26 routes
• 10,375 validation examples from 5 routes
• 8,594 test examples from the newest 5 completely held-out routes
• Complete forward-in-time route separation
• No frame from a validation or test route enters training
• Model inputs use only information available at or before the newest frame
• Labels are GPS-derived maneuver proxies approximately 20 seconds ahead
• Primary metric: macro F1
• 95% confidence interval: 500 route-clustered bootstrap samples

Headline results

Metric	YellowCab
Macro F1	0.3916
Route-bootstrap 95% CI	0.3476–0.4281
Balanced accuracy	0.3916
Accuracy	0.5755
Weighted F1	0.5807
Log loss	1.0841
Brier score	0.5563
ECE, 15 bins	0.0118

Raw accuracy is not the primary metric because continue is the majority class. A majority classifier reaches 0.6232 accuracy while scoring only 0.1536 macro F1.

Baselines

Model	Macro F1	Balanced accuracy	Accuracy
YellowCab: temporal vision + telemetry	0.3916	0.3916	0.5755
Telemetry gradient boosting	0.3414	0.4217	0.4147
Current-frame vision linear model	0.3050	0.2968	0.5752
Past-motion rule	0.2235	0.2238	0.4550
Telemetry logistic regression	0.1887	0.2872	0.1963
Majority class	0.1536	0.2000	0.6232

YellowCab improves macro F1 over every tested baseline, but the gradient-boosted telemetry baseline has higher balanced accuracy. We report both rather than claiming uniform dominance.

Per-class results

Class	Precision	Recall	F1	Support
Continue	0.7595	0.7394	0.7493	5,356
Slow	0.4031	0.3544	0.3772	951
Stop	0.3754	0.3423	0.3581	634
Turn left	0.2447	0.3157	0.2757	833
Turn right	0.1897	0.2061	0.1975	820

Turn performance is the clearest weakness. YellowCab should be used to produce ranked candidates or analytics, not treated as a reliable maneuver oracle.

Confidence-based abstention

The default confidence threshold is 0.45. On this held-out set it retains 69.5% of examples, with 0.6583 selective accuracy and 0.4327 selective macro F1 among retained examples. Raising the threshold trades coverage for higher accuracy:

Threshold	Coverage	Selective accuracy	Selective macro F1
0.00	100.0%	0.5755	0.3916
0.35	91.0%	0.6007	0.4061
0.45	69.5%	0.6583	0.4327
0.55	47.2%	0.7324	0.4702
0.65	30.2%	0.8092	0.5275

This makes the model more useful for candidate retrieval and review queues, but these thresholds must be recalibrated after domain shift. Full results are in selective_metrics.csv.

The complete checkpoint-specific report is in evaluation.json. Recomputable, privacy-sanitized held-out probabilities are in eval_predictions.csv, and the metric script is in recompute_metrics.py. Additional methodology and limitations are documented in EVALUATION.md.

Checkpoint identity

• Version: v0.1-causal
• Published checkpoint SHA-256: af1a98cd96bb68d642f7a0adeedfacac37b230917afc15f3c87230a99a15b8a2
• Training manifest SHA-256: 8b92ad563892776cc25148f39fec6bf207e1e442c618b58fb07af7e2b899364d
• Split SHA-256: bfc5e960c8af32d0ea75a5e293bb95b3125111ace2cf8452eb42feb4bc2a1a5e
• Training seed: 20260726

The causal-manifest audit replaced unavailable current headings with the neutral encoding (sin=0, cos=1) rather than deriving them from future motion. The published checkpoint was trained and evaluated on that corrected manifest.

Input

Requests require exactly three base64-encoded JPEG or PNG frames ordered oldest to newest. Optional frame timestamps must be strictly increasing.

All telemetry fields are required:

current_speed_mps, past_speed_mps, acceleration_mps2, past_turn_degrees, past_yaw_rate_deg_s, heading_sin, heading_cos, gps_accuracy_m, hour_sin, hour_cos, dow_sin, and dow_cos.

{
  "inputs": {
    "frames": ["<base64 t-20>", "<base64 t-10>", "<base64 now>"],
    "frame_timestamps": [1721990000.0, 1721990010.2, 1721990020.1],
    "telemetry": {
      "current_speed_mps": 7.2,
      "past_speed_mps": 8.1,
      "acceleration_mps2": -0.045,
      "past_turn_degrees": 3.1,
      "past_yaw_rate_deg_s": 0.15,
      "heading_sin": 0.5,
      "heading_cos": 0.8660254038,
      "gps_accuracy_m": 4.0,
      "hour_sin": -0.7071067812,
      "hour_cos": 0.7071067812,
      "dow_sin": 0.7818314825,
      "dow_cos": 0.6234898019
    },
    "abstention_threshold": 0.45
  }
}

The custom EndpointHandler validates the request and returns the predicted label, ordered class probabilities, confidence, and abstention state. This custom multimodal architecture is not compatible with the standard image-classification pipeline() or Hub widget.

Local use

python -m venv .venv
# Activate the environment for your shell.
python -m pip install --requirement requirements.txt

from handler import EndpointHandler

predict = EndpointHandler(".")
result = predict(request_body)

Training data and privacy

The model was trained on privacy-redacted imagery collected from the operator's taxi fleet under contributor agreements authorizing model training. No source imagery, exact GPS records, route manifests, or personal identifiers are distributed in this repository.

The published evaluation predictions use sequential example identifiers and generic held-out route groups. Capture identifiers, timestamps, grid cells, and locations have been removed.

Intended uses

• Offline fleet-video indexing and search
• Maneuver analytics and aggregate research
• Candidate-event retrieval followed by human review
• Multimodal classification and calibration research
• Prototyping and collection-planning experiments

Out-of-scope uses

Do not use YellowCab:

• as the sole input to steering, braking, throttle, or dispatch decisions;
• as a safety-certified driver-assistance component;
• to identify drivers, passengers, pedestrians, or locations;
• to infer intent, fault, liability, or legal compliance; or
• outside a target domain without measuring performance and calibration again.

Limitations

• Ground truth is GPS-derived silver labeling, not independent human annotation.
• The evaluation covers five held-out routes from one operating domain.
• Geography, weather, camera placement, traffic, and telemetry quality can shift performance.
• Rare classes are substantially weaker than continue.
• Temperature scaling used the validation split rather than a separate calibration split.
• No membership-inference, model-inversion, or differential-privacy assessment has been completed.
• Benchmark results are author-reported and have not been independently replicated on a public dataset.

License

Unless otherwise noted, the model weights and original code, configuration, and documentation are licensed under the Apache License 2.0. Third-party components remain subject to their respective terms; see THIRD_PARTY_NOTICES.md.

Citation

@software{yellowcab_v0_1_causal_2026,
  author = {{The General Data Corporation}},
  title = {YellowCab: Multimodal Taxi Maneuver Forecasting},
  year = {2026},
  version = {v0.1-causal},
  url = {https://huggingface.co/generaldata/YellowCab}
}