jarrarx/nyc311-72h-shadow

NYC 311 72-hour closure prediction — shadow demo

Operational status: HOLD_FOR_OPERATIONAL_USE. This is a learning and shadow-evaluation model. It must not determine who receives city services or replace human review.

What it predicts

At request creation, the model estimates the probability that a NYC 311 service request will not be closed within 72 actual hours. Label 1 means late; label 0 means closed within 72 hours. The classroom decision threshold is 0.20.

Inputs are complaint_type, borough, and month, hour, weekday derived from the NYC-local created_date. closed_date, status, and resolution fields are never model inputs.

Model and data

The artifact is a scikit-learn histogram gradient boosting pipeline. It was selected on January–February 2026 validation and refit on 2025 training plus that validation. Its final evaluation used March–April 2026 requests. The source is NYC Open Data 311 Service Requests from 2020 to Present; the course saved immutable daily snapshots and split manifests in Google Drive.

• Model ID: 20260926T143549799163Z
• Split ID: 20260926T142339294958Z
• Experiment ID: 20260926T143147282446Z
• Decision ID: 20260926T143947829974Z
• Final model SHA-256: af4151fbde0a19001bd38f7189cbb2e5448f8c87efe2e719d337d98167c95611
• Python: 3.13.15; pandas 2.2.3; NumPy 2.1.3; scikit-learn 1.6.1; joblib 1.6.0

Final test results

On 64,751 March–April 2026 requests: accuracy 0.845, precision for late cases 0.672, recall for late cases 0.871, F1 0.759, F2 0.823. The model found 15,773 late cases, missed 2,327, and produced 7,709 false alarms.

Critical limitation: For HEAT/HOT WATER, it flagged zero of 4,887 requests as late and missed all 552 late cases. Snow or Ice had only two late test cases, so its test recall is too uncertain for a firm conclusion. The saved release decision is HOLD_FOR_OPERATIONAL_USE. Overall recall does not remove this subgroup failure.

The test period has already been used and must not be reused to select a revised model. A future candidate needs a later untouched test period and a new release decision.

Loading

Install the versions in requirements.txt under Python 3.13.15. Verify model.joblib against the SHA-256 above before loading. joblib deserialization can execute code; load only this trusted, verified artifact.

import hashlib, joblib, pandas as pd
assert hashlib.sha256(open("model.joblib", "rb").read()).hexdigest() == "af4151fbde0a19001bd38f7189cbb2e5448f8c87efe2e719d337d98167c95611"
model = joblib.load("model.joblib")
row = pd.DataFrame([{"complaint_type": "HEAT/HOT WATER", "borough": "BRONX",
                     "month": 9, "hour": 10, "weekday": 5}])
probability_late = float(model.predict_proba(row)[0, 1])

This example is a smoke test, not a reliable HEAT/HOT WATER decision. The evidence/ folder contains the locked plan, final test report, and release decision. No raw request records are included.