BJyotibrat/BanTRel-v1.0.0

BanTRel (Bangalore Traffic Reinforcement Learning)

A Proximal Policy Optimization (PPO) agent trained to adaptively control a 4-way traffic signal intersection under simulated Bangalore traffic conditions, benchmarked against a fixed-cycle baseline using SUMO.

Model Details

Model Description

BanTRel is a reinforcement learning agent that learns to control a single 4-way traffic intersection (12 turning movements, 2 lanes per approach) to minimize vehicle waiting time and delay. It uses a shared-backbone actor-critic network trained with PPO (clipped surrogate objective + Generalized Advantage Estimation) inside a custom SUMO (Simulation of Urban MObility) environment modeling realistic Bangalore traffic volumes — 16,400 vehicles/day across night, morning rush, mid-morning, afternoon, evening rush, and evening periods, with a 60% car / 30% motorcycle / 10% auto-rickshaw vehicle mix.

The agent selects between 2 discrete actions (NS-green / EW-green phase) based on an 8-dimensional state vector, and was evaluated against a fixed-cycle baseline policy across 9 traffic performance metrics.

• Developed by: Bindupautra Jyotibrat (BJyotibrat)
• Model type: Reinforcement learning — Proximal Policy Optimization (PPO), actor-critic, discrete action space
• License: Apache 2.0
• Trained from scratch: Yes (no pretrained base model)

Model Sources

• Repository: https://github.com/Jyotibrat/BanTRel

Uses

Direct Use

The model can be used directly for inference in a SUMO-simulated 4-way intersection matching the training topology (8-dim state, 2 discrete actions), to select traffic signal phases at each control step. It is intended for research and educational use in traffic signal control and reinforcement learning, not for deployment on real, physical traffic infrastructure.

Out-of-Scope Use

• Not validated on multi-intersection networks, different intersection geometries, or real-world traffic sensor data.
• Not suitable for direct deployment to physical traffic signal hardware without further validation, safety testing, and regulatory review.
• Not intended for any safety-critical decision-making outside of simulation.

Bias, Risks, and Limitations

• Trained and evaluated on a single synthetic 4-way intersection topology; performance on different intersection layouts or real-world networks is untested.
• Traffic demand is derived from aggregate daily volume estimates for Bangalore rather than granular real-world traffic count data, so learned behavior reflects the assumptions of this synthetic schedule.
• As with any RL policy trained in simulation, there is a sim-to-real gap — real-world deployment would require retraining or fine-tuning against real sensor data and safety constraints.

Recommendations

Users should treat this as a research artifact demonstrating RL-based traffic signal control, not a production-ready traffic control system. Any real-world application would need domain adaptation, safety validation, and traffic engineering sign-off.

How to Get Started with the Model

import torch
import torch.nn as nn

class ActorCritic(nn.Module):
    def __init__(self, state_dim=8, action_dim=2, hidden=64):
        super().__init__()
        self.backbone = nn.Sequential(
            nn.Linear(state_dim, hidden), nn.Tanh(),
            nn.Linear(hidden, hidden), nn.Tanh(),
        )
        self.actor = nn.Linear(hidden, action_dim)
        self.critic = nn.Linear(hidden, 1)

    def forward(self, x):
        features = self.backbone(x)
        return self.actor(features), self.critic(features)

model = ActorCritic()
model.load_state_dict(
    torch.load("Model/Phase 2/PyTorch Model/ppo_bangalore.pt", map_location="cpu")
)
model.eval()

state = torch.randn(1, 8)  # replace with a real 8-dim state from your SUMO env
with torch.no_grad():
    logits, value = model(state)
    action = torch.argmax(logits, dim=-1).item()  # 0 = NS-green, 1 = EW-green

Or, without needing the class definition, using the TorchScript export:

import torch
model = torch.jit.load("Model/Phase 2/PyTorch Model/ppo_bangalore_scripted.pt")
model.eval()

Or, you can use the onnx model:

import onnxruntime as ort
import numpy as np

session = ort.InferenceSession("ppo_bangalore.onnx")
state = np.random.randn(1, 8).astype(np.float32)
logits, value = session.run(None, {"state": state})
action = int(np.argmax(logits, axis=-1)[0])

Training Details

Training Data

No external dataset — training data is generated on-the-fly through interaction with a custom SUMO simulation environment. Traffic demand is derived from observed Bangalore intersection vehicle counts (16,400 vehicles/day), converted into SUMO flow definitions across 7 time-of-day periods.

Training Procedure

Two-phase on-policy training: 100 episodes of initial training (Phase 1), followed by 50 episodes of fine-tuning from the best Phase 1 checkpoint (Phase 2), for 150 episodes total. Rollouts of 1,024 steps were collected before each PPO update, with 8 epochs of mini-batch (256) updates per rollout. The learning rate was linearly annealed from 3e-4 to 3e-5 over training. Rewards (negative sum of edge waiting times) were normalized by dividing by 1,000 and clipping to [-10, 0] for training stability.

Training Hyperparameters

Hyperparameter	Value
Algorithm	PPO (clipped surrogate + GAE)
Discount factor (gamma)	0.95
GAE lambda	0.95
Clip epsilon	0.2
Learning rate	3e-4 → 3e-5 (linear anneal)
Optimizer	Adam (eps=1e-5)
Epochs per update	8
Rollout length	1,024 steps
Mini-batch size	256
Entropy coefficient	0.05
Value function coefficient	0.5
Gradient clip norm	0.5
Weight initialization	Orthogonal (gain=√2 backbone, 0.01 actor, 1.0 critic)
Training regime	fp32

Speeds, Sizes, Times

• Total training episodes: 150 (100 Phase 1 + 50 Phase 2)
• Checkpoint size (state_dict): See Model/Phase 2/PyTorch Model/ppo_bangalore.pt in the repository

Evaluation

Testing Data, Factors & Metrics

Testing Data

Evaluation was run in the same custom SUMO environment as training, over the full 24-hour simulated traffic schedule, comparing the trained PPO policy against a Fixed-Cycle baseline policy.

Metrics

Nine metrics were evaluated, following the 2WSI-RL benchmark protocol plus additional traffic-flow metrics: total stopped vehicles, total waiting time, mean waiting time, mean speed, average waiting time, average travel time, queue length, throughput, and delay vs. free-flow. A tie threshold was applied per metric so that negligible differences aren't misattributed as a win for either policy.

Results

See the full results table and per-metric figures in the GitHub repository (Results/final_results_table.txt, Results/Phase 2/).

Summary

PPO was compared against a Fixed-Cycle baseline across all 9 metrics; see the repository results table for the win/loss/tie breakdown and per-metric mean values.

Environmental Impact

Training was run entirely on CPU (Google Colab, CPU-only runtime), not GPU/TPU. Standard carbon-estimation tools such as the ML CO2 Impact calculator are built around GPU/TPU instance types and don't currently offer a CPU-only Colab option, so a standardized emissions estimate isn't available for this run. CPU-only training draws substantially less power than GPU/TPU training, so the impact is expected to be comparatively low, but no formal figure is reported here.

• Hardware Type: CPU (Google Colab, CPU-only runtime)
• Cloud Provider: Google Colab
• Carbon Emitted: Not formally estimated — see note above

Technical Specifications

Model Architecture and Objective

Shared-backbone actor-critic network: Linear(8, 64) → Tanh → Linear(64, 64) → Tanh, with a separate actor head (Linear(64, 2)) producing action logits and a critic head (Linear(64, 1)) producing the state-value estimate. Trained with the PPO clipped surrogate objective combined with a value function loss and an entropy bonus for exploration.

Compute Infrastructure

Hardware

CPU only (Google Colab CPU runtime) — no GPU/TPU used.

Software

• Python 3.12.13
• PyTorch 2.11.0+cpu
• SUMO (Simulation of Urban MObility)
• traci, sumolib

Citation

If you use this model, please cite the GitHub repository:

@misc{bantrel2026,
  author = {Bindupautra Jyotibrat},
  title = {BanTRel: PPO-based Adaptive Traffic Signal Control for Bangalore Traffic},
  year = {2026},
  publisher = {GitHub},
  howpublished = {\url{https://github.com/Jyotibrat/BanTRel}}
}

Model Card Contact

• Email: bjyotibrat@gmail.com
• GitHub: https://github.com/Jyotibrat