CARMA

CARMA is a PyTorch checkpoint for camera and radar BEV object detection on nuScenes. The model uses six surround cameras and radar sweeps at inference time, then predicts object centers and 3D boxes in a bird's eye view representation.

This repository currently contains the trained checkpoint:

CARMA_map0.4922.pth

The checkpoint is intended for research, dissertation experiments, qualitative visualization, and reproducibility of the CARMA detector. It is not a standalone Hugging Face Transformers model and is not configured for hosted Inference API deployment.

Model Overview

CARMA combines image and radar information in a BEV detection pipeline:

• Camera branch: six nuScenes RGB cameras are encoded and lifted into a BEV representation.
• Radar branch: radar points and sweeps are encoded into BEV features.
• Fusion module: modality projection, deformable alignment, bidirectional cross attention, and gated residual fusion combine camera and radar features.
• Detection head: a center based heatmap predicts object centers, while regression channels decode 3D box geometry, orientation, and velocity.
• Auxiliary training heads: LiDAR derived supervision is used during training for depth, camera BEV, and related auxiliary branches. These LiDAR derived targets are not required at inference time.

Final inference inputs:

• Six camera images.
• Camera intrinsics and extrinsics.
• Radar points or radar BEV tensors.

Not required at inference time:

• LiDAR point clouds.
• LiDAR occupancy maps.
• LiDAR derived depth supervision targets.

Classes

The checkpoint predicts the standard 10 nuScenes detection classes used by this project:

car, truck, construction_vehicle, bus, trailer,
barrier, motorcycle, bicycle, pedestrian, traffic_cone

Evaluation

The checkpoint was selected using the project validation metric at a 2 m center distance threshold.

Important note: the metrics below are from the CARMA project validation script. They are custom center distance metrics and should not be reported as the official nuScenes detection benchmark, NDS, or official nuScenes mAP unless the official nuScenes evaluation protocol is run separately.

nuScenes v1.0 trainval validation

Validation configuration:

• Dataset: nuScenes v1.0 trainval validation split.
• Decode setting: raw heatmap at score threshold 0.4.
• Main matching threshold: 2 m center distance.
• Selected checkpoint: CARMA_map0.4922.pth.

Metric	Value
Precision at 2 m	0.7970
Recall at 2 m	0.5736
F1 at 2 m	0.6671
mAP at 0.5 m	0.1892
mAP at 1 m	0.3573
mAP at 2 m	0.4922
mAP at 4 m	0.5548
Mean localization distance	0.526 m
True positives	89969
False positives	22909
False negatives	66877

Per Class Results

Class	AP	Precision	Recall	F1	Mean error
car	0.629	0.855	0.645	0.736	0.46 m
truck	0.517	0.672	0.598	0.633	0.67 m
construction_vehicle	0.389	0.653	0.444	0.529	0.83 m
bus	0.560	0.610	0.679	0.643	0.88 m
trailer	0.347	0.546	0.488	0.516	1.01 m
barrier	0.582	0.830	0.611	0.704	0.58 m
motorcycle	0.503	0.731	0.521	0.608	0.53 m
bicycle	0.453	0.745	0.471	0.577	0.51 m
pedestrian	0.364	0.779	0.386	0.516	0.54 m
traffic_cone	0.578	0.761	0.615	0.680	0.43 m

nuScenes Mini Validation Sanity Check

The same checkpoint was also evaluated on the nuScenes mini validation split as a quick sanity check.

Metric	Value
Precision at 2 m	0.7783
Recall at 2 m	0.5729
F1 at 2 m	0.6600
mAP at 0.5 m	0.2830
mAP at 1 m	0.4804
mAP at 2 m	0.5707
mAP at 4 m	0.6278
Mean localization distance	0.524 m

Usage

Install Git LFS before cloning the model repository:

git lfs install
git clone https://huggingface.co/AdrianNaziru/CARMA

The checkpoint can be inspected with PyTorch:

import torch

checkpoint = torch.load("CARMA/CARMA_map0.4922.pth", map_location="cpu")
print(checkpoint.keys() if isinstance(checkpoint, dict) else type(checkpoint))

To run inference, use the CARMA project code that defines the model architecture, preprocessing, nuScenes dataset loading, and decoding utilities. The checkpoint file alone is not sufficient without the matching source code and configuration.

Typical local commands from the CARMA source tree:

python realtime_bev_inference_pro_fast.py

Expected local data layout:

data/nuscenes/v1.0-trainval

The main runtime settings used for this checkpoint are:

NUSCENES_VERSION=v1.0-trainval
BEV grid=192 x 192
point cloud range=[-51.2, -51.2, -5.0, 51.2, 51.2, 3.0]
image size=256 x 704
radar sweeps=13
decode heat=raw
score threshold=0.4 for the reported validation metrics

Limitations

• This checkpoint is for research and academic analysis only.
• It is not validated for safety critical autonomous driving deployment.
• It depends on the exact CARMA source code, nuScenes preprocessing, calibration handling, and decoding logic used during training.
• Reported metrics are custom project metrics and are not official nuScenes leaderboard metrics.
• Performance may degrade outside the nuScenes sensor setup, camera calibration, weather, geography, or object distribution.
• LiDAR supervision improves training signals, but LiDAR is not used during final inference.

Citation

If you use this checkpoint, please cite the CARMA dissertation or the associated project repository when available.

License

This model repository is released under the Apache 2.0 license.