dlrv-ppointnet-exp6

PointNet++ 6DoF object pose estimator for the fifteen LineMOD objects, trained on masked RGB-D point clouds. This is the exp6 checkpoint, the variant trained on a diameter-normalised ADD objective. It is the most accurate configuration on synthetic validation and the most evenly balanced across object classes.

Produced for a Deep Learning for Robot Vision project at Hochschule Bonn-Rhein-Sieg.

The idea behind it

The ADD benchmark accepts a pose when the average model point distance falls below 0.1 x the object's diameter. The threshold therefore scales with the object, but a loss computed on absolute ADD distance does not. Training on raw distance consequently favours physically large objects, whose thresholds are the most forgiving: under that objective the lamp reaches 92.5% and the ape 7.0%.

Dividing the loss by the object diameter aligns the objective's scale with the metric's. Across the twelve asymmetric objects this leaves mean accuracy essentially unchanged while cutting per-class standard deviation from 30.4 to 6.3 percentage points, a factor of 4.8, and lifting the weakest class from 7.0% to 45.2%.

The mechanism is confirmed by a counterintuitive observation: this model has a higher mean absolute ADD error than the unnormalised one (69 mm against 34.6 mm) while being more accurate. That is only possible if error moved from objects with tight thresholds to objects with loose ones.

Architecture

Backbone	PointNetPlusPlus, SA1 to SA3, 1024 input points
Head	PoseEstimationHead, 512-d feature, separate rotation and translation branches
Parameters	0.86 M
Input	1024 points, metres, centroid-centred

Rotation is regressed in the continuous 6D representation of Zhou et al.; translation as a residual from the observed point cloud centroid.

Results

Trained on the synthetic PBR split only, with no domain adaptation.

Split	ADD(-S)@0.05d	ADD(-S)@0.1d	ADD(-S)@0.2d	Mean	Median
Synthetic validation		58.41%
Real BOP test, 3,000 images	6.37%	33.33%	57.33%	45.99 mm	30.13 mm

Read this before choosing between exp6 and exp5

Normalisation wins on synthetic data by 1.05 points and loses on real data by 2.90 points. Directing capacity toward small objects helps in distribution, but small objects are exactly the ones whose real depth observations are sparsest, so the gain does not survive the domain shift.

Reducing per-class variance in distribution is not the same thing as robustness. If you want the better model on real images, use jan024/dlrv-pointnet-exp5 at 36.23%. Use this one if you care about balanced per-class behaviour on data that matches the training distribution.

Training

--experiment exp6_normadd --loss_type normalized_add --epochs 30
--batch_size 32 --lr 0.001 --backbone_lr 0.0005

The backbone learning rate matches exp5 so that the objective is the only variable between the two.

Usage

The checkpoint is a dict with keys epoch, val_add, backbone and pose_head. Load it with the model definitions from the project repository:

import torch
from pointnet.model import PointNetPlusPlus
from pose.head import PoseEstimationHead

ckpt = torch.load('exp6_normadd_best.pth', map_location='cuda')

backbone = PointNetPlusPlus(n_classes=15, n_points=1024).cuda()
backbone.load_state_dict(ckpt['backbone'])

pose_head = PoseEstimationHead(feature_dim=512).cuda()
pose_head.load_state_dict(ckpt['pose_head'])

To reproduce the 33.33% figure end to end:

python -u src/pose/eval_test_set.py \
    --checkpoint exp6_normadd_best.pth \
    --data_root /path/to/lm \
    --output_json results.json

Roughly two minutes on an RTX A5000. LineMOD in BOP format is required and is not distributed here; get it from https://bop.felk.cvut.cz/datasets/.

Limitations

• Trained on synthetic data only, and loses more to the domain shift than the surrogate-trained exp5 does.
• Requires a ground-truth object mask. No detection or segmentation stage is included.
• Evaluation subsamples the point cloud and the CAD vertices without a seed, so per-object figures move by a few points between runs.