ernestchu/jetlink-diorama

Jetlink Diorama models

The segmentation model Jetlink's Diorama runs on the comma's warped camera frames, exported to ONNX by python -m diorama export (see docs/diorama.md in the Jetlink repository). The Jetlink Mac and iPhone apps download this repo's diorama.json and the files it names.

File	Model	Source	License
diorama_seg.onnx	Mask2Former, ResNet-50, Mapillary Vistas semantic	Mask2Former model zoo, model id 49189528_1	Code MIT. Weights CC BY-NC 4.0; they were trained on Mapillary Vistas, whose terms apply to them too.
diorama_seg.mlpackage	the same model as a native CoreML program (fp16), which the apps download instead	as above	as above
diorama.json	the model's classes and camera, and its files	Jetlink	MIT
r50-semantic/mask2former/	the same weights converted to transformers (with a ResNet-50 backbone) by Jetlink's python -m diorama.convert_r50, which python -m diorama export takes	as above	as above
r50-semantic/	the same model's files, as first tried	as above	as above
diorama_dd3d.onnx, diorama_dd3d.mlpackage	DD3D, VoVNet-99, KITTI, from older exports; no longer run	TRI-ML/dd3d	MIT (Toyota Research Institute)

The model takes the packed 6×128×256 YUV frame as float16 and carries its post-processing inside. Earlier versions of this repo had Mask2Former Swin-L (Mapillary Vistas panoptic) here instead; its files are in the repo's history.