MCAP: A Next-Generation File Format for ROS Recording
John Hurliman
john@foxglove.dev

How can we make rosbag recordings
more crash resilient and improve write
throughput?

How can we make rosbag recordings
more crash resilient and improve write
throughput?

How can we leverage recording SDKs
and bag tooling outside the ROS
ecosystem?

How can we make rosbag recordings
more crash resilient and improve write
throughput?

How can we leverage recording SDKs
and bag tooling outside the ROS
ecosystem?

How can we make recordings selfcontained and remotely streamable?

Design Goals

Design Goals◻ Serialization AgnosticDesign Goals◻ Serialization Agnostic◻ Determinism

Design Goals◻ Serialization Agnostic◻ Determinism◻ Self-Describing

Design Goals◻ Serialization Agnostic◻ Determinism◻ Self-Describing◻ Big and Small Data

Design Goals◻ Serialization Agnostic◻ Determinism◻ Self-Describing◻ Big and Small Data
◻ Write-Optimized

Design Goals◻ Serialization Agnostic◻ Determinism◻ Self-Describing◻ Big and Small Data
◻ Write-Optimized◻ Dynamically Add New Data Streams

Design Goals◻ Serialization Agnostic◻ Determinism◻ Self-Describing◻ Big and Small Data
◻ Write-Optimized◻ Dynamically Add New Data Streams◻ Corruption ResilientDesign Goals◻ Serialization Agnostic◻ Determinism◻ Self-Describing◻ Big and Small Data
◻ Write-Optimized◻ Dynamically Add New Data Streams◻ Corruption Resilient◻ Indexing

Design Goals◻ Serialization Agnostic◻ Determinism◻ Self-Describing◻ Big and Small Data
◻ Write-Optimized◻ Dynamically Add New Data Streams◻ Corruption Resilient◻ Indexing◻ Standards Compatible

Why a New File Format?
● We evaluated many existing formats:○ rosbag1○ rosbag2 (SQLite)○ Length-delimited Protobuf○ Avro○ HDF5○ Parquet○ EBML
mcap.dev/motivation/evaluation-of-robotics-data-recording-ﬁle-formats

Introducing: MCAP

MCAP Features

Heterogeneous data
● Store messages encoded in multiple serialization formats in a single ﬁle● Can store ROS 1, ROS 2 (CDR), Protobuf, Flatbuﬀer, JSON, etc● Include metadata and attachments

MCAP Features

Performant writing● Write-optimized for fast recording on resource-constrained robots● Append-only structure● Recover partially-written ﬁles when recording is interrupted● Writes can be streamed across a network● Optional compression

MCAP Features

Eﬃcient seeking● Extract data without scanning or decompressing the entire ﬁle● Fast access to indexed summary data
● Remote reading via HTTP Range requests

MCAP Features

Self-contained ﬁles
● Embed all message schemas in the ﬁle● No extra dependencies needed for decoding● CRCs ensure data integrity per chunkDesign Checklist✅ Serialization Agnostic✅ Determinism✅ Self-Describing✅ Big and Small Data
✅ Write-Optimized✅ Dynamically Add New Data Streams✅ Corruption Resilient✅ Indexing✅ Standards Compatible

Simple Speciﬁcation

Benchmark: Throughput

Benchmark: Throughput

Benchmark: Throughput

Benchmark: File SizeCommand Line Interface
ROS 1

ROS 2 + SQLite

ROS 2 + MCAP

⏺ J GK: 9? � J = ; GJ <ros2 bag recordr os 2 ba g r e c or d - s mc a pℹ J GK: 9? � AF>Gr os 2 ba g i nf omc a p i nf o

▶ J GK: 9? � HD9Q

r os 2 ba g pl a yr os 2 ba g pl a y✅ r os ba g c he c k🚫🚫

mc a p doc t or🔧🔧 r os ba g f i x

🚫🚫

mc a p r e c ove r✂ J GK: 9? � >ADL= Jr os 2 ba g c onve r t

mc a p f i l t e r🗜🗜 J GK: 9? � ; GEHJ = KKr os 2 ba g c onve r t

mc a p c ompr e s s

🎈🎈 r os ba g de c ompr e s sr os 2 ba g c onve r t

mc a p de c ompr e s s

🗂🗂 J GK: 9? � J = AF<= Pr os 2 ba g r e i nde x

mc a p r e c ove r

Tooling● rosbag2● rviz
● rqt_bag● Foxglove Studio● PlotJuggler

LibrariesPythonC++TypeScriptGoSwiftRustExample: Python Reader

Example: Python WriterExample: C++ Reader

Example: C++ WriterUse MCAP Today

ROS2○

Install: sudo apt install ros○

Record: ros2 bag record○

Use mcap- ros2 - support○Convert existing .db3 files with mcap convert- $ROS_DISTRO- rosbag2 - storage - mcap
- s mcap
API to read and write ROS2 messages programmaticallyor ros2 bag convert

ROS1○Use mcap- ros1 - support

API to read and write ROS1 messages programmatically○Convert existing .bag files with mcap convert○Recording can be backported to ROS1 if there’s demandmcap.dev

Appendix

Tuning For Diﬀerent Use Cases

No CRCs, Fast Compression, Larger Blocks
FG! 0 ! ~� LJ M=; @MFC1 AR= ~� [ V[ [ YS[ � � µ� í [ + ì; GEHJ = KKAGF~� â 8 KL<â; GEHJ = KKAGF* = N= D~� â $ 9 KL= KLâMinimal Overhead
FG! 0 ! ~� LJ M=
FG! @MFCAF? ~� LJ M=
FG1 MEE9 J Q~� LJ M=