DOI: 10.5281/zenodo.11475245
title: Data from MovingCables: Moving Cable Segmentation Method and Dataset

description: Manipulating cluttered cables, hoses or ropes is challenging for both robots and humans. Humans often simplify these perceptually challenging tasks by pulling or pushing tangled cables and observing the resulting motions. We would like to build a similar system -- in accordance with the interactive perception paradigm -- to aid robotic cable manipulation. A cable motion segmentation method that densely labels moving cable image pixels is a key building block of such a system. We present MovingCables, a moving cable dataset, which we hope will motivate the development and evaluation of cable motion (or semantic) segmentation algorithms. The dataset consists of real-world image sequences automatically annotated with ground truth segmentation masks and optical flow.
description: The composed dataset

We provide the composed MovingCables dataset in two packages, full and small. Both packages contain all the 312 composed video clips. The full package contains all the 187187 images (ca. 600 images per clip, 60 FPS). The small package contains ten times fewer images per clip, i.e. ca. 60 images per clip, 6 FPS.



MovingCables_full.tar



size: 114.6 GiB

sha256sum: ce5841f1d706b5a4e76d85b21b3e9397b7235e3e76250d999eb0d8681368a641

60 FPS. 600 images per clip.



MovingCables_small.tar



size: 11.5 GiB

sha256sum: 37fae4b64de416ec5f36c1f0c9e2fa2f6c2ac85a76f64be165d97a481d9633c8

6 FPS. 60 images per clip.



MovingCables_sample.tar



size: 80.7 MiB

sha256sum: b0d5fcff8eed3d380f9bb812c0e0ffe7ab25a03e82cfbad6120e46b2552dc45a

This package contains only the clips test/0003 and test/0006 from MovingCables_small.




Data format

All the composed dataset files are PNG images. They are stored in directories according to the pattern:

split_name/image_type/clip_number/image_number.png



split_name can be test, train, or validation

image_type can be one of:



flow_first_back - ground truth optical flow images

normal_flow_first_back - ground truth normal flow images

rgb_clips - color images

stick_masks - binary images of poking stick segmentation masks



clip_number - clip number padded by zeros to four digits, starting from one (0001, 0002, 0003, ...)

image_number - image number padded by zeros to eight digits


The published composed dataset contains only odd-numbered images (00000001.png, 00000003.png, ...). These images were captured with the white light turned on. (In a raw recorded sequence, each such white-lit image was preceded and followed by even-numbered UV-lit images (00000000.png, 00000002.png, 00000004.png, ...). The first (00000000.png) and the last image of a raw recorded sequence was always captured with the UV light turned on. These UV-lit images are not in the composed dataset.)

The flow images have three 16-bit channels. The first two channels encode the ground truth optical flow. The third channel stores the ground truth cable instance segmentation label of each pixel. The background label is 0, integers greater than zero (1,2,3,...) label individual cable instances (1,2,3,...). To load the optical flow or normal flow images from the PNG files, one can use the Python function load_flow_png_float:

import cv2    # OpenCV
import numpy as np

def load_flow_png_float(fn):
    """Load a uint16 PNG optical flow image as a numpy float array."""
    im = cv2.imread(fn, cv2.IMREAD_UNCHANGED)
    if im is None:
        return None
    assert(im.dtype==np.uint16)
    im = im[...,::-1] # Convert from reversed 'BGR' order to original 'RGB'.
    im = im.astype(np.float32)
    im[...,0:2] = (im[...,0:2] - 2**15)*(1.0/64.0)
    return im

The key line (in Python) for converting the 16-bit unsigned integer (uint16) values to floating point (float) flow values in pixels is flow_float[...,0:2] = (flow_uint16[...,0:2] - 2**15)*(1.0/64.0).

 

The source dataset

We also provide the source dataset consisting of the postprocessed recorded clips for generating new compositions. It contains 177 clips with optical and normal flow ground truth and cable and poking stick segmentation masks. The images are RGBA with the alpha channel already generated by chroma key. The source dataset package also contains the background images we used to generate the composed MovingCables dataset above. The dataset compositing code can sample and create new composite clips from this source dataset.



MovingCables_src.tar



size: 59.6 GiB

sha256sum: 7639c617de58746c0ea1c6ca2c9c8b29fd53bf5c12476aa958dea07bf310c866

60 FPS. 600 images per clip.




The source dataset contains the following folders:



background_images/vga_cc0 - The VGA background images. They are further divided into clutter and distractors.

flow_first_back - Ground truth optical flow images.

normal_flow_first_back - Ground truth normal flow images.

rgba_clips - RGBA images showing the cables and the poking stick. The alpha channel masks the background.

rgba_clips_stick - RGBA images showing the cables and the background. The alpha channel masks the poking stick.

stick_masks - Binary images of poking stick segmentation masks.


The marker traces for each source dataset clip are saved in JSON format in a separate zip archive. They can serve as a cable tracking ground truth. One can also use them to create a new optical flow ground truth with different target or reference frames.



MovingCables_traces.zip



size: 23 MiB

sha256sum: 9c84fea2c3d741f0aa1c61be2aeabef64562d0448372e7f38d7dff16e223b833

The package contains one traces.json JSON file for each source dataset clip.

Source dataset image file name numbers of a clip clip_name directly index into the marker traces array loaded from traces_json/clip_name/traces.json. (The traces array stores the marker positions for both the UV-lit and the white-lit images.)

The indexing of each traces array is traces[image_index, marker_index, coordinate_index], where coordinate_index = 0 means the vertical coordinate and coordinate_index = 1 means the horizontal coordinate of the marker position in the image.