DOI: 10.18419/darus-5674
title: MultiCamAssembly: Distributed Robotic Vision System Software Framework

description: &lt;h1&gt;MultiCamAssembly&lt;/h1&gt;

&lt;p&gt;
MultiCamAssembly is a distributed software framework for multi-camera perception and coordination in automated crane-based timber construction.
It orchestrates multiple, distributed assembly robots (cambots) with embedded vision that detect geometric features in prefabricated timber elements and transmit observations to a central coordinator for multi-view pose estimation.
The fused pose estimate enables vision-guided crane motion during on-site assembly.
&lt;/p&gt;

&lt;h2&gt;Structure&lt;/h2&gt;

&lt;ul&gt;
 &lt;li&gt;&lt;b&gt;base/&lt;/b&gt; — UDP server, device management, plotting tools, CLI helpers&lt;/li&gt;
 &lt;li&gt;&lt;b&gt;cambots/&lt;/b&gt; — camera utilities, YOLO integration, motor control, and programs&lt;/li&gt;
 &lt;li&gt;&lt;b&gt;overlay_tracker/&lt;/b&gt; — geometry, detections, tracking, rendering, and CLI&lt;/li&gt;
 &lt;li&gt;&lt;b&gt;ssh/&lt;/b&gt; — SSH config (e.g., lights-off profile)&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;Project Structure&lt;/h2&gt;

&lt;p&gt;
For a full folder tree with short descriptions for every file, see &lt;code&gt;STRUCTURE.md&lt;/code&gt;
&lt;/p&gt;

&lt;h2&gt;Key Workflows&lt;/h2&gt;

&lt;h3&gt;Main App Workflow&lt;/h3&gt;

&lt;ol&gt;
 &lt;li&gt;Power on remote robots and connect to the local network&lt;/li&gt;
 &lt;li&gt;Verify SSH config (e.g. &lt;code&gt;ssh/config_lightsoff&lt;/code&gt;) matches the network&lt;/li&gt;
 &lt;li&gt;Run &lt;code&gt;base/programs/App.py&lt;/code&gt;&lt;/li&gt;
 &lt;li&gt;
 Guided steps:
 &lt;ol&gt;
 &lt;li&gt;Device discovery and selection&lt;/li&gt;
 &lt;li&gt;Remote code updates&lt;/li&gt;
 &lt;li&gt;Component selection&lt;/li&gt;
 &lt;/ol&gt;
 &lt;/li&gt;
&lt;/ol&gt;

&lt;h3&gt;Model Training Workflow&lt;/h3&gt;

&lt;ol&gt;
 &lt;li&gt;Create labeled data using Label Studio&lt;/li&gt;
 &lt;li&gt;Update dataset paths in &lt;code&gt;yolo/yolo_train.yaml&lt;/code&gt;&lt;/li&gt;
 &lt;li&gt;Set hyperparameters and epochs in &lt;code&gt;yolo/yolo_train.py&lt;/code&gt;&lt;/li&gt;
 &lt;li&gt;Train model with &lt;code&gt;yolo/yolo_train.py&lt;/code&gt;&lt;/li&gt;
 &lt;li&gt;Copy trained weights to &lt;code&gt;cambots/yolo/models/&lt;/code&gt;&lt;/li&gt;
 &lt;li&gt;Generate TensorRT engines on remote devices&lt;/li&gt;
 &lt;li&gt;Validate inference on remote devices&lt;/li&gt;
&lt;/ol&gt;

&lt;h3&gt;Pushing and Pulling Workflow&lt;/h3&gt;

&lt;ol&gt;
 &lt;li&gt;Push code via &lt;code&gt;deploy.bat&lt;/code&gt; (Windows) or &lt;code&gt;deploy.sh&lt;/code&gt; (Linux)&lt;/li&gt;
 &lt;li&gt;Command: &lt;code&gt;./deploy.sh lightsoff-j01 [config_file]&lt;/code&gt;&lt;/li&gt;
 &lt;li&gt;Pushes &lt;code&gt;cambots/&lt;/code&gt; without deleting remote-only files&lt;/li&gt;
 &lt;li&gt;Pull data via &lt;code&gt;fetch.bat&lt;/code&gt; (Windows) or &lt;code&gt;fetch.sh&lt;/code&gt; (Linux)&lt;/li&gt;
 &lt;li&gt;Command: &lt;code&gt;./fetch.sh lightsoff-j01 [-d]&lt;/code&gt;&lt;/li&gt;
 &lt;li&gt;Retrieves data from remote &lt;code&gt;cambots/save/&lt;/code&gt; to local &lt;code&gt;save/&lt;/code&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;h3&gt;Camera Calibration Workflow&lt;/h3&gt;

&lt;ol&gt;
 &lt;li&gt;Required only if cameras change&lt;/li&gt;
 &lt;li&gt;Measure checkerboard cell size and update &lt;code&gt;square_size&lt;/code&gt;&lt;/li&gt;
 &lt;li&gt;Run &lt;code&gt;calibrate_camera.py&lt;/code&gt; on remote devices&lt;/li&gt;
 &lt;li&gt;Capture 20 checkerboard images&lt;/li&gt;
 &lt;li&gt;Copy &lt;code&gt;camera_calibration_data.npz&lt;/code&gt; to each remote device&lt;/li&gt;
&lt;/ol&gt;

&lt;h3&gt;Logging Workflow&lt;/h3&gt;

&lt;ol&gt;
 &lt;li&gt;Enable logging in &lt;code&gt;base/programs/App.py&lt;/code&gt;&lt;/li&gt;
 &lt;li&gt;UDP messages logged as JSONL files in &lt;code&gt;logs/&lt;/code&gt;&lt;/li&gt;
 &lt;li&gt;Used for post-processing and algorithm prototyping&lt;/li&gt;
&lt;/ol&gt;