An Open-Source Soft Robotic Platform for Autonomous Aerial Manipulation in the Wild
Authors: Erik Bauer
Marc Blöchlinger
Pascal Strauch
Arman Raayatsanati
Cavelti Curdin
Robert K. Katzschmann
Venue: CoRL 2024 (2024)
Keywords: Aerial Manipulation
Learning-Based Grasping
Autonomous Flight
Robotic Systems
Soft Grasping
TL;DR: We propose an open-source soft robotic platform for autonomous aerial manipulation in the wild that uses only onboard perception for self- and target localization.

Abstract: Aerial manipulation combines the versatility and speed of flying platforms with the functional capabilities of mobile manipulation, which presents significant challenges due to the need for precise localization and control. Traditionally, researchers have relied on off-board perception systems, which are limited to expensive and impractical specially equipped indoor environments. In this work, we introduce a novel platform for autonomous aerial manipulation that exclusively utilizes onboard perception systems. Our platform can perform aerial manipulation in various indoor and outdoor environments without depending on external perception systems. Our experimental results demonstrate the platform's ability to autonomously grasp various objects in diverse settings. This advancement significantly improves the scalability and practicality of aerial manipulation applications by eliminating the need for costly tracking solutions. To accelerate future research, we open source our modern ROS 2 software stack and custom hardware design, making our contributions accessible to the broader research community.

An Open-Source Soft Robotic Platform for Autonomous Aerial Manipulation in the Wild
Erik Bauer", Mare Blöchlinger", Pascal Strauch", Arman Raayatsanati", Curdin Cavelti, Robert K. Katzschmann'
Soft Robotics Laboratory
Department of Mechanical and Process Engineering
ETH Zurich
(erbauer, mbloechli, pstrauch, araayatsa, cucavelti, rkk) ch
Abstract: Aerial manipulation combines the versatility and speed of flying plat- forms with the functional capabilities of mobile manipulation, which presents sig- nificant challenges due to the need for precise localization and control. Tradi- tionally, researchers have relied on offboard perception systems, which are lim- ited to expensive and impractical specially equipped indoor environments. In this work, we introduce novel platform for autonomous aerial manipulation that ex- clusively utilizes onboard perception systems. Our platform can perform aerial manipulation in various indoor and outdoor environments without depending external perception systems. Our experimental results demonstrate the platform's ability to autonomously grasp various objects in diverse settings. This advance- ment significantly improves the scalability and practicality of aerial manipulation applications by climinating the need for costly tracking solutions. To accelerate future research, we open source? our ROS 2 software stack and custom hardware design, making our contributions accessible the broader research community.
Keywords: Aerial Manipulation, Learning-Band Based Grasping, Autonomous Flight, Robotic Systems, Soft Grasping
Visual SLAM
Compliant Soft Gripper
Indoor Grasping
In-the-Wild Aerial Manipulation
Onboard Perception
Object Segmentation
Zero-Shot Object Picking
Outdoor Grasping
Figure 1: Our autonomous aerial platform demonstrates its aerial manipulation capabilities with onboard-only perception.
'These authors contributed equally to this work
I Cooresponniig author: tkk@ethz.ch httpg/iiwww.coaaaaaaeaa////rrree
8th Conference Robot Learning (CoRL. 2024), Munich, Germany.

---

Introduction
Aerial manipulation, which combines the agility of aerial vehicles with the functional capabilities of manipulators, poses a significant challenge in robotics. Traditional mobile manipulation platforms, typically composed of mobile base and a manipulator, are designed to interact with their envi- ronment and perform various tasks. Aerial manipulators, using multicopters their mobile base, offer significant advantages over ground-base platforms such as legged robots [1] and wheeled robots [2], particularly in scenarios requiring high traversal speeds or navigation through impass- able terrain, such as during floods or in debris-strewn areas.
In recent years, aerial manipulation has seen significant advances using soft robotic manipula- tors [3, 4, 5, 6]. The compliant properties of soft manipulators relax the otherwise high constraints on precisely positioning the manipulator when grasping objects. Furthermore, soft manipulators also dampen the impact on the dynamics of the flying platform, which eases the burden on flight controllers during each grasp.
However, significant limitation of existing aerial manipulation platforms is their dependence on external perception systems, such motion capture systems [4, 5, 7]. fiducial markers [3, 8], and VIVE trackers [9], to perform self-localization and target object localization. These systems simplify the research problem by providing nearly perfect pose information, but also present two core issues.
First, external perception systems, particularly for self-localization, assume access to almost flawless pose data. Such strong assumptions often limit the applicability of research findings, increasing the barrier to adaptation in real-world scenarios. Second, motion capture systems can be prohibitively expensive, creating high financial barrier that restricts initial research efforts and limits the repro- ducibility of existing studies.
To overcome these limitations, we propose a new platform that relies exclusively on onboard per- ception sensors for aerial manipulation. For self-localization, we use an existing simultaneous lo- calization and mapping (SLAM) system [10], allowing the platform to estimate its state using visual observations alone. We implement a learning- based pipeline localize target objects through RGB- D segmentation and point cloud analysis to determine target grasp locations.
Our approach enables autonomous aerial outdoor manipulation without requiring significant prior knowledge about the environment or target objects. This work introduces a novel method that relies solely on onboard perception, distinguishing itself from previous approaches by operating without strong object priors such as computer-aie aided design (CAD) models or semantic object categories. To our knowledge, this represents the first demonstration of such capability in autonomous acrial outdoor manipulation, allowing the platform adapt diverse and unpredictable scenarios. With our platform, we aim to facilitate future research in key areas of aerial manipulation, including hardware optimization, perception, planning, and control, by providing an open-source, modular framework that researchers can easily build upon and adapt.
1.1 Contributions
Our contributions can be summarized as follows:
1. We propose an autonomous soft robotic aerial manipulation platform utilizing soft robotics, SLAM for self-localization, and learning-based target localization. This platform demon- strates zero-shot manipulation capabilities for diverse objects across indoor and outdoor environments.
2. We open source our ROS 2 software stack, our custom onboard power electronics stack, and our custom hardware design to facilitate further research on autonomous aerial manip- ulation.
3. We validate the platform's effectiveness and versatility through flight and grasping exper- iments, demonstrating its ability autonomously grasp number of objects with high success rate.

---

2
Related Work
The integration of aerial systems with grasping mechanisms has gained considerable attention in recent years [11]. Research has demonstrated the potential of aerial grasping using soft grippers, which adapt the shapes of objects due their passive compliance [3]. Further advances include rapid grasping platforms [4] and efforts to achieve more autonomous grasping without relying fiducial markers [12].
However, many existing systems still depend on motion capture systems to provide the precise lo- cation of the aerial platform or the object. For example, several works have focused on improving flying manipulators with stiffness control to improve object interaction dynamics [13] and devel- oping lightweight, low activation foree grippers designed specifically for aerial grasping applica- tions [14]. In addition, Wu et. al. [15] has investigated optimizing quadcopter designs to enhance aerial grasping performance.
Despite these advancements, the reliance on motion capture systems for self-localization and object priors for target localization remains a common theme. More recent work has demonstrated suc- cessful aggressive aerial grasping using onboard perception by employing visual-inerte odometry for seff-localization and CAD models for object recognition [5]. Our approach differs from this by leveraging earning-based target localization system for zero-shot video segmentation integrated with a SLAM pipeline. This method eliminates the need for prior assumptions regarding object geometry or the environment, enhancing the flexibility and applicability of aerial grasping systems unstructured settings.
3 System Architecture
In this section, we motivate the key design choices that form the architecture of our proposed aerial system. Due to the highly specialized nature of aerial grasping. we have designed custom mod- ular quadrotor frame and power electronics stack to facilitate platform development. Additionally, we have developed a custom software stack for seamless and modular development on ROS 2 in conjunction with the PX4 flight-control stack, We open source our custom hard- and software as an accessible starting point for other researchers in the field.
3.1 Hardware Stack
3.1.1 Platform
The platform is designed with focus modularity and maintainability. Compared to consumer- grade "ready-to-fly" platforms, which are commonly modified to serve as research platforms, our design allows for simpler customization to accommodate various downstream applications, includ- ing aerial manipulation.
The platform has an x-shaped configuration optimized for minimized and symmetric inertia in roll and pitch. Our multi-level design requires vertical assembly: gripper components are located at the bottom, electronic circuit boards and flight controllers occupy the middle level, and the onboard computer and gripper controller are mounted above. This layout simplifies access to components that require frequent adjustments. Acrylic plates provide visual access while protecting the internal components. Electronic speed controllers (ESCs) are integrated within carbon profile arms, and frame connections are made of 3D-printed polylactic acid. The takeoff weight of the platform, including the battery, is 2479 g, resulting in payload capacity of approximately 500
custom-developed power management board offers overcurrent protection, weighs less, and is more compact than tff-the-shelf components. The gripper features two palms, both with two fin- gers inspired by the Fin Ray effect [16]. The fingers leverage their compliance to bend around target objects upon contact, creating reliable grasps regardless of object geometry. The fingers are 3D- 3D-printed with 85A thermoplastic polyurethane to offer the necessary flexibility and strength for effective manipulation.

---

Visual Perception
Planning
Control
quad pose estimate
flihh trajectory gripper angles
SLAM
target pose estimate
Mission Control
Flight Control
Target Locallizaiio
Trajectory Planning
Gripper Control
The overall pipeline of our acrial manipulation system. The visual perception module includes SLAM for estimating the system's 's pose and target localization to identify objects. The planning module encompasses mission control and trajectory planning based on visual estimates. Finally, the actuation module involves flight and gripper control to execute the planned trajectory and manipulate objects.
Figure 2:
Onboard Odroid H3+
Offboard PC
OAK-D
Camera Interface
Mission Control
Serial
Wi-Fi
Arduino Nano
Gripper Interface
Trajectory Planning
Radio
Pixhawk
SLAM
Target Localization
Figure 3: The component architecture with corresponding connections. The onboard components, including the OAK-D, Arduino Nano, and Pixhawk 6C, are connected the Odroid H3+ via USB connections. The onboard and offboard computers maintain a WiFi connection for exchanging data between the modules. The offboard PC also establishes a radio connection to the Pixhawk 6C flight controller, allowing the offboard modules to communicate directly.
3.2 Middleware
Our software design follows the typical autonomy pipeline, including perception, planning, and control routines. The logic flow illustrated in Figure 2, and the architecture the software components is shown in Figure 3. For low-latency communication, we ROS 2 [17] instead of ROS I [18], which is still the predominant middleware in most current works [3, 7, 13, 19, 20).
Unlike ROS 1, ROS 2 supports real-time communication, which is highly advantageous for applica- tions requiring aggressive flight maneuvers [3, 4, 5].
3.3 Planning, Perception, Control
Our visual perception system uses an OAK-D S2 camera, which provides aligned grayscale and depth images at 30 Hz to both the SLAM pipeline and the target localization module. The SLAM pipeline also incorporates IMU data streamed at 200 Hz. Using single camera for both SLAM and downstream perception, reduce overall weight and complexity compared setups that separate cameras for visual-inertial odometry (VIO) SLAM and downstream perception [5].
For planning, a mission control module manages the overall mission logic, observes the current state, and detects and handles exception states or failure modes. Working closely with this module, a trajectory planner computes the subsequent flight maneuvers based on the current mission state and specifies reference trajectories the Pixhawk flight controller. The Pixhawk 6C runs PX4 Autopilot [21], open-source flight control system that communicates over MAVLink [22]. To control the gripper in parallel, separate ROS interface receives commands over the network and forwards them to an Arduino Nano via UART serial communication.
During testing, we frequently employ a VICON motion capture system to obtain ground-trute -ruth pose information on the quadcopter and the object. These ground-truth estimates are also used test various modules in isolation. To access the VICON system measurements and publish them to the ROS 2 network, employ open-source repository from OPT4SMART [23]. The VICON system is configured to provide measurements at 200 Hz.

---

3,4 Self-Localization
Achieving robust motion estimation through visual observations remains a challenging problem. While different open-source methods exist that perform well in synthetic benchmarks [24, 25, 26, 27, 28], we find that most of these published methods suffer from unsatisfactory robustness across different real-world scenarios. Multiple problems arise: high sensitivity to the camera and IMU calibration values makes challenging to get repeatable localization results due to minor differences in calibration. Other parameters are often fine-tuned for specific synthetic benchmarks, necessitating lengthy search to make pipelines work across different real-world environments.
The modular design of our pipeline allows for seamless integration of various SLAM algorithms. We conducted a comprehensive comparison of several open-source options, including SVO [28],
Kimera [27], Rovio [24], and RTAB-Map [29]. Our evaluation revealed that Spectacular Al [10]
consistently provided the most robust and precise state estimates. Notably, the modular nature of
system enables researchers to easily substitute any alternative SLAM pipelines as baselines when developing new state estimation algorithms. Further comparison details can be found in the
Appendix (Section 6.3).
3.5 Target Localization
We propose novel target localization pipeline, which can be used for zero-shot grasping of differ- objects. In contrast existing pipelines for aerial manipulation [5, 12], this approach requires neither CAD models semantic class label of the target objects, which enables strong general- ization for zero-shot grasping.
The pipeline is visualized in Figure 4. The process is initialized with input from the operator, who selects point of the target object in a livestream of the onboard camera. To produce a segmentation mask across subsequent image frames, we use SAM2 [30]. Given the segmentation mask and its corresponding depth image, we obtain a partial point cloud of the object.
We select the grasping points [31] using the following strategy. First, we estimate the centroid of the point cloud and determine its base. We then duplicate the point cloud and rotate it by 180 degrees (assuming symmetry of the object) to estimate its front and back. Using this assumed symmetry, we determine a new centroid for the object. Finally, we determine set of candidate points for grasping. We do so by taking the intersection of the point cloud and a cutting plane that goes through the estimated centroid and is normal to the longest axis.
4 Experiments
To evaluate the capabilities of our proposed system, conduct two experiments. First, we com- pare the accuracy of the SLAM system that of motion capture ground truth to quantitatively determine the quality of the state estimates using only onboard perception. Second, we show comprehensive overview of grasp success rates for 8 commonly used household objects, using only onboard perception to perform all grasps without any explicit prior knowledge these objects.
4.1 Flight Experiments
First, aim investigate the pose errors induced by the Spectacular AI SLAM pipeline. To compute the relative pose error (RPE), we assume that the quadcopter pose in the motion capture system is the ground truth. In Figure 5, we visualize exemplary grasping trajectory expressed in the north-east-down convention, along with ground-truth data from a motion capture system. The error appears to correlate with the platform's velocity, observing a peak RPE of 0.042 during a fast descent toward the object.
Overall, real-world tracking performance meets the high- seeehspreiioon requirements of aerial manipula- tion tasks as long as it initializes well. To ensure sufficient image features are present in the camera
5

---

RGB
Depth
RGB
Depth
RGB
Depth
SAM2
SAM2
SAM2
Operator Imput
Mask
Mask
Mask
Grasp Planning
Grasp Planning
Grasp Planning
Target Point
Target Point
Target Point
RANSAC
Output
t=
t=i+1
π
Figure 4: A high-level overview of the target localization pipeline. The target localization process starts at timestep t = i, with the operator selecting the target object in an RGB livestream of the onboard camera. We use SAM2 to segment and track the target object across subsequent frames. Each mask is fused with the corresponding depth image to produce partial point cloud of the object, which then used for grasp planning. The individual target point estimates are fused via I 1-point RANSAC to compute a robust estimate of the target pose.
0.042
reference
(00)
estimate
estmate
-2.5
Re relstive Pose Error
7(m)
11.5
-1.0
0.0
(00)
16
0.0 (m)
1.0
-05
(m)
380
380
400
410
420
(19]
0.000
(a) Reference (motion capture) and estimate (SLAM) (b) Relative pose error (RPE) in between the reference translation
and estimate trajectory
Figure 5: We evaluate the accuracy of the Spectacular AI SLAM system against motion capture ground truth during a sample grasping mission. All coordinates are expressed in NED (north- cast-down) convention. The two trajectories have been aligned using Umeyama alignment with EVO [32]. We observe a peak RPE of 0.042 during fast descent towards the object, whereas the mean RPE is 0.028.
frames at SLAM initialization, we find that increasing the platform's 's initial position height to at least 80 cm above ground significantly boosts the quality of SLAM initialization.
4.2 Grasping Experiments
To demonstrate the autonomous capabilities of our platform, we performed nine indoor experiments
in controlled setting to ensure repeatability. The general setup consists of a predefined search area
with a target object arbitrarily placed within. The platform must first search for, grasp the object
within the search area, and end the mission by dropping the object predefined location. We employ simple search algorithm, visualized in Figure 6.

---

Cbjject Search Takeoff
Object Grasp
Object Approach
(a) Illustrated standard mission for searching and grasping an object.
(b) Same mission in a). but doing it outdoors with plush toy instead bottle.
The platform executes the mission indoors and outdoors. First, the platform iter- atively advances through a predefined search field. Upon receiving input from our segmentation system (section 3.5, the platform approaches the target object to refine the pose estimate. Based the refined estimate, it performs a grasping maneuver.
Figure 6:
DNTTOAFT
Figure 7: The eight objects we use for grasping experiments shown Table From left to right: plush toy, styrofoam object, cardboard box, ramen cup, bottle, kitchen roll, pouch, ball. The objects differ distinctly in size, weight, shape and surface friction. The bar denotes length of 30 cm.
4.3 Results
In Table 1, we present the success rates for nine experiments obtained in the indoor setting using onboard perception only. Across a total of 144 grasp attempts, our system exhibits an average grasping success rate of 85%, which falls in line with previous works that rely on external perception systems [3, 4, 12]. contrast to these works, we show zero-shot generalization to different objects by climinating the need for explicit object priors in our grasp planning algorithm.
We can observe different failure modes. The most common failure modes are horizontal position eттотs incurred by the increasingly difficult control as the platform approaches the ground. These position errors can then cause the gripper to miss the target object. Secondly, we can observe that for objects with low height, the platform rarely closes the gripper above the object, which could be attributed to increased thrust closer to the ground.
Furthermore, we find that the ability to grasp objects autonomously transfers to other environments, such as outdoor settings, as shown in Figure 6, given that there are no significant disturbances that impact the controllability of the platform (e.g. wind gusts).
Overall, there appear be few to no downsides concerning the efficacy of autonomous grasping using onboard perception. In contrast, the upsides of exclusively relying on onboard perception constitute significant step closer scalable real-world applications of aerial manipulators across different scenarios, allowing for reliable flight without installing extensive infrastructure.

---

Object
Success Rate
Attempts
Weight [g]
Dimensions [cm]
Bottle (upright)
75%
16
248.9
10 10 x 29
Bottle (sideways)
81%
16
248.9
10 x 29 x 10
Plush Toy
75%
68.8
10 x 12 x 26
Pouch
94%
16
240.9
8 25 15
Styrofoam Object
94%
88.7
9 x 13
Kitchen Roll
81%
16
140.1
9 27
Ramen Cup
81%
16
120.4
10 10 x
Ball
88%
16
143.8
14 14 14
Cardboard Box
100%
16
93.0
8 x
Total
85%
144
Table 1: Success rates, number of attempted grasps, weight, and dimensions for test objects in the indoor setup ensure repeatable tests.
4.4 Limitations
We occasionally encountered issues with initializing the platform's 's starting pose, leading to unex- pected rotations at takeoff. This resulted in a misalignment between the actual search pattern and the expected search sector, potentially causing the platform to miss the target object. To mitigate this problem, we placed the platform on an elevated surface to enhance SLAM initialization.
Due to the gripper's placement below the quadcopter, all grasped objects are subjected to significant aerodynamic downwash. Consequently, the object's position may shift during the grasping attempt when it is out of the camera's ' field of view. To minimize this issue, most grasping objects were placed on surfaces with artificially increased friction or mounted on stable supports (e.g. duet tape).
Our grasp planning algorithm operates on partial object point clouds and assumes approximate sym- metry of target objects. This assumption limits the grasp planning capabilities of the platform, which could be addressed using point cloud reconstruction techniques [33] Furthermore, long-term video segmentation and tracking is not possible with SAM2, as the amount of memory and compute time increases with each frame. While sufficient for short grasping missions, using only short-term video segmentation (less than 400 frames) could be a limitation for other usages.
Conclusion
In this work, we have introduced a new autonomous robotic acrial manipulation platform that can localize and grasp targets indoors and outdoors. Using an off-the-shelf SLAM system for self- localization and a learning- based pipeline for target object localization, our platform can operate independently of external perception systems. This autonomy removes the constraints imposed by external motion capture systems, enabling the platform to perform aerial manipulation in various environments, including challenging outdoor settings. With 144 flights for nine zero-shot grasping experiments, we demonstrate that our platform shows competitive grasping efficacy (85%) compared to previous state-of-the-art works using external perception systems.
Our approach addresses significant limitations in the current state of aerial manipulation research, particularly the reliance on pose information provided by external systems and the high costs and limited mobility associated with such equipment. By eliminating these dependencies, broadens the applicability of aerial manipulation. In addition, the open-source release of our ROS 2 software stack, onboard power electronics, and custom hardware design aims to lower the initial in- vestments to facilitate research and development in autonomous acrial manipulation with soft robots.
platform
In summary, our contributions demonstrate the feasibility of autonomous aerial manipulation using onboard perception and pave the way for future research and real-world applications in this domain. Future work could address non-symmetric objects in grasp planning and downwash effects.

---

Acknowledgments
We would like to thank Aashi Kalra for her valuable support in leading the design of the power management board, its assembly, and testing. She has been valuable member of the team and the project. We also express our gratitude to Jero Barahona, who assisted the design of the power management board.
References
[1] D. Bellicoso, K. Krämer, M. Stäuble, D. V. Sako, F. Jenelten, M. Bjelonic, and M. Hutter. Alma articulated locomotion and manipulation for a contol-contoollable robot. 2019 International Conference on Robotics and Automation (ICRA), pages 8477-8483, 2019.
[2] V. Klemm, A. Morra, C. Salzmann, F. Tschopp. K. Bodie, L. Gulich, N. Küng, D. Mannhart, C. Pfister, M. Vierneisel, F. Weber, R. Deuber, and R. Y. Siegwart. Ascento: A two-wheeled jumping robot. 2019 International Conference on Robotics and Automation (ICRA), pages 7515-7521,2011
[3] J. Fishman, S. Ubellacker, N. Hughes, and L. Carlone. Dynamic grasping with "soft" drone: From theory to practice. 202/ IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4214-4221, 2021.
[4] A. X. Appius, E. Bauer, M. Blochlinger, A. Kalra, R. Oberson, A. Raayatsanati, Strauch, S. Suresh, M. von Salis, and R. K. Katzschmann. Raptor: Rapid aerial pickup and transport of objects by robots. 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 349-355, 2022.
[5] S. Ubellacker, A. Ray, J. M. Bem, J. Strader, and L. Carlone. High-speed aerial grasping using a soft drone with onboard perception. npj Robotics, 2024.
[6] H. C. Cheung, C.-W. Chang, B. Jiang, C.-Y. Wen, and H. K. Chu. modular pneumatic soft gripper design for aerial grasping and landing. 2024 IEEE 7th International Conference Soft Robotics (RoboSoft), pages 82-88, 2023.
[7] M. Xu, Huang, R. He, D. Yu, and H. Wang. Aerial shooting manipulator for distant grasping- IEEE Robotics and Aulomation Letters, 8:1991-1998, 2023.
[8] L. Bianchi, D. Carnevale, F. D. Frate, R. Masocco, Mattogno, F. Romanelli, and A. Tenaglia. novel distributed architecture for unmanned aircraft systems based on robot operating system IET Cyber-Systems and Robotics, 2023.
[9] M. Lieret, J. Lukas, M. Nikol, and J. Franke. lightweight, low-cost and self-diagnosing mechatronic jaw gripper for the aerial picking with unmanned aerial vehicles. Procedia Man- ufacturing, 51 430, 2020.
[10] Spectacular AI. Spectacular ai: Slam algorithm pkg on github. https://github.com PhectacularAI/sdaee trankk-exampee,, 2024.
[11] H. Zhong, J. Liang, Y. Chen, H. Zhang, J. Mao, and Y. Wang. Prototype, modeling, and control of aerial robots with physical interaction: A review. IEEE Transactions Automation Science and Engineering, 2024.
[12] E. Bauer, B. G. Cangan, and R. K. Katzschmann. Autonomous marker-less rapid aerial grasp- ing. 2023 IEEE/RSJ International Conference Intelligent Robots and Systems (IROS), pages 6395 6402, 2022.
[13] M. J. Sewalt and K. T. Woo. Enhancing uav capabilities for object picking: A flying ma- nipulator with stiffness control. 2023 8th International Conference Control, Robotics and Cybernetics (CRC), pages 42-46, 2024.

---

[14] P. A. Kremer, H. R. Nohooji, J. L. Sanchez-Lopez, and H. Voos. A lightweight universal gripper with low activation force for aerial grasping. ArXiv, abs/2208. 10768, 2022.
[15] Y. Wu, F. Yang, Wang. K. Wang. Y. Cao, C. Xu, and Gao. Ring-rotor: A novel retractable quadrotor with aerial grasping and transportation capability. IEEE Robotics and Automation Letters, 8:2126-2133, 2023.
[16] W. Crooks, G. D. Vukasin, M. O'Sullivan, W. C. Messner, and C. Rogers. Fin ray@ effect inspired soft robotic gripper: From the robosoft grand challenge toward optimization. Frontiers Robotics AI, 3:70, 2016.
[17] S. Macenski, T. Foote, B. P. Gerkey, C. Lalancette, and W. Woodall. Robot operating system 2: Design, architecture, and uses the wild. Science Robotics, 7, 2022.
[18] M. Quigley. Ros: an open-source robot operating system. IEEE International Conference on Robotics and Automation, 2009.
[19] P. Ramón-Soria, B. C. Arrue, and A. Ollero. Grasp planning and visual servoing for out- doors aerial dual manipulator. Engineering, 2020.
[20] A. S. Nedungadi and M. Saska. Design of an active-reliable grasping mechanism for au- tonomous unmanned aerial vehicles. In International Workshop on Modelling and Simulation for Autonomous Systems, 2019.
[21] L. Meyer. Px4 autopilot, 2011. URL
[22] MAVLink. Mavlink micro air vehicle protocol, 2009, URL bttts://github..r mavlink/navlink.
[23] OPT4SMART. ros2-vicon-receever:: C++ package to publish vicon data in ros 2. https: git, 2024.
/ wttp://TSS/AAA////2220000----000
[24] M. Bloesch, M. Burri, S. Omari, M. Hutter, and R. Y. Siegwart. Iterated extended kalman filter based visual-inertial odometry using direct photometric feedback. The International Journal of Robotics Research, 36:1053. 2012,2017
[25] R. Mur-Artal, J. M. M. Montiel, and J. Tardós. Orb-slam: A versatile and accurate monoc- ular slam system. IEEE Transactions Robotics, 31:1147- 2015.
[26] T. Qin, S. Cao, J. Pan, and S. Shen. A general optimization-based framework for global pose estimation with multiple sensors. ArXiv, abs/1901.03642, 2019.
[27] A. Rosinol, M. Abate, Y. Chang, and I.. Carlone. Kimera: an open-source library for real-time metric-semantic localization and mapping. 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 1689-1696, 2019.
[28] C. Forster, Z. Zhang. M. Gassner, M. Werlberger, and D. Scaramuzza. Svo: Semidirect visual odometry for monocular and multicamera systems. IEEE Transactions Robotics, 33:249- 265, 2017.
[29] M. Labbé and F. Michaud. Rtab-map as an open-source lidar and visual simultaneous local- ization and mapping library for large-scale and long-term online operation. Journal of Field Robotics, 36(2):416 446, 2019.
[30] N. Ravi, Gabeur, Y.TT Hu, R. Hu, C. Ryali, T. Ma, H. Khedr, Rädle, Rolland, L. Gustafson, Mintun, Pan, K. V. Alwala, N. Carion, C.-Y. Wu, R. Girshick, P. Dollár, and C. Feichtenhofer. Sam 2: Segment anything in images and videos. arXiv preprint ha////0s//4000 /408 00714.
arXiv:2408.00714 2024. URL
org/abs/
10

---

[31] B. S. Lapata-Impats P. Gil, J. Pomares, and F. Torres. Fast geometry-based computation of grasping points on three-dimensional point clouds. International Journal of Advanced Robotic Systems, 16, 2019.
[32] M. Grupp. evo: Python package for the evaluation of odometry and slam. https://githul: com//iichaelCruppeerro 2017.
[33] W. Yuan, T. Khot, D. Held, C. Mertz, and M. Hebert. Pen: Point completion network. In 2018 International Conference 3D Vision (3DV), pages 728-737, 2018.

---

Appendix
6.1 Parts List
Additional resources for replicating our hardware platform are available in our public repository at pettps:/githhnn com/srr-ethz/oppree cut plates and an assembly guide can found there. Given an RC Control unit, a workshop, and miscellaneous small parts such as screws, the total cost should around $ 1274. See Table 2 for the item list.
All files for sketches, 3DD-printed linkages, laser-
Count
Name
Price per Unit
Odroid H3+
$ 165
NVME SSD
$ 50
16GB DDR4 RAM
$99
OAK-D
$ 250
Pixhawk 6C
$ 140
Motor 2216-920KV
$ 20
Propeller 1045
$
20A ESC
$
LiPo 4300 mAh
$ 80
Arduino Nano
$ 20
Telemetry Transmitter
$ 60
Custom PCB
$
3D printed parts
$ 30
Carbon Arms
$
3
Acrylic Plates
$ 10
Total Price
$ 1274
Table 2: Total Price of the Platform
6.2 Power Management Board
(a) Top side of the custom PMB: a) on-off switches for power circuits b) DC/DC converters c) Arduino Nano d) power connector onboard PC
(b) Bottom side of the custom PMB: a) servo con- nectors b) 5 V connectors c) lithium polymer battery input
Figure 8: 3D visualization of the custom power management board (PMB), designed to centralize and regulate power distribution for multiple voltage requirements across the platform.
Our custom PMB, shown Figure 8, supplies power to the onboard computer (19 V up to 2 A), the gripper's servo motors (6 V at up A), and various onboard electronics at up 2 A). The Arduino Nano is directly connected to the gripper's servos to relay setpoints from the onboard computer.
12

---

6.3 Comparison of SLAM Algorithms
In the following, we go over different SLAM algorithms which we tested on our onboard computer. To support the use of ROS we utilize the rosl bridge package. We used each SLAM system with the provided default parameter set, adjusting overview is not exhaustive and further performance gains could likely be obtained by spending more time tuning the different algorithms. An in-depth quantitative overview and comparison of different SLAM algorithms can be explored in future work with stronger focus self-localization.
needed to a reasonable degree. To this end, this
SVO Pro [28]: very sensitive camera calibration. Very little documentation. Compatible with ROS 1.
Kimera [27]: precise for slow maneuvers, but does not run fast enough for faster maneuvers without sacrificing accuracy. Compatible with ROS 1.
Rovio [24]: similar to Kimera, does sacrificing accuracy. Compatible with ROS 1.
fast enough fly agile maneuvers without
RTAB-MAP [29]: similar to Kimera, does not fast, relocalization capabilities robust enough for agile drone flight. Compatible with ROS 1.
Spectacular AI [10]: offers out-of-the-box SLAM, which runs fast and offers good preci- sion. Template wrappers are provided for ROS 1 and ROS 2. Closed-source core algorithm.
Other common tracking solutions include the Intel RealSense T265 camera, which offers out-of- the-box tracking, but no longer produced and suffers from poor availability. Another off-the- shelf alternative is the VIO pipeline provided by ZED. However, the ZED SDK requires CUDA on the host computer, which greatly constrains the choice of onboard computers because it requires CUDA-compatble graphical processing unit.
13


Reviews:
[{"review_id": "lZzNML36cA", "reviewer": "Reviewer_ouWQ", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper presents an open source system for aerial manipulation research that relies solely on onboard sensors for perception. This includes the choice of quadruped, OAKD camera, laser range finder, choice of SLAM algorithm, object detection stack, and hardcoded grasping routine. The authors demonstrate grasping results on two objects - a plushy and a water bottle.", "summary_of_recommendation": "I vote to weak reject this paper because in my opinion the proposed platform is not yet ready for use by the wider robotics community due to its reliance on proprietary SLAM + limited demo results.", "technical_quality": 2, "clarity_of_presentation": 3, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 3, "questions_for_rebuttal": "- When conducting grasping experiments, to what degree is the initial pose of the object varied? Eg. what happens if the bottle is kept on its side instead of upright?\n- What are the main failure modes observed in the case of the plushy and the bottle?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8rWtw5gBrw", "reviewer": "Reviewer_KZo7", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 4, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper proposes an interesting aerial manipulation problem setup, tested in the wild using an onboard perception sensor. While traditional methods rely on off-board vision such as using motion capture, this work uses an onboard sensor, allowing it to efficiently perform the task in both indoor and outdoor settings in a cheap and effective way without relying on expensive tracking solutions.", "summary_of_recommendation": "This paper presents some novel and interesting results in the field of aerial manipulation along with hardware experiments. The technical contributions are significant, however the experimental results could have been more diverse and exhaustive, displaying the compelling efficacy of the proposed approach. I propose to give it a weak acceptance.", "technical_quality": 4, "clarity_of_presentation": 5, "potential_impact": 4, "robotics_focus": 4, "quality_of_limitations": 3, "questions_for_rebuttal": "I request the authors to address the following concerns:\n\n1. It is unclear how the agent deals with the uncertainty estimates from the time of detection when the object is in the field-of-view of the onboard camera to actually grasping it when the camera cannot see it anymore. How does it affect based on the size of the object and hence the precision required to successfully grasp it? \n\n2. Does the agent behavior change based on the changes in the physical properties of the object such as its mass, odd/asymmetric shape as well as the coefficient of friction?\n\n3. How does poor depth under inappropriate lighting conditions or holes in the depth image under near proximity affect the visual sensing of the agent?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "etUWCFirFx", "reviewer": "Reviewer_BFE8", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 2, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper introduces an opensource quadrotor platform that carries a soft gripper for aerial manipulation. The platform carries stereo cameras for localization and target detection. The entire autonomy pipeline is introduces as well as the hardware components onboard. I have a hard time summarizing the contributions in this works because I think the major efforts were on the integration existing solutions to build this platform. No specific learning-relevant algorithms or designs have been discussed.", "summary_of_recommendation": "I recommend rejecting this paper because of lack of sufficient contributions.", "technical_quality": 4, "clarity_of_presentation": 4, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 1, "questions_for_rebuttal": "None.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The authors propose an open platform for robotic manipulation. The system is unique, and will hopefully enable more future work. The authors showed that the system is versatile and produce a system that should be reproducible, and can grasp a wide variety of different objects. They describe many different potential research directions enabled by their device, and go to lengths to make sure it is reproducible. The paper is clear and reasonably well-written.

Strengths:
- Grabbing objects with a gripper on a flying drone is interesting and a challenging problem
- System seems very novel and proposes a unique problem
- The system is mostly open-source and seems like it will be reproducible by other labs

Weaknesses:
- Authors should better describe what research questions they think the platform can be used for
- Experiments don't show it being used on very many objects
- Authors should clarify their novelty and contributions
Decision: Accept