ROS-PyBullet Interface: A Framework for Reliable Contact Simulation and Human-Robot Interaction
Authors: Christopher Mower
Theodoros Stouraitis
João Moura
Christian Rauch
Lei Yan
Nazanin Zamani Behabadi
Michael Gienger
Tom Vercauteren
Christos Bergeles
Sethu Vijayakumar
Venue: CoRL 2022 Poster (2022)
Keywords: Contact-rich Simulation
Manipulation
Teleoperation
Human-Robot Interaction

Abstract: Reliable contact simulation plays a key role in the development of (semi-)autonomous robots, especially when dealing with contact-rich manipulation scenarios, an active robotics research topic. Besides simulation, components such as sensing, perception, data collection, robot hardware control, human interfaces, etc. are all key enablers towards applying machine learning algorithms or model-based approaches in real world systems. However, there is a lack of software connecting reliable contact simulation with the larger robotics ecosystem (i.e. ROS, Orocos), for a more seamless application of novel approaches, found in the literature, to existing robotic hardware. In this paper, we present the ROS-PyBullet Interface, a framework that provides a bridge between the reliable contact/impact simulator PyBullet and the Robot Operating System (ROS). Furthermore, we provide additional utilities for facilitating Human-Robot Interaction (HRI) in the simulated environment. We also present several use-cases that highlight the capabilities and usefulness of our framework. Our code base is open source and can be found at https://github.com/ros-pybullet/ros_pybullet_interface.

ROS-PyBullet Interface: A Framework for Reliable Contact Simulation and Human-Robot Interaction
Christopher E. Mower" .133, Theodoros Stouraitis" .233 João Moura Christian Rauch
Lei Yan", Nazanin Zamani Behabadi, Michael Gienger', Tom Vercauteren', Christos Bergeles' and Sethu Vijayakumar
King's s College London, London, UK. Honda Research Institute, Offenbach, Germany. The University of Edinburgh, Edinburgh, UK. "Harbin Institute of Technology,
Shenzhen, China. Thhe Alan Turning Institute, London, UK. christopher mower ac uk, theosttou@honda-ii.com
Abstract: Reliable contact simulation plays key role in the development of (semi))autonomous robots, especially when dealing with contact-rich manipula- tion scenarios, an active robotics research topic. Besides simulation, components such as sensing, perception, data collection, robot hardware control, human in- terfaces, etc. are all key enablers towards applying machine learning algorithms or model-based approaches in real world systems. However, there is a lack software connecting reliable contact simulation with the larger robotics ecosys- tem (i.e. ROS, Orocos), for a more seamless application of novel approaches, found in the literature, to existing robotic hardware. In this paper, we present the ROS-PyBullet Interface, a framework that provides a bridge between the reliable contact/impact simulator PyBullet and the Robot Operating System (ROS). Fur- thermore, we provide additional utilities for facilitating Human-Robon Interaction (HRI) in the simulated environment. We also present several use-cases that high- light the capabilities and usefulness of our framework. Our code base is open source and can be found at github.com/ros wow/rro--yyuuullttroooo pybullet interface.
Introduction
Dealing with contacts is key requirement for robots to become effective in our daily lives and valu- able assets in industry [1]. Examples of contact-rich tasks include pick and place [2], locomotion [3, 51, wiping [6], pushing [7. 8), dyadic co-manipulation [9], and robot surgery [10, 11, 12, 13]. De- veloping approaches for (semi-)autonomous robots involving contact is thwart with practical issues (e.g. slippage. high impulsive forces, model mismatch, etc.) that may cause failure and damage to the robot or yield safety concerns for humans in close proximity.
Development of machine learning approaches require facility to collect large datasets. However, collection on scale is non-trivial. Learning from demonstration [14] is a popular technique for en- dowing robots with new skills where human provides examples. According to one paradigm, kinaesthetic teaching (e.g. [15]). the human interacts directly with the robot. However, this method requires physical system which is cumbersome and has potential safety concerns. A second ap- proach utilizes teleoperation (e.g. [16]). which has the benefit that the human operator can either interact with simulated robot considerations for our framework are that the virtual world can include human model via telep- resence and that the human operator experience the virtual forces generated by the simulator, through interface to haptic devices. Furthermore, to ease issues when deploying methods on robot hardware, we provide software features easily map targets the robot control commands.
distance with the physical system (ensuring safety). Two key
In robotics, the system's 's complexity, need for several sub-processes, and multi-machin operations motivate modular system design and message parsing functionality [17, 18, 19]. There are many packages for the Robot Operating System (ROS) [18] integrating useful functionalities for research and commercial systems, including useful data structures, control interfaces, inverse kinematics (IK) and motion planning, perception tools, etc. [20, 21, 22, 23, 24]. Additionally, various visual-
"The majority of this work was done while at The University of Edinburgh.
6th Conference on Robot Learning (CoRI. 2022), Auckland, New Zealand.

---

ROS
ROS- ROS-PyBullet Interface
Contact rich physics simulation
Virtual world Sec. 2.3
Virtual robots
Machine learning
Control
Objects
Data collection
Dynamic
Collision
Visual
Sensing
PHYSICS
Sec, 2.2
ROS Services
Extra utilities
Human interaction
Time stepping
Move to state
ik.ros
Force rendering
Env. configuration
Human interfaces
Add/remove objects
Motion planning
IK solvers
Telepresence
Robots
Safe operation
Sec. 2.6
Sec. 2.4
Human input
Figure I: Outline of our framework for bridging a reliable contact simulator within the ROS ecosystem, and HRI interfaces. Colored boxes indicate the specific parts of proposed framework.
izers [25, 26, 27] and physics simulators [28, 29, 30, 31, 32] allow for the development and testing of algorithms, prior to execution robot hardware. However, popular and reliable libraries for contact-rich scenarios, such as PyBullet [31], Drake [30], and MuloCo [29), lack integration with the ROS framework.
1.1 Contributions
In this paper, our contributions are
framework (Figure 1) that enables research in contact-rich manipulation scenarios al- lowing for scamless collection of contact-rich data and human demonstrations within simulated environment.
Our system enables transference of robot behaviors (leamed otherwise) from the PyBul- let simulation environment (a reliable contact/impact simulator) to hardware using ROS. Implementation of several HRI interfaces, e.g. keyboard, mouse, joystick, 6D-mouse, hap- tic devices, Xsens suit that enable easy interaction with virtual environments for HRI and haptic setups.
Several use-cases to demonstrate the capabilities and usefulness of the framework, includ- ing a variety of robots (e.g. Kawada Nextage humanoid, KUKA LWR robot arm, Kinova arm, and dual-arm KUKA IIWA), and full documentation (see supplementary material).
Our framework exploits modularity and extensibilty by implementing several ROS nodes and object types using class hierarchy design patterns. Furthermore, we include additional features such as: utilities for Model Predictive Control (MPC) design and development, motion planning, safe robot operation (for ensuring safety limits are satisfied before commanding the real robot), and inverse kinematics. Several recent works use the proposed framework [6, 33, 8, 34, 35].
2
Proposed Framework
To enable easy prototyping, implementation, and integration with hardware we implement several tools/features in the framework. This section describes these and our design decisions.
2.1
Framework features
Figure shows an overview for our framework highlighting it as central interface between the ROS ecosystem and PyBullet, human interaction, IK solvers, and real robots. The main features of the framework are listed as follows.
1. Online, full-physics simulation using reliable contact simulator. The framework relies on PyBullet to enable well established contact simulation for rigid/deformable bodies.
2. Integration with the ROS ecosystem. Robot simulation and visualization of real robots/objects (utilizing sensing) are integrated via ROS. Furthermore, this enables (i) ROS packages to be integrated with PyBullet and (ii) straightforward way to port developed algorithms to real systems.
3. Several interfaces enabling HRI with virtual worlds and telepresence. We provide facilities for human's 's to provide examples in simulated environment via several popular interfaces (including haptic devices).

---

Examples of the ROS POS-Pyyullet Interface: (left-right): a Kinova arm reaching for a cup, Kuka LWR arm simulating wiping task, human model waving, Talos humanoid robot reaching for target while maintaining balance, the Nextage robot simulating a smart factory scenario, and simulated RGB-D data visualized in RViz.
Figure 2:
4. Sensor simulation. Robot joint force-torque sensors and RGB-D cameras (i.c. point clouds) are provided for sensing-based control and can be seamlessly interchanged with real world sensor streams via ROS.
5. Modular and extensible design. Our framework adopts modular (i.e. several ROS nodes) and highly extensible design paradigm (i.e. class hierarchy) using the Python pro- gramming language. This makes easy to quickly develop new features for the framework.
6. Data collection with standard ROS tools. Since the framework provides an interface to ROS, can leverage common tools for data collection such ROS bags [36] and data processing to common formats in machine learning applications, i.c. rosbag pandas [37].
7. Integration with robot and sensing hardware. Tools are provided to easily remap the virtual system to physical hardware and integrate real sensing apparatus the PyBullet simulation (e.g. vicon).
2.2 Full-Physics Contact-rich Simulation
Several simulators exist for full-physics simulation e.g. Gazebo [27], ODE [28], Nvidia Isaac [38], DART [39], PyBullet [31]. Drake [30], and MuJoCo [29]. We chose to PyBullet [31] since free and open source, a well-known library with an active community, easy to install, well documented, and is in Python. A more detailed comparison is given in Section 4. To ensure framework is extensible, we develop several classes that interface with PyBullet and establish com- munication links with ROS utilizing publishers, subscribers, timers, and services.
2.3 Robots and virtual worlds building
Our framework provides several tools build virtual worlds. The main ROS-PyBullet Interface node must specify a main YAML configuration file containing a list of robots/objects to load into PyBullet, parameters, RGB-D sensor configuration, and visualizer options robots/objects can also be added/removed through ROS services. Various examples of robots/tasks are shown in Figure 2. Several object types (Section 2.3.I) were developed with different interaction properties and various communication channels with ROS. The specification for each object is defined in separate YAML. files. See the documentation provided in the supplementary material for full list of parameters for the configuration files, ROS topics published/subscribed. and ROS services provided.
2.3.1 PyBullet Objects
Robot Incorporating robots in our framework is simple. In a configuration file the user will specify the URDF file name, and several parameters (e.g. base position, inital joint configuration, etc). The robot base frame with respect to the world frame (named rpbi/world) is set by a transform broadcast using the ROS TF library [20]. The interface optionally publishes the robot joint/link states to ROS. Mobile and floating-bass robots are also supported by our framework given initial state (i.e. pose and linear/angular velocity). ROS services expose PyBullet's IK features, allow the user to move the the robot to given joint/end-effector state?, and return robot information (e.g. joint/link names, number of degrees of freedom, etc).
Visual robot A visualization of a robot, that does not interact with the PyBullet environment, can parameter True the robot configuration file.
be instantiated by setting the is.visual_robeieee The main utility of this feature is to allow the user to map a real robot state to the PyBullet environ- ment enabling them easily compare the real robot with simulated robot representing the target configuration. Only the robot information and IK services are available for visual robots.
2 When end- andefffector state is given, the corresponding joint state is found using PyBullet's IK features.

---

(a)
(b)
(c)
Figure 3: RGB-D observation of a scene with two objects: (a) colour image, (b) depth image, (c) projected colour point cloud RViz.
Visual object A common requirement for simulators is to visualize objects. In PyBullet, these objects do not affect other bodies in the scene, nor react to those. Visual objects are listed in the main configuration file under the parameter visual.objects. A visual object was used to visualize the real pushing box (tracked with Vicon) in [8].
Collision object Modeling static objects, that cause momentum changes for other bodies upon col- lision, such as floors, ceilings, and walls are often necessary. These objects are listed under the parameter collision witlssinoooseeeeeeoeeeeeeeeeeeee objects in the main configuration. During the development of the experi- ments presented in [6]. collision object was used to represent a surface.
Dynamic object Objects whose pose evolution is completely determined by the simulator's 's physics engine are a key requirement for development of control algorithms. Dynamic objects, listed as dynamic_objecta in the main configuration file, can be used inside PyBullet to simulate object. A simulated pushing box was used to develop the controller in [8].
Soft object All previously described object types are rigid bodies. We also provide an interface to PyBullet deformable objects. These are similar dynamic objects in that their evolution is defined by PyBullet and can be specified using the soft t objects parameter.
Load from URDF Finally, robots and objects can also be loaded directly into the PyBullet environ- ment using the urdfs parameter. The evolution of these objects are defined by PyBullet. However, since the usage of these objects is ambiguous their communication with ROS is limited.
2.3.2 Sensor simulation
Many control, and planning algorithms rely on sensory feedback. Integrating multiple sensory in- puts, such as tactile and vision, is underdeveloped robotic manipulation [40]. To enable future research in realistic simulated environments, we provide an interface to several sensing modalities. F/T sensor Through the configuration file for the robot it is possible instantiate a simulated force- torque sensor attached to any joint on the robot. These virtual sensors publish joint reaction forces, read from PyBullet, as ROS wrench-stamppe
messages a user-defined sampling frequency.
RGB-D camera Color and depth perception is a key sensing capability for object state estimation in contact-rich manipulation tasks. An RGB-D camera can be instantiated and attached frame through the main configuration file. The color and depth images (image) are published together with the intrinsic camera parameters (camera info). The camera intrinsic parameters are derived from the OpenGL projection matrix and can be used to back-project the images to colored point cloud (Figure 3). This is natively sup- ported in ROS, e.g. via the RViz DepthCl oud plugin or via the rgbd launch package. Optionally, the interface can compute and publish the point cloud data directly.
On a discrete GPU (NVIDIA GeForee GTX 1650 Mobile) this will achieve about 27 Hz, while on an integrated GPU (Intel UHD Graphics 630) this will reduce to about 18 Hz.
2.4 Human interaction
Developing contact-aware algorithms a key aspect of future work for the robotics commu- nity [40, 41]. Clearly there are safety concerns for HRI tasks. Simulation is necessary step in any development cycle involving robots. This means the only way a human can interact with the virtual environment is through developed a plugin for several human interfaces so that the human can be realized the virtual environment and also receive virtual feedback from that environment.
interface (e.g. haptic device). To remedy this issue, have
Telepresence Physical HRI has obvious safety concerns [42, 43], this motivates including the hu- man in the simulation environment. We provide several interfaces to incorporate human into the

---

simulation environment. A ROS driver for the Xsens suit (see Section 3.3) and an avatar (top-right image Figure 2) is provided incorporate a human into the simulation. Additionally, several ROS drivers for haptic devices are provided that enable virtual forces to be rendered to the human.
Control mapping Teleoperation requires the human to interact with the system via an interface. The signals from the device need to be mapped to a control space, the choice of mapping however is non-trivial [44, 45, 46]. We provide ROS nodes in the operator _noe package that take raw interface signals as input and maps them to control space of the users specification. Two options are provided that are typical in robotics applications: (1) Several systems utilize joysticks, often the scaled value of a joystick axis defines velocity in certain dimensions [47, 48]. We provide scale node. py that appropriately orders and scales the interfaces axes. (2) Cartesian/task space control for teleoperation is common, e.g. [44, 45, 49, 6, 33]. The individual axes of the interface often in the range 1,, If we scaled the joystick axes, as before, then the magnitude of the maximum velocity is non-uniform for all interface states. We provide isometric sommetriccoodd....oeee py that ensures the maximum velocity magnitude is isometric.
Logging signals There are several advantages for an intermediary node mapping driver signals to operator commands. First, modularity allows the user easily swap out interfaces/mappings to compare modes. Second, methods utilizing moving horizon estimation (e.g. [50, 51, 61) need to track window of signals. We provide a node that enables this functionality in the operator -interffcee logger node. Third, such structure enables collection/comparison of data streams using ROS bags removing the need for extensive post- post-processing.
2.5 Interfacing with real hardware
Interfacing with hardware is straightforward using our framework. Each object can pub- lish/broadcast its state in several formats (e.g. joint states, float arrays, transforms, wrenches). This means the simulator can act, at development time, as the real system. When porting to the real hardware we provide nodes that remap the current system states to the required robot/hardware drivers: see the
and remap-joint custom package. Setup for these is simple remapping topics in launch file or enabling a ROS re-mapper. The framework can also visualize the current state of physical robots/objects in PyBullet. This is very useful when debugging software and hardware.
nodes in the
2.6 Additional utilities
We provide several utilities facilitate easy transference of robot controllers (learned or otherwise) from simulation to hardware; these are described below.
Model Predictive Control When developing MPC methods [7, 6, 8], it is useful to slowly iterate through individual MPC iterations. Start, stop, and step ROS services are provided that allow the user to easily debug MPC controllers (see rpbi.controls node py in the rpbi.utils package). Inverse Kinematics Inverse kinematics is a key requirement for robotic systems with several es- tablished libraries [52, 53, 54, 23, 21]. A developer may want to investigate various IK problem formulations by changing constraints and/or cost terms. Henee, we provide standardized interface ik ros that allows the user to switch between several IK solvers (local, global, trajectory-based etc) [53, 52, 31] and also define their own problem formulations. The implementation is extensi- ble so that additional solvers can be easily included. Further, a popular integrated solution for IK, and motion planning is Movelt [21] which, via ROS, could be accessed by the PyBullet community using our framework. In future work, we plan to link proposed framework with Movelt.
Interpolation Control often requires smaller time resolution than planning. In this case, interpola- tion between knot points is necessary. The framework provides the interpolation node, py in the rpbi utils package that performs interpolation of planned trajectories.
Time-syne ROS with PyBullet Synchronizing time between processes and a simulated time can be useful. We provide an option so that the user can synchronize the ROS clock with PyBullet's simulation time.
Safe robot operation Safely operating robots is of paramount importance especially when con- ducting HRI experiments. We provide a package safe robot that ensures safe robot motions acting as guard between target and commanded states every target is checked (e.g. link and joint posi- tion/velocity limits, and self-collision) prior to being commanded on the real system.
5

---

Force feedback (N)
0.6
0.4
WW
FW
1.0
1.4 2.0
Time (5)
(c)
Figure 4: A human interacting with virtual world. (a) The experimental setup where the human interacts with the virtual world using a haptic device. (b) The Kuka LWR robot in PyBullet about to interact with static collision object including the coordinate frames (the frames are not presented the user). (c) The force-feeback evolution that is rendered to the human via haptic device.
3 Use-cases
This section describes four use-cases highlighting the features of the proposed framework: (i) human interaction with virtual world, (ii) learning from demonstration using dynamic movement primi- tives, (iii) full-hody telepresence, and (iv) hardware realization. The code for running examples (i) and (ii) has been made fully open source. We plan make the driver for the Xsens suit, used in (iii), open source along with an example.
3.1 Human interaction with virtual worlds
Developing robust learning and control techniques in contact-rich scenarios utilizing human input requires the human to interact with the virtual world. In order to prototype contact-rich machine learning and optimization-based methods with realistic interaction sequences requires such a haptic interface and simulator for generating realistic force-feedback.
In this use-case, we present the user with a haptic interface, a simulated Kuka LWR robot arm, and static collision box object (Figure 4a). Using the interface, the user controls target position defined in the zw axis that is constrained in the zw, Vw axes (Figure 4b). The task for the robot is minimize the distance between the end-effector position and the target while keeping the zæ and zw axes aligned. The joint motion is generated using the IK features in PyBullet, we interface with this functionality using the ik_ros package described in Section 2.6. A simulated Force-Torque sensor is attached to the robot at the wrist joint. When the end-effector comes into contact with the static collision object, the force measured by the sensor in the gg axis is rendered the user via force-feedback. The evolution of the force feedback for a single interaction with the the box object shown in Figure 4c. Key to this use-case to demonstrate that realistic contact force feedback (rendered to the user via a haptic device) can be obtained from PyBullet. To run this example, attach haptic device (3D Systems Touch X), and execute the command roslaunch rpbi examples human interaction. launch.
3.2 Learning from demonstration
Dynamic movement primitives (DMPs) [55] are widely used mathematical formulation for mod- elling motor control of biological systems. Over recent years they have become a key component of learning from demonstration [56]. Many packages. examples, and code exists for learning and executing DMPs - several have been integrated ROS. To highlight the flexibility and potential for learning from demonstration utlizing our framework we leverage standard ROS package for learning DMPs [57].
The goal in this section is to demonstrate how to learn a DMP from teleoperated demonstration using our framework. In this use-case, the user interfaces with the system using the keyboard. A Kuka LWR robot arm is presented in PyBullet, and controlled in position control mode (Figure 5). The interface commands h are mapped to end-effector velocity two dimensions. We use the EXOTica [23] plugin in the ik ros package to perform inverse kinematics the box and end-effector states Id, Ud are saved as the human demonstration. The goal is for the human demonstrate how to push a box from a starting location to the goal position Zy; this behavior is then learned from the human demonstration uy using a DMP Ø. A random starting location for the end effector chosen and the DMP is used to plan a motion ă, î. When the DMP is executed, starting position is chosen randomly. contrast to customized data collection schemes, with this use-case, we demonstrate that data can be collected using standard ROS tools and processed to a Pandas data frame for analysis. To run this example, open terminal and execute roslaunch
lfd. launch.

---

Figure 5: Pipeline for the learning from demonstration use-case.
Human input
Plan DMP
2gg
Start/Gool pose
Zdd
Execute motion
Teleoperation demo
Learn DMP
dld
3.3 Full-body telepresence
In this use-case, we show that not only can we setup various telcoperation examples for the devel- opment of learning and control, but also we can integrate a realization of virtual human that can interact with the PyBullet environment. We equip a human with an XSens suit and publish the 3D positions of each human body part into ROS as transformation. Consecutively, we align each link of the human model in PyBullet with the positions of each body part of the human to realize full- body virtual-pre pretueceeeeence of a humanoid figure. The setup is shown via an assistive dressing scenario in Figure 6a, full details of the work can be found in [35]. This use-case demonstrates how physical HRI task can validated in simulation, and then easily transferred to the real world.
3,4 Infegration with hardware and MPC
A key feature of our framework is its ability easily integrate with robot hardware and develop methods for MPC [6, 8J. In this use-case we highlight this feature of our framework using push- ing task deployed on the Kawada Nextage humanoid robot. The setup is shown Figure 6c. A typical development cycle for research is to develop first in simulation, and then port the work to the real system. Due the complexity of robotic systems, data collection, hardware issues, etc., the latter step can be quite time consuming. Our goal during development of the framework was to minimize this difficulty. We developed the facility to remap target joint states to several formats required by real systems in our lab, and several object types that can interface with real sensors (e.g- Vicon, AprilTags) or broadcast transforms (similar to a real object equipped with Vicon markers or AprilTaggs. The interface makes it simple to swap between testing/prototyping the system in simu- lation, and deploying on the real system our demonstration reduces to setting a flag (real robot
True False). Furthermore, this use-case highlights the utility of the time-stepping feature of the interface for developing MPC algorithms, described in Section 2.6. Typically this can only be achieved in simulation, however since system maps the simulator state the real robot and has the facility to track objects using online sensing, e.g. we used Vicon, the framework can execute an MPC iteration on the real system by the user clicking button. This significantly reduces the development time for laborious tasks such as parameter tuning.
Related work
There are many physics simulators available to the robotics community, e.g. Drake [30], Gazebo [27], Isaac Sim, MuJoCo [29], and PyBullet. However, there does not exist single simula- tor that performs best in all desired features for all domains [58]. Table 1 provides a comparison of our framework against other potential open-source solutions. Each simulator is typically developed for a specific purpose or with certain application in mind, e.g. manipulation, medical, marine, soft robotics, locomotion, etc. Our requirement was simulator that could reliably model contact and impact for manipulation, as well as haptic interactions. Collins et al. compare the accuracy of manipulation tasks of simulators with respect to real world data [59]. Their comparison showed that PyBullet performed better than V-RIEP/CopeliiSSim and MuloCo. In terms of contacts, Chung and Pollard show that Bullet (version including the generalized coordinate approach, in PyBullet) outperformed Dart and ODE for a contact task [60]. A recent study by Acosta [61] compared Py-
(a)
(b)
Figure 6: Integrating robot hardware and sensing. (a) and (b) Real-time virtual presence of a human in the sim- ulation environment during assistive dressing. (a) The configuration of the human is sensed with the Xsens suit. (b) humanoid figure that corresponds the human shown, along with green visual object (Section 2.3.1) used to illustrate the estimated region occluded human elbow. (b) The Kawada Nextage humanoid robot performing a pushing task.
virtual-presence
7

---

ROS
Languages
Deform. Obj.
Hardware
HRI
Photo-realistic
Drake
X
C++/Python
X
Gazebo
C++
X
x
X3
Nvidia Isaac
Python
X
X
MuloCo
X
C++/Python
X
X
ROS-PyBullet
Python
X
Table 1: Comparison of our proposed framework against other potential open source solutions.
Bullet, MuloCo, and Drake for impact accuracy compared to real world data. PyBullet and Drake were shown to be preferable options for simulating contacts/impacts.
In terms of transference to hardware, Gazebo [27] is arguably one of the most popular simulators in robotics [58, Fig. 2]. Since it's incarnation. Gazebo has supported four backend physics simulators (ODE [27, 28). Dart [39], Bullet, and SimBody). Yet. the current Bullet backend uses maximal coordinate rigid bodies, that is not well suited for robotics (since it allows joint constraint viola- tions). On the other hand, Bullet with the generalized coordinate approach based on Featherstone's algorithm [63] is particularly suitable for robotics (integrated in PyBullet) [60]. Recently, there has been efforts to bridge Nvidia Isaac with ROS/Gazebo. However, as discussed by Gonzalez-Badee al. [64] was found that PhysX outperformed Bullet for assemblies involving simple objects, while Bullet performed better with complex objects. In summary, we argue that PyBullet, with the generalized coordinate is one of the best engines for contact and dynamics modeling and haptic interactions, which is also popular in the learning community (e.g. [65, 66, 67]).
Limitations
We implemented the framework using Python, a popular programming language with numerous libraries libraries-mmakin it suitable for our goals of easy prototyping and extensiblity. Despite Python be- ing unsuitable for real-time systems, the robotics community has broadly adopted it as the language for implementing robotic experiments. Furthermore, we have found latency issues in all of our experimental setups, running at frequencies of up to 200 Hz.
Currently, the framework only supports ROS Noetic-thus the only way to run the framework with ROS2 is via the ROSI bridge [68]. Porting the framework to ROS2 is, at the time of writing this manuscript, in-development.
High quality synthetic image rendering and switching out different physics engines are important features for learning applications. The new Kubric library [62] uses Blender to add photo-realistic rendering to Pybullet. Potential future work of ours is to incorporate the Kubric library and provide a backend interface of our framework to other physics engines, e.g. MuloCo.
Conclusions
In this paper, we have proposed framework for simulating/collecting data for contact-rich manipu- lation scenarios including: full physics simulation using PyBullet (known for reliable impact/contact modeling), easy transference from simulation real hardware, integration with the ROS ecosys- tem,and several telcoperation interfaces and robots. Our focus has been to enable research in contact- rich scenarios involving HRI and haptics, interfacing with virtual worlds. We have chosen a physics simulator that reliably models contact/impact interactions and provide bridge for the learning co- munity to transfer their learned policies to real hardware via ROS. We have specifically designed the implementation to exploit modularity and extensibility and to be highly flexible, easily developed, and be able to interface with common machine learning tools/libraries.
We hope students, researchers, and industry will make use this framework to facilitate the devel- opment of their control algorithms and machine learning approaches in scenarios involving con- lact. The supplementary material contains full documentation. The main code base is open source github.com/ros py bullet/ros-pybulllt bybullet-interface interface which contains several examples, documentation, and videos. Dependencies are linked to from the documentation and system requirements are de- tailed. The framework is released under the LGPL license.
Possible via additional plugins. Isaac Sim was recently integrated with Gazebo. Until recently Mu- JoCo required license. Possible with new Kubric library [62].
8

---

Acknowledgments
This research is supported by The Alan Turing Institute, United Kingdom and has received funding from the European Union's Horizon 2020 research and innovation programme under grant agree- ment No 101017008, Enhancing Healthcare with Assistive Robotic Mobile Manipulation (HAR- MONY). This work was supported by core funding from the Wellcome/EPSRC [WT203148/Z/166) NS/A000049/1]. This project has received funding from the European Union's 's Horizon 2020 re- search and innovation programme under grant agreement No 101016985 (FAROS project). This research is supported by Kawada Robotics Corporation and the Honda Research Institute Europe. For the purpose of open access, the authors have applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission.
References
[1] I. Kao, K. Lynch. and J. W. Burdick. Contact Modeling and Manipulation, pages 647-669. Springer Berlin Heidelberg, Berlin, Heidelberg, 2008.
28.
[2] R. A. Brooks. Planning collision- free motions for pick-and-place operations. The Interna- tional Journal of Robotics Research, 2(4):19 44, 1983. 1bf::000177702777444330000000000
[3] A. W. Winkler, C. D. Bellicoso, M. Hutter, and J. Buchli. Gait and trajectory optimization for legged systems through phase-based end-effector parameterization. IEEE Roboties and Automation Letters, 3(3):1560-1567 2018. doi: 10. 109/ 271.22885.....
[4] C. Mastalli, R. Budhiraja, W. Merkt, G. Saurel, B. Hammoud, M. Naveau, J. Carpentier, I.. Righetti, S. Vijayakumar, and N. Mansard. Crocoddyl: An Efficient and Versatile Frame- work for Multi-Contace Optimal Control. In IEEE International Conference on Robotics and Automation (ICRA), 2020. 100.L1////CA444422200000..
[5] M. Posa, C. Cantu, and R. Tedrake. A direct method for trajectory optimization of rigid bodies through contact. The International Journal of Robotics Research, 33(1):69-81, 2014. doi: 10.1 10.1177/0273344913555555
[6] C. E. Mower, J. Moura, and S. Vijayakumar. Skill-based Shared Control. In Proceedings of Robotics: Science and Systems (R:SS), Virtual, July 2021. doi:10. 15..0../........22 200........21.....222.2.
[7] R. Hogan and A. Rodriguez. Reactive planar non- non-prehensile manipulation with hybrid model predictive control. The International Journal of Robotics Research, 39(7):755-773, 2020. doi: 10.1177/02833649209 13938.
[8] Moura, T. Stouraitis, and S. Vijayakumar. Non- -onheeehe planar manipulation via trajectory optimization with complementarity constraints. In Proceedings of IEEE International Confer- ence on Robotics and Automation (ICRA), Jan. 2022. doi: 1099111A446399.2222222 2022)399922099999 1942.
[9] T. Stouraitis, Chatzinikolaidis, M. Gienger, and S. Vijayakumar. Online hybrid motion plan- ning for dyadic collaborative manipulation via bilevel optimization. IEEE Transactions Robotics, 36(5):1452-1447) 1471, 2020. doi: 2T1.:/TR..0000.9922222
[10] Rydén and H. J. Chizeck. Forbidden-region virtual fixtures from streaming point clouds: Remotely touching and protecting beating heart. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 6386012.
2012. doi:10. 101111000//R001111
[11] J. Troccaz, G. Dagnino, and G.-Z. Yang. Frontiers of medical robotics: From concept to systems to clinical translation. Annual Review of Biomedical Engineering, 21(1):193-118, 2019. doi: 10. 1tl11166annneee 000
[12) H. Saeidi, J. D. Opfermann, M. Kam, S. Wei, S. Leonard, M. H. Hsieh, J. U. Kang, and
A. Krieger. Autonomous robotic laparoscopic surgery for intestinal anastomosis. Science Robotics, 7(62), 2022. doi:10.1 122//cinnobotessabb22998
[13] K.-W. Kwok, H. Wurdemann, A. Arezzo, A. Menciassi, and K. Althoefer. Soft robot-assisted minimally invasive surgery and interventions: Advances and outlook. 11(())871-892 2022. doi: 10.
Aaotook,Proceellee Proceedings of the IEFF, 2.000 1109/PPRCC.2222.33677733

---

[14] B. D. Argall, S. Chernova, M. Veloso, and B. Browning. A survey of robot learning from demonstration. Roboties and Autonomous Systems, 57(5):469445 4833, 2009. doi.ttps:/doii.or// 10. 10.1016/j.rohot.2000 200.0o...00....022.... 10.024.
[15] L. Armesto, J. Moura, V. Ivan, M. S. Erden, A. Sala, and S. Vijayakumar. Constraint-aware learning of policies by demonstration. The International Journal of Robotics Research, 37 (13-14):1673-1699 2018. doi: 001:001177/022333918883333
[16] E. Jang, A. Irpan, M. Khansari, D. Kappler, Ebert, C. Lynch, S. Levine, and Finn. BC-z: Zero-shot task generalization with robotic imitation learning. In 5th Annual Conference Robot Learning (CoRL), 2021. URL /tsttps://ppnreeeee
[17] A. S. Huang, E. Olson, and D. C. Moore. LCM: Lightweight communications and marshalling. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4057- 4062, 2010.
201.
[18] M. Quigley, K. Conley, B. Gerkey, J. Faust, T. Foote, J. Leibs, R. Wheeler, A. Y. Ng, et al. ROS: an open-source robot operating system. In ICRA workshop on open source software, volume 3, page 5. Kobe, Japan, 2009.
[19] H. Bruyninckx, P. Soetens, and B. Koninckx. The real-time motion control core of the Orocos project. In IEEE International Conference on Robotics and Aulomation (ICRA), volume 2, pages 2766-2771 vol.2, 2003. 10.1 100110.109/R0B0T.200..22222.
[20] T. Foote. tf: The transform library. In IEEE Conference on Technologies for Practical Robot Applications (TePRA). pages 1-6, 2013. 000110..109//PPAA2213..6666
[21] D. Coleman, I. Sucan, S. Chitta, and N. Correll. Reducing the barrier to entry of complex robotic software: a Movelt! case study, 2014. URL https: htt.aaivvv//aaaaaa/////
orgg ab/// 1404. 3785.
[22] S. Chitta, E. Marder-Eppstein, W. Meeussen, V. Pradeep, A. R. Tsouroukdissian, J. Bohren, D. Coleman, B. Magyar, G. Raiola, M. Lüdtke, and E. F. Perdomo. ros_core vos_control: A generic and simple control framework for ros. Journal of Open Source Software, 2(20))456, 2017. doi: 10.1100111110 00.21105/joss.00455
[23] V. Ivan, Y. Yang, W. Merkt, M. Camilleri, and Vijayakumar, EXOTica: An Extensible Optimization Toolset for Protolyping and Benchmarking Motion Planning and Control, pages 211-240. Springer International Publishing, Cham, 2019. ISBN 3189999...... doi: 10.10077978-3-1 3-319-91590-6-77
[24] R. B. Rusu and S. Cousins. 3D is here: Point Cloud Library (PCL). In IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China, May 9-13 2011. IEEE. 100.1110//CRA.221..555
[25] W. Schroeder, K. Martin, Lorensen, and I. Kitware. The Visualization Toolkit: An Object- oriented Approach to 3D Graphics. Kitware, 2006. ISBN 9781930934191. URL https: //books. google.........o.. co. (ooookks?di=xAAPPAAAAAA..
[26] H. R. Kam, S.-H. Lee, T. Park, and C.-H. Kim. RViz: A toolkit for real domain data visualiza- tion. Telecomman. Syst., 60(2):337-345, oct 2015. 1oi:10.1107/: 1235-015-003455.
[27] N. Koenig and A. Howard. Design and use paradigms for Gazebo, an open-source multi-robot simulator. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 2149 1154,, Sendai, Japan, Sep 2004. doi:
[28] R. Smith et al. Open dynamics engine. ht http: // ode. org, 2005.
[29] E. Todorov, T. Erez, and Y. Tassa. Mujoco: A physics engine for model-based control. In JEFE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5026-
5033, 2012. doi:10.r 109//R0S.2011.33866
[30] R. Tedrake et al. Drake: planning, control, and analysis toolbox for nonlinear dynamical systems, 2014.
10

---

[31] E. Coumans and Y. Bai. Pybullet, python module for physics simulation for games, robotics and machine learning. http: http://pybullet.com org. 2016-2020.
[32] K. Werling, D. Omens, J. Lee, I. Exarchos, and C. K. Liu. Fast and Teature-Complee Differen- tiable Physics Engine for Articulated Rigid Bodies with Contact Constraints. In Proceedings of Robotics:
Science and Systems (R:SS), Virtual, July 2021. doi:10. 1...0.5./ 2011..01221.
[33] C. E. Mower. An Optimization-busee Formalism For Shared Autonomy Dynamic Environ- ments. PhD thesis, The University of Edinburgh, 2021.
[34] Stouraitis. Dyadic Collaborative Manipulation Formalism for Optimizing Human-Robot Teaming. PhD thesis, The University of Edinburgh, 2021.
[35] Li, T. Stouraitis, M. Gienger, Vijayakumar, and J. A. Shah. Set-based state estimation with probabilistic consistency guarantee under epistemic uncertainty. IEEE Robotics and Au- tomation Letters, pages 1-1, 2022. doi: 10011000100111A22020..1111
[36] T. Field, J. Leibs, J. Bowman, and D. Thomas. rosbag. http://wiki: ros. http:///ilk..ros.rrrrrogggg 2020. Online; accessed 14 June 2022.
[37] A. Taylor. rosbag-pandas.
wwt.:iiii.i.oorro.........o....e http:///i....rrrrooooooogpppaas. 2019. Online; accessed
14 June 2022.
[38] NVIDIA Isaac Sim. http::///eeellopee.viiiiiaaooo 2022.
Accessed: August
[39] J. Lee, M. X. Grey, S. Ha. T. Kunz, S. Jain, Y. Ye, S. S. Srinivasa, M. Stilman, and C. K. Liu. Dart: Dynamic animation and robotics toolkit. Journal of Open Source Software, 3(22):500, 2018. doi: (05/osss00000,
[40] N. Fazeli, M. Oller, J. Wu, Z. Wu, J. B. Tenenbaum, and A. Rodriguez. See, feel, act: Hicrar- chical learning for complex manipulation skills with multisensory fusion. Science Robotics, (26), 2019. 10 126/scirobotics.aav3111 333333
[41] A. Rodriguez. The unstable queen: Uncertainty, mechanics, and tactile feedhack. Science Robotics, 6(54), 2021. doi:10. 1226/cinnhhoticsaaa4446...
[42] P. A. Lasota, T. Fong, and J. A. Shah. A survey of methods for safe human-robot interaction. Foundations and Trends@ in Robotics, 5(4):261-34,, 2017. ISSN 1935-8253. 1oi:10.1561/ 2300000052.
[43] M. Selvaggio, M. Cognetti, S. Nikolaidis, S. Ivaldi, and B. Siciliano. Autonomy in physical human-robot interaction: A brief survey. IEEE Robotics and Automation Letters, 6(4):7989- 7996, 2021. doi: R...
[44] E. You and K. Hauser. Assisted teleoperation strategies for aggressively controlling a robot arm with 2d input. In Proceedings of Robotics: Science and Systems, Los Angeles, CA, USA, June 2011.
20....1004. VII.045.
[45] A. E. Leeper, K. Hsiao, M. Ciocarlie, L. Takayama, and D. Gossow. Strategies for human- lnop robotic grasping. In Proceedings of the Seventh Annual ACM/IEEE International Conference on sociation for Computing Machinery. ISBN 9781450310635, doi: 1411100144//115769225555000
Interaction, HRI 122, page 1-8, New York, NY, USA, 2012. As-
[46] C. E. Mower, W. Merkt, A. Davies, and S. Vijayakumar. Comparing alternate modes of teleop- eration for constrained tasks. In 2019 IEEE 15th International Conference on Automation Sci- ence and Engineering (CASE), pages 1497-1504, 2019.
[47] W. Merkt, Y. Yang, Stouraitis, C. E. Mower, M. Fallon, and Vijayakumar. Robust shared autonomy for mobile manipulation with continuous scene monitoring. In 13th IEEE Conference on Automation Science and Engineering (CASE), pages 130-137, 2017. 10.10090CAAS..D01..555000
doi:
11

---

[48] T. Klamt, D. Rodriguez, M. Schwarz, C. Lenz, D. Pavlichenko, D. Droeschel, and S. Behnke. Supervised autonomous locomotion and manipulation for disaster response with a centaur- like robot. In JEEF/RSI International Conference on Intelligent Robots and Systems (IROS), pages 1-8. IEEE, 2018. 109.0.11119RRSS... 20000..201..8800 0.1888994509
[49] A. Gijbels, E. B. Vander Poorten, Stalmans, H. Van Brussel, and D. Reynaerts. Design of a teleoperated robotic system for retinal surgery. In 2014 IEEE International Conference on Robotics and Automation (ICRA), pages 2357-2363, 2014. doi: 10w...1199A....1....... 186.
[50] T. Callens, A. van der Have, S. Van Rossom, J. De Schutter, and E. Aertbeliën. Online phase speed estimation of human motion using a moving horizon estimator. In ICRA Workshop on Integrating Multidisciplinaay Approaches to Advance Physical Human-Roue Interaction, 2020.
[51] O. Adiyatov, Rakhim, A. Zhakatayev, and H. A. Varol. Sensor reduction of variable stiff- ness actuated robots using moving horizon estimation. IEEE Transactions on Control Systems Technology. 28(5) 2020. doi: 10. 209/TCST.200992292450
[52] Beeson and B. Ames. TRAC-IK: An open-source library for improved solving of generic inverse kinematics. In IEEE-RAS 15th International Conference on Humanoid Robots (Hu- manoids), pages 928 935, 2015.
[53] M. L.. Felis. RBDL: an efficient rigid-body dynamics library using recursive algorithms. Au- tonomous Robots, pages 1-17, 2016. doi: 10. 10110007/s155 1000051-0116 1-----9577-
[54] D. Rakita, B. Mutlu, and M. Gleicher. RelaxedIK: Real-time Synthesis of Accurate and Feasi- ble Robot Arm Motion. In Proceedings of Robotics: Science and Systems (R:5S), Pittsburgh, Pennsylvania, June 2018
[55] S. Schaal. Dynamic movement primitives-a framework for motor control in humans and hu- manoid robotics. In Adaptive motion of animals and machines, pages 261-280. Springer, 2006. 10i: 10070004344--3333111 23.
[56] M. Saveriano, F. J. Abu-Dakka, A. Kramberger, and L. Peternel. Dynamic movement primi- tives in robotics: A tutorial survey, 2021.
[57] S. Niekum. dmp. http::/wiiit ros. org/dmg 2015, Online; accessed 14 June 2022.
[58] J. Collins, S. Chand, A. Vanderkop. and D. Howard. A review of physics simulators for robotic applications. IEEE Access, 9:51416-51431, 2021. doi: 1199//1CESS 33066666
[59] J. Collins, D. Howard, and J. Leitner. Quantifying the reality gap robotic manipulation tasks. In 2019 International Conference on Robotics and Automation (ICRA), pages 6706- 6712, 2019. doi: 10.:10.11 11.11.119990 (T/0///CCAA.11.8799999999
[60] S.-J. Chung and N. Pollard, Predictable behavior during contact simulation: a comparison of selected physics engines. Computer Animation and Virtual Worlds, 27(3-4):262 2702222700220 2016.
[61] B. Acosta, W. Yang, and M. Posa. Validating robotics simulators real-world impacts. IEEE Robotics and Automation Letters, pages 1-1, 2022. doi: 10. 1101001100LRRAA222222143344
[62] K. Greff, Belletti, I.. Beyer, C. Doersch, Y. Du, D. Duckworth, D. J. Fleet, D. Gnanapra- gasam, F. Golemo, C. Herrmann, T. Kipf, A. Kundu, D. Lagun, I. Laradji, H.-T. D. Liu, H. Meyer, Y. Miao, D. Nowrouzezahrai, C. Oztireli, E. Pot, N. Radwan, D. Rebain, S. Sabour, M. S. M. Sajjadi, M. Sela, V. Sitzmann, A. Stone, D. Sun, S. Vora, Z. Wang, T. Wu, K. M. Yi, F. Zhong, and A. Tagliasacchi. Kubric: a scalable dataset generator. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
[63] R. Featherstone, Rigid body dynamics algorithms. Springer, 2014.
[64] G. Conzalez-Badillo, H. Medellin-Castillo, Lim, Ritchie, and S. Garbaya. The devel- opment of physics and constraint-based haptic virtual assembly system. Assembly Au- tomation, 34(1):41-55, Jan 2014. ISSN 0144-5154. 100..01100//AA0-2001..00 URL. https://doi. org/10. 1100/11-000------ 04000//4400002222202-3--00
12

---

[65] B. Delhaisse, L. Rozo, and D. G. Caldwell. Pyrobolearn: A python framework for robot learning practitioners. In L. P. Kaelbling, D. Kragic, and K. Sugiura, editors, Proceedings of the Conference on Robot Learning, volume 100 of Proceedings of Machine Learning Research, pages 1348-1358. PMLR, Oct-01 Nov 2020. URL set.Lttts://roceeeiii mlr. ppres// 1..0/delhaissee0a. html.
[66] X. B. Peng, E. Coumans, T. Zhang, T.-W. Lee, J. Tan, and S. Levine. Learning Agile Robotic Locomotion Skills by Imitating Animals. In Proceedings of Robotics: Science and Systems, Corvalis, Oregon, USA, July 2020. doi: 20.11567/RSS
[67] S. James, P. Wohlhart, M. Kalakrishnan, D. Kalashnikov, A. Irpan, J. Ibarz, S. Levine, R. Hadsell, and K. Bousmalis. Sim-to-real via sim-to-sim: Data-efficient robotic grasping via randomized-to-canonice adaptation networks. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12627-12637, 2019.
[68] D. Thomas and J. Perron. rosl bridge. https://inee ros brriog//_briir/..oe 2022. Online; accessed 14 June 2022.
13


Reviews:
[{"review_id": "ySn1wjIZ-em", "reviewer": "Reviewer_bKas", "summary": "This paper presents a ROS-PyBullet Interface bridging between the popular physics simulator PyBullet with Robot Operating System (ROS). Different tools are supported in addition to the native PyBullet and ROS functions such as Inverse Kinematics interface and interfaces for human interaction.  ", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n* The paper clearly outlines the features in ROS-PyBullet and made it very clear what features are supported in the current frame work. \n* The framework presented can be a very useful tool for the robotics community. There’s definitely value in developing systems like this.\n* Demonstration of hardware integration is helpful. \n\n**Weaknesses:**\n* It is not entirely clear what the contribution of this work is after removing the pre-existing modules. In the contribution section, it is claimed that one contribution of the work is the proposed framework that contains different pieces of software, which demonstrates the engineering effort went into the work, but does not speak for the research contribution. \n* Related to the previous point, seems like section 2.4, 2.5 and 2.6 are not related to integrating ROS with PyBullet, they are more like standalone ROS nodes (they can be very useful nonetheless). \n* It is also not clear how the proposed system helps different projects (Does it achieve things that was not possible before, or does it make things so much easier?) Section 3 listed different use cases for the ROS-PyBullet interface. While it definitely helps to see the example usage, what might be even more helpful is to see how these examples are done differently in the proposed system compared to other systems. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "ROS and PyBullet are both tools commonly used in the robotics community, and integrating a physics simulation engine with ROS could potentially benefit future research. While the authors clearly achieved the goal of integrating these two tools and even provided a few other tools to make it more user friendly, it is not entirely clear how this integration is fundamentally better than other existing frameworks (or a combination of frameworks). The engineering effort and quality of this work are clear but the research contribution is not well presented.", "technical_quality": "Fair: Significant additional evidence required to support the claims.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "2: Unsure, difficult to assess.", "reviewer_expertise": "5: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature"}, "extra_text": {"issues": "* This work seems like an integration of multiple pre-existing tools and the contribution section also more or less listed the features of the framework rather than stating the contributions.  \n* It might be helpful to clearly separate what parts of in this framework are pre-existing and what parts are developed in this work. \n* As a paper proposing a system consisting of existing resources (which could be okay), it is very important to demonstrate why it is better than everything else that’s out there. This can be done through experiments comparing with other software, or a dedicated subsection clearly stating the difference and improvement. \n"}}, {"review_id": "u8sCxhR05Yl", "reviewer": "Reviewer_E5qq", "summary": "This paper presents a framework for generating and simulating multiple scenarios that involve contact data with human demonstrations. The work leverages the pybullet simulation environment and develops robot-centric components for control and sensor measurement. In addition, the work integrates ROS into the the pybullet interface which readily enables the use of human feedback for collecting demonstrations. ", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper presents several use-cases for their proposed framework. This is presented by the plenty ways to interface and use the presented framework. The breadth of robots and interfaces is very useful when prototyping and developing robotic experiments. \n\nThe paper can relate and compare itself more closely with existing simulators that offer similar interfaces with ROS. One example is Gazebo which not only has similar interfaces, but can also choose between contact simulators. Ideally, this work could benefit from allowing other simulators to interface with the ROS environment and the utilities created. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The work provides a necessary tool for robotics that can facilitate research progress, especially with human demonstrations. However, there are other tools that claim similar libraries that attain the same contributions. It would strengthen the paper to illustrate where their library stands-out and why this is a preferred way of developing robot simulation utilities. ", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "2: Unsure, difficult to assess.", "issues": "A comparison with Gazebo (and other similar libraries).\nComprehensive overview of other robotics toolbox-libraries, what they provide, and the gap that this fills in. ", "reviewer_expertise": "3: The reviewer is fairly confident that the evaluation is correct"}, "extra_text": {}}, {"review_id": "4GX1a8aK5PI", "reviewer": "Reviewer_VFxS", "summary": "The authors present a bridge between PyBullet and ROS. This empowers researchers to connect an easy-to-work with simulator that’s suitable for simulating contact-rich tasks directly into robotics contexts, where ROS is a dominant middleware. The authors describe the components of their bridge and then present four use cases that highlight the value of the system.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n* Serves a real need in the community. I am aware of several researchers that have independently built subsets of this bridge to suit their needs. \n\n* Well motivated use cases. While simulation solutions oriented toward data generation for learning are the predominant use case in this community, the author’s use cases illustrate some of the main scenarios where simulation is a key part during execution on a real robot system.\n\nWeaknesses\n\n* Limited comparison or contextualization in the landscape of comparable solutions. The paper does not have a related works section, nor does it allude to alternative solutions. The framework tackles a smattering of technical needs (see list on 48-54), which means that readers will come with a variety of comparisons in mind. A table that highlights comparable features and limitations of different options would help.\n    * OSRF recently worked with NVIDIA to bridge Gazebo and Isaac Sim (which is PhysX-based). How does this work relate?\n    * Readers of CoRL papers may be looking for a contextualization wrt simulators for learning vision-based policies. As the authors call out in 137-140, lots of research is being done that leverages realistic simulated environments. PyBullet doesn’t include a good renderer out of the box. Even discounting the vision use case, nicer renderers are an enabler for work that asks users to interact with the simulation. \n    * You may also get readers who are primarily ROS users. They will find the comparisons to Gazebo provided unilluminating, inaccurate, and incomplete. You should highlight why Gazebo can't easily address the needs of certain use cases. In my assessment, it has much more to do with Gazebo requiring the user to know C++ than simply that it's default backend isn't good enough. These readers will also pause at the description of an independent set of kinematics solvers and motion planners, when they are likely most familiar with MoveIt as an integrated solution here. Highlight the benefit you get, be it in ease of use or in capability, compared to simply loading the same URDF into MoveIt and using it's solvers.\n    * Necessarily, every effort to enhance robotics simulation platforms will be lacking in some feature, but the onus is on the authors to help prospective users understand how their solution fits in the landscape.\n\n* The motivations for some features may be niche to many readers. There are a couple places (169, 176) where the authors motivate a feature by citing a use case from their own work, indicated by the anonymization of the reference. If there aren’t independent works that have a similar need or use case, it’s an indication that the audience for a particular feature is limited. The authors should consider whether it is worth providing more exposition on the value of these features, or to frame them in an alternate way that more readers will connect with.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The authors provide a set of tools which will be valuable either as a reference or even as an immediate aide to researchers. The specific implementation path they took is pragmatic and maintains the strengths of using a common middleware like ROS while enabling some tasks that wouldn’t be possible without a well-behaved simulator in the loop. I think there is room to improve in regards to identifying the key selling points of the system and highlighting contrasts with other options that readers may be considering.\n\n**Update after rebuttal:**\n\nThe authors have provided a reasonable qualitative characterization of the landscape of alternatives. I believe this will greatly help readers understand this work and how it fits into the broader landscape. I have raised my recommendation to accept.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "5: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature"}, "extra_text": {"issues": "* Contextualize the work. Provide qualitative comparisons to alternatives. Think about the people you would most like to use your system, and the people who are most likely to come across it, and highlight the value for them.\n\n* Gazebo supported four simulation backends from its earliest days. In fact, in its recent incarnations, DART has become the default. If the other backends are deprecated, or lack a specific feature, point to a notice of the deprecation or documentation of the limitation. You should clarify specifically which version you are referring to, as they have muddied the branding considerably while flip-flopping on their Ignition rename."}}]

Author rebuttal:

Decision: Accept (poster)