RADU: Realsense D435 Installation Troubles with ROS2'
Tags: robots, ros, realsense
Published: 2022-04-11 05:01:45

In the last article, I explained the various visual sensors that you can equip on your robot. After comparing the various sensors, I decided to use the Intel Realsense D435 camera. It provides an RGB camera, a depth camera, has a perfect form factor, and comes with an officially maintained ROS wrapper.

So, I started the journey to get the D435 camera working with ROS2. The journey was long, unexpected, and ultimately failed. Using `Ubuntu 20.04 LTS` with latest `ros2-foxy`, and a combination of `ros2-realsense v3.2.2 && librealsense-v2.48.0` or `ros2-realsense v3.2.0 && librealsense-v2.45.0`, I could not get it working. Nonetheless, I decided to document my approach in this article so that others might save time when trying to get it running.

_This article originally appeared at my blog [admantium.com](https://admantium.com/blog/robo11_ros2_and_realsense_d435/)_.

## First Approach: Installing and Running Ubuntu Packages

The first approach uses pre-build Ubuntu Packages. You will read how to install the SDK and the library, how to start the ROS2 node and the RViz visualization.

### SDK and Library Installation

The official [realsense-ros](https://github.com/IntelRealSense/realsense-ros/tree/ros2) documentation explains how to install the librealsense SDK and the ros2-realsense ROS2 package by using prebuild packages.

Run the following commands:

```bash
sudo apt-get install ros-$ROS_DISTRO-realsense2-camera
sudo apt-get install ros-$ROS_DISTRO-realsense2-description
```

When its finished, connect your Realsense camera an execute `dmesg`. You should see an output similar to this:

```bash
[ 9985.717873] usb 1-1.3: new high-speed USB device number 3 using xhci_hcd
[ 9985.822961] usb 1-1.3: New USB device found, idVendor=8086, idProduct=0b07, bcdDevice=50.cd
[ 9985.822978] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 9985.822991] usb 1-1.3: Product: Intel(R) RealSense(TM) Depth Camera 435
[ 9985.823002] usb 1-1.3: Manufacturer: Intel(R) RealSense(TM) Depth Camera 435
[ 9985.826872] uvcvideo: Unknown video format 00000050-0000-0010-8000-00aa00389b71
[ 9985.827127] uvcvideo: Found UVC 1.50 device Intel(R) RealSense(TM) Depth Camera 435  (8086:0b07)
[ 9985.830442] input: Intel(R) RealSense(TM) Depth Ca as /devices/platform/scb/fd500000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0/usb1/1-1/1-1.3/1-1.3:1.0/input/input1
[ 9985.832009] uvcvideo: Found UVC 1.50 device Intel(R) RealSense(TM) Depth Camera 435  (8086:0b07)
```

This looks good. Let’s start the ROS nodes.

### Starting the ROS2 node

The documentation lists several starting options. Let’s use the official one:

```bash
ros2 launch realsense2_camera rs_launch.py device_type:=d435
```

However, this command does not work and creates the following error:

```bash
realsense2_camera_node-1]  03/07 11:15:46,387 ERROR [281472753719712] (handle-libusb.h:51) failed to open usb interface: 0, error: RS2_USB_STATUS_ACCESS
[realsense2_camera_node-1]  03/07 11:15:46,387 ERROR [281472994937248] (sensor.cpp:526) acquire_power failed: failed to set power state
[realsense2_camera_node-1]  03/07 11:15:46,388 WARNING [281472994937248] (rs.cpp:306) null pointer passed for argument "device"
^C[WARNING] [launch]: user interrupted with ctrl-c (SIGINT)
[realsense2_camera_node-1] [INFO] [1625310949.972289490] [rclcpp]: signal_handler(signal_value=2)
```

The Github repository is well maintained and has a well-supported issue list. Searching here, I found this [issue](ttps://github.com/IntelRealSense/realsense-ros/issues/1408) and saw the main error: The UDEV rules were missing. You can install them manually as outlined there, and then restart the ROS node.

This time, no errors popped up, and I could see see the topics.

```bash
$> ros2 topic list
/camera/color/camera_info
/camera/color/image_raw
/camera/depth/camera_info
/camera/depth/color/points
/camera/depth/image_rect_raw
/camera/extrinsics/depth_to_color
/clicked_point
/goal_pose
/initialpose
/parameter_events
/rosout
/tf
/tf_static
```

### Visualization with RViz

When RVIZ is started, you can click on "Add", then "By Topic" and select the topic and the plugin you want to use. I added the Camera plugin to see images from the topic `/camera/color/image_raw`. However, as soon as I selected this topic, I could see several errors in the terminal.

```bash
[INFO] [1625329665.271601068] [rviz]: Message Filter dropping message: frame 'camera_color_optical_frame' at time 1625329664,860 for reason 'Unknown'
control_transfer returned error, index: 768, error: No data available
```

Searching for this RVIZ error, I found an related issue [#1547](https://github.com/IntelRealSense/realsense-ros/issues/1547). In this particular case, the solution involved using and explicit argument to reset the device during startup, and also to revert to an earlier version of the library.

The explicit reset is done with the following command:

```bash
ros2 launch realsense2_camera rs_launch.py device_type:=d435 initial_reset:=true --debug
....
[realsense2_camera_node-1] [INFO] [1625385699.954313972] [RealSenseCameraNode]: RealSense ROS v3.2.1
[realsense2_camera_node-1] [INFO] [1625385699.954474877] [RealSenseCameraNode]: Built with LibRealSense v2.45.0
[realsense2_camera_node-1] [INFO] [1625385699.954518413] [RealSenseCameraNode]: Running with LibRealSense v2.45.0
```

However, the problem persisted with `librealsense-v2.45.0`. Reading more documentation and issues on Github, I learned that the `ros2-realsense v3.2.2` actually requires `librealsense-v2.48.0`. Since this version is not available as an Ubuntu package, I would need to manually compile it, or use `ros2-realsense v3.2.0` in combination with `librealsense-v2.45.0`. I thought "The Newer, the Better" and started the manual installation.

## Second Approach: Building, Installing and Running from Source

### Librealsense SDK

The librealsense installation from source is [documented on Github](https://github.com/IntelRealSense/librealsense/blob/master/doc/installation.md). The essential steps are:

- Install build essentials for your Linux system
- Clone the GitHub repository
- Run cmake
- Build and install

If you are using an Ubuntu version earlier then 20.04, you need to add additional packages and patch your Kernel. The configuration with cmake is important, you need to use [appropriate cmake flags](https://dev.intelrealsense.com/docs/build-configuration) when building. For example on a Raspberry Pi it is advised to use `FORCE_RSUSB_BACKEND` . When you want to use the native commands for [streaming realsense2 pictures](https://dev.intelrealsense.com/docs/open-source-ethernet-networking-for-intel-realsense-depth-cameras), use `-DBUILD_NETWORK_DEVICE=ON`.

The complete compilation can take a while - on my QuadCore Linux Workstation, it was about 40 minutes.

### ROS2 Realsense Package

The [documentation](https://github.com/IntelRealSense/realsense-ros#method-2-the-realsense-distribution) only explains the step for ROS1, but i could figure out how to get it running on ROS1 as well.

Follow these steps ...

- install the latest Realsense SDK
- Create a ROS2 package with `ros2 pkg create --build-type ament_cmake ros2-realsense`
- Clone the desired ros2-librealsense [v3.2.2](https://github.com/IntelRealSense/realsense-ros/releases/tag/3.2.2) into the `src` folder
- Configure and compile the sources with `colcon build`

... or execute these commands in your shell:

```bash
mkdir -p ~/foxy_ws/src
cd ~/foxy_ws/src/
git clone https://github.com/IntelRealSense/realsense-ros.git
cd realsense-ros/
git checkout tags/3.2.2 #Be sure to use latest tag!
cd ..
colcon build --symlink-install
```

It took me about two minutes to build the package.

```c
Starting >>> realsense2_camera_msgs
[1.382s] WARNING:colcon.colcon_core.prefix_path.colcon:The path '/home/ubuntu/realsense-ros/install' in the environment variable COLCON_PREFIX_PATH doesn't exist
[1.382s] WARNING:colcon.colcon_ros.prefix_path.ament:The path '/home/ubuntu/realsense-ros/install/realsense-ros' in the environment variable AMENT_PREFIX_PATH doesn't exist
[1.383s] WARNING:colcon.colcon_ros.prefix_path.catkin:The path '/home/ubuntu/realsense-ros/install/realsense-ros' in the environment variable CMAKE_PREFIX_PATH doesn't exist
Finished <<< realsense2_camera_msgs [25.5s]
Starting >>> realsense2_camera
Starting >>> realsense2_description
Finished <<< realsense2_description [4.27s]
[Processing: realsense2_camera]
Finished <<< realsense2_camera [2min 18s]

Summary: 3 packages finished [2min 44s]
```

### Starting the ROS2 node

The compilation was successful. We can now source the installation folder with `. install/setup.bash` and starting the librealsense-ros2 node.

```c
ros2 launch realsense2_camera rs_launch.py enable_pointcloud:=true device_type:=d435 initial_reset:=true enable_sync:=true --debug

[realsense2_camera_node-1] [INFO] [1625394768.207055050] [RealSenseCameraNode]: RealSense ROS v3.2.2
[realsense2_camera_node-1] [INFO] [1625394768.207213882] [RealSenseCameraNode]: Built with LibRealSense v2.45.0
[realsense2_camera_node-1] [INFO] [1625394768.207256325] [RealSenseCameraNode]: Running with LibRealSense v2.48.0
[realsense2_camera_node-1] [WARN] [1625394768.207289788] [RealSenseCameraNode]: ***************************************************
[realsense2_camera_node-1] [WARN] [1625394768.207318417] [RealSenseCameraNode]:** running with a different librealsense version **
[DEBUG] [launch.launch_context]: emitting event synchronously: 'launch.events.process.ProcessStderr'
[DEBUG] [launch]: processing event: '<launch.events.process.process_stderr.ProcessStderr object at 0xffffa8320a60>'
[DEBUG] [launch]: processing event: '<launch.events.process.process_stderr.ProcessStderr object at 0xffffa8320a60>' ✓ '<launch.event_handlers.on_process_io.OnProcessIO object at 0xffffa8320e20>'
[realsense2_camera_node-1] [WARN] [1625394768.207345676] [RealSenseCameraNode]:** than the one the wrapper was compiled with!   **
[realsense2_camera_node-1] [WARN] [1625394768.207373546] [RealSenseCameraNode]: ***************************************************
```

The error message confused me, but was quickly resolved: There still was `ros-foxy-librealsense2` package was still installed, and during string the ros2-realsense node, it would load the older librealsense! After deinstallation and launching the node, I could continue.

```c
ros2 launch realsense2_camera rs_launch.py device_type:=d435 initial_reset:=true --debug

[160431.153805] usb 2-1: new SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[160431.174617] usb 2-1: New USB device found, idVendor=8086, idProduct=0b07, bcdDevice=50.cd
[160431.174622] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[160431.174625] usb 2-1: Product: Intel(R) RealSense(TM) Depth Camera 435
[160431.174628] usb 2-1: Manufacturer: Intel(R) RealSense(TM) Depth Camera 435
[160431.174630] usb 2-1: SerialNumber: 020223022670
[160431.180002] uvcvideo: Found UVC 1.50 device Intel(R) RealSense(TM) Depth Camera 435  (8086:0b07)
[160431.181574] uvcvideo 2-1:1.0: Entity type for entity Intel(R) RealSense(TM) Depth Ca was not initialized!
[160431.181579] uvcvideo 2-1:1.0: Entity type for entity Processing 2 was not initialized!
[160431.181581] uvcvideo 2-1:1.0: Entity type for entity Camera 1 was not initialized!
[160431.181779] input: Intel(R) RealSense(TM) Depth Ca as /devices/pci0000:00/0000:00:15.0/usb2/2-1/2-1:1.0/input/input33
[160431.182552] uvcvideo: Found UVC 1.50 device Intel(R) RealSense(TM) Depth Camera 435  (8086:0b07)
[160431.183971] uvcvideo 2-1:1.3: Entity type for entity Processing 7 was not initialized!
[160431.183974] uvcvideo 2-1:1.3: Entity type for entity Extension 8 was not initialized!
[160431.183977] uvcvideo 2-1:1.3: Entity type for entity Camera 6 was not initialized!
```

The terminal output showed new warnings, but the node was finally up.

```c
devcon@giga:/home/work/development/core$ ros2 node info /camera/camera
/camera/camera
  Subscribers:
    /parameter_events: rcl_interfaces/msg/ParameterEvent
  Publishers:
    /camera/color/camera_info: sensor_msgs/msg/CameraInfo
    /camera/color/image_raw: sensor_msgs/msg/Image
    /camera/depth/camera_info: sensor_msgs/msg/CameraInfo
    /camera/depth/color/points: sensor_msgs/msg/PointCloud2
    /camera/depth/image_rect_raw: sensor_msgs/msg/Image
```

Also, the `rqt` graph looked good.

![](https://admantium.com/images/blog/robo11_rqt_graph.png)

### Visualization with Rviz

Its time to see the results of the invested effort. Start RViz, add the camera plugin and configure it to listen to the topic `/camera1/color/image_raw.`

However, immediately after I could see the following message in the terminal:

```c
Message Filter dropping message: frame 'camera_color_optical_frame'
```

And no data was shown in RViz.

![](https://admantium.com/images/blog/robo11_ros2_rviz_no_data.png)

At this time, I created the new issue [#1970](https://github.com/IntelRealSense/realsense-ros/issues/1970) on Github. Quickly, other community members reached out to me, and I learned that others were having similar issues ROS2 and the D435. And I gave up.

## Conclusion

My first attempt to get the D435 running with ROS2 failed. I invested several hours searching for issues, re-reading documentation, re-compiling, and just trying different startup parameters. I could not get it to work. But because I wanted to use this particular sensor, I tried to get it running with ROS1. After not more than 2 hours, including getting and solving other error messages, I could finally see some pictures. This particular part will be explained in a future article.