Repository: ouster-lidar/ouster-ros (language: cpp)

Issue:
Increase RCVBUF_SIZE to prevent package loss 
We experience package losses when it comes to some system load on our vanilla Ubuntu 20.04.
We reproducibly tested it with maxing out the system's load with `$ yes` while running:
```
roslaunch ouster_lidar sensor.launch sensor_hostname:=$OUSTER_OS1_SENSOR_HOSTNAME udp_dest:=$OUSTER_OS1_UDP_DESTINATION lidar_port:=$OUSTER_OS1_UDP_PORT_LIDAR 
``` 
The applications were compiled with `-O3` and `release` flags beforehand.
However, adapting the system's `rmem_default` and `rmem_max` to e.g., 20MB did not help, since the ouster_client sets the receive-buffer explicitly to 256kB in the [client.cpp](https://github.com/ouster-lidar/ouster_example/blob/d73079865ebadd961d7af81b235bb34747f7cb6a/ouster_client/src/client.cpp#L56)

Increasing `RCVBUF_SIZE` to 20MB as well (this value was just a random guess BTW!) solved all our problems and we don't experience any frame drops from now on.

However, I think that just increasing `RCVBUF_SIZE` is a bad practice.
I think, that this value should be exposed as an argument such that it can be configured concerning the user's demand.

**What do you think?** 

Example with package loss:
![image](https://github.com/ouster-lidar/ouster_example/assets/3391191/5599e288-c35f-4e6b-8e13-04d23f72e146)

Example with increased `RCVBUF_SIZE ` buffer and no package loss:
![image](https://github.com/ouster-lidar/ouster_example/assets/3391191/5db30de0-cd41-4b73-894f-f9d36838f1e3)



Discussion / hints:
Definitely something to try and see if it would actually help in all scenarios that users complain about packets drop. May I ask though which ROS version are you currently using?
ROS Noetic. All the other configurations are pretty much vanilla.
Thanks, before trying your suggestion I would want to try out some of the changes that I implemented [here](https://github.com/ouster-lidar/ouster-ros/pull/146) and see if that helps with the issue. Unfortunately, the changes at this moment target the ROS2 branch but soon I will port some of these improvements to  ROS1. I will update you on that front.
Sure, but I think that an adaptable or increased receive buffer is inevitable. We have a quite solid hardware setup (core i7 6000, 64 GB Ram, Intel 550 NIC) and even get package losses when we record the Ouster's UDP datastream with tools like tshark when we introduce load to the system. Only increasing tshark's standard 2MB buffer prevents package losses. Maybe the Ouster's sending behavior utilizes the connection too much. It makes 30MByte/s and each frame seems to be send in burst mode.
> Sure, but I think that an adaptable or increased receive buffer is inevitable. We have a quite solid hardware setup (core i7 6000, 64 GB Ram, Intel 550 NIC) and even get package losses when we record the Ouster's UDP datastream with tools like tshark when we introduce load to the system. Only increasing tshark's standard 2MB buffer prevents package losses. Maybe the Ouster's sending behavior utilizes the connection too much. It makes 30MByte/s and each frame seems to be send in burst mode.

I agree, I just wanted to check if packet loses are due to driver not polling data from the buffer fast enough. In the current ROS1 branch we poll the data using a ROS timer and publish the packets at the same time. So the pace at which we poll from the client is dependent on ROS. The PR I put out last week (currently only for ROS2) removes this dependency and it ensures we constantly poll from the client. But yeah I would still want to look into increasing recv buffer size and what size to use a default.
I also had problems with packet loss in Ouster Studio and tshark. I got frustrated and just build a custom udp capture tool just with recv + fwrite for capturing and fread+sendto for replaying.
buffer increase included PR ouster-lidar/ouster_example#565

Fix patch:
diff --git a/.github/workflows/docker-image.yml b/.github/workflows/docker-image.yml
index ebeaf4db..b723fafd 100644
--- a/.github/workflows/docker-image.yml
+++ b/.github/workflows/docker-image.yml
@@ -15,7 +15,6 @@ jobs:
       fail-fast: false
       matrix:
         ros_distro:
-          - melodic
           - noetic
     steps:
       - uses: actions/checkout@v3
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
index 677f7e71..4b36cef0 100644
--- a/CHANGELOG.rst
+++ b/CHANGELOG.rst
@@ -2,26 +2,62 @@
 Changelog
 =========
 
-[unreleased]
-============
+ouster_ros v0.10.0
+==================
+
 ouster_ros(1)
 -------------
-* breaking: publish PCL point clouds destaggered.
+* [BREAKING]: updated ouster_client to the release of ``20231031`` [v0.10.0]; changes listed below.
+* [BREAKING]: with this release the ouster-ros driver is no longer compatible with ROS melodic
+* [BREAKING]: publish PCL point clouds destaggered.
 * introduced a new launch file parameter ``ptp_utc_tai_offset`` which represent offset in seconds
   to be applied to all ROS messages the driver generates when ``TIME_FROM_PTP_1588`` timestamp mode
-  is used.
-* fix: destagger columns timestamp when generating destaggered point clouds.
-* fix: gracefully stop the driver when shutdown is requested.
+  is used. 
+* [BUGFIX]: destagger columns timestamp when generating destaggered point clouds.
+* [BUGFIX]: gracefully stop the driver when shutdown is requested.
+
+ouster_client
+-------------
+* [BREAKING] Updates to ``sensor_info`` include:
+    * new fields added: ``build_date``, ``image_rev``, ``prod_pn``, ``status``, ``cal`` (representing
+      the value stored in the ``calibration_status`` metadata JSON key), ``config`` (representing the
+      value of the ``sensor_config`` metadata JSON key)
+    * the original JSON string is accessible via the ``original_string()`` method
+    * The ``updated_metadata_string()`` now returns a JSON string reflecting any modifications to
+      ``sensor_info``
+    * ``to_string`` is now marked as deprecated
+* [BREAKING] The RANGE field defined in `parsing.cpp`, for the low data rate profile, is now 32 bits
+  wide (originally 16 bits).
+    * Please note this fixes a SDK bug. The underlying UDP format is unchanged.
+* [BREAKING] The NEAR_IR field defined in `parsing.cpp`, for the low data rate profile, is now 16
+  bits wide (originally 8 bits).
+    * Plase note this fixes a SDK bug. The underlying UDP format is unchanged.
+* [BREAKING] changed frame_id return size to 32 bits from 16 bits
+* An array of per-packet timestamps (called ``packet_timestamp``) is added to ``LidarScan``
+* The client now retries failed requests to an Ouster sensor's HTTP API
+* Increased the default timeout for HTTP requests to 40s
+* Added FuSA UDP profile to support Ouster FW 3.1+
+* Improved ``ScanBatcher`` performance by roughly 3x (depending on hardware)
+* Receive buffer size increased from 256KB to 1MB
+* [bugfix] Fixed an issue that caused incorrect Cartesian point computation in the ``viz.Cloud``
+  Python class
+* [bugfix] Fixed an issue that resulted in some ``packet_format`` methods returning an uninitialized
+  value
+* [bugfix] Fixed a libpcap-related linking issue
+* [bugfix] Fixed an eigen 3.3-related linking issue
+* [bugfix] Fixed a zero beam angle calculation issue
+* [bugfix] Fixed dropped columns issue with 4096x5 and 2048x10
 
 
-ouster_ros v0.10.0
+
+ouster_ros v0.9.0
 ==================
 
 ouster_ros(1)
 -------------
-* breaking change: update to ouster_client release 20230403
 * EOL notice: ouster-ros driver will drop support for ``ROS melodic`` by May 2023.
-* bugfix: Address an issue causing the driver to warn about missing non-legacy fields even they exist
+* [BREAKING]: update to ouster_client release 20230403
+* [BUGFIX]: Address an issue causing the driver to warn about missing non-legacy fields even they exist
   in the original metadata file.
 * added a new launch file ``sensor_mtp.launch`` for multicast use case (experimental).
 * added a technique to estimate the the value of the lidar scan timestamp when it is missing packets
@@ -46,72 +82,16 @@ ouster_ros(1)
   the sensor one which is in line with the current behavior.
 * added the ability to change the names of ``sensor_frame``, ``lidar_frame`` and ``imu_frame``
 * added a placeholder for the ``/ouster/reset`` (not implemented for ROS1).
-* breaking: switched back to using static transforms broadcast but with ability to select the frames
-  to be updated dynamically and at what rate through the two new launch file arguments
+* [BREAKING]: switched back to using static transforms broadcast but with ability to select the
+  frames to be updated dynamically and at what rate through the two new launch file arguments
   ``dynamic_transforms_broadcast`` and  ``dynamic_transforms_broadcast_rate``.
 * updated RVIZ color scheme for point clouds to match with the ROS2 version of the driver.
 
-ouster_ros(2)
--------------
-* MVP ouster-ros targeting ros2 distros
-* introduced a ``reset`` service to the ``os_sensor`` node
-* implemented a new node named ``os_driver`` which combines the functionality of ``os_sensor``,
-  ``os_cloud`` and ``os_image`` into a single node.
-* added support to parse the same parameters provided by the ``ros2_ouster_driver``, the parameters
-  are ``lidar_ip``, ``computer_ip``, ``proc_mask`` and ``use_system_default_qos``; the parameters
-  are fully functional and similar to what the ``ros2_ouster_driver`` provides.
-* for convenience introduced a new launch file ``driver_launch.py`` that is compatible with the 
-  ``ros2_ouster_driver`` in terms of parameters it accepts and the name of published topics.
-* introduced a new parameter ``point_cloud_frame`` to allow users to select which frame to use when
-  publishing the point cloud (choose between ``sensor`` and ``lidar``).
-* breaking: ``lidar`` frame is the default frame used when publishing point clouds.
-* added the ability to choose between ``SensorDataQoS`` or ``SystemDefaultQoS`` across all published
-  topics with ``SensorDataQoS`` selected by default for live sensor mode and ``SystemDefaultQoS``
-  enabled for record and replay modes.
-* introduced a new topic ``/ouster/scan`` which publishes ``sensor_msgs::msg::LaserScan`` messages
-
-
-ouster_client
--------------
-* added a new method ``mtp_init_client`` to init the client with multicast support (experimental).
-* the class ``SensorHttp``  which provides easy access to REST APIs of the sensor has been made public
-  under the ``ouster::sensor::util`` namespace.
-* breaking change: get_metadata defaults to outputting non-legacy metadata
-* add debug five_word profile which will be removed later
-* breaking change: remove deprecations on LidarScan
-
-
-[20230114]
-==========
-
-ouster_ros(2)
--------------
-* MVP ouster-ros targeting ros2 distros (beta release)
-* introduced a ``reset`` service to the ``os_sensor`` node
-* breaking change: updated to ouster sdk release 20230403
-* EOL notice: ouster-ros driver will drop support for ``ROS foxy`` by May 2023.
-* bugfix: Address an issue causing the driver to warn about missing non-legacy fields even they exist
-  in the original metadata file.
-* added a new launch file ``sensor_mtp.launch.xml`` for multicast use case (experimental).
-* added a technique to estimate the the value of the lidar scan timestamp when it is missing packets
-  at the beginning
-* add frame_id to image topics
-* fixed a potential issue of time values within generated point clouds that could result in a value
-  overflow
-* added a new ``/ouster/metadata`` topic that is consumed by os_cloud and os_image nodes and save it
-  to the bag file on record
-* make specifying metadata file optional during record and replay modes as of package version 8.1
-* replace ``tf_prefix`` from ``os_cloud`` with ``sensor_frame``, ``lidar_frame`` and ``imu_frame``
-  launch parameters.
-* bugfix: fixed an issue that prevents running multiple instances of the sensor and cloud components
-  in the same process.
-* switch to using static transform publisher for the ros2 driver.
-
 ouster_client
 -------------
 * added a new method ``mtp_init_client`` to init the client with multicast support (experimental).
 * the class ``SensorHttp``  which provides easy access to REST APIs of the sensor has been made public
   under the ``ouster::sensor::util`` namespace.
-* breaking change: get_metadata defaults to outputting non-legacy metadata
+* [BREAKING]: get_metadata defaults to outputting non-legacy metadata
 * add debug five_word profile which will be removed later
-* breaking change: remove deprecations on LidarScan
+* [BREAKING]: remove deprecations on LidarScan
diff --git a/ouster-sdk b/ouster-sdk
index d7307986..2898060f 160000
--- a/ouster-sdk
+++ b/ouster-sdk
@@ -1,1 +1,1 @@
-Subproject commit d73079865ebadd961d7af81b235bb34747f7cb6a
+Subproject commit 2898060fb47a69b4fbf9807d3c17f6fdc8249379
diff --git a/package.xml b/package.xml
index b0d260f8..965cec66 100644
--- a/package.xml
+++ b/package.xml
@@ -1,7 +1,7 @@
 <?xml version="1.0"?>
 <package format="3">
   <name>ouster_ros</name>
-  <version>0.9.3</version>
+  <version>0.10.0</version>
   <description>Ouster ROS driver</description>
   <maintainer email="oss@ouster.io">ouster developers</maintainer>
   <license file="LICENSE">BSD</license>


Test patch: