HOW DOES ROS CAREABOUT QUALITY?

1© ROSIN – ROS-Industrial Quality-Assured Robot Software Components

HOW DOES ROS CARE ABOUT QUALITY?
A preliminary study of ROS Quality Assurance (QA) practices and the nature of ROS bugs

rosin-project.eu
2© ROSIN – ROS-Industrial Quality-Assured Robot Software Components

This project is funded by the European Union‘s Horizon 2020 research and innovation
programme under grant agreement No 732287

Preliminary Studyn Objectives:n Understand the As Is
n What are the current quality assurance (QA) and quality control practices?
n What are the current quality assurance challenges?
n What are the charachtiristics of ROS bugs?

n Methods:
n Intensive interviews conducted with community members
n Analysis of ROS Wiki and online documentationsn The analysis of 177 reported bugs representative collection of ROS code
repositories

3© ROSIN – ROS-Industrial Quality-Assured Robot Software Components

This is what we learned from the community
1. THIS IS WHAT WE LEARNED FROM THE COMMUNITY
It’s a role-based environment:(1) Core Development,(2) Driver and Reusable Packages Development,(3) Application DevelopmentROS cares about quality: some industry established practices havebeen implemented but experiencing some challenges.
Quality Assurance Challenges
Core Development
vLack of overview for newcontributors
vHeterogeneous quality
criteria
vMaintenance effort
vLack of maintainers
vUnmaintained packages
vLate detection of errors

4© ROSIN – ROS-Industrial Quality-Assured Robot Software Components

Driver and Reusable Packages
Development
vConnection to hardware
vQuality of architectural design
vLack of established processes
Application Development
vComplexity
vSelecting the right module
vInterdisciplinary domain
vUser Interfaces

This is what we learned about ROS bugs
n ROS Bugs Longevity
100%
90%
80%

Percentage of bugs fixed

70%
60%
50%
40%
30%
20%
10%
0%050

100

150200250300No. of days
5© ROSIN – ROS-Industrial Quality-Assured Robot Software Components

This is what we learned about ROS bugs
n ROS Bugs Detection Phasen ROS Bugs Classification (i.e. types)

CONCURRENCYERRORS
3%
COMPILERERRORS
10%

Compile-time
11%

MISCELLANEOUS ERRORS
4%

Deployment-time
8%

Build-time
12%

DEPENDENCYERRORS
33%
FUNCTIONALERRORS
50%§§

Runtimeinitialization
16%

Finding the functional errors is likely to take a wide collection of different techniques and tools. It
appears to be worthwhile to add a wide collection of different code scanners to the continuous
integration service.
One out of seven bugs is reported by users.

6© ROSIN – ROS-Industrial Quality-Assured Robot Software Components

Runtimeoperation
53%

How does ROS care for quality?
n ROS cares about quality:
n Established QA practices are in placen However:n The execution and implementation of these practices is experiencing some challenges.
n It seems that these practices are not ingrained in the community, lacks of consistency
in the application of QA practices.
n Tools and methods in practice need to be expanded (i.e. code scanning techniques)
and further analysis required to identify the right fits.
n Next:n Work closely and collaboratively with the community:n To address the current challenges
n Implement the recommendations (i.e. CI, code scanning techniques, etc.)
n Extend this study to more participants to understand further the current challenges

7© ROSIN – ROS-Industrial Quality-Assured Robot Software Components

HOW DOES ROS CARE ABOUT QUALITY?
A preliminary study of ROS Quality Assurance (QA) practices and the nature of ROS bugs

Poster session for questions, feedback and discussion
Adam Alami
adaa@itu.dk8© ROSIN – ROS-Industrial Quality-Assured Robot Software Components