ROS 2: The Right Framework to Choose When Building an Autonomous Vehicle
Written by Dejan Pangercic
March 25, 2020
At Apex.AI, we develop certified software for autonomous mobility. ROS 2 with DDS is a core element of Apex.OS®. Apex.OS is a safe and certified fork of ROS2. Apex.OS abstracts complexity of underlying hardware, middleware, kernel, interfaces and drivers into simple to use, robust, reliable, safe, secure APIs.
Based on the Data Distribution Service™ (DDS™) communications framework, ROS 2 expands the Robot Operating System (ROS) into commercial use cases. It is a heterogeneous and complex system that consists of several components. The performance of ROS 2 greatly depends on multiple architectural layers: The hardware, the operating system kernel and scheduler, the DDS implementation underneath, and the application above. Since ROS 2 uses DDS for the communication, the ROS 2 components can be freely intermixed with native DDS components, delivering a rich set of functionality and performance.
I’ve read a few ROS 2 performance reports which have been incomplete; some have been erroneous. Since we are in the early stages of using ROS 2 for autonomous vehicles development, we need a consistent, methodical and fair process to evaluate the technology, similar to that which has been done for PascalVOC and Kitti.
At Apex.AI, we have been working on unbiased and reproducible performance testing for ROS 2 based on the framework we have built at Apex.AI. This work is now documented in our new white paper, Performance Testing in ROS 2.
The white paper focuses on how the non-functional performance of software component in a system – such as the latency of the perception software pipeline, memory consumption, CPU usage, etc. - can be measured and maintained. In our experience, we find that many developers already track the functional performance of their systems, so we concentrated on the testing for the system components.
We start with the terminology relevant for performance testing, emphasizing the parameters that heavily influence ROS 2 performance. Next, we present a standardized experiment setup and discuss a working implementation of an end-to-end performance testing system that will guarantee a standardized, unbiased and reproducible evaluation.
Some of our findings include:
- Tuning the Quality of Service (QoS) to a particular workload and environment is critical for gaining the best performance. Generally, DDS has the lowest latency and highest throughput when configured to use BEST_EFFORT reliability. When using the RELIABLE QoS setting, it is important to ensure that the resource limits are large enough to support the intended publication rate. When published messages are dropped due to the subscriber exhausting its resource limits, the publisher has to retransmit them, resulting in higher latency.
- Consider the Transport mechanisms. At Apex.AI, we use RTI Connext DDS Micro, which uses three types of transports: Intra-process, shared memory, and UDP. RTI Connext DDS Micro boosts performance through zero-copy and flat data optimizations that reduce the overhead of copying and serializing messages respectively.
- While Discovery in DDS uses multicast, application data can be sent by the UDP transport mechanism using unicast or multicast. By default, RTI Connext DDS Micro uses unicast, which is simpler and interacts more efficiently with network switches and other network infrastructure. In certain applications that publish larger messages to many subscribers, using multicast transport can be beneficial.
- Performance Tuning - After running the tests with different parameters, we found ways to improve performance. For example, there is a lot of CPU utilization and higher latency when using the ROS 2 PollingSubscription plugin, which uses the rclcpp wait-set. The rclcpp waitset_wait wrapped the rcl wait-set implementation underneath, which was quite slow. After switching the implementation to use the Connext DDS wait-set directly, we found performance improvements in terms of reduced latency.
- Finally, we wanted to share that the underpinning DDS layer matters. In testing an open-source DDS implementation, we experienced lock-up issues on the Renesas R-Car H3. These types of issues are difficult to reproduce. The performance test tool made it easier to detect the fault because performance test is run multiple times through continuous integration (CI). We were able to systematically apply fixes, and eventually found the solution which is documented internally at Apex.AI.
In summary, our research and testing proved that ROS 2 demonstrates superior performance for autonomous vehicle production. I invite you to review our approach for unbiased and reproducible performance testing for ROS 2 based on the framework we have built at Apex.AI. The software has been released in our public GitLab repository: https://gitlab.com/ApexAI/performance_test/
About the author
Dejan Pangercic is CTO and Co-Founder of Apex.AI. Prior to founding Apex.AI, he was the Perception and Framework Lead at Faraday Future and the CTO at Deepfield Robotics, developing an autonomous asparagus harvesting robot and most recently a 4D-scanning-robot for plant emergence counting and 3D leaf area measurement.
Dejan also worked as a Senior Research Engineer at the Bosch Research and Technology Center in Palo Alto and as a research assistant at the Technical University of Munich. He is an author of several papers and publication in the field or robotics and robotic perception and the author of two patent applications in the area of field robotics. He is also a contributor to the Point Cloud Library (PCL) and one of the early developers of ROS. Dejan is a member of the ROS 2 Technical Steering Committee. He has organized various workshops and a frequent reviewer for leading conferences. Dejan earned a Ph.D. in computer science from the University of Bremen, Germany, and a master’s and bachelor’s degree in electrical engineering from the University of Ljubljana in Ljubljana, Slovenia.