ROS 2 AS BACKBONE FOR AUTONOMOUSOUTDOOR FORKLIFT OPERATIONSJohannes Huemer
Christoph Fröhlich
Patrik Zips

Research & Technology Organisation

Research questions

Science / Research

17/12/2023

Solutions

Business / Industry

2

AIT Austrian Institute of Technology

Ownership

49 %
51 %

17/12/2023

3

Seibersdorf
Labor
GmbH

AIT Austrian Institute of Technology

EnergyHealth &Bioresources

Digital Safety &Security

Vision,Automation &Control

NuclearEngineering
Seibersdorf
GmbH

1.400
Employees

Low-Emission
Transport

17/12/2023

Technology
Experience

Innovation
Systems &Policy

183 M€
Total Revenues
as of Y 2022

4

OUR CENTER
Vision, Automation & Control
Center for Vision, Automation & Control
Prof. Kugi & DI Vrabl

High-Performance Vision
Systems

17/12/2023

Complex DynamicalSystems

Assistive & Autonomous
Systems

5

MOTIVATION FOR AUTOMATION IN
LOGISTICS• Labour shortage• Efficiency and productivity• Safety and risk reduction• Cost savings• Intralogistics vs. outdoor environments

17.12.23

Foto: Pixabay - marcinjozwiak

7

PALFINGER CRAYLER BM 214• Remotely controlled forklift• Motivation for this vehicle• Outdoor capability• Robust construction• Hydraulic system••••

17.12.23

steeringbending
mast actuationdriving the wheels

Project Overview

8

PROBLEM STATEMENT:TECHNICAL POINT OF VIEW
Hardware & Sensor Integration• Power supply, processing hardware, …
• Proprioception: Wheel speed, hydraulic pressure, …
• Exteroception: LiDAR, cameras, ultrasonic, …
Control Systems• Modelling & controlling the system ensuring stability and safety
Navigation and Mapping• Efficient mapping and navigation solutions for partly unknown environments
Obstacle Detection and Avoidance• Development of robust obstacle detection algorithms•

17.12.23

Ensure safe navigation in dynamic environments

10HARDWARE ARCHITECTURE
Remotecontrol

Handheld

GNSS
Septentrio
mosaic-H

CustomStereocamera

Livox Mid-70

Rugged PC

Intel NUC
OusterOS1-128

2x Zed2i

TCP/IP

B&R PLC

Safe
remotecontrol

Motor On/Off
Hydraulic
Joint Control

Pressure
sensors
17.12.23

Draw-wire
encoders

eHMI (LED, Alarm, …)

Brake
Rotaryencoders

Linearpotentiometers

Processing
Input
Output11

SOFTWARE ARCHITECTURE
Localization &Mapping

High-level
Control

17.12.23

Object Detection

Low-level
Control

Navigation

Task Planning

Simulation

12

AI-BASED OBJECT DETECTION

Feature extraction

Objects of interest: Pallets (EPAL)

Left

Right
Stereo matching

Disparity/ Depth
17.12.23

C. Beleznai, et. al, "Automated pallet handling via occlusion-robust recognition learned from synthetic data," 2023 IEEE
Conference on Artificial Intelligence (CAI), Santa Clara, CA, USA, 2023, pp. 74-75, doi: 10.1109/CAI54212.2023.00039.

13

AI-BASED OBJECT DETECTION
Training on synthetic data

17.12.23

14

AI-BASED OBJECT DETECTION
Detection results
PART-BASED VOTINGedgeg
lonshort edge

Input representation

17.12.23

15

FACTOR GRAPH BASED LOCALIZATIONFusing measurements• Wheel odometry, GNSS positions• Pallet + Slot detections

17.12.23

16

NAVIGATION
Initial situation

UnsafeSafe, Infeasible

Vehicle kinematics

17.12.23

17

VEHICLE KINEMATICS
Modelling the articulated vehicle
Kinematic model:
𝑥̇ !
𝑦̇ !
Θ̇ !
𝛾̇

=cos Θ!
sin Θ!
sin 𝛾
𝑙! 𝑐𝑜𝑠 𝛾 + 𝑙"
000𝑙"
𝑙! sin 𝛾 + 𝑙"
1𝑣!
𝛾̇𝑣! … velocity of front axle𝛾̇ … steering rate
𝑥! , 𝑦! , Θ! … vehicle front axle coordinates𝑙! , 𝑙" … center-to-axle lengths
Source: Nayl, Thaker (2013).

17.12.23

18

NAVIGATION
Initial situation

17.12.23

UnsafeSafe, Infeasible

Safe, Feasible

Vehicle kinematics

19

PATH PLANNING
Concept:• Two dimensional path planning using Hybrid A*• Combination of A* and Dubins/Reeds-Shepp curves

Dubins• Dubins:• Shortest curve that connects two poses (𝑥, 𝑦, 𝜃)• Constrained by path curvature• Single direction driving

ReedsShepp• Reeds-Shepp:• Extension of Dubins with reversing
17.12.23

20

MOTION PLANNING SYSTEM

PLC

IPC
EN 61131-3: STPathPlanner

𝑥 𝑠 , 𝑦[𝑠]Hybrid A*𝜃𝑠

Pathtracking
Controller*𝑥,- 𝑦,- 𝜃.

𝑣! 𝑘𝛾̇ ! 𝑘𝑹𝑢"𝑢$̇

𝜮𝑣, 𝛾PID-Controls

Localization
17.12.23* Bian, Y., Yang, M., Fang, X. et al, "Kinematics and Path Following Control of an Articulated Drum Roller," 2017
Chin. J. Mech. Eng. 30, pp. 888–899. https://doi.org/10.1007/s10033-017-0102-8

21

DIGITAL TWIN AND SIMULATION

Hybrid A*

17/12/2023

22

PROTOTYPE TESTING AND VERFICATION

17.12.23

24

POSSIBLE CONTRIBUTIONSAIT + ROS 2 SUMMARY
Summary• ROS 2 is great as common language for roboticists with different background
Possible Contributions for ROS 2 community• TCP/IP-based hardware component for ros2_control• Articulated vehicle path-tracking controller as ros2_controller• nav2_costmap_2d::CostmapLayer plugin, which adds polygons from ROS messages
Reach Out• Large-Scale Robotics Lab• AIT as cooperation partner for co-funded projects (Horizon Europe, national programs, …)
17/12/2023

25

PROJECT TEAM