Enhancing Consistent Ground Maneuverability by Robot Adaptation to Complex Off-Road Terrains
Abstract: Terrain adaptation is a critical ability for a ground robot to effectivelytraverse unstructured off-road terrain in real-world ﬁeld environments such asforests.
However, the expected or planned maneuvering behaviors cannot alwaysbe accurately executed due to setbacks such as reduced tire pressure.
This inconsistency negatively affects the robot’s ground maneuverability, and can cause slowertraversal time or errors in localization.
To address this shortcoming, we proposea novel method for consistent behavior generation that enables a ground robot’sactual behaviors to more accurately match expected behaviors while adapting toa variety of complex off-road terrains.
Our method learns offset behaviors in aself-supervised fashion to compensate for the inconsistency between the actual andexpected behaviors without requiring the explicit modeling of various setbacks.
Toevaluate the method, we perform extensive experiments using a physical groundrobot over diverse complex off-road terrain in real-world ﬁeld environments.
Experimental results show that our method enables a robot to improve its groundmaneuverability on complex unstructured off-road terrain with more navigationalbehavior consistency, and outperforms previous and baseline methods, particularlyso on challenging terrain such as that which is seen in forests.
Introduction
When ground robots are deployed in these environments, their actual behaviors often do not match the expected behaviors, e.g., due to wheel slip.
The inconsistency often causes slower traversal time or errors in robot state estimation.
Therefore, the capability of consistent behavior generation is essential for maneuverability while ground robots navigate over unstructured off-road terrains.
Over the past several years, autonomous groundrobots have been increasingly deployed in offroad ﬁeld environments to address real-worldapplications, including disaster response, homeland defense, and planetary exploration [1, 2, 3].
Field environments are challenging for groundrobots to navigate over because the terrain is unstructured and cannot be fully modeled beforehand, as depicted in Fig.
Terrain adaptation,the robot’s ability to adjust its behaviors to perceived terrains, is therefore an essential abilityto traverse over unstructured terrains [4, 5].
Given its importance, robot terrain adaptationhas recently been widely investigated.
Previouslearning-based methods can be divided into twobroad categories: terrain classiﬁcation and terrain adaptation.
The ﬁrst category uses a robot’sexteroceptive and proprioceptive sensory data toclassify terrain types and estimate traversabilityfor robot navigation over the terrain [6, 7, 8, 9, 10].
This category also includes techniques that modelterrain complexity for navigation planning [5, 11].
The second category of methods focus on directlygenerating adaptive behaviors according to terrain in order to successfully complete navigation tasks [5, 12, 13, 14].
Speciﬁcally, learning from demonstration (LfD) is widely used to transferhuman expertise to robots in order to achieve human-level robot navigational control [15, 14, 16].
However, the expected navigational behaviors generated by previous approaches cannot alwaysbe executed accurately by a ground robot when traversing over unstructured off-road terrain, i.e.,the actual and expected behaviors may not be consistent.
This inconsistency is mainly caused bysetbacks [17, 18] that are deﬁned as factors that increase the difﬁculty for a ground robot to achieve itsexpected behaviors.
Example setbacks include wheel slip, reduced tire pressure, and heavy payload.
Existing learning-based methods for robot navigation generally ignore these setbacks, which oftenleads to the robot not being able to consistently execute the learned behaviors.
The challenge of howto learn consistent navigational behaviors in a self-supervised fashion has not been well addressed.
To address this shortcoming, we develop an approach for consistent navigational behavior generation.
Our approach learns offset behaviors in a self-supervised fashion, allowing the robot to compensatefor the inconsistency between the actual and expected behaviors without explicitly modeling thesetbacks, while also adaptively navigating over changing terrain.
In addition, our method is ableto integrate multi-modal features to characterize terrain and estimate the importance of features toenable terrain-aware ground navigation.
This is all implemented in a uniﬁed regularized optimizationframework with a theoretical convergence guarantee.
The key novelty of this paper is the introduction of a method to enhance consistent ground maneuverability, which advances the state-of-the-art by enabling a ground robot’s actual behaviors to accuratelymatch its expected behaviors while adapting to a variety of complex unstructured off-road terrain.
The speciﬁc novel contributions include:
• We propose a novel mathematical formulation to generate consistent navigational behaviorsby learning offset behaviors in a self-supervised fashion.
We also introduce new regularization terms to learn important terrain features from multi-sensory observations and fuse themtogether to improve robustness of robot adaptation to unstructured terrain.•
We propose a new optimization
algorithm to address the formulated regularized optimization problem with dependent variables and non-term regularization terms, which holds atheoretical guarantee to effectively converge to the global optimal solution.
As an experimental contribution, we provide a comprehensive performance evaluation of learningbased terrain adaptation methods by designing a set of robot navigation scenarios over a wide varietyof individual and complex unstructured off-road terrains.
Related Work
Related research on robot terrain adaptation can be broadly classiﬁed into two categories, includingterrain classiﬁcation and robot adaptation.
Terrain classiﬁcation methods use sensory data from a robot to classify the terrain.
Many earliermethods were developed to address the speciﬁc needs of larger vehicles [19, 20] and the classiﬁcationprocess was typically performed in a manual or pre-selected fashion.
Some methods used a preexisting terrain map and terrain ruggedness data to achieve high-speed terrain navigation [6, 7, 9, 10,21, 22].
Learning-based methods are commonly used to classify terrain for navigation.
Color-basedterrain classiﬁcation was performed to generate navigational behaviors by labeling obstacles [23].
More recently, semantic segmentation neural network architectures [24] have been successfully usedto classify off-road terrain [25, 26] and perform robot navigation tasks accordingly.
However, thesemethods rely on a discrete categorization of terrain types.
The methods typically do not characterizecomplex unstructured off-road terrain well and cannot directly enable robot adaptation in real-worldenvironments, where environments have a wide variety of characteristics (as seen in Fig. 1).
Robot adaptation methods focus on enabling robots to intelligently adapt to various unstructured terrain.
The general problem of robot adaptation is commonly studied in robotics [27, 28, 29, 30, 31, 32]using high-level behavior models.
Earlier works considered ground speed as the optimization variableand formulated a method for trading progress and velocity with changing terrain characteristics[33, 34].
Learning-based methods for terrain adaptation have gained attention because of theireffectiveness and ﬂexibility [35].
Early work addressed terrain adaptation from the perspective ofonline learning that updates model parameters in the execution phase [36, 37, 38].
However, these methods lacked the ability to quickly adapt to sudden terrain changes.
Accordingly, methods weredeveloped that generate navigational behaviors using open-loop controllers according to the predictedterrain characteristics [39].
Methods to mimic expert controls were also developed aiming to achievehuman-level maneuverability [15, 14].
Recently, navigational affordances are learned from experts foroff-road navigation [40].
External disturbances were also considered in the learning-based methods[41, 42, 43].
However, these methods often require modeling of robot dynamics or explicit humandemonstrations, and cannot generate consistent ground navigation that adapt to diverse terrains.
Approach
Notation: We denote scalars using lowercase italic letters (e.g., m ∈ R ), vectors using boldfacelowercase letters (e.g., m ∈ R p ), matrices using boldface capital letters, e.g., M = { m ij } ∈ R p × qwith its i -th row and j -th column denoted as m i and m j , respectively.
We use boldface capital Eulerscript letters to denote tensors (i.e., 3D matrices), e.g., M = { m i ( k ) j } ∈ R p × q × r .
Unstacking tensor
M along its height ( p ), width ( q ) and depth ( r ) provides slices of matrices M i ∈ R q × r ,
M j ∈ R p × rand M ( k ) ∈ R p × q , respectively [44].
3.1 Problem Formulation for Terrain-Aware Navigation
As a robot traverses over terrain, at each time step, we extract multi-modal features from observationsacquired from multiple sensors installed on the robot including visual camera, LiDAR, and IMU.
We concatenate all features extracted at time step t into a vector and denote it as x ( t ) ∈ R d , whered = (cid:80) mj =1 d j and d j is the dimensionality of the j -th feature modality with m as the number ofmodalities.
We stack features extracted from a sequence of consecutive c time steps into a matrixand represent it as a terrain feature instance denoted as X
=
[ x ( t ) ; . . .
; x ( t − c ) ] ∈ R d × c .
We furtherdenote the set of n feature instances that are obtained as a robot traverses over various terrains, anddenote this set as a terrain feature tensor X =
[ X 1 , . . .
, X n ] ∈ R d × n × c .
We use Y =
[ y 1 , . . .
, y n ] ∈ R r × n to denote the robot’s expected navigational behaviors associatedwith X , where y i ∈ R r is a vector of r behaviors corresponding to X i .
Behaviors are encoded bythe control variables (e.g., linear and angular velocities) that decide the robot’s motion at the presenttime t .
Due to momentum, a robot often has continuous motion and observations; thus, considering ahistory of past c observations can provide more information to generate navigational behaviors at thepresent time.
Accordingly, we estimate the robot’s behaviors
y i using X i =
[ x ( t ) i ; . .
; x ( t − c ) i ] , takinginto account the history of c observations.
Then, the problem of navigational behavior generation canbe formulated as:
[Equation]
where W ∈ R d × r × c is a weight tensor used to encode the importance of each element in X towardsestimating navigational behaviors.
Each tensor element w i ( k ) j ∈ W denotes the weight of the i -thterrain feature from the k -th past time step to recognize the j -th behavior type.
The operator ⊗denotes the tensor product and ⊗ 3 is deﬁned as the tensor product that performs the sum of mode-3multiplication [44] between feature tensor X and W .
In Eq.
(1), the tensor product ⊗ 3 takes eachterrain feature instance X
i ∈
X , and multiplies it with the weight tensor W .
The ﬁrst term in Eq.
(1) is a loss function that encodes the error of using terrain features in X toestimate the robot behaviors, through the learning model parameterized by W .
Furthermore, our lossfunction encodes the non-linear nature of robot navigational behavior generation as a linear functionof terrain features X .
This can be achieved because: i) for short periods of time (i.e., c = [1 , 30] ), thedynamics of the robot do not dramatically change [45, 46] and thus the non-linearity in the robot isnot severe, and ii) with high-dimensional features (e.g., d > 10000 ), learning non-linear tasks can belifted to a linear space [47, 48].
The second term in Eq.
(1) is a regularization term named the featuremodality norm and is mathematically deﬁned as:
[Equation]
where (cid:107) .
(cid:107) F is the Frobenius norm and W i ∈
R b × c is the slice of the matrix obtained by unstackingthe weight tensor W along its height d .
The feature modality norm groups together weights withina feature modality and enforces sparsity among different modalities, thus, identifying the mostdescriptive features for behavior generation.
This is a critical capability for ground robot navigationsince different features typically capture different characteristics of the unstructured terrains (e.g.,color, slope, and roughness), and have different effects toward generating navigational behaviors.
The trade-off hyperparameter λ 1 in Eq.
(1) is used to balance the loss and the regularization term.
The problem formulation in Eq.
(1) allows ground robots to adapt their navigational behaviorsaccording to different terrain features.
However, due to setbacks that reduce the effectiveness ofrobot navigation, such as wheel slip, heavy payload, and reduced tire pressure [49], the robot’s actualbehaviors may not match the expected behaviors.
[Figure]
3.2 Consistent Navigational Behavior Generation
[Figure]
The key novelty focuses on a principled methodfor ground robots to generate consistent navigational behaviors that adapt to unstructured terrain.
As illustrated by Fig.
2, besides generatingterrain-aware expected behaviors, our methodmonitors the difference between the actual andexpected navigational behaviors caused by setbacks, and computes an offset to reduce the difference.
This allows our approach to achieveconsistent robot behaviors without the requirement of explicitly modeling all of the setbacks.
Mathematically, we denote the actual behaviorsas A =
[ a 1 , . .
. , a n ] ∈ R r × n , where a i ∈ R r denotes the actual behaviors executed by the robotwhen observing the i -th terrain feature instance X
i .
The actual robot behaviors can be estimatedusing a pose estimation technique, e.g., based on SLAM or visual odometry [50].
Setbacks can causethe robot’s actual behaviors to not match its expected behaviors.
This difference in navigationalbehaviors over the past c -time steps is computed as E =
[( a ( t ) − y ( t ) ); . . .
; ( a ( t − c ) − y ( t − c ) )]
∈ R r × cand the behavior differences for all the instances in X is denoted as a behavior difference tensor E =
[ E 1 , . . .
, E n ] ∈ R r × n × c .
We then introduce a loss function to encode consistent behavior generation as follows:
[Equation]
where U =
[ U 1 , . .
, U r ] ∈ R r × r × c is a weight tensor with U j ∈ R r × c indicates the importanceof behavior differences E towards generating the j -th offset behaviors.
Using data from a historyof c time steps allows our method to consider inertia (i.e., resistance to change in behaviors) duringnavigation.
Mathematically, this implies that U ( k ) , k = 1 , . . .
, c ; has non-zero elements as opposedto each element in U ( k ) being 1 at k = 1 and 0 at k (cid:54)
= 1 when robot inertia is not considered.
The loss function in Eq.
(3) models the actual behavior by considering both terrain features X andthe behavior differences E to achieve consistent navigational behaviors.
Because of inertia, historicaldata from different past time steps may contribute differently towards generating offset behaviors(e.g., a heavier robot with larger inertia often needs to consider a longer history).
Thus, we proposea new regularization term to explore which time steps in the historical data are more importantfor generating the offset behaviors.
We name this regularization term the temporal norm, which isexpressed by:
[Equation]
(4) where U ( k ) =
[ u ( k ) 1 , . .
, u ( k ) r ] ∈ R r × r is the weight matrix, and u ( k ) j indicates the importance of thej -th behavior difference a ( k ) j − y
( k ) j from k -the past time step towards generating offset behaviors.
This norm groups together weights for the vector of behavior differences at each time step, andenforces sparsity between weights at different time steps to identify the most important time steps.
Using both norms to generate consistent robot navigational behaviors while identifying importantfeature modalities and historical time steps, the ﬁnal objective function becomes:
[Equation]
where λ 1 ≥ 0 and λ 2 ≥ 0 are trade-off hyper-parameters.
After computing the optimal values of the weight tensors W and U according to Algorithm 1, inthe training phase, a robot can apply our self-reﬂective terrain-aware adaptation method to generateconsistent navigational behaviors during execution.
At each time step t 0 in the execution phase, therobot extracts multi-modal features X t 0 from observations obtained from its on-board sensors overthe past c -time steps.
Then, the robot also estimates the corresponding actual behaviors (measuredusing a pose estimation technique based upon SLAM or visual odometry [50]) and computes thematrix of behavior differences at time t 0 as E t 0 .
Then our approach can be used by the robot togenerate consistent actual navigational behaviors as:
[Equation]
The ﬁrst term in Eq.
(6) generates the expected navigational behaviors according to the terrain ittraverses, which allows a ground robot to adapt its navigational behaviors to unstructured terrains.
The second term in Eq.
(6) provides offset behaviors based on monitoring the difference betweenactual and expected behaviors in order to compensate for the setbacks.
Optimization Algorithm
The optimization problem in Eq.
(5) is challenging to solve because the regularization terms arenot smooth and because the objective function includes dependent variables.
Thus, we derive a newiterative optimization algorithm to obtain the optimal solution to Eq.
This algorithm is shownin Algorithm 1, which provides an alternating minimization algorithm that alternatively updates theparameter tensors in each iteration until convergence.
To solve for the optimal weight tensor W , we minimize Eq.
(5) with respect to W ( k ) resulting in:
[Equation]
where each Q ( k ) ∈ R d i × d i is a diagonal matrix with the i -th diagonal block computed by
[Equation]
W i ( k ) (cid:107) F I d i and I
d
i is an identity matrix.
Then, we compute each W ( k ) as:
[Equation]
[Equation]
Because each of the block-diagonal matrices Q ( k ) are dependent on W and also each slice of matrix
W ( k ) is dependent on corresponding Q ( k ) , an iterative algorithm is required to compute them.
To compute the optimal U , we calculate the derivative of the objective function in Eq.
(5) with respectto U ( k ) and set the equation to zero as:
[Equation]
where P ∈ R r × r is a diagonal matrix with the k -th diagonal block as
[Equation]
[Equation]
F I r , with I r being anidentity matrix.
Then, we compute each U ( k ) in a closed-form solution as:
[Equation]
This U is then used to calculate W in the next iteration.
Similar to Q ( k ) and W , P and U areinterdependent.
Thus, we develop an iterative algorithm to solve the formulated optimization problem,which is described in Algorithm 1.
Convergence.
Algorithm 1 is guaranteed to converge to the global optimal solution to the formulatedregularized optimization problem in Eq.
The proof is provided in the supplementary material.
Complexity.
As the optimization problem in Eq.
(5) is convex, Algorithm 1 converges fast (e.g.,within tens of iterations only).
In each iteration, computing Steps 3 and 5 is trivial.
Steps 4 and 6 canbe computed by solving a system of linear equations with quadratic complexity.
Experiments
We utilize the Clearpath Husky robot in our ﬁeld experiments.
The robot is equipped with an Intel
Reasense D435 color-depth camera and an Ouster OS1-64 LiDAR.
The robot also has a variety ofsensors to measure its internal states, including IMU readings, wheel odometry, motor speed, andbattery status.
Linear interpolation is used to get a steady 30 Hz frame rate from all sensors.
To represent unstructured terrain, we implement multiple visual features extracted from color imagesto describe different terrain characteristics, including Histogram of Oriented Gradients (HOG)
[51]to describe the shape and Local Binary Patterns (LBP)
[52] to describe texture.
We also computean elevation map [53] from LiDAR data to represent the grid-wise elevation of the terrain aroundthe robot.
In the training phase, expected navigational behaviors are provided by remote control thatnavigates the robot over unstructured terrains as fast as possible while maintaining safety.
Duringtraining and execution phases, actual behaviors are estimated from LiDAR-based SLAM [50] as robotpose.
We use a sequence of 15 frames (i.e., c = 15 ), and λ 1 = 0 .
1 and λ 2 = 10 for all experiments.
More details on our algorithm’s implementation and training/testing procedures are provided in thesupplementary material.
We compare our approach with several previous state-of-the-art learning-based robot navigationtechniques, including Learning from Demonstration (LfD) for robot navigation [16], multi-modal LfD(MfD) [54]
[Table]
and Terrain Representation and Apprenticeship Learning (TRAL) [15].
To quantitativelyevaluate the performance of robot navigation, we use four metrics:
[Table]
L.
[Table]
• Failure Rate (FR) :
This metric is deﬁned as the number of times the robot fails to completethe navigation task across a set of experimental trials.
If a robot ﬂips or is stopped by aterrain obstacle, it is considered a failure.
Lower values of FR indicate better performance.
• Traversal Time (TT) :
This metric is deﬁned as the time taken to complete the navigation task
over given terrain.
Smaller values of TT indicate better performance.
• Inconsistency : This metric is deﬁned as the error between the expected behavior and the
actual behavior in terms of robot poses (linear position and angular).
Lower values of
inconsistency indicate better performance.
• Jerkiness : This metric is deﬁned as the average sum of the acceleration derivatives along all
axes, with lower values indicating better performance.
Jerkiness indicates how smooth a
robot can traverse over a terrain.
Because state estimation and SLAM methods (e.g., basedon Kalman ﬁlters) may assume smooth robot motions, jerkiness is a useful metric.
5.1 Navigating over Individual Types of Unstructured Terrain
In this set of experiments, the robot navigates over individual terrain using off-road tracks.
Eachtrack is made up of one type of terrain and is approximately ten meters long.
Five types of terrain areused in our experiments, which are illustrated in Fig.
Our approach is trained on data collectedwhile the robot is manually controlled by an expert to traverse the terrain.
Then, the learned model isdeployed on the robot to autonomously navigate over the terrain.
Evaluation metrics for each methodare computed across ten trials on each type of individual terrain track.
The quantitative results achieved by our approach and the comparison to other methods are presentedin Table 1.
For simple individual terrains, such as grass and gravel, all methods allow the robot tosuccessfully traverse over the terrain.
However, for more challenging terrain, especially large rocks,both LfD and MfD have a high failure rate, whereas, TRAL and our approach have a low failurerate.
Our approach only has one failure over the difﬁcult large-rock terrain and outperforms othertested methods.
The presented traversal time is computed by averaging the traversal time across allsuccessful runs, i.e., it excludes the failed trials captured by the FR metric.
It is observed that allmethods have a similar traversal time.
In successful runs, both LfD methods show less traversal timecompared to other methods over rocky terrain, although they also have a much higher failure rate.
Thus, the emphasis on high-speed traversal used by both the LfD methods produces an unreliablesystem when the robot traverses unstructured off-road terrain in real-world ﬁeld environments.
Table 1 also presents the quantitative results for the inconsistency and jerkiness metrics.
We observethat both LfD and MfD methods do not perform well and have higher values of inconsistency overindividual types of terrain, especially on the large-rock terrain.
TRAL has lower inconsistency andperforms the best over the grass terrain.
Our proposed method outperforms the previous approachesand obtains the lowest averaged inconsistency value.
Finally, we also evaluate the tested methodsusing the jerkiness metric.
An observation is that the medium-rock terrain causes the largest jerkinessmeasure.
This is because medium-rocks terrain as compared to the large-rocks produce much morevibrations for even slow maneuvers.
[Table]
5.2 Navigating over Complex Off-road Unstructured Terrains
In the second set of experiments, we evaluate our approach when the robot navigates over complex offroad unstructured terrain.
The tracks in these experiments either show transitions between differentterrain types (i.e., grass to large rocks: Gr.
L.R, and grass to medium rocks: Gr.M.R) or a mixture ofdifferent terrain types in real off-road environments (i.e., Mixed Terrain I: M.T-I and Mixed Terrain II:
M.T-II), as shown in Fig.
In the experiments, no additional training is performed and the previouslytrained model from individual types of unstructured terrains is used directly.
Table 2 presents the quantitative results obtained by our approach and the comparison with othermethods.
It is observed that each of the methods have a much higher failure rate in general, especiallyover the M.T-II and the Gr.
L.R terrains.
Our approach signiﬁcantly outperforms LfD and MfD interms of failure rate.
Similar to the experiments over individual types of terrain, we observe thatboth LfD methods have slower traversal time for successful runs, but they have a signiﬁcantly higherfailure rate.
Moreover, LfD and MfD are outperformed with respect to inconsistency and jerkiness,with MfD performing worst among all tested methods, especially on the jerkiness metric.
Although both TRAL and our method obtain promising performance,as the main goal is to enhance consistent maneuverability, our approachobtains an average of 15.83% less on inconsistency over TRAL (with7.66% less over traversal time, 4.50% less over failure rate, and 4.56%less on jerkiness).
The p-value for inconsistency improvement is 0.004,indicating that the improvement is statistically signiﬁcant.
This improvement is most likely to be caused by the closed loop feedback, as it is thebiggest fundamental difference between the methods.
5.3 Discussion
Hyperparameter Analysis: The hyperparameters λ 1 and λ 2 in Eq.
(5)are implemented to balance the loss function and regularization terms.
Fig.
5 depicts how the inconsistency metric changes given varying λvalues based on cross-validation during training.
It is observed thatλ 1 ∈ (0 . 1 , 10) and λ 2 ∈ (1 , 10) result in good performance in general.
The best result is obtained when λ 1 = 0 .
1 and λ 2 = 10 .
These valuesof hyperparameters are then used for all the experiments.
Dependence on Frame Sequences: Our approach uses a sequence ofhistorical frames with length c to generate consistent behaviors.
Fig.6 shows the change of the inconsistency metric according to c .
It isobserved that our approach generally performs well when c ∈ (15 , 20) ,and we observe that the inconsistency metric is worst when either a smallnumber ( c < 5 ) or a big number ( c > 30 ) is used under the sensingframerate of 30 Hz.
These values can be mainly affected by the robot’sspeed and can differ between robotic platforms.
Convergence: Experimental results in Fig.
7 illustrate the fast, monotonically decreasing convergence of Algorithm 1, which validates ourtheoretical analysis.
Conclusion
In this paper, we introduce a novel approach for consistent behavior generation that enables groundrobots’ actual behavior to more accurately match expected behaviors while adapting to a variety ofunstructured off-road terrain.
Our approach learns offset behaviors in a self-supervised fashion tocompensate for the inconsistency between the actual and expected navigational behaviors withoutthe need to explicitly model various setbacks, and learns the importance of the multi-modal featuresto improve the representation of terrain for better adaptation.
Our proposed approach is extensivelyevaluated in real-world off-road environments.
Experimental results have shown that our approachenables a robot to improve its ground maneuverability when traversing over complex unstructuredoff-road terrain with more behavior consistency and smoothness compared to previous methods.