3-D shape control of deformable linear objects for branch
handling using an adaptive Lyapunov-based scheme

Omid Aghajanzadeh, Mohammadreza Shetab-Bushehri, Miguel Aranda, Juan Antonio Corrales Ramon, Christophe Cariou, Roland Lenain, Youcef Mezouar

► To cite this version:

Omid Aghajanzadeh, Mohammadreza Shetab-Bushehri, Miguel Aranda, Juan Antonio Corrales Ramon, Christophe Cariou, et al.. 3-D shape control of deformable linear objects for branch handling using an adaptive Lyapunov-based scheme. Computers and Electronics in Agriculture, 2025, 232, pp.109931. 10.1016/j.compag.2025.109931. hal-04963112

HAL Id: hal-04963112

https://hal.inrae.fr/hal-04963112v1

Submitted on 10 Dec 2025

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
3-D shape control of deformable linear objects for
branch handling using an adaptive Lyapunov-based
scheme

Omid Aghajanzadeh¹ª, Mohammadreza Shetab-Bushehriᵇ, Miguel Arandaᶜ,
Juan Antonio Corrales Ramonᵈ, Christophe Cariouª, Roland Lenainª,
Youcef Mezouarᵇ

<sup>a</sup>Université Clermont Auvergne, INRAE, URTSCF, 9 av. Blaise Pascal CS 20085,
F-63178 Aubière, France

<sup>b</sup>Université Clermont Auvergne, CNRS, Clermont Auvergne INP, Institut Pascal,
F-63000 Clermont-Ferrand, France

<sup>c</sup>Instituto de Investigación en Ingeniería de Aragón (I3A), Universidad de
Zaragoza, E-50018 Zaragoza, Spain

dCentro Singular de Investigación en Tecnoloxías Intelixentes (CiTIUS), Universidade
de Santiago de Compostela, 15782 Santiago de Compostela, Spain

**Abstract**

Despite its various applications, robotic manipulation of deformable ob-
jects in agriculture has experienced limited development so far. This is due to
the specific challenges in this domain, i.e., the variety of objects in this field
is wide, and the deformation properties of the objects cannot be easily rec-
ognized in advance. In addition, deformable objects generally have complex
dynamics and high-dimensional configuration space. In this paper, the ma-
nipulation of deformable linear objects (DLOs) is addressed by considering
these challenges. Concretely, a new indirect adaptive control method is pro-
posed to manipulate DLOs by controlling their shape in 3-D space towards
previously defined targets, with a specific focus on agricultural applications.
The proposed method can follow a desired dynamic evolution of the shape
with a smooth deformation that brings about a stable gripper motion. This
property of the method can protect the object from possible damages, even
under large deformations, which is crucial in agricultural scenarios. An adap-
tation law is leveraged for estimating the system parameters, and Lyapunov

<sup>1</sup>Corresponding author: Omid Aghajanzadeh, Email: omid.aghajanzadeh@inrae.fr
analysis is employed to study the validity of the proposed control scheme.
The scheme can be applied to diverse objects that can be modeled as lin-
ear, including tree branches or other rod-like structures. The effectiveness of
the scheme is demonstrated through various experiments where, using shape
feedback obtained from a 3-D camera, a robotic arm controls the shape of a
flexible foam rod and of branches of different plants.

Keywords:

deformable linear object, robot manipulation, adaptive control, tracking of
deformation, parameters estimation, agricultural robotics.

1. Introduction

The necessary transition of agricultural practices avoiding the use of
chemicals to limit environmental impact has become a major concern, with
a significant impact on agriculture (Agovino et al. 2019). Several new agri-
culture practices have then been proposed to tackle an environment-friendly
production (Springmann et al. 2018) (such as Precision Agriculture, Organic
Farming, and more generally the application of agroecological principles).
Nevertheless, such practices are difficult to be applied at large scale and re-
quire crop monitoring and frequent treatments implying a huge amount of
manpower. As a result, robotics arises at a promising solution (McGlynn
and Walters 2019), and many progresses have been made, particularly with
respect to autonomous navigation in the field. However, contrarily to au-
tonomous cars, agricultural robots are not devoted to the sole navigation
task and have to achieve actual actions on soil and crops (such as sowing,
weeding, pruning, or harvesting). To this aim, robots have to be able to
manipulate vegetation, which can be considered as soft and fragile objects
from a robotic point of view.

Until recent years, most robotic manipulation studies have focused on
rigid objects, i.e., objects that maintain their shape during manipulation.
As opposed to rigid objects, there is a wide range of objects whose deforma-
tion cannot be neglected during the manipulation process, such as vegetables,
metal sheets, and clothes. These types of objects are known as deformable
objects. Since deformable objects are ubiquitous, automating their manip-
ulation has been the focus of attention in recent years. Examples range
from agricultural applications (for instance, in fruit harvesting) to industrial
applications (such as wire harnessing). However, when it comes to robotic
27

28

29

30

31

manipulation, effectively deforming these types of objects is very challenging. This is due to the fact that they have an infinite number of degrees of freedom, they can go under self-occlusions, there are local deformations in each part of these objects, and finally, the types of deformations vary from one object to another (Sanchez et al. 2018).

32

33

34

35

36

37

38

39

40

41

42

43

44

Most early research on this topic aimed to precisely model the deformable objects to predict their deformation (James and Pai 2002; Lin et al. 2014; Lin et al. 2015; Marchesseau et al. 2010). The main drawback of these methods is that they require offline information to model deformable objects, which is not possible in many applications such as agriculture where deformation properties change from one natural product to another. Later, several studies proposed manipulation methods without an exact deformation model of deformable objects. They mainly used approximations of the deformation behavior (Lagneau et al. 2020; Zhu et al. 2021; Shetab-Bushehri et al. 2022). These methods require other offline or online methods to calculate a Jacobian matrix (i.e., relating the motion of the robot with the changes in deformation of the grasped object), which sometimes could be a challenging task. They also cannot be used for large deformations.

45

46

47

48

49

50

51

52

53

54

55

56

Many researchers have made efforts to design appropriate controllers to manipulate deformable objects. One of the first successful model-free works to control the deformation of an object using a robot can be found in (Navarro-Alarcon et al. 2014). This paper presents two vision-based adaptive methods that do not need prior identification of the object deformation model and the camera parameters. The studied object is assumed to be quasi-static and purely elastic. Although the controllers successfully performed the desired tasks, the main drawback of this work is that it can only be used for small deformations. This is not the case in the agricultural context, where the objective is precisely to have large deformation. This is for instance the case for the pruning application, or for clearing vegetation that interferes with observation or actions to be carried out.

57

58

59

60

61

62

63

64

Later, an adaptive controller was presented in (Navarro-Alarcon et al. 2016) to deform an elastic object in 3-D, also without requiring the deformation model. Based on manipulator's input motion and the velocity of deformable feature points, an estimation of the deformation model was computed, and a velocity controller was presented. However, the proposed method required a proper choice of the feature points, and it was not able to handle small curvature deformations appropriately. (Navarro-Alarcon and Liu 2017) proposed a vision-based controller to manipulate the entire shape
65

66

67

68

69

70

71

72

73

74

75

76

of the object using its 2-D shape based on the Fourier series. This method was well adapted to the cases where the shape of the object can be correctly described with a reduced number of Fourier coefficients. In (Alambeigi et al. 2018), a robust data-driven control framework was presented that was capable of indirectly manipulating heterogeneous 3-D compliant objects in the presence of unknown internal and external disturbances. The Jacobian matrix that relates the motion of the grasping point to the motion of feature points was estimated online using the Broyden update rule. (Delgado Rodríguez et al. 2017) introduced a model-free approach for deformable objects manipulation according to tactile images. The proposed algorithm introduced a way to represent and use tactile information based on a combination of dynamic Gaussians defined from the sensor values.

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

A particular subdomain in the robotic manipulation of deformable objects that has attracted the most attention in the past few years is the study of deformable linear objects (DLOs) manipulation (Qi et al. 2022; Koessler et al. 2021; Aghajanzadeh et al. 2022a; Bretl and McCarthy 2014). In this regard, motion of a flexible beam is controlled using a simple PD controller in (Dadfarnia et al. 2004). A nonlinear model-based energy shaping controller was presented in (Gandhi et al. 2016) to solve the simplified model of an ultra-flexible inverted pendulum on a cart. The objective of that study was to maintain the flexible inverted pendulum in the upright position while the cart was stopped at a desired location. (Zhu et al. 2018) proposed an algorithm to control the 2-D shape of a DLO (a wire) using two robotic manipulators. In that paper, the DLO's shape is estimated by a Fourier series. A least-squares minimization scheme is adopted to calculate the Jacobian matrix. The authors proposed to control Fourier coefficients which are a simplified representation and do not represent the actual shape as precisely. A feed-forward control approach is proposed and evaluated in (Kater and Meurer 2019) to develop a solution to the motion planning problem for a structure consisting of coupled flexible bending beams. In (Aghajanzadeh et al. 2022c), a control framework for manipulating DLOs with a robotic arm without knowing the object's properties on a 2-D workspace is proposed.

97

98

99

100

101

102

A specific feature in manipulating plants is that each object has different characteristics for elasticity, and may be broken by an inappropriate deformation motion. As a result, it is crucial to adapt the deformation strategy to the object's properties. In that paper, an optimal controller based on a model-based Jacobian is then proposed to automatically drive the entire shape of DLOs toward the target shapes. (Aghajanzadeh et al. 2022b) developed a
controller that used an offline geometric model to drive a set of shape features on DLOs toward their targets. This technique does not require perceiving the full shape of the object or to estimate/simulate a deformation model at run time. However, it is limited to 2-D deformations, and the type of objects it may be applied to is confined. (Li et al. 2019) proposed a method to reshape the object by moving several points on the object toward their corresponding desired positions. (Kinio and Patriciu 2012) used indirect deformable object manipulation to move a few points on the object to their corresponding targets. Also in our previous work (Aghajanzadeh et al. 2022a), a novel adaptive controller method was proposed to move an arbitrary point along the length of the object toward a target. Although this methodology was supported by formal analysis and validated in various experiments, it is limited to 2-D space and can only adjust one point's pose. In comparison, the approach proposed in the current paper is capable of controlling the entire shape of the object. It is worth mentioning that recent studies attempted to use machine learning to address DLO manipulation problems (Yu et al. 2022; Yan et al. 2020). However, these methods need significant efforts to train an algorithm to deform the objects. In contrast with this group of methods, our approach does not need any training or offline information on objects' deformation.

While the previously mentioned studies offer valuable insights into the manipulation of DLOs across various applications, they largely ignored the specific challenges and needs associated with agricultural applications. The main goal of the current work is to extend previous shape servoing approaches to agriculture applications, such as pruning trees (Davenport 2006; You et al. 2020; Zahid et al. 2021; Tinoco et al. 2021), fruit detection (Zhuang et al. 2018; Kang and Chen 2020), and harvesting (Davidson et al. 2020; Stavridis et al. 2022).

In most existing agricultural manipulation studies, robotic harvesters employ a single arm that grasps and detaches crops by cutting or by performing a detachment motion (Stavridis et al. 2022). Recently, (Stavridis et al. 2022) proposed a bimanual control methodology, which coordinates the end-effector velocities of a camera arm to reveal the stem while a grasping arm manipulates the surrounding foliage to create space. However, no specific controller has been developed to precisely control the shape of the stem to avoid damage in these works. Agricultural manipulation tasks often involve selecting a robotic manipulator (arm and gripper) that is suitable to handle and apply the necessary forces based on the task such as fruit or vegetable
detachment (Davidson et al. 2020). Thus, the method is only applicable for
certain tasks. Robotic manipulation in agriculture typically treats objects as
rigid. For instance, Yandun et al. (2020) modeled tree branches as a series
of rigid objects, while in reality, a significant portion of stems and trees is
flexible.

In this work, we begin by focusing on the manipulation of agricultural branches, which are relevant for tasks such as fruit harvesting and tree pruning. Specifically, we investigate agricultural objects that can be classified as DLOs. These objects are flexible, continuous structures characterized by a one-dimensional shape, such as plant stems, that can be manipulated along their length in response to external force. In the agricultural applications, it may be necessary to bend and move these type of objects, such as green branches, aside to access fruits or cut dead branches. Preventing damage to the green branches is crucial, as improper bending can cause major harm to the tree and shorten its lifespan. The main challenges are: first, the diversity of objects in this field is vast and consequently a specific model and controller cannot be used to model and manipulate all branches; second, determining the properties of objects is not always possible; and third, objects are mostly very fragile and a stable robotic motion is essential.

The state-of-the-art methods previously described had not considered the
aforementioned challenges in this field. To the best of the authors' knowl-
edge, this topic (i.e., robotic manipulation of deformable objects in the agri-
cultural field) has not received significant attention in the former studies.
One of the very first works in agriculture that assumed robots are dealing
with deformable objects can be found in (Tanner et al. 2001). In that study,
the object is assumed to be a rod modeled using the finite element method
(FEM). However, that work mainly focused on obtaining equations of a mul-
tiple mobile manipulator system, and no shape controllers were presented.
Inspired by that work, we assume that the branches can be considered as
DLOs. However, we do not need an FEM representation of the DLOs as in
(Tanner et al. 2001) but only a set of controllable points. Recently, we set
a new milestone in this domain and attempted to control an arbitrary point
along the length of the stems and branches in the article (Aghajanzadeh
et al. 2022a), discussed above. That work focused on controlling a single
point along the length of a DLO in 2-D without considering the final shape
of the object. Therefore, in that paper, it was possible to achieve various
final shapes for the same target point. Although that method was effective
in achieving its objective, it was recognized that it did not consider the full
shape of the object and was not designed to work in 3-D. Thus, in the current
paper, we propose an extension of our previous work (Aghajanzadeh et al.
2022a) to enable the control of DLO shapes in 3-D. This new method can
enhance the practical applicability and expands the scope of potential use
in various applications, including agriculture, which is a novel application in
the field of DLO control.

In particular, the agricultural field presents a significant challenge due to
the large number of deformable objects involved. Our objective is to propose
a general method that can be applied to a wide range of objects without
requiring additional computation or modification, which represents a signif-
icant advantage. Furthermore, we aim to open a new path for research in
the agricultural field since no method has been proposed for manipulating
the entire shape of agricultural deformable objects. To validate the effec-
tiveness of the proposed approach, we report on experiments where, using
shape tracking data from a 3-D camera, the shapes of a flexible foam rod
and branches of different plants are controlled with a robotic arm.

1.1. *Our contribution*

This study proposes an indirect model-free deformation method of DLOs
in 3-D space to be used in the agricultural field. Apart from agriculture,
this type of object (DLO) is also one of the most common objects of interest
in other industrial applications (Cao et al. 2020; Brown et al. 2004; Kang
and Wen 2002; Wang et al. 2015). Therefore, the proposed method is also
applicable in other domains.

We aim to control the full shape of a DLO by using a robotic gripper
grasping the object at one of its ends (i.e., indirect control), without knowing
the object's properties (i.e., model-free). An adaptive control strategy is
proposed to achieve the stated objective. Our contributions can be listed as
follows:

• The adaptive method introduced in (Aghajanzadeh et al. 2022a) is extended to deform the shape of DLOs in 3-D space. In particular, the proposed method is suitable for manipulating stems or branches for pruning, plant inspection, or fruit harvesting operations.

• The proposed method works in real-time and is suitable for large deformations. Our proposed method considers and controls the full shape of the object and not a partial or simplified representation of it.

• In contrast to (Li et al. 2019) and related adaptive deformation control approaches (Lagneau et al. 2020; Navarro-Alarcon and Liu 2017; Navarro-
Alarcon et al. 2014; Zhu et al. 2018), the proposed method can be used to track a desired dynamic evolution of the entire shape of the object, rather than simply drive fixed set-points to their corresponding targets. This gives better control over the evolution path of the object and the completion time of the task. It also provides a smooth deformation since time-varying trajectories are tracked instead of fixed distant targets, which helps to have a deformation without sudden changes. This can protect the object from possible damages.

• The proposed method does not require any prior information regarding object's deformation. In addition, there is no need to compute any offline Jacobian matrix to know how displacements of the manipulator are mapped to deformations of the object.

## 1.2. Paper organization and notation

The remainder of the paper is organized as follows: section 2 describes the studied problem; the methodology and controller scheme are described in section 3; the experimental design and experimental results are presented in sections 4 and 5; a discussion is provided in section 6; and finally, the conclusion is summarized in section 7.

Throughout this paper, the following mathematical notation is employed: capital letters are used to represent vectors and matrices, e.g., $M \in R^{q \times h}$ while scalar parameters and variables are expressed by lower-case letters, e.g., $x$.

<figure><img src="image_2.png" /><figcaption>Figure 1: Representation of the DLO in the current study. From left, the first image depicts the object under study (e.g., a plant branch); the second image shows the discretized object; and the third image displays the distributed controlled points along the object's length ($L$), representing the object's entire shape.</figcaption></figure>
## 2. Problem definition

In this study, our main focus is to control deformable objects in the field of agriculture that can deform like an elastic linear rod (such as branches and twigs of plants and trees). We consider these objects as DLOs, and we put our effort into designing a method for controlling their shapes. Since we deal with branches of plants, we presume one of the ends is fixed (connected to the ground by the roots), while the other end is grasped by the robot, as illustrated in Fig. 1. The DLO is represented discretely with a set of points, which are ordered consecutively starting from the fixed end of the object to capture the entire shape. We adopt $p_i$ ($i \in \{1, ..., 6\}$) to represent the six coordinates (position and orientation) of each point along the length of the object, as depicted in the figure. This is a suitable representation for the types of deformations considered in our scenario. It is assumed that the position and orientation of each point during manipulation can be obtained. To achieve this, a sensor (an Intel RealSense 3-D camera) is used in our experiments to track the object's shape and determine the state of the controlled points.

In this paper, it is assumed that using a robotic gripper, we are able to deform the whole object by controlling the state of the endpoint. $G$ is the state of the object at the grasped point $g$ (where $G = [p_{1g}, p_{2g}, p_{3g}, p_{4g}, p_{5g}, p_{6g}]^T$). The full state of each controlled point is $[p_{1l}, p_{2l}, p_{3l}, p_{4l}, p_{5l}, p_{6l}]^T$ (position and orientation of controlled point number $l$ in 3-D). We have $n$ controlled points, indexed $1, ..., n$. $p_{il}$ represent the states of any point along the length of the object that is not being grasped (i.e., the controlled (servoed) points $m_l$). $l$ is used to represent the servoed point's number, where $l \in \{1, ..., n\}$. We define $M_l$ as the state of $m_l$, which contains the information of the point number $l$ along the object's length.

Our objective is to deform the object from an initial shape $I_s$ to a target shape $F_s$ (as shown in Fig. 2). Note that each of these shapes can be represented by the states of the controlled points at that shape. Hence, to achieve the mentioned objective, we should drive the error $E_l$ for every $l$ to zero, where $E_l$ is defined as:

$$E_l = M_l - M_{lf}, \qquad (1)$$

and the set of $M_{lf}$ expresses the target shape of the controlled points $m_l$. The target should be within the workspace of the used robotic arm, in a way that the object can reach this target. It is also assumed that the deformation
<figure><img src="image_3.png" /><figcaption>Figure 2: The initial ($I_s$) and target ($F_s$) shape of the studied object.</figcaption></figure>

process will not result in any significant sharp deformation along the length of the object. $E_l$ indicates the error between the current and the target shapes. By leading this error to zero, the object is deformed to the target shape.

<table><caption>Table 1: Variables used in the proposed controller scheme</caption><tr><td>$M_l = [p_{1l}, p_{2l}, p_{3l}, p_{4l}, p_{5l}, p_{6l}]^T$</td><td>State of the controlled points $l \in \{1, ..., n\}$</td></tr><tr><td>$M_s = [p_{1s}, p_{2s}, p_{3s}, p_{4s}, p_{5s}, p_{6s}]^T$</td><td>Average of the controlled points states</td></tr><tr><td>$M_{sf} = [p_{1sf}, p_{2sf}, p_{3sf}, p_{4sf}, p_{5sf}, p_{6sf}]^T$</td><td>Average of the target points states</td></tr><tr><td>$M_{sdes} =$</td><td>Desired trajectories</td></tr><tr><td>$[p_{1sdes}, p_{2sdes}, p_{3sdes}, p_{4sdes}, p_{5sdes}, p_{6sdes}]^T$</td><td></td></tr><tr><td>$\tilde{M}_s = [\tilde{p}_{1s}, \tilde{p}_{2s}, \tilde{p}_{3s}, \tilde{p}_{4s}, \tilde{p}_{5s}, \tilde{p}_{6s}]^T$</td><td>Tracking errors</td></tr><tr><td>$P_i$</td><td>Vectors of the states $i \in \{1, ..., 6\}$</td></tr><tr><td>$\dot{C}_i$</td><td>Adaption laws</td></tr><tr><td>$\hat{C}_i$</td><td>Vectors of the estimation of the parameters</td></tr></table>

### 3. Methodology

In this section, a controller is developed to drive every $E_l$ to zero. Since the gripper fixes the velocity of the grasped point, the controller input $U = [u_1, u_2, u_3, u_4, u_5, u_6]^T$ is set as follows:

$$U = \dot{G}. \qquad (2)$$
For the set of points used to represent the object's shape, there is a relation between their displacements and also their velocities with the motion of the gripper $\dot{G}$ (Aghajanzadeh et al. 2022a; Navarro-Alarcon et al. 2014; Ogden 1997). It is important to note that the map between the position and velocity of the control points and the motion of the gripper is not static but is dependent on the object's deformation. Previous approaches (Navarro-Alarcon and Liu 2017; Zhu et al. 2018) calculated this relation using a deformation Jacobian at each time step. In our proposed approach, the objective is to simplify this relationship that can be used to design an adaptive controller that can estimate the map between the velocity of the gripper and the motion of the object without requiring a deformation Jacobian at each time step. Accordingly, one can write:

$$ \dot{G} = F(M_l, \dot{M}_l \text{ (for all } l \in \{1, \dots, n\} \text{)}). \qquad (3) $$

To provide a method to move these points to their corresponding targets, we assume a relation (i.e., (3)), between the current state of the points along the length of the object ($p_{il}$) and the controller inputs ($u_i$, where $i \in \{1, \dots, 6\}$) at each time instant. It is assumed that the deformation process should occur quasi-statically and not at high speed.

This study deals with an underactuated system since the number of actuated degrees of freedom is lower than the number of controlled degrees of freedom. In this context, it has been observed that the system is only locally stable with respect to a fixed desired state. This is analogous to standard control strategies in underactuated control systems, and in particular, to some existing works in deformable object shape control (Navarro-Alarcon et al. 2013; Zhu et al. 2018). In this paper, this problem is assumed by controlling a new variable, denoted as $M_s = [p_{1s}, p_{2s}, p_{3s}, p_{4s}, p_{5s}, p_{6s}]^T$, where $p_{is}$ (as shown below in (4)) represents the average of the state (position and orientation) of all the controlled points $p_{il}$:

$$ p_{is} = \frac{\sum_{l=1}^{n} p_{il}}{n}. \qquad (4) $$

In other words, the value of $M_s$ contains information about the complete state of the object. Since the system is locally stable and operates quasi-statically, we control the object locally at each time step. Thus, by regulating $M_s$, we achieve control over the entire shape.
In this section, an adaptive control method is designed to drive $M_s$ toward its target which results in effectively controlling the full shape of the deformable object. This is due to the fact that $M_s$ is obtained based on the average of all the controlled points, and by controlling it, each $E_l$ for all servoed points $l \in \{1, ..., n\}$ can be driven to zero. To do that, we draw inspiration from model-free schemes that have been proposed for deformable object manipulation (Navarro-Alarcon and Liu 2017; Zhu et al. 2018). However, to design our method, we also use some additional terms. These terms allow us to track a desired deformation, instead of just driving a group of points to their corresponding fixed targets. The proposed controller formulation uses adaptive control techniques (Slotine et al. 1991). Therefore, in this study, for each possible desired shape of the object, the following relation between the grasped point $g$ and $p_{is}$ is assumed:

$$ \dot{p}_{ig} = a_{i0}\dot{p}_{is} + \Omega_i \tilde{M}_s, \qquad (5) $$

where $\dot{p}_{ig}$ is an element of $\dot{G}$, which means $\dot{G} = [\dot{p}_{1g}, \dot{p}_{2g}, \dot{p}_{3g}, \dot{p}_{4g}, \dot{p}_{5g}, \dot{p}_{6g}]^T$. $\Omega_i$ is a $1 \times 6$ vector that relates the controlled points to the grasped point (namely $a_{ij}$ ($j \in \{1, ..., 6\}$)). $a_{i0}$ has been defined as a non-zero positive parameter which permits to account for the speed of $p_{is}$ during the process. $\tilde{M}_s$ represents the tracking errors and is introduced as:

$$ \tilde{M}_s = M_s - M_{sdes}. \qquad (6) $$

In (6), $M_{sdes}(t)$ is the desired trajectory of $M_s$ at each time step (t) of the deformation process. $p_{isdes}$ is the ith component of the $M_{sdes}$, as presented in Table 1. We intend to develop an adaptive algorithm to track these trajectories, defined as bounded and differentiable. To achieve a smooth and uniform dynamic evolution of the DLO's shape, the following trajectories are proposed to be used as the desired position trajectories for each point:

$$ p_{ildes} = (p_{il0} - p_{ilf}) \exp(-\kappa t) + p_{ilf}, \qquad (7) $$

$p_{ilf}$ is the target and $p_{il0}$ is the initial value of the $p_{il}$. The constant parameter $\kappa$ (the exponential coefficient) stands for a predefined positive constant. The exponential trajectory used in this study offers a simple and straightforward way to achieve smooth trajectories without the need for complex planning algorithms. Moreover, the speed of the trajectory smoothly reduces to zero, providing a smooth deformation. While trajectory (7) may
theoretically require an infinite amount of time to reach the target precisely, this is generally not a practical concern, as errors can be made negligible within a finite time frame by appropriately adjusting the exponential parameter $\kappa$. This brings about a good control over the completion time of the task. It links implicitly all the points $m_l$ by making them evolve in the same exponential fashion. From (7), we obtain $p_{isdes}$ based on the average of all the desired points $p_{ildes}$ and their angles.

For each $i \in \{1, ..., 6\}$, (5) can be rewritten to obtain the following equation, which expresses the assumed relation between controlled points and gripped point used to derive our controller:

$$ \dot{p}_{ig} = a_{i0}\dot{p}_{is} + \sum_{j=1}^{6} a_{ij}\tilde{p}_{js} = R_i C_i^T, \quad (8) $$

where $R_i$ are vectors that have the following form:

$$ R_i = [\dot{p}_{is}, \tilde{p}_{1s}, \tilde{p}_{2s}, \tilde{p}_{3s}, \tilde{p}_{4s}, \tilde{p}_{5s}, \tilde{p}_{6s}], \quad (9) $$

where $\tilde{p}_{is}$ is $p_{is} - p_{isdes}$. The vectors $C_i$ contain unknown parameters:

$$ C_i = [a_{i0}, a_{i1}, a_{i2}, a_{i3}, a_{i4}, a_{i5}, a_{i6}], \quad (10) $$

where $a_{ik}$ ($k \in \{0, 1, ..., 6\}$) are parameters that depend on the current and desired shape of the object, which are representative of the rod deformation. It is presumed that these parameters are approximately constant in a local neighborhood of the current shape at each time instant. They must be estimated online to find the relation between the motion of the gripper and $M_s$.

By defining the controller based on the expression (8) and using $p_{isdes}$, we can track the deformation evolution closely while converging to the target shape. This characteristic of the proposed method can be helpful to have a smooth deformation without sudden movement, which is essential to protect the object against possible damages. This trajectory (7) also allows approaching the desired target of $M_s$ in a limited time. Furthermore, by using the proposed method of trajectory tracking of a time-varying desired state, we are not limited to operating in a local neighborhood of a fixed desired state (as in (Zhu et al. 2018; Navarro-Alarcon et al. 2013)). Indeed, our approach is capable of controlling larger, non-local deformations, associated with the defined time evolution of the desired state since at each time
instant, we can assume that the system is in a local neighborhood of the desired value.

### 3.1. Adaptive controller design

This section aims to develop an adaptive Lyapunov-based technique to find estimations $a_{ik}$ of the parameters of (10), and define a control law based on these estimated parameters. The block diagram of the proposed controller is shown in Fig. 3. As depicted in that figure, the controller aims to develop a method capable of manipulating a branch based on shape tracking data acquired from an external sensor (in this case a 3-D camera). Thus, it is assumed that we are able to track the object's shape during the control process.

<figure><img src="image_4.png" /><figcaption>Figure 3: Schematic diagram of the proposed control strategy.</figcaption></figure>

The goal is to design a controller that can deform the object based on desired trajectories. To this end, we change the form of the speed of $\dot{p}_{is}$ in (8) to $\dot{p}_{isdes} - \alpha \tilde{p}_{is}$, where $\alpha$ is a positive constant. We use this modification to derive our controller which helps to generate desired trajectories for the velocities to be used in the controller design. It also helps us to ensure the stability of the closed-loop system which is validated using Lyapunov analysis. Instead of $a_{ik}$, we use in (5) the estimations $\hat{a}_{ik}$ which will be updated using an adaptation law.

The accent $\hat{\cdot}$ is used for the estimated parameters of the system. These estimations allow us to control the object's shape without requiring detailed knowledge of its physical properties. Therefore, the adaptive control law $u_i$ is defined as:

$$u_i = \hat{a}_{i0}(\dot{p}_{isdes} - \alpha \tilde{p}_{is}) + \sum_{j=1}^{6} \hat{a}_{ij} \tilde{p}_{js}. \qquad (11)$$
As explained in the introduction, most of the works in this area have been done by estimating the Jacobian matrix or having an object model, while using the adaptive controller (11) with the proposed scheme, we do not need the mentioned information.

To rewrite (11) in a similar form as (8), we replace $R_i$ and $C_i$ with new forms $P_i$ and $\hat{C}_i$. Therefore, (11) can be rewritten as:

$$u_i = P_i \hat{C}_i^T, \qquad (12)$$

where:

$$P_i = [(\dot{p}_{isdes} - \alpha \tilde{p}_{is}), \tilde{p}_{1s}, \tilde{p}_{2s}, \tilde{p}_{3s}, \tilde{p}_{4s}, \tilde{p}_{5s}, \tilde{p}_{6s}]. \qquad (13)$$

$\hat{C}_i$ are the vectors of system parameters' estimations:

$$\hat{C}_i = [\hat{a}_{i0}, \hat{a}_{i1}, \hat{a}_{i2}, \hat{a}_{i3}, \hat{a}_{i4}, \hat{a}_{i5}, \hat{a}_{i6}]. \qquad (14)$$

The adaption laws for updating the estimated parameters can be expressed as:

$$\dot{\hat{C}}_i = -P_i T_i \tilde{p}_{is}, \qquad (15)$$

where the adaptation gains $T_i$ are constant non-singular positive definite matrices of size 7 × 7. $\tilde{p}_{is}$, introduced earlier, is the ith element of $\tilde{M}_s$, as presented in Table 1. The adaption laws will be analyzed in the next section using Lyapunov's formalism.

## 3.2. Lyapunov analysis

In this section, Lyapunov's theorem (Slotine et al. 1991) is used to investigate the validity of the proposed adaptive control (11) and adaptation laws (15). To this end, the closed-loop dynamics of the system are obtained. Doing so, we substitute the control law (11) in the relation between the object's shape and grasped point (8). By performing several simple mathematical operations, we have:

$$\begin{gathered} a_{i0}\dot{p}_{is} + \sum_{j=1}^{6} a_{ij}\tilde{p}_{js} = \\ \hat{a}_{i0}(\dot{p}_{isdes} - \alpha\tilde{p}_{is}) + \sum_{j=1}^{6} \hat{a}_{ij}\tilde{p}_{js} + a_{i0}(\dot{p}_{isdes} - \alpha\tilde{p}_{is}) - a_{i0}(\dot{p}_{isdes} - \alpha\tilde{p}_{is}). \end{gathered} \qquad (16)$$

By reordering the terms in (16), one can write:
$$ a_{i0}(\dot{p}_{is} - (\dot{p}_{isdes} - \alpha \tilde{p}_{is})) = (\dot{p}_{isdes} - \alpha \tilde{p}_{is})(\hat{a}_{i0} - a_{i0}) + \sum_{j=1}^{6} \tilde{p}_{js}(\hat{a}_{ij} - a_{ij}). \quad (17) $$

Using equations (10), (13), and (14) in (17), the system's closed-loop dynamics for every i can be found as:

$$ a_{i0}(\dot{\tilde{p}}_{is} + \alpha \tilde{p}_{is}) = P_i \tilde{C}_i^T, \quad (18) $$

where $\tilde{C}_i = \hat{C}_i - C_i$. By simplifying (18), the closed-loop dynamics of the controllers for every i are reformulated as:

$$ \dot{\tilde{p}}_{is} = -\alpha \tilde{p}_{is} + \frac{1}{a_{i0}} P_i \tilde{C}_i^T. \quad (19) $$

A positive definite Lyapunov function candidate is used as follows to analyze the system stability and the tracking convergence using the proposed controller.

$$ V = \frac{1}{2} \left( \sum_{i=1}^{6} \tilde{p}_{is}^2 + \frac{1}{a_{i0}} \tilde{C}_i T_i^{-1} \tilde{C}_i^T \right). \quad (20) $$

The time derivative of $V$ is determined as:

$$ \dot{V} = \sum_{i=1}^{6} (\tilde{p}_{is} \dot{\tilde{p}}_{is} + \frac{1}{a_{i0}} \dot{\tilde{C}}_i T_i^{-1} \tilde{C}_i^T), \quad (21) $$

where as explained earlier, $\dot{\tilde{C}}_i$ is negligible compared to $\tilde{C}_i$, so $\dot{\tilde{C}}_i = \tilde{C}_i$. Therefore, via (19), (21) is transformed into:

$$ \dot{V} = \sum_{i=1}^{6} \left( -\alpha \tilde{p}_{is}^2 + \frac{1}{a_{i0}} P_i \tilde{C}_i^T \tilde{p}_{is} + \frac{1}{a_{i0}} \dot{\tilde{C}}_i T_i^{-1} \tilde{C}_i^T \right). \quad (22) $$

Using the parameters' adaptation laws (15), the time derivative of the Lyapunov function (22) finally has the following expression:

$$ \dot{V} = \sum_{i=1}^{6} -\alpha \tilde{p}_{is}^2 \leq 0. \quad (23) $$
Since $\alpha$ is a positive constant, the time derivative of the Lyapunov function is negative semi-definite. It should be noted that the only way to have $\dot{V} = 0$ is to have all errors, i.e., $\tilde{p}_{is}$ for all $i \in \{1, ..., 6\}$, equal to zero. According to (20), the Lyapunov function $V$ is positive definite in terms of $\tilde{p}_{is}$ and $\tilde{C}_i$. The time derivative of the Lyapunov function in (22) is negative semi-definite $\dot{V} \le 0$. Therefore, $V$ is bounded and consequently $\tilde{p}_{is}$ and $\tilde{C}_i$ remain bounded. Based on Barbalat's lemma (Slotine et al. 1991), if $w$ is a uniformly continuous function and the limit of the integral $\lim_{t \to \infty} \int_0^t w(\eta)d\eta$ exists and has a finite value, it is concluded that:

$$ \lim_{t \to \infty} w(t) = 0. \qquad (24) $$

Now, considering $w(t) = \sum_{i=1}^6 \alpha \tilde{p}_{is}^2$ (where $\tilde{p}_{is}$ was proved to be bounded), $\dot{V}$ in (23) can be written as

$$ \dot{V} = - \sum_{i=1}^{6} \alpha \tilde{p}_{is}^2 = -w(t). \qquad (25) $$

By integrating both sides of (25) from $t = 0$ to $t \to \infty$, it is obtained:

$$ V(0) - V(\infty) = \lim_{t \to \infty} \int_{0}^{t} w(\eta) d\eta. \qquad (26) $$

As shown in (23), $\dot{V} = dV/dt \le 0$. Therefore, $V(0) - V(\infty)$ is positive and finite. Accordingly, $\lim_{t \to \infty} \int_0^t w(\eta)d\eta$ in (26) exists and has a finite positive value because of the positiveness of $w(t)$. As a result, based on Barbalat's lemma (Slotine et al. 1991), one can write:

$$ \lim_{t \to \infty} w(t) = \lim_{t \to \infty} \sum_{i=1}^{6} \alpha \tilde{p}_{is}^2 = 0. \qquad (27) $$

Since $\tilde{p}_{is}^2 \ge 0$ and $\alpha$ is a positive non-zero constant, (27) ensures the convergence to $\tilde{p}_{is} \to 0$ as $t \to \infty$ for all $i$. Thus, the proposed adaptive controller achieves its objective which is tracking the desired trajectories ($p_{is} \to p_{isdes}$). The variable $M_s$ containing all the $p_{is}$ tracks the desired trajectory, which means the tracking error $\dot{M}_s$ (6) goes to zero.
<figure><img src="image_5.png" /><figcaption>Figure 4: Different objects used in the performed experiments: (a) a foam rod (L = 0.87 m), (b) a tree branch (L = 0.60 m), (c) a plant branch (L = 0.38 m).</figcaption></figure>

4. Experimental design

The performance of the proposed adaptive control strategy is validated
through various experiments on real deformable objects<fnref n="2" />. The objects of in-
terest are a foam rod and two different plants, as shown in Fig. 4. We chose
objects with different lengths and stiffness, to validate the capability of our
control strategy to handle objects with diverse and unknown characteristics.
The experiments are performed with a UR10 arm of a mobile manipulator.
In our experiments, we initially tuned the control parameters to some arbi-
trary values that allowed us to demonstrate the effectiveness of our proposed
approach. We chose them using some pre-performed simulations. However,
a more thorough investigation and optimization of the control parameters
could lead to an enhanced performance of the proposed controller.

A tracking pipeline is used for tracking the object deformation in real-
time. This tracking pipeline works based on a 3-D template of the object
created offline. The template is a 3-D mesh of the object at its rest shape;
a rectangular cuboid for the foam rod and a cylinder for the plants. The
resolution of the meshes along the object is chosen in such a way that we
have 20 controlled points (i.e., *n* = 20). This value was determined as a trade-
off between shape representation accuracy and computational efficiency. An
Intel RealSense 3-D camera is used to provide a point cloud of the scene as
the input for the tracking pipeline. The camera is externally calibrated with
the robot. Additionally, it is well-positioned to capture the entire shape of
the object during manipulation. The tracking process starts by setting the

<footnote n="2">²A video of our experimental results is attached and can be found at: https://www.youtube.com/watch?v=oD9Tasz7YxI</footnote>
object in a known shape in front of the camera and triggering the tracking. The tracking pipeline uses the inferred shape of the object in the previous frame and the point cloud captured in the current frame to infer the object shape in the current frame. This is done by: (i) using an intensity filter to filter the pixels and, consequently, the point cloud belonging to the object, (ii) applying an ICP algorithm to rigidly transfer the object mesh to the point cloud, (iii) applying an ICP-like algorithm proposed in (Petit et al. 2015) to find correspondences between the object and the point cloud, and (iv) applying a deformation constraint using position-based dynamics (Müller et al. 2007) to the mesh of the object. The last step obtains the inferred shape of the object according to the correspondences found in the step (iii).

The tracking pipeline and the controller are both written in C++ and run on two standard Dell laptops, each with an Intel Core i7 CPU. ROS (Robot Operating System) is used to connect the controller and tracking nodes to the arm. The experimental setup is shown in Fig. 5. With the tracking pipeline running, the object is deformed to a desired shape. We save this desired shape to be used later by the control method. We then deform the object to its initial shape. We, finally, start the controller to drive the object from the initial shape to the desired shape. The results of the experimental tests are detailed in the next section.

## 5. Experimental results

<figure><img src="image_6.png" /><figcaption>Figure 5: The experimental setup used in this work.</figcaption></figure>
We carried out multiple experimental tests using the objects depicted in Fig. 4 to validate the effectiveness of the proposed methodology. To present the servoing errors for the experiments, the following metric is defined from (1):

$$ \text{Servoing error} = \frac{1}{n} \sqrt{\sum_{l=1}^{n} (p_{1l} - p_{1lf})^2 + (p_{2l} - p_{2lf})^2 + (p_{3l} - p_{3lf})^2}. \quad (28) $$

In (28), the position errors of all the points used to represent the object's shape are included. The summation of the errors of these points converging toward zero means that the object reaches its target and the controller objective is completed. Moreover, to show how the object tracks the desired trajectories, we plot the evolution of the states representing the object's shape.

<figure><img src="image_7.png" /><figcaption>Figure 6: Experiment 1. The used object is a deformable foam rod. (a) The initial shape and final shape. The target shape is shown in red. (b) The servoing error of this experiment. (c) The evolution of the shape (average of its states $p_{1l}$, $p_{2l}$, $p_{3l}$) with respect to its desired trajectory shown in black.</figcaption></figure>

To validate the methodology, we first conducted an experiment using a flexible foam rod and subsequently applied our approach to deform branches
<figure><img src="image_8.png" /><figcaption>Figure 7: Experiment 2. The used object is a tree branch. (a) The initial shape and final shape. The target shape is shown in red. (b) The servoing error of this experiment. (c) The evolution of the shape (average of its states $p_{1l}, p_{2l}, p_{3l}$) with respect to its desired trajectory shown in black.</figcaption></figure>

from various plants with diverse levels of flexibility. The results of all the experiments can be seen in Fig. 6 to Fig. 9. We choose different initial and target shapes for each experiment as presented in the plot (a) of Fig. 6 to Fig. 9. At each instant, the state of the grasped point is updated using (12) and (15). The objects' shapes are assumed to remain stable during the entire control process.

The servoing errors of these experiments and the evolution of the shape (average of its states $p_{1l}, p_{2l}, p_{3l}$) regarding its desired trajectory are illustrated in the plot (b) of Fig. 6 to Fig. 9. As can be seen, by applying the controller during the deformation process, the errors converge to zero, and the objects are successfully deformed toward their desired shapes. However, it is worth noting that due to differences in the characteristics of the objects, the convergence rates may vary among different cases. Note that the deformations achieved are in 3-D space, not being restricted to a fixed plane. It should be pointed out that the deformations considered in our experiments involve changes in target shapes, which consequently impact the positions and orientations of the controlled points. Nevertheless, we only show the evaluation of the position in our servoing error and in the plots, since these plots (i.e, plot (c) of Fig. 6 to Fig. 9) are sufficient to demonstrate that we can control the entire shape of the object, given that the object is continuous. As presented in the figures, the controller uses the desired trajectories
of each state to drive the shape toward the targets. To demonstrate the motion of the robotic gripper in the experiments, we show its velocities, which represent the control inputs, in the plot (d) of Fig. 8 and Fig 9 (experiments 3 and 4).

<figure><img src="image_9.png" /><figcaption>Figure 8: Experiment 3. The used object is a tree branch. (a) The initial shape and final shape. The target shape is shown in red. (b) The servoing error of this experiment. (c) The evolution of the shape (average of its states $p_{1s}$, $p_{2s}$, $p_{3s}$) with respect to its desired trajectory shown in black. (d) The controller inputs ($u_1$ to $u_6$). In comparison to the previous experiment, in this experiment multiple variables, including the target shape, initial shape, and object, have been changed to evaluate the efficacy of the presented control scheme.</figcaption></figure>

# 6. Discussion

The effectiveness and versatility of the proposed methodology in manipulating deformable objects towards desired shapes are demonstrated by our experimental results. Objects such as foam rods and plant branches with
<figure><img src="image_10.png" /><figcaption>Figure 9: Experiment 4. The used object is a plant branch. (a) The initial shape and final shape. The target shape is shown in red. (b) The servoing error of this experiment. (c) The evolution of the shape (average of its states $p_{1t}, p_{2t}, p_{3t}$) with respect to its desired trajectory shown in black. (d) The controller inputs ($u_1$ to $u_6$). In comparison to the previous experiment, in this experiment multiple variables, including the target shape, initial shape, and object, have been changed to evaluate the efficacy of the presented control scheme.</figcaption></figure>

varying characteristics, including length and stiffness, were successfully de-
formed, showcasing the capability of our control strategy to handle objects
with diverse and unknown properties. Furthermore, the potential practical
applications of our approach in agricultural tasks (e.g., tree pruning, fruit
detection and harvesting) are highlighted by the ability to accurately track
and deform objects in real-time. The proposed approach is capable of making
the objects track prescribed deformation trajectories leading to final target
shapes that are defined in 3-D space. This important set of capabilities is
not available using prior approaches.

The proposed methodology has applicability limitations due to the as-
sumptions we made. For instance, it may not be suitable for situations
where deformations occur rapidly. This is because our methodology assumes
that the deformation process occurs in a quasi-static manner, implying that it evolves slowly enough to be controlled accurately. Additionally, perceiving the full shape of the considered types of objects is not easy, leading to noises in the results that can affect the performance of the controller. The methodology was validated in a controlled laboratory setting; it is clear that implementation in, e.g., real agricultural scenarios would encounter more complex conditions. Our results underline the importance of ongoing research efforts to overcome these existing limitations, ultimately optimizing applicability in real-world scenarios.

The main limitation of the current work is that no force feedback from the object is considered. This can pose a significant challenge for the controller in effectively handling various objects with different characteristics and stiffness in more complex scenarios. Additional tuning may be required on the controller gains to adapt to different objects. By incorporating force feedback, it could be possible to dynamically adjust the controller based on the properties of the object being manipulated. This enhancement could lead to better and faster control of the object while ensuring its integrity and minimizing the risk of damages. All these are very relevant aspects for application in agricultural tasks.

# 7. Conclusion

In this paper, the problem of controlling deformable linear objects (DLOs) in the agricultural field was investigated. A method to control the shape of these kinds of objects with a robotic gripper in 3-D was presented. An indirect adaptive control method was developed to reshape the DLOs based on predefined desired shapes. The proposed method is able to track the evolution of the desired trajectories in real-time. The method does not require prior knowledge of the object's deformation properties. Furthermore, there is no need to calculate the Jacobian matrix based on an offline model to obtain the relationship between the displacements of the manipulator and the deformations of the object.

The proposed method controls the entire shape of the object while following a desired trajectory towards the target shape. The adaptation law to update the estimations of the system parameters and the states of the controlled points during the control process was presented. The validity of the proposed adaptation law was investigated using Lyapunov's theorem and Barbalat's lemma. The controller was successfully validated using various ex-
584

585

586

587

periments on a flexible foam rod and branches of various plants, assuming that the object's shape can be tracked. These results show the controller's potential for being used in agricultural applications involving branch handling, such as tree pruning, fruit detection, and harvesting.

588

589

590

591

592

593

594

595

596

597

For future steps, an attempt will be made to propose a method that utilizes force data to determine stress along the object's length as feedback while controlling its shape. Other interesting future steps include designing a detection method that can work in outdoor environments and optimizing the desired trajectory according to the deformation task. In the current work, the exponential coefficient $\kappa$ was chosen based on the task's risk. Still, our long-term objective is to provide a way to relate the exponential coefficient based on the task and flexibility of the object. We are also interested in exploring the problem of automatically determining a suitable grasping configuration for a given control task.

598

## Acknowledgement

599

600

601

602

603

604

605

606

607

This work was sponsored by a public grant overseen by the French National Research Agency as part of the "Investissements d'Avenir" through the IMobS3 Laboratory of Excellence (ANR-10-LABX-0016), the IDEX-ISITE initiative CAP 20-25 (ANR-16-IDEX-0001) and the European Union's Horizon 2020 research and innovation programme under grant agreement No 869855 (Project "SoftManBot"). It is developed within the context of the project "SyncEA" ("Dynamic SYNChronization of mobile manipulators for new Environmental and Agricultural tools") of CAP 20-25 Challenge 2 (Domain: Agro Technologies).

608

609

610

611

M. Aranda was supported via project REMAIN - S1/1.1/E0111 (Interreg Sudoe Programme, ERDF) and via a María Zambrano Fellowship funded by the Spanish Ministry of Universities and the European Union-NextGenerationEU.

612

613

614

615

J. A. Corrales Ramon was funded by the Spanish Ministry of Universities through a 'Beatriz Galindo' fellowship (Ref. BG20/00143) and by the Spanish Ministry of Science and Innovation through the research project PID2020-119367RB-I00.

616

## References

617

618

[1] Aghajanzadeh, O., Aranda, M., Corrales Ramon, J.A., Cariou, C., Lenain, R., Mezouar, Y., 2022a. Adaptive deformation control for elastic
619

linear objects. Frontiers in Robotics and AI, 113.

620

621

622

623

[2] Aghajanzadeh, O., Aranda, M., López-Nicolás, G., Lenain, R., Mezouar, Y., 2022b. An offline geometric model for controlling the shape of elastic linear objects, in: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2175-2181.

624

625

626

627

[3] Aghajanzadeh, O., Picard, G., Ramon, J.A.C., Cariou, C., Lenain, R., Mezouar, Y., 2022c. Optimal deformation control framework for elastic linear objects, in: 2022 IEEE 18th International Conference on Automation Science and Engineering (CASE), IEEE, pp. 722-728.

628

629

630

[4] Agovino, M., Casaccia, M., Ciommi, M., Ferrara, M., Marchesano, K., 2019. Agriculture, climate change and sustainability: The case of EU-28. Ecological Indicators 105, 525-543.

631

632

633

634

[5] Alambeigi, F., Wang, Z., Hegeman, R., Liu, Y.H., Armand, M., 2018. A robust data-driven approach for online learning and manipulation of unmodeled 3-D heterogeneous compliant objects. IEEE Robotics and Automation Letters 3, 4140-4147.

635

636

637

[6] Bretl, T., McCarthy, Z., 2014. Quasi-static manipulation of a kirchhoff elastic rod based on a geometric analysis of equilibrium configurations. The International Journal of Robotics Research 33, 48-68.

638

639

[7] Brown, J., Latombe, J.C., Montgomery, K., 2004. Real-time knot-tying simulation. The Visual Computer 20, 165-179.

640

641

642

643

[8] Cao, L., Li, X., Phan, P.T., Tiong, A.M.H., Kaan, H.L., Liu, J., Lai, W., Huang, Y., Le, H.M., Miyasaka, M., et al., 2020. Sewing up the wounds: A robotic suturing system for flexible endoscopy. IEEE Robotics & Automation Magazine 27, 45-54.

644

645

646

[9] Dadfarnia, M., Jalili, N., Liu, Z., Dawson, D.M., 2004. An observer-based piezoelectric control of flexible cartesian robot arms: theory and experiment. Control Engineering Practice 12, 1041-1053.

647

648

649

[10] Davenport, T.L., 2006. Pruning strategies to maximize tropical mango production from the time of planting to restoration of old orchards. HortScience 41, 544-548.
650

651

652

653

[11] Davidson, J., Bhusal, S., Mo, C., Karkee, M., Zhang, Q., 2020. Robotic manipulation for specialty crop harvesting: A review of manipulator and end-effector technologies. Global Journal of Agricultural and Allied Sciences 2, 25-41.

654

655

656

657

[12] Delgado Rodríguez, Á., Corrales Ramón, J.A., Mezouar, Y., Lequievre, L., Jara, C.A., Torres, F., 2017. Tactile control based on gaussian images and its application in bi-manual manipulation of deformable objects. Robotics and Autonomous Systems 94, 148-161.

658

659

660

[13] Gandhi, P.S., Borja, P., Ortega, R., 2016. Energy shaping control of an inverted flexible pendulum fixed to a cart. Control Engineering Practice 56, 27-36.

661

662

663

664

[14] James, D.L., Pai, D.K., 2002. Real time simulation of multizone elastokinematic models, in: Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No. 02CH37292), IEEE. pp. 927-932.

665

666

667

[15] Kang, H., Chen, C., 2020. Fast implementation of real-time fruit detection in apple orchards using deep learning. Computers and Electronics in Agriculture 168, 105108.

668

669

670

[16] Kang, H., Wen, J.T., 2002. Robotic knot tying in minimally invasive surgeries, in: IEEE/RSJ International conference on intelligent robots and systems, IEEE. pp. 1421-1426.

671

672

673

[17] Kater, A., Meurer, T., 2019. Motion planning and tracking control for coupled flexible beam structures. Control Engineering Practice 84, 389-398.

674

675

676

677

[18] Kinio, S., Patriciu, A., 2012. A comparative study of H∞ and PID control for indirect deformable object manipulation, in: 2012 IEEE International Conference on Robotics and Biomimetics (ROBIO), IEEE. pp. 414-420.

678

679

680

681

[19] Koessler, A., Roca Filella, N., Bouzgarrou, B.C., Lequièvre, L., Corrales Ramon, J.A., 2021. An efficient approach to closed-loop shape control of deformable objects using finite element models, in: 2021 IEEE Int. Conf. on Robotics and Automation (ICRA), pp. 1637-1643.
682

683

684

[20] Lagneau, R., Krupa, A., Marchal, M., 2020. Automatic shape control of deformable wires based on model-free visual servoing. IEEE Robotics and Automation Letters 5, 5252-5259.

685

686

687

688

[21] Li, X., Wang, Z., Liu, Y.H., 2019. Sequential robotic manipulation for active shape control of deformable linear objects, in: 2019 IEEE International Conference on Real-time Computing and Robotics (RCAR), IEEE. pp. 840-845.

689

690

691

[22] Lin, H., Guo, F., Wang, F., Jia, Y.B., 2014. Picking up soft 3-D objects with two fingers, in: 2014 IEEE International Conference on Robotics and Automation (ICRA), IEEE. pp. 3656-3661.

692

693

694

[23] Lin, H., Guo, F., Wang, F., Jia, Y.B., 2015. Picking up a soft 3-D object by "feeling" the grip. The International Journal of Robotics Research 34, 1361-1384.

695

696

697

698

[24] Marchesseau, S., Heimann, T., Chatelin, S., Willinger, R., Delingette, H., 2010. Fast porous visco-hyperelastic soft tissue model for surgery simulation: application to liver surgery. Progress in biophysics and molecular biology 103, 185-196.

699

700

[25] McGlynn, S., Walters, D., 2019. Agricultural robots: Future trends for autonomous farming. Int. J. Emerg. Technol. Innov 6, 944-949.

701

702

703

[26] Müller, M., Heidelberger, B., Hennix, M., Ratcliff, J., 2007. Position based dynamics. Journal of Visual Communication and Image Representation 18, 109-118.

704

705

706

[27] Navarro-Alarcon, D., Liu, Y.H., 2017. Fourier-based shape servoing: a new feedback method to actively deform soft objects into desired 2-D image contours. IEEE Transactions on Robotics 34, 272-279.

707

708

709

[28] Navarro-Alarcon, D., Liu, Y.H., Romero, J., Li, P., 2013. Model-free Visually Servoed Deformation Control of Elastic Objects by Robot Manipulators. IEEE Transactions on Robotics 29, 1457-1468.

710

711

712

713

[29] Navarro-Alarcon, D., Liu, Y.H., Romero, J.G., Li, P., 2014. On the visual deformation servoing of compliant objects: Uncalibrated control methods and experiments. The International Journal of Robotics Research 33, 1462-1480.
714

715

716

717

[30] Navarro-Alarcon, D., Yip, H.M., Wang, Z., Liu, Y.H., Zhong, F., Zhang, T., Li, P., 2016. Automatic 3-D manipulation of soft objects by robotic arms with an adaptive deformation model. IEEE Transactions on Robotics 32, 429-441.

718

719

[31] Ogden, R.W., 1997. Non-linear elastic deformations. Courier Corporation.

720

721

722

723

[32] Petit, A., Lippiello, V., Siciliano, B., 2015. Real-time tracking of 3-D elastic objects with an rgb-d sensor, in: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE. pp. 3914-3921.

724

725

726

727

[33] Qi, J., Ma, G., Zhu, J., Zhou, P., Lyu, Y., Zhang, H., Navarro-Alarcon, D., 2022. Contour moments based manipulation of composite rigid-deformable objects with finite time model estimation and shape/position control. IEEE/ASME Trans. Mechatr. 27, 2985-2996.

728

729

730

731

[34] Sanchez, J., Corrales, J.A., Bouzgarrou, B.C., Mezouar, Y., 2018. Robotic manipulation and sensing of deformable objects in domestic and industrial applications: a survey. The International Journal of Robotics Research 37, 688-716.

732

733

734

[35] Shetab-Bushehri, M., Aranda, M., Mezouar, Y., Özgür, E., 2022. As-rigid-as-possible shape servoing. IEEE Robotics and Automation Letters 7, 3898-3905.

735

736

[36] Slotine, J.J.E., Li, W., et al., 1991. Applied nonlinear control. Prentice hall Englewood Cliffs, NJ.

737

738

739

740

[37] Springmann, M., Clark, M., Mason-D'Croz, D., Wiebe, K., Bodirsky, B.L., Lassaletta, L., De Vries, W., Vermeulen, S.J., Herrero, M., Carlson, K.M., et al., 2018. Options for keeping the food system within environmental limits. Nature 562, 519-525.

741

742

743

[38] Stavridis, S., Papageorgiou, D., Droukas, L., Doulgeri, Z., 2022. Bi-manual crop manipulation for human-inspired robotic harvesting. arXiv preprint arXiv:2209.06074.

744

745

[39] Tanner, H., Kyriakopoulos, K., Krikakis, N., 2001. Advanced agricultural robots: kinematics and dynamics of multiple mobile manipulators
746

747

handling non-rigid material. Computers and electronics in agriculture 31, 91–105.

748

749

750

751

[40] Tinoco, V., Silva, M.F., Santos, F.N., Rocha, L.F., Magalhães, S., Santos, L.C., 2021. A review of pruning and harvesting manipulators, in: 2021 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), IEEE. pp. 155–160.

752

753

754

[41] Wang, W., Berenson, D., Balkcom, D., 2015. An online method for tight-tolerance insertion tasks for string and rope, in: 2015 IEEE International Conference on Robotics and Automation (ICRA), IEEE. pp. 2488–2495.

755

756

757

[42] Yan, M., Zhu, Y., Jin, N., Bohg, J., 2020. Self-supervised learning of state estimation for manipulating deformable linear objects. IEEE Robotics and Automation Letters 5, 2372–2379.

758

759

760

761

[43] Yandun, F., Silwal, A., Kantor, G., 2020. Visual 3D reconstruction and dynamic simulation of fruit trees for robotic manipulation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 238–247.

762

763

764

765

[44] You, A., Sukkar, F., Fitch, R., Karkee, M., Davidson, J.R., 2020. An efficient planning and control framework for pruning fruit trees, in: 2020 IEEE International Conference on Robotics and Automation (ICRA), IEEE. pp. 3930–3936.

766

767

768

769

[45] Yu, M., Zhong, H., Li, X., 2022. Shape control of deformable linear objects with offline and online learning of local linear deformation models, in: 2022 International Conference on Robotics and Automation (ICRA), pp. 1337–1343.

770

771

772

773

[46] Zahid, A., Mahmud, M.S., He, L., Heinemann, P., Choi, D., Schupp, J., 2021. Technological advancements towards developing a robotic pruner for apple trees: A review. Computers and Electronics in Agriculture 189, 106383.

774

775

776

777

[47] Zhu, J., Navarro, B., Fraisse, P., Crosnier, A., Cherubini, A., 2018. Dual-arm robotic manipulation of flexible cables, in: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE. pp. 479–484.
778

779

780

781

[48] Zhu, J., Navarro-Alarcon, D., Passama, R., Cherubini, A., 2021. Vision-based manipulation of deformable and rigid objects using subspace projections of 2-D contours. Robotics and Autonomous Systems 142, 103798.

782

783

784

785

[49] Zhuang, J., Luo, S., Hou, C., Tang, Y., He, Y., Xue, X., 2018. Detection of orchard citrus fruits using a monocular machine vision-based method for automatic fruit picking applications. Computers and Electronics in Agriculture 152, 64–73.