# Insect-Inspired Robots: Bridging Biological and Artificial Systems

Poramate Manoonpong, Luca Patanè, Xiaofeng Xiong, Ilya Brodoline, Julien

Dupeyroux, Stéphane Viollet, Paolo Arena, Julien Serres

► To cite this version:

Poramate Manoonpong, Luca Patanè, Xiaofeng Xiong, Ilya Brodoline, Julien Dupeyroux, et al. Insect-Inspired Robots: Bridging Biological and Artificial Systems. Sensors, 2021, 21 (22), pp.7609. 10.3390/s21227609. [https://arxiv.org/abs/2009.07609]

HAL Id: hal-03430897

https://amu.hal.science/hal-03430897

Submitted on 16 Nov 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Review

# Insect-Inspired Robots: Bridging Biological and Artificial Systems

Poramate Manoonpong <fnref n="1" />,<fnref n="2" />,*,† ID, Luca Patanè <fnref n="3" />,*,† ID, Xiaofeng Xiong <fnref n="1" /> ID, Ilya Brodoline <fnref n="4" /> ID, Julien Dupeyroux <fnref n="5" /> ID,
Stéphane Viollet <fnref n="4" /> ID, Paolo Arena <fnref n="6" />,*,† ID and Julien R. Serres <fnref n="4" />,*,† ID

<figure><img src="image_2.png" /></figure>

check for
updates

**Citation:** Manoonpong, P.; Patanè, L.; Xiong, X.; Brodoline, I.; Dupeyroux, J.; Viollet, S.; Arena, P.; Serres, J.R.
Insect-Inspired Robots: Bridging Biological and Artificial Systems.
*Sensors* **2021**, *21*, 7609.
https://doi.org/10.3390/s21227609

Academic Editor: Vittorio M.N.
Passaro

Received: 9 September 2021

Accepted: 27 October 2021

Published: 16 November 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

<figure><img src="image_3.png" /></figure>

**Copyright:** © 2021 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

**Abstract:** This review article aims to address common research questions in hexapod robotics. How can we build intelligent autonomous hexapod robots that can exploit their biomechanics, morphology, and computational systems, to achieve autonomy, adaptability, and energy efficiency comparable to small living creatures, such as insects? Are insects good models for building such intelligent hexapod robots because they are the only animals with six legs? This review article is divided into three main sections to address these questions, as well as to assist roboticists in identifying relevant and future directions in the field of hexapod robotics over the next decade. After an introduction in section (1), the sections will respectively cover the following three key areas: (2) biomechanics focused on the design of smart legs; (3) locomotion control; and (4) high-level cognition control. These interconnected and interdependent areas are all crucial to improving the level of performance of hexapod robotics in terms of energy efficiency, terrain adaptability, autonomy, and operational range. We will also discuss how the next generation of biorobotics will be able to transfer knowledge from biology to robotics and vice versa.

**Keywords:** hexapod; legged robotics; biomimicry; biomimetism; bionics; biorobotics

## 1. Introduction

Legged robots represent a unique opportunity to understand locomotion in the animal kingdom [1,2]. Ever since robotics emerged in the 20th century, legged robots have aroused enthusiasm and curiosity from both researchers and the general public. In addition to their contributions to our understanding of locomotion, legged robots offer a great alternative to their wheeled counterparts, showing better abilities to navigate uneven terrains. Indeed, most animals have legs, allowing them to move, explore, and adapt to their direct environment. Similarly, legged robots can operate on many types of surfaces, including rough terrains, and also jump over obstacles, and climb structures. The following article is focused on hexapod robots, i.e., robots equipped with six actuated legs. The choice of hexapod design is strongly motivated by the need for insectoid robots to mimic the animal model as far as possible, so that the results obtained from experiments can be fairly compared to insects (locomotion, navigation, object manipulation, etc.) [3–5]. On the other hand, and from a purely robotic perspective, hexapod robots represent an optimum

<footnote n="1">¹ Embodied Artificial Intelligence and Neurorobotics Laboratory, SDU Biorobotics, The Mærsk Mc-Kinney Møller Institute, University of Southern Denmark, 5230 Odense, Denmark; xizi@mmmi.sdu.dk</footnote>

<footnote n="2">² Bio-Inspired Robotics and Neural Engineering Laboratory, School of Information Science and Technology, Vidyasirimedhi Institute of Science and Technology, Rayong 21210, Thailand</footnote>

<footnote n="3">³ Department of Engineering, University of Messina, 98100 Messina, Italy</footnote>

<footnote n="4">⁴ Department of biorobotics, Aix Marseille University, CNRS, ISM, CEDEX 07, 13284 Marseille, France;
ilya.brodoline@univ-amu.fr (I.B.); stephane.viollet@univ-amu.fr (S.V.)</footnote>

<footnote n="5">5 Faculty of Aerospace Engineering, Delft University of Technology, 52600 Delft, The Netherlands;
j.j.g.dupeyroux@tudelft.nl</footnote>

<footnote n="6">6 Department of Electrical, Electronic and Computer Engineering, University of Catania, 95131 Catania, Italy</footnote>

<footnote>* Correspondence: poma@mmmi.sdu.dk (P.M.); lpatane@unime.it (L.P.); paolo.arena@unict.it (P.A.);
julien.serres@univ-amu.fr (J.R.S.)</footnote>

<footnote>† These authors contributed equally to this work.</footnote>
compromise between overall stability and energy cost. With fewer legs, such as in bipedal or quadrupedal robots, the locomotion pattern implies that whenever the robot is walking, its stability decreases during the leg transfer phase: a bipedal robot will have to stay stable with only one leg, and a quadrupedal robot is inherently unbalanced because of the symmetry of the legs configuration. Alternatively, octopod robots are stable, but less energy efficient. To summarize, hexapod robots offer different ranges of relatively stable walking gaits, while optimizing energy cost.

This review article was written to address a common research question in the field of hexapod robotics: *what is the best way to build autonomous hexapod robots that can exploit their biomechanics, morphology, and computational systems to achieve autonomy, adaptability, and energy efficiency comparable to a small living creature, such as an insect?* To help researchers to find relevant directions over the next decade in the field of hexapod robotics, we will divide this review in three main sections following the logic of Figure 1. These are: Section 2, “Biomechanics in hexapod robotics”, which will be focused on the design of smart legs; Section 3, “From biomechanics to locomotion”, which will be focused on insect and robot locomotion control; and lastly Section 4, “From locomotion to cognition”, which will be focused on planning body actions to reach complex goals. Figure 1 illustrates the links between these three sections as they relate to the above main research question in italic, which is quite complex. To address it, it requires to divide it into more specific research questions about how we can transfer knowledge derived from biological agents to artificial agents. In addition, according to the scaling factor between a biological agent (i.e., an insect) and an artificial agent (i.e., a robot), the transferability of biologically-based knowledge will be discussed from a robotic point of view, because there are occasions when such transferabilities are unfortunately not relevant. As a result, we have deliberately limited the current state-of-the-art as seen in Table 1 to provide an overview of hexapod robots developed over the last 20 years in the range of 1–27 kg.

<figure><img src="image_4.png" /><figcaption>**Figure 1.** Overview of three main components underlying complex locomotion and cognition: biomechanics, locomotion control, and high-level cognitive control. Left (right) shows examples of the three components in insects (robots). (a) Left (right) is the insect central brain (high-level neural cognitive control model). (b) Left (right) is the insect thoracic ganglia (modified from [6]) (robot locomotion control model). (c) Left (right) is a biomechanical insect (robot) leg.</figcaption></figure>

Section 2 aims to survey a significant quantity of studies reported in the literature produced over the two last decades in legs, structure, and morphology in hexapod robotics. The design of legged robots is a technical and technological challenge in many respects. Since the very first robots were built in the late 1960s [7–11], an outstanding body of work has been realized to improve both the mechatronic structure and the locomotion gait towards auto-adaptability to the terrain. Nowadays, after barely more than 50 years of research and development, the principles of robots are well mastered and leverage multiple opportunities in terms of field implementation (for a review of hexapod robots up to 2010s,
see [12,13]). Over the past ten years, the increasing use of 3D printing has significantly boosted the development of brand new mechatronic designs for inexpensive hexapod robots, which are easy, for academics and students, to build, duplicate, and fix. This section will deal with passive materials, actuators, and force sensing in legs design to improve the level of autonomy of hexapod robots over complex terrain allowing future hexapod robots to maintain their speed over an extended period or range. Section 2 will be divided into the following three subsections:

* Section 2.1. Robotic leg design;

* Section 2.2. From legs to robots;

* Section 2.3. Hexapod robots' accessibility criteria for academics.

<table><caption>**Table 1.** Overview of the state-of-the-art hexapod robots developed over the last 20 years in the range 1~27 kg. The size, given in meters, corresponds to the largest dimension between width and height. DOF stands for Degrees Of Freedom for the entire robot (note that this can include extra actuation for head control and body control). The speed represents the maximum speed measured, in meters per second. The symbol "-" represents missing data.</caption><thead><tr><th>Year</th><th>Ref.</th><th>Hexapod</th><th>Size [m]</th><th>Mass [kg]</th><th>DOF</th><th>Compliant</th><th>Speed [m/s]</th><th>Task</th></tr></thead><tbody><tr><td>2021</td><td>[14]</td><td>HAntR</td><td>0.50</td><td>2.9</td><td>24</td><td>×</td><td>0.43</td><td>Locomotion</td></tr><tr><td>2019</td><td>[15,16]</td><td>MORF</td><td>0.60</td><td>4.2</td><td>18</td><td>✓</td><td>0.70</td><td>Locomotion</td></tr><tr><td>2019</td><td>[17]</td><td>Daisy</td><td>1.10</td><td>21</td><td>18</td><td>✓</td><td>0.13</td><td>Locomotion</td></tr><tr><td>2019</td><td>[18,19]</td><td>Drosophibot</td><td>0.80</td><td>1</td><td>18</td><td>✓</td><td>0.05</td><td>Locomotion</td></tr><tr><td>2019</td><td>[3,20]</td><td>AntBot</td><td>0.45</td><td>2.3</td><td>18</td><td>×</td><td>0.90</td><td>Navigation</td></tr><tr><td>2019</td><td>[21]</td><td>Corin</td><td>0.6</td><td>4.2</td><td>18</td><td>×</td><td>0.10</td><td>Locomotion</td></tr><tr><td>2018</td><td>[22]</td><td>AmphiHex-II</td><td>0.51</td><td>14</td><td>6</td><td>✓</td><td>0.36</td><td>Locomotion</td></tr><tr><td>2018</td><td>[23]</td><td>CRABOT</td><td>0.70</td><td>2.5</td><td>24</td><td>×</td><td>0.05</td><td>Locomotion</td></tr><tr><td>2017</td><td>[24,25]</td><td>PhantomX AX</td><td>0.50</td><td>2.6</td><td>18</td><td>×</td><td>0.29</td><td>Locomotion</td></tr><tr><td>2017</td><td>[20,24]</td><td>Hexabot</td><td>0.36</td><td>0.68</td><td>18</td><td>×</td><td>0.35</td><td>Navigation</td></tr><tr><td>2016</td><td>[26]</td><td>Weaver</td><td>0.35</td><td>7</td><td>30</td><td>×</td><td>0.16</td><td>Locomotion</td></tr><tr><td>2016</td><td>[27]</td><td>MX Phoenix</td><td>0.80</td><td>4.8</td><td>18</td><td>×</td><td>0.50</td><td>Locomotion</td></tr><tr><td>2015</td><td>[28]</td><td>Phoenix 3DOF</td><td>0.37</td><td>1.3</td><td>18</td><td>×</td><td>0.25</td><td>Locomotion</td></tr><tr><td>2015</td><td>[29]</td><td>HexaBull-1</td><td>0.53</td><td>3.4</td><td>18</td><td>✓</td><td>-</td><td>Locomotion</td></tr><tr><td>2015</td><td>[30,31]</td><td>MantisBot</td><td>0.74</td><td>6.1</td><td>28</td><td>×</td><td>-</td><td>Navigation</td></tr><tr><td>2015</td><td>[32]</td><td>Snake Monster</td><td>0.70</td><td>4.6</td><td>18</td><td>✓</td><td>-</td><td>Locomotion</td></tr><tr><td>2015</td><td>[33]</td><td>BionicANT</td><td>0.15</td><td>0.105</td><td>18</td><td>×</td><td>-</td><td>Swarming</td></tr><tr><td>2014</td><td>[34-36]</td><td>HECTOR</td><td>0.95</td><td>13</td><td>18</td><td>✓</td><td>-</td><td>Navigation</td></tr><tr><td>2014</td><td>[37,38]</td><td>Messor II</td><td>0.30</td><td>2.5</td><td>18</td><td>×</td><td>0.09</td><td>Locomotion</td></tr><tr><td>2014</td><td>[39,40]</td><td>LAURON V</td><td>0.90</td><td>42</td><td>24</td><td>✓</td><td>-</td><td>Navigation</td></tr><tr><td>2014</td><td>[41]</td><td>CREX</td><td>1</td><td>27</td><td>24</td><td>×</td><td>0.17</td><td>Locomotion</td></tr><tr><td>2012</td><td>[42]</td><td>Octavio</td><td>1</td><td>10.8</td><td>18</td><td>×</td><td>-</td><td>Locomotion</td></tr><tr><td>2011</td><td>[43]</td><td>-</td><td>0.46</td><td>3</td><td>18</td><td>✓</td><td>0.03</td><td>Navigation</td></tr><tr><td>2011</td><td>[44,45]</td><td>EduBot</td><td>0.36</td><td>3.3</td><td>6</td><td>✓</td><td>2.50</td><td>Locomotion</td></tr><tr><td>2010</td><td>[46]</td><td>X-RHex</td><td>0.57</td><td>9.5</td><td>6</td><td>×</td><td>1.54</td><td>Locomotion</td></tr><tr><td>2008</td><td>[47]</td><td>DLR-crawler</td><td>0.50</td><td>3.5</td><td>18</td><td>✓</td><td>0.20</td><td>Locomotion</td></tr><tr><td>2006</td><td>[48,49]</td><td>AMOS-WD06</td><td>0.40</td><td>4.2</td><td>19</td><td>×</td><td>0.07</td><td>Locomotion</td></tr><tr><td>2006</td><td>[50,51]</td><td>Gregor I</td><td>0.30</td><td>1.2</td><td>12</td><td>×</td><td>0.03</td><td>Locomotion</td></tr><tr><td>2005</td><td>[52,53]</td><td>BILL-Ant-a</td><td>0.33</td><td>2.3</td><td>18</td><td>×</td><td>0.03</td><td>Locomotion</td></tr><tr><td>2001</td><td>[54]</td><td>RHex</td><td>0.54</td><td>7</td><td>6</td><td>✓</td><td>0.55</td><td>Locomotion</td></tr></tbody></table>
Section 3 focuses on the considerable amount of research over the last decades on locomotion control in insects and hexapod robots. An insect's central nervous system can efficiently combine information from a variety of sensor modalities to achieve interlimb and intralimb coordination, as well as control joint compliance for locomotion in complex terrains and object manipulation. Understanding such a control system and implementing it on a hexapod robot are still difficult tasks. To date, significant progress on the investigation of insect locomotion control and the development of hexapod locomotion control has been made. In this context, Section 3 provides the key findings of insect locomotion control and their translation to robot locomotion control (i.e., bio-inspired control). It also covers conventional engineering-based control and advanced machine learning-based control methods for hexapod locomotion generation. This is to update and guide our research community with future research directions in locomotion control. Section 3 will be divided into the following two subsections:

* Section 3.1. Insect locomotion control;

* Section 3.2. Robot locomotion control.

Section 4 explores a considerable amount of research produced over the last decades, which has concentrated on unraveling the main features of the insect brain and deriving the main guidelines for new approaches to high level behavior emergence in engineering. Insects will be presented as creatures able to show complex behaviors in very small brains. We then assume the possibility of the physical implementation of similar behaviors within a currently existing hardware device. Of course, to perform such a task, a deep level of knowledge about the loop linking the neural-functional-behavioral levels is needed. As with higher brains, complex behaviors in insects typically arise from the coordinated action of concurrent control systems, but the details of the neural mechanisms in charge of each function, and how these are coordinated, still remain largely unknown in a lot of cases. Therefore, Section 4 reviews some successful examples in which this task was performed, identifying, in reference to specific tasks, the responsible neural sites, the neuro-engineered models designed, and the final robotic structures showing the corresponding behaviors. Of course, a huge amount of work still remains to be done, especially as regards our capability to design suitable experimental setups leading to the explicit stimulation of other interesting behaviors. Section 4 will be divided into the following three subsections:

* Section 4.1. The fly brain and cognition;

* Section 4.2. The insect brain structure;

* Section 4.3. Insect brain functional models, implementations and robotic experiments.

Section 5 will deal with lessons learned from this review, and a conclusion will be drawn in Section 6 to give new lines of research and future directions in hexapod robotics for the next 10 years.

## 2. Biomechanics in Hexapod Robotics

Biomechanics are at the heart of the biomimetic embodiment of a hexapod robot (Figure 1a) because legs are constrained by mechanical stress when they are in contact with the ground, and they have to follow various locomotion patterns dictated by the locomotion control system (Figure 1b). Mechanical strength and execution fidelity of the pattern of locomotion by adding force sensing to joints are the two key parameters in biomechanics.

Below we introduce the recent developments in hexapod robot design reported over the last 20 years (Table 1), illustrating the outstanding achievements in size reduction, leg compliance, and speed in hexapod robotics. From a purely biomechanical point of view, three main research questions are explored in hexapod robotics:

* How can we improve the level of performance of hexapod robots in terms of traveled distance?

* Is the study of insects helpful in the design of highly efficient hexapod mobility?

* Are robotic legs as intelligent as insect legs over complex terrain?
In Section 2.1, we introduce the variety of designs, morphology, actuation, cost of transport, and force sensing in robotic legs. In Section 2.2, we present the various hexapod robotic designs and the effect of the scaling factor on the level of performance. In Section 2.3, we show those hexapod robots, which are available on the market or as open-source projects for academics.

## 2.1. Robotic Leg Design

### 2.1.1. Design and Morphology

The design of an insect-like hexapod robot is, in itself, a complex task, requiring compromises and the establishment of a balance between desired performance and true feasibility limited by technical progress. Two main aspects define a hexapod robot leg: the number of degrees of freedom (DOF) and the structure type. Each of these aspects are selected and designed for the uses to which the robot will be put.

* **Number of degrees of freedom (DOF):** determines the operating space of the robot. By increasing of DOFs, the robot can achieve more complex trajectories. The number of DOFs also has a direct impact on robot characteristics, such as its autonomy, mass, and cost. Therefore, the number of DOFs should not be neglected in the design process. Currently, insect-inspired robotic legs are designed with between one and five DOFs per leg.

With one DOF per leg, the robot's maneuverability is highly limited. Depending on both leg and control designs, a one DOF per leg robot can perform a straight line walk [55] and also achieve simple rough terrain navigation if is equipped with whegs [46,56,57], comprising elements of both wheels and legs. These whegs equipped robots cannot really be considered as insect-like robots regarding their body structure. Their development tends to target navigation tasks over locomotion studies.

With two DOFs per leg, a simplified hexapod robot can be built [58]. This choice is a good compromise between energetic cost and mobility. However, this type of robot walks mainly over flat terrains and can only perform curved leg trajectories, generating body oscillations.

Previously, insect-based hexapod robots were built with three DOFs per leg (Table 1, in which the trochanter segment joint is merged with the femur and each joint only comprises one DOF, see Figure 2). Reflecting the standard insect leg model [59], this type of leg permits roaming in a slightly rough or slopped terrain in addition to a flat terrain walk.

More DOFs in leg morphology improve maneuverability and adaptation to challenging terrains [39,60]. Additional actuators help to adjust robot orientation according to the slope in order to increase stability [26]. Experiments have shown that 4 or 5 joints per leg enable robots to cope with high gradient slopes in any orientation (e.g., up to 50° slopes, see [26,61], or up to 43° slopes, see [14]). Unfortunately, these improvements increase the level of complexity of control commands and the robot's price and weight (Table 2), they also concomitantly, reduce autonomy due to the high power consumption of numerous actuators.

To sum up, from the large number of robots based on three-DOF legs, this appears to be the right compromise to walk on a flat terrain. Despite the three DOFs per leg trend, from a biological point of view, an insect possesses more than three joints with one DOF per joint [62,63], allowing it to overcome large obstacles and cross sloped and rough terrain (e.g., up to 54° see [64]). More complex models based more closely on insect leg kinematics are being developed [65]. Dung beetle like legs were built in 2018, the leg design was based on micro-CT scans of a real dung beetle [66]. A pair of beetle-like legs comprising four DOFs, allowing both manipulation and transportation was tested [66].

In 2017, a hexapod robot, called Cassino Hexapod III (~3 kg), composed of hybrid legs on a modular anthropomorphic architecture with omni-wheels, as feet at the extremities, was designed and built [67]. Each hybrid leg was built with three DOFs
with the third being dedicated to rotating the wheel at the tip of the leg. This kind of hybrid locomotion is relevant for efficient rolling mobility on moderate terrain and walking mobility on extreme terrain, such as non-terrestrial exploration [68]. Hybrid locomotion by walking or by rolling allows hexapod robots to save energy, and this hybrid locomotion is a combination of an engineering approach and a bio-inspired approach. Hybrid locomotion has not been developed in this review, which is focused on the biomimetic approach.

<figure><img src="image_5.png" /><figcaption>**Figure 2.** A standard hexapod robot 3-DOF leg, based the *Cataglyphis fortis* ant scale 1:30. Angle α corresponds to the thorax-coxa joint position, angle β corresponds to coxa-femur joint (up to now, robotic designs have often fused the trochanter-femur joint), and angle γ represents femur-tibia position. Illustration: ©Camille Dégardin & Ilya Brodoline (2021).</figcaption></figure>

* **Structure type:** this describes how leg joints are linked to each other. Two major leg designs are used on inspect-inspired robots: serial multi-shaped legs or single shape legs.

Serial multi-shaped legs are the most common structures encountered for locomotion and navigation. By definition, an insect leg is composed of five segments (coxa, trochanter, femur, tibia, and tarsus), arranged in a particular toggled zig-zag shape, forming a *sprawled* posture, reducing and distributing the forces on every joint of the leg [69,70]. However, in most robotic cases, this structure is simplified to three segments per leg (coxa, femur, tibia) comprising three joints per leg, each one with only one DOF (see previous point). In this arrangement, the trochanter segment of the leg is merged with the femur, and the tarsus is generally removed. However, the tarsus makes an important contribution to the insect's walk, serving as an adhesive pad [62,71] and allowing a better ground forces transmission with a passive spring effect. Moreover, some insects (e.g., leafhoppers) possess particular tarsal structures allowing them to jump from smooth surfaces [72]. Looking over the last decade of hexapod robots, presented in Table 1, the tarsus is often neglected, even though it represents more than 30% of the leg length [73]. Currently, few artificial tarsus designs have been developed to efficiently walk on complex terrains [74].

Single shape legs are used on *whegs* robots, origami, and compliant joint robots. The specifications of these types of robots, require the absence of most tiny mechanical parts such as bearings, shafts, screws, and nuts, and involves a simplicity of manufac-
turing, scale and cost reduction and, backlash and structural robustness improvements. The development of single shape legs follows the advances in new materials and manufacturing techniques such as multi-material 3D printing, which allows the building of soft joint robots [75]. Particularly, the 3D printing of legs appears to be a good way to develop and simplify standard joint designs by using properties of these new materials, such as flexibility or heat deformation [76]. In this way, hexapod robot legs are tending to become closer to real insect legs, in terms of relative dimensions and mass. An important point to notice for insects, e.g., cockroaches, is that a leg corresponds to approximately 2% of the body mass [77], allowing them low inertia, high frequency strides during a walk. In comparison, insect-like robot legs represent at least 10% of the overall mass (estimated for a 2 kg robot, from Table 1). Apart from 3D printing, some materials could take over from standard aluminum or molded plastic legs, e.g., chitosan–fibroin material, inspired by insect cuticle structure [78]. Furthermore, some other original structures have been designed; they were mainly developed when a specific animal behavior, such as jumping [79,80], is to be replicated or to satisfy some sought after design specifications like posture changes [81]. At first glance, leg design is highly dependent on actuator technologies. However, an impressive number of improvements are still possible through subtle structural modifications, allowing huge performance improvements. Independently of the structure type, observing the current state of the art in leg design, a question presents itself: why are all the legs of a hexapod robots the same? Insect legs are different in size (Figure 3, see [82]), and not built like robot legs, wherein the six are often identical, except for a few robots mimicking insect morphology in detail (Drosophibot [18,19] and MantisBot [30,31]). In response to this question and with the technologies now available, in the 2020s, leg design is likely to become increasingly based on available micro-CT scans of real insects (e.g., [66,83] dung beetles, [84] flies, or [65] ants) in order to improve the level of complexity, fidelity, and bio-inspiration.

<figure><img src="image_6.png" /><figcaption>**Figure 3.** Scale, segments, and joints in ant *Messor barbarus*. Photographic credits: Hugo Merienne, Centre de Recherches sur la Cognition Animale (CRCA UMR 5169), Toulouse, France. The relative scale of each segment (coxa, femur, tibia, tarsus) w.r.t. the coxa leg of each leg (R1, R2, R3) comes from [73]. Adapted from [82] under CC-BY License, 2019.</figcaption></figure>
### 2.1.2. Cost of Transport

To compare the level of performance between robotic designs, the cost of transport (CoT) is now a common adimensional metric (Equation (1)) used to evaluate any legged robots [14,85–87].

$$CoT = \frac{\bar{P}}{m \cdot g \cdot \bar{v}} \qquad (1)$$

where $\bar{P}$ is the mean power consumption, $m$ is the robot weight, $\bar{v}$ is the mean speed, and $g$ is the gravitational acceleration ($g = 9.81 \text{ m·s}^{-2}$). The CoT depends linearly on the weight in log-log space with a negative slope [88–90] for both animals and robots. In the animal kingdom, biologists evaluate the efficiency locomotion with the mass-specific energy per unit distance (in J/(m·kg)), which can be defined by the gravitational acceleration-CoT product. With such an approach, we can estimate the ant's CoT ~ 39, and also the CoT of a 2–3 kg animal as close to ~2. The current CoTs of hexapod robots weighting 2–3 kg is similar those of animals (Table 2). As a result, we can conclude that the locomotion efficiency of hexapod robots is currently no better than that of animals.

The CoT of hexapod robots is lower in tripod locomotion on a flat terrain [14]. A tetrapod or pentapod locomotion deteriorates the CoT value, likewise a sloping or rough terrain. As a result, to compare hexapod platforms, they must be evaluated using the same task and environmental conditions, here in tripod locomotion on a flat terrain and in a room with the air temperature set at 25 °C (Table 2).

<table><caption>**Table 2.** Level of performance in terms of cost of transport (CoT) in hexapod robots walking in tripod locomotion over a flat terrain. The speed is the maximum speed.</caption><thead><tr><th>Robot</th><th>Actuators</th><th>#Actuators</th><th>Mass (kg)</th><th>Speed (m/s)</th><th>CoT</th></tr></thead><tbody><tr><td>Daisy</td><td>X-series<br>X8-9 and X8-16</td><td>18</td><td>21</td><td>0.13</td><td>3.7</td></tr><tr><td>HAntR</td><td>Dynamixel AX-12A</td><td>24</td><td>2.9</td><td>0.43</td><td>1.5</td></tr><tr><td>AntBot</td><td>Dynamixel AX-18A</td><td>18</td><td>2.3</td><td>0.90</td><td>6.2</td></tr><tr><td>CRABOT</td><td>Dynamixel AX-18A</td><td>24</td><td>2.5</td><td>0.05</td><td>-</td></tr><tr><td>Hexabot</td><td>Dynamixel XL-320</td><td>18</td><td>0.93</td><td>0.35</td><td>-</td></tr><tr><td>Weaver</td><td>Dynamixel<br>MX-64 and MX-106</td><td>30</td><td>7</td><td>0.16</td><td>1.5–1.8</td></tr><tr><td>EduBot</td><td>DC motor</td><td>6</td><td>3.3</td><td>2.5</td><td>0.5–1.6</td></tr><tr><td>Messor II</td><td>Dynamical RX-28</td><td>18</td><td>2.5</td><td>0.09</td><td>-</td></tr><tr><td>BionicANT</td><td>Trimorphic piezo-ceramic</td><td>18</td><td>0.105</td><td>-</td><td>-</td></tr></tbody></table>

### 2.1.3. Actuation of the Legs

Many actuator technologies can be used to control the joints between the leg segments: servomotors, brushless motors, or artificial muscles, which are under developments in laboratories.

* **Servomotors:** the main issues with servomotors are their weight and energy efficiency. A servomotor heats up easily until it surpasses its maximum operating temperature of 70 °C, then it stops working. In addition, a servomotor is composed by definition, of a motor with high ratio gearing used to make it as stiff as possible. In this sense, such servo-based actuators differ significantly from biological actuators that may have variable stiffness and adaptable compliance. One way to implement variable stiffness is to use springs to make variable impedance actuators (VIA). As summarized in [91], VIA actuators can be classified into three categories: spring pre-loaded variation, transmission ratio changing and spring physical property alteration. VIA is
certainly an approach of great interest for the design of future hexapod robots able to dynamically change the stiffness of their joints.

* **Brushless motors**: recent developments in smart rotating actuators based on brushless motors will permit the design of direct drive joints without gearing. The maximum specific power of electric motors with permanent magnets is 300 W/kg, which is about the same order as biological muscle [92]. Companies, such as HEBI Robotics [17] or IQ Motion, have developed integrated rotating actuators for robotic applications and for the development of mobile robots of various sizes. As the electronic driver and angular sensor are integrated into the motor, it drastically simplifies the wiring and complexity of the overall hardware architecture, which can be crucial when designing robots like hexapods that require the control of 18 actuators.

* **Artificial muscles**: the design of future insect-inspired robots will certainly depend on the availability of actuators able to mimic the functionalities of biological muscles. Their properties of viscoelasticity and energy dissipation leading to high compliance is the holy grail of insect-inspired actuators. Among the broad repertoire of new artificial actuators for robots (see review by [93]), non-conventional actuators like pneumatic artificial muscles (PAMs), shape memory alloys (SMAs), and electroactive polymers (EAPs) are of great interest. One particular case is HASEL actuators, which are composed of a series of pouches made of a flexible and inextensible shell that is filled with a liquid dielectric. Electrodes cover a portion of each pouch so as to progressively close when a voltage is applied thus squeezing the pouches to increase their volume [94]. HASEL actuators can be implemented in different ways and can feature a bandwidth as high as 126 Hz for the quadrant-donut HASEL and even a specific energy twice as high as mammalian skeletal muscles for the planar HASEL actuator [94]. HASEL actuators mimic the muscle-like performance of dielectric actuators (DEAs), which can be highly effective for robotic applications. They can lift more than 200 times their weight and have a peak specific power of 585 W/kg [95]. Moreover, it is worth noting that a toolkit has been developed to aid designs using HASEL actuators [96]. In addition, electro-ribbon technology, with an ability of lifting 1000 times its own weight and a contraction by 99.8% of its length, is also very promising [97]. Finally, five-DOF soft dielectric elastomer actuators have been shown to be very useful in the implementation of soft legged robots [98] which are able to walk with an alternative tripod gait as fast as 52 mm/s for a 7 Hz actuation frequency.

### 2.1.4. Force Sensing in Robotic Legs

An overview of the state-of-the-art in force sensing for multi-legged walking robots is available in this study [99], but where is the right location in a leg for sensing a mechanical action: is it at the tip of the leg, within the leg structure, or directly in the joints?

* **Leg tip/TARSE sensing**: sensing at the end of the leg can be done by a tactile sensor, a pressure sensor, a three-axis force/torque sensor [100], or a compliant force sensor made with a spring [15,39]. Leg tip sensing can be easily implemented by adding an attachment point to the leg tip without requiring any modification of the robot's structure. The cost of these tip sensors can be expensive depending on the chosen technology, but recent research has developed low-cost designs [100]. Tip force sensing is useful because it provides the robot with a terrain description. Force measurements allow the robot to understand which of its legs are in contact with the ground, or to evaluate the terrain slope, in order to both adjust its gait and plan its path [101].

* **Force sensing in actuators**: sensing coming from the state of actuators by current measurement [26,29,37,43] or dedicated sensors in the joints. This category of sensors simplifies the robot's design, since the sensor is integrated within the actuator forming a compact structure. The complexity of the estimation of forces from the currents generated by the actuators is based on the robot's leg model. To obtain an accurate leg movement, the mathematical model should reflect the robot as closely as possible,
and take account of any structural deformation under various loads since no material
is perfectly rigid.

*   **Legs with compliant structures:** compliant mechanisms exploit the deformation properties of the leg segments, deformations that could be a disadvantage in other legs. Stiffer legs appear to narrow the region of stable gaits while preventing tripod contact with the ground. However, compliant legs are more capable of absorbing energy even if the leg touches down early, thus minimizing the severity of ground reaction on legs. This solution had been developed for one-joint C-shaped legs [44,45,54]. Whegs do not possess any force sensors on their legs. Compliant legs offer the possibility of placing the force sensors along the segments (such as the femur or the tibia) [101,102]. This type of sensor placement mimics the force measurements in insects, as done by campaniform sensilla mechanoreceptors [103,104].

Currently, three distinct groups of sensors exist and are implemented on board hexapod robots. A force sensing in actuators may be chosen if no structural modification is allowed. Alternatively, with a slight modification of the tarse, leg tip sensors provide the required information for achieving accurate locomotion control over rough terrains. Lastly, compliant legs permit the implementation of force sensing although this will increase the complexity of the design.

## 2.2. *From Legs to Robots*

### 2.2.1. Body Morphology

Although hexapod robot leg morphology seems to be a similar, albeit simplified, representation of insect morphology, the overall body structure of most of the common robots is far from that of animals. Firstly, few robot designs feature a multi-segment thorax. While this is present in robots such as HECTOR [34–36] or MantisBot [30,31], the benefit of this type of design has still not been demonstrated. This only appears to help during tight turns or high obstacle climbing [105]. Thus, the increase in weight and energy consumption of this type of design, due to additional joints and body parts, justifies the more general choice of single shape body morphology. The second point is another aspect of body geometry. Various geometries are used in hexapod robot developments: circular, hexagonal, rectangular, etc. The main difference between these geometries is the necessity of performing a specific turning gait for the rectangular shape, whereas circular or hexagonal bodies allow, through their symmetry, an omnidirectional walk (see the review [13] for more details).

### 2.2.2. Scale Effect on Level of Performance

Insect-like robot design process aims to validate biological hypotheses in navigation or locomotion, or to mimic behaviors and sensory systems (as shown in Table 1). The scale should then be taken into account throughout the development process as this will affect kinematic changes in robot locomotion and mechanics material [106]. Therefore, the question of the scale’s influence is raised. How will the scale affect the performance level? If a behavior is present in a small insect, is it possible to reproduce it at a given scale of robot? To answer these questions, it is necessary to discuss the kinematic changes in robot locomotion taking account of material mechanics and energetic aspects.

From the point of view of the material, the larger the robot is, the larger are going to be the internal deformations of its segments. To compensate for deformations, larger beams have to be used, so the weight of the leg segments and of the thorax are increased. Moreover, the scale factor of the robot proportionally affects the step size. However, not all kinematic parameters vary linearly with scale. From mechanical equations, some scale factors have been set [107,108], giving a scaling effect on essential physical values like mass, frequency, stiffness, damping, velocity, and power. Thus, a small size robot with a scale close to that of an insect can possess a light structure, fine legs, and walk at a high frequency. Conversely, for a large scale robot, higher mass and a slower frequency walk, due to inertia
increase, are going to be observed. Higher actuator power will also be required, to set the robot in motion, and this implies a manufacturing price increase.

As a brief conclusion, the development of large scale hexapod robots is not justified. The advantages of working with a high scaling factor are their improvements in energetic values and technological limits (see Table 2). Regarding power consumption, represented by the CoT (see Section 2.1.2), the relationship between different scales of the same robot morphology has not yet been precisely fixed [89,108]. Basically, the global trend shows that, the more massive the hexapod, the lower its CoT; thus, it has a better walk efficiency. However, and by definition, the CoT does not take account of animal scale and its interaction with environment. The high CoT in insects must be considered in terms of their lifestyle, they do not need to save more energy when traveling. Accordingly, a robot's size should be selected according to the type of mission (environment relief, distance to travel, payload, etc.), then corresponding to a given CoT value, roboticists should be able to compute the ideal robot mass.

Moreover, there are difficulties in manufacturing reduced scale robots. For instance, common methods of 3D printing have an average deviation error of 0.4 ± 0.2 mm [109], which limits the size of printed mechanics. Batteries also have technological limits. Commonly used lithium batteries have a low energy density (around 150 Wh/kg) [110]. Moreover, battery cells have a high mass, difficult for a light robot. In the near future, the development of new bio-inspired power sources should bridge this energetic gap, with a higher energy density (298 Wh/kg [110]), and this would allow more compact robot designs [111].

## 2.3. Hexapod Robot Accessibility Criteria for Academics

Over the past 10 years, many new designs have been proposed to the general public. These hexapod robots are generally available in *do-it-yourself* kit form, thus promoting opportunities for customization such as modifications in actuation and the addition of sensors to enhance the robots' locomotion skills. For instance, the small inexpensive hexapod robot Hexy, produced by ArcBotics, features 18 DOFs achieved by means of Fitec servomotors (0.12 s per 60°), and is mainly intended for educational applications (Figure 4a). A leader in the market of robot kits for the general public, Lynxmotion™ offers a wide range of relatively small designs, such as the Phoenix, which features 18 DOFs (Hitec HS-645 servomotors) with a maximum speed of 25 cm/s (Figure 4b), or T-Hex for which the 24 DOFs allow more complex motion (Figure 4d). Recently, EZ-Robot released Six, a fully 3D-printed hexapod robot endowed with 12 DOFs (Figure 4e). As with Hexy, this robot was released with educational purposes in mind, allowing students to tackle complex tasks such as visual-based object tracking and artificial intelligence with machine learning applied to gait generation. The 18-DOF PhantomX AX Metal Mark III hexapod robot (Figure 4c), developed by Interbotix Labs., offers promising dynamic performance with a maximum speed measured at 80 cm/s. Generally, these robots could be considered as inexpensive, with prices ranging from 250 to 3000 US dollars, but others require a significant budget, such as the Daisy Hexapod Robot Kit from HEBI Robotics [17], available from 83k US dollars (Figure 4f). The variations in price are almost exclusively the result of the quality and number of servomotors used in their design. On average, hexapod robots equipped with servos using only plastic gears are cheaper, but have a significantly reduced lifespan compared to their metal counterparts. Although standard servomotors are not optimized for such applications, they offer a quick solution for benchmarking new designs of hexapod robots in academia. Open-source projects by private individuals have also been reported. For instance, the MX Phoenix hexapod robot [27] and the MorpHex double hexapod robot were developed by Kare Halvorsen, who was inspired by robotic projects from Jim Frye (founder of Lynxmotion) and from Matt Dentons (Mantis project manager and founder of Micro Magic Systems). MorpHex's unusual design features two hexapod robots connected in a top to tail fashion. Their legs are covered with spherical parts in such a way that when the robot refolds, it forms a perfect sphere. This mechatronic design allows MorpHex to either crawl or roll on almost any kind of terrain.
<figure><img src="image_7.png" /><figcaption>**Figure 4.** Open-source hexapod robots and educational robots available in kit form. (A) Hexy (Credits: ArcBotics). (B) Phoenix (Credits: Lynxmotion). (C) PhantomX (Credits: Interbotix Lab., Trossen Robotics). (D) T-Hex (Credits: Lynxmotion). (E) Six (Credits: EZ-Robot). (F) HEBI Robotics' 18-DoF Daisy Hexapod, built using HEBI's X-Series actuation hardware (Credits: ©HEBI Robotics 2021).</figcaption></figure>

The aforementioned robots represent a unique set of opportunities for academia in both research and education since they are relatively inexpensive and easy to obtain, either in kit form or by replication based on open-source models. Fast customization for adaptation to the experimental context is another important feature of these new products. This recent trade has been intensified with the emergence of online sharing platforms such as *GitHub*, *Thingiverse*, and *Instructables*. A wide range of open-source projects can be found on these very popular platforms, supported by an extensive international community of robotics designers, engineers, and researchers. The recent democratization of 3D printing is also greatly contributing to the rise in open-source projects. Lastly, the development of a unique framework for robot programming, i.e., robot operating system (ROS), ensures the standardization of software developments, such as locomotion firmware for hexapod gait generation.

### 3. From Biomechanics to Locomotion

In the previous sections of this article, the relevance of biomechanical aspects has been mainly summarized in relation to actuation, robot leg, and morphology design (see Section 2). Supplementing biomechanical developments, locomotion control (Figure 1) is an important ingredient for interlimb and intralimb coordination as well as joint compliance (Figure 5). Interlimb coordination concerns the relationship between the legs for generating gaits, while intralimb coordination deals with the relationship between the joints within one leg for generating swing and stance, as well as adapting leg movements to deal with complex terrain [112]. Joint compliance can be beneficial for impact force absorption, payload compensation, disturbance rejection, and energy efficient locomotion [113,114].

In Section 3.1, we will introduce insect locomotion control for interlimb and intralimb coordination and joint compliance control. In Section 3.2, we will present different robot locomotion control methods including bio-inspired, engineering-based, and machine learning-based control. Finally, in Section 5.2, we will suggest some future directions in locomotion control for hexapod robotics.
<figure><img src="image_8.png" /><figcaption>**Figure 5.** (a) Overview of locomotion control involving interlimb coordination (coordination between legs), intralimb coordination (coordination between joints in a leg), and joint compliance (a property of joint with variable stiffness). (b) Example of joint compliance generated by an adaptive bio-inspired muscle model (modified from [115]). Using this model, a leg joint can behave like dampers, struts, brakes, or springs (modified from [115]). (c) Example of intralimb coordination in the swing and stance phases of the leg and different reflexes for walking on uneven terrain (modified from [116–118]). (d) Example of interlimb coordination which results in different gaits (modified from [119]). The main observed gaits from slow to fast [120] include: wave gait (only one leg lifts at any given time (swing) while the remaining legs stay on the ground (stance), and the wave travels from back to front), transition gait (front and back legs of the opposite side lift together at a given time while the other legs stay on the ground), tetrapod gait (diagonal pairs of legs lift together at a given time while at least four legs stay on the ground), and tripod gait (front and back legs of one side and the middle leg of the opposite side lift together at a given time while the remaining three legs stay on the ground). While the articles specify specific gaits, it is important to note that certain insects (such as stick insects [121,122], cockroaches [123], and flies [124]) frequently change their gaits depending on their locomotion speeds and situations [125].</figcaption></figure>

### 3.1. *Insect Locomotion Control*

Insect locomotion control is based on neural mechanisms with a distributed control architecture [126,127] located in the insect's thoracic ganglia. It is known that there are special neural circuits, called central pattern generators (CPGs) [128,129], with spiking and non-spiking interneurons for generating basic rhythmic motor activities. Activity can still continue even in the simultaneous absence of both afferent feedback and rhythmic inputs from other neural circuits. Biological investigations reveal that each hemisegment has independent CPG modules that regulate the motor neurons and muscles of the leg joints. As described in [129], CPG non-spiking interneurons (NSI E4, NSI I4, and NSI5) contribute to the generation of various leg movements. They can reset the phase and influence the frequency of the rhythmic neuron activity in leg motor neurons [130]. Specifically, NSI E4 was observed to deactivate the stance phase and activate the swing phase while NSI I4 was observed to induce searching movements [131].

Although CPGs are the basis of leg movement generation, sensory feedback is important for interlimb coordination and adaptation. For instance, joint movement signals are used to coordinate joints and switch leg movement states from swing to stance and vice versa. Furthermore, a combination of joint movement signals [132] and load signals [133] can entrain the CPG. This is known as entrainment. Studies on stick insects and cockroaches have identified interlimb coordination where neural signals from a front
leg can entrain the CPG controlling the middle leg. Sensory feedback to the CPG and coordination for gait transition can occur at two levels: one is direct sensory input to the central descending command of the CPG (high-level feedback) [134] and another is local sensory feedback or reflexes which influence the CPG phase (low-level feedback) [135]. Typically, for interlimb coordination, it seems that slow-walking insects (e.g., stick insects) rely more on sensory feedback [136] than on the central coupling between CPGs, while fast-walking insects (e.g., cockroaches) rely more on centrally coupled CPG coordination (for more details see [129]).

In addition to CPGs and sensory feedback, some biological investigations have shown that forward models [137] also play an important role in insect locomotion control. The forward model is the neural mechanism that transforms motor commands (efference copies) into an expected sensory input in order to compare it to the actual incoming sensory feedback. It is used for state estimation or sensory prediction. This information can allow for movement adaptation. In the stick insects *Aretaon asperrimus*, when ground contact is lost at the end of the swing phase while climbing over extremely wide gaps, they immediately alter their leg stepping pattern [138]. This would indicate that ground contact is expected on a frequent basis. Other findings that support the concept of forward model predictions [116] show that stick insect responses to barriers are influenced by an internal state during the swing phase. In locusts [139] and fruit flies [140], they seem to have, when tested in flight simulators, forward models that can predict changes in their sensory feedback and adaptively modify the motor control parameters to cope with these changes. Current animal studies (mentioned above) indicate that CPGs, multimodal sensory feedback, and forward models are used in insect locomotion control for interlimb and intralimb coordination, movement adaptation, and joint compliance (all described below). The contributions of these components, however, vary across species.

*   **Interlimb coordination:** biological studies have revealed rules for interlimb coordination of insect locomotion. For instance, Wilson [119] proposed five rules. Rule 1: a wave of swing runs from hind (posterior) to front (anterior) legs. Rule 2: contralateral legs of the same segment alternate in phase. Rule 3: protraction (swing) time is constant. Rule 4: frequency varies (stance decreases as frequency increases). Rule 5: the intervals between steps of the hind leg and middle leg and between the middle leg and fore leg are constant, while the interval between the foreleg and hind leg steps varies inversely with frequency. These rules have been translated to neural mechanisms for hexapod locomotion control, which can generate various insect-like gaits [141,142].

Subsequent research by Cruse et al. [116] introduced six rules for insect walking (called WalkNet). The rules were derived from behavioral experiments with stick insects. Rule 1: posterior swing inhibits start of anterior swing. Rule 2: start of posterior stance excites anterior swing (posterior reaches a given anterior extreme position (AEP)). The AEP is the anterior transition point from swing to stance in a forward walking animal. Rule 3: caudal positions of anterior stance excite start of posterior swing (anterior reaches a given posterior extreme position (PEP)). The PEP is the posterior transition point from stance to swing. Rule 4: end position of anterior stance influences end position of posterior swing (called targeting). Rule 5: increased resistance increases force and increased load prolongs stance phase. Rule 6: the information from the anterior leg's reflex stimulation is passed on to the posterior leg. Recently, Schilling and Cruse [143] introduced the realization of these rules as an artificial neuronal network with an antagonistic structure (called neuroWalknet controller). The controller can generate diverse robot walking behaviors including different gait patterns emerging from different velocities, curve negotiation, and backward walking.

In addition to the aforementioned rules for insect locomotion, a recent study from Leung et al. [144] analyzed and identified four underlying rules for interlimb coordination of dung beetle ball rolling gaits. Rule 1: front legs alternately step on the ground. The rule describes the relationship between the two front legs in the gait.
Rule 2: each middle leg steps similarly to its contralateral hind leg. The rule describes the synchronization of the contralateral middle and hind legs. Rule 3: an ipsilateral pair of middle and hind legs seldom lift together. Rule 4: a contralateral pair of middle or hind legs rarely lift together. In principle, a pair of legs following the third and fourth rules tend not to lift together. A partial implementation of the rules as modular neural control with a CPG was performed and tested on a simulated dung beetle-like robot [4]. The controller can generate four different robot behaviors including forward walking, backward walking, level-ground ball rolling, and sloped-ground ball rolling.

*   **Intralimb coordination:** In addition to the biological studies of the relationship between legs (interlimb coordination) in insects, some studies have further investigated individual leg movements and adaptations during normal and rough terrain walking in insects. The leg movements basically reflect intralimb coordination. For instance, Pearson and Franklin [117] proposed locusts' reflex strategies for leg movements when walking over rough and complex terrain. As described by them, the strategies include (1) rhythmic searching movements; (2) local searching movements; and (3) elevator reflex.

Based on [117], the rhythmic searching movements are to search for a ground contact, if the animal has not located it by the end of its swing phase. The searching movements show rhythmic patterns including fast elevation and depression movements of the leg. While searching, the animal also extends the search range from the body to explore the supporting points around the leg, e.g., up to eight searching cycles. The searching typically stops either when the animal stops walking, the leg gets stuck, or ground contact is found. The local searching movements are small rhythmic leg movements from point to point on a potential supporting ground. These movements occur either at the beginning of a stance phase, after the rhythmic searching movements and/or an elevator reflex (described below). The local searching movements are required if the potential support surface is smooth where the leg action needs to push the animal forward. The elevator reflex is a rapid elevation and extension of the leg to step over an obstacle, followed by placing the leg where the obstacle can be used as a support. The elevator reflex can be activated when the leg gets stuck during the swing phase. It can also occur during searching movements. In rough terrain walking experiments on locusts, the elevator reflex was mostly observed in the fore and middle legs while the hind legs moved behind the animal. This makes it difficult to distinguish between an elevator reflex and a passive pulling of the hind legs up onto the obstacle while moving forward. Examples of the implementation of these reflex strategies for adaptive hexapod robot locomotion on rough and complex terrain can be seen at [118,142].

*   **Joint (mechanical) compliance:** this is the interactive relationship between kinematic changes and the resulting dynamics of joints [145]. One of its key components, stiffness, refers to the ratio between joint torque and angle changes, which are related to muscle activation. Compliance control of insect muscles is very important in facilitating adaptive and robust locomotion over natural terrain [146,147]. Computerized (computational) muscle models can be used to enhance the understanding of neuromechanical control principles underlying insect locomotion [148]. The Hill muscle is one of the most influential 'seed' models that has inspired many successors [149]. For instance, Proctor and Holmes built a neuromechanical model to study feedback effects of perturbated insect locomotion [150], in which 24 neural oscillators and 48 pairs of Hill muscles were used. Guo et al. proposed a neuro-musculo-skeletal model to reproduce gait pattern in virtual insects [151]. Naris et al. analyzed a closed-loop neuromechanical simulation of insect joint control driven by a pair of Hill muscles [152]. However, most of these studies were limited to numerical simulations, because a greater number of parameters needed to be offline optimized based on nonlinear differential equations. Therefore, they failed to account for intrinsically delayed feedback in real insect locomotion dynamics. This failure may cause a misin-
terpretation of the neuromechanical control principles in insect locomotion. Therefore, questions remain open whose answers may decode insect muscle intelligence in dynamic robust locomotion.

- **Multifunctional muscles:** insects exhibit different muscle functions in flying and walking [153], which are characterized by the work loop technique [154]. These functions may facilitate the decoding of muscle compliance in dynamic insect locomotion. Interestingly, some preliminary results show that muscles act as brakes and springs when their passive stiffness and damping are tuned in computational simulations [155]. Tuning muscle stiffness and damping properties based on the work-loop technique, can be a key to understanding and translating muscle intelligence between engineering applications and neuromechanical models [115,156].

- **Predictive muscle tuning:** muscle compliance can be tuned in terms of sensory feedback. However, this feedback is intrinsically subject to noise and delays owing to high levels of dynamics of insect locomotion. Therefore, it may be assumed that insects, and their robot counterparts use internal models to predict sensory outputs for tuning insect muscle compliance [114].

- **Engineering-inspired muscle intelligence:** biological muscle control principles have been borrowed to enhance robot designs and control for many years. This research approach can be flipped, i.e., robots as tools for decoding muscle compliance in insect locomotion [157]. For instance, an insect-like robot was used to test a simplified muscle control hypothesis, i.e., proximodistal gradient [114]. It showed that this gradient reduces the number of controlled variables and enhances walking stability. Engineering-inspired methods can close the research loop of insect muscle intelligence, providing new hypotheses for biological experiments on insect locomotion.

## 3.2. Robot Locomotion Control

While insect locomotion control (Figures 1b and 5) has been studied and thoroughly investigated as described above, translation from biological investigation to robot implementation remains a challenge. To date, different locomotion control methods for interlimb and intralimb coordination have been proposed (Figures 1b and 5). The key methods from different domains include (i) bio-inspired control (e.g., pure CPG(s), pure reflexes, or their combination) (Figure 6a); (ii) engineering-based control (e.g., kinematic and dynamic models) (Figure 6b); (iii) machine learning-based control (Figure 6c); and (iv) a combination of these key methods. This review will focus on bio-inspired control while the other control techniques will be briefly discussed.

<figure><img src="image_9.png" /><figcaption>**Figure 6.** Different control approaches to robot locomotion. (a) Bio-inspired control. The upper-inset shows distributed decentralized CPGs with force feedback [158–160]. The middle inset shows the Walknet-control (modified from [143,161]). The lower inset shows the CPG-based control and a simplified model with six oscillators for interlimb coordination (modified from [162,163]). (b) Engineering-based control [66,164]. (c) Machine learning-based control [165,166].</figcaption></figure>
### 3.2.1. Bio-Inspired Control

One of the standard bio-inspired methods is CPG-based control where CPG rhythmic signals are used to generate individual leg movements (intralimb coordination) and create coordination between the legs (interlimb coordination) [167] (Figure 6a, upper and lower insets). A number of CPGs have been explored for locomotion control from minimal to maximal CPGs (i.e., all legs and joints controlled by a minimal single CPG [168], each leg by one CPG [169,170] (six CPGs in total), or maximally each joint by one CPG [171] (18 CPGs in total)). Various CPG models have been developed (Figure 7) from conceptual biological CPG models based on a half-center oscillator [172], biophysical models using Hodgkin-Huxley neurons [173], connectionist models using simplified neurons with various activation functions [174–176] to abstract models using nonlinear coupled oscillators. Note that here CPG(s) and oscillator(s) are used interchangeably through out this section.

<figure><img src="image_10.png" /><figcaption>**Figure 7.** Timeline of the development of various CPG models from 1914 to 2020. The key CPG models include a half-center oscillator in 1914 [172], the Van der Pol oscillator in 1926 [177], the Matsuoka oscillator in 1985 [178], the Matsuoka oscillator with entrainment in 1991 [179], a biophysical oscillator based on Hodgkin-Huxley neurons in 1992 [173], a connectionist oscillator based on leaky-integrator neurons in 1993 [174], a connectionist oscillator based on a cellular neural network (CNN) in 2000 [175], an SO2 neural oscillator in 2003 [176], an adaptive frequency oscillator (AFO) in 2006 [180], a neural oscillator with magnitude adaptation in 2007 [49], an adaptive chaotic oscillator in 2010 [141], an oscillator with phase resetting [181], an oscillator with continuous phase modulation [182], an oscillator with frequency adaptation through fast dynamical coupling (AFDC) in 2017 [183], a neural oscillator with fast online-error based learning in 2019 [184], and a neural oscillator with dynamical state forcing (DSF) in 2020 [185].</figcaption></figure>

In principle, the CPG acts as an open loop control since it does not require sensory feedback to generate its periodic patterns. Most of the CPG models can generate only periodic patterns. To achieve a variety of complex patterns, including periodic and chaotic ones for complex robot locomotion, Steingrube et al. [141] modified the SO2 oscillator to become a chaotic CPG and applied a novel adaptive chaos control that exploits neural dynamics embedded in the chaotic CPG and uses time delay feedback mechanisms to control the dynamics. Although none of these abstract models require any external input or sensory feedback to produce basic rhythmic activity, they do need sensory feedback to adapt and tune their frequency, phase, and magnitude for efficient locomotion control to deal with different situations. Thus, different feedback techniques have been developed for frequency, shape, and phase adaptations.

For the frequency adaptation, typically, joint angle feedback and foot contact feedback are used to entrain the frequency of an oscillator (entrainment [186]). If the oscillator gets entrained by the feedback and adapts to it, even if only temporarily, i.e., the feedback only has a short term effect, such an oscillator is considered as reactive [177,179]. In this case, if the feedback is switched off the CPG system immediately returns to its intrinsic dynamics, there is no memory of the input and no lasting change to the dynamics. To maintain the effect of the feedback after the feedback has been removed, Righetti et al. [180] introduced a frequency adaptation schema, which permanently modifies the intrinsic frequency of an oscillator. An oscillator with this schema is considered to be an adaptive frequency
oscillator (AFO). However, AFOs can suffer from significantly long adaptation times as has been shown in many robotic applications. To obtain fast as well as precise adaptations, Nachstedt et al. [183] proposed frequency adaptation through fast dynamical coupling (AFDC). It is based on dynamically adapting the coupling strength of sensory feedback to an oscillator. While the AFO and AFDC can automatically adapt the oscillator frequency to match the frequency of an external periodic signal (i.e., robot joint angle feedback, which can identify robot locomotion), they do not deal with the tracking error that may occur between the actual robot's motion and oscillator output. This could lead to the loss of precision, unwanted movement, or energy-inefficient locomotion. To address this, recently Thor and Manoonpong [184] proposed online error-based learning for frequency adaptation of oscillators. The learning mechanism can reduce tracking and steady-state errors as well as perform fast and stable learning. Frequency adaptation is basically used for robot locomotion enhancement [16].

For the shape adaptation, a typical technique is to use additional premotor neuron networks for shaping oscillator signals [49,187]. Different premotor neuron network models have been proposed. For instance, feedforward neural networks with a hyperbolic tangent activation function or a radial basis activation function were used to shape or translate oscillator signals into complex locomotion patterns [49,188] for climbing over or avoiding an obstacle [189,190]. To obtain motor memory for robust locomotion patterns, reservoir-based recurrent neural networks were applied [191]. While the typical technique uses various types of premotor networks and a learning method, recently Chuthong et al. [185] proposed an alternative approach that exploits the entrainment-like dynamics for CPG shape adaptation. This technique, called dynamical state forcing CPG (DSF-CPG), behaves as a reactive CPG that can temporarily adapt the geometry/shape of the CPG signals without using any learning or premotor networks. The DSF-CPG approach can promote robot compliance, by changing the target dynamics according to external perturbations.

For the phase adaptation, typically ground reaction force (GRF) feedback is employed to reset or continuously modulate the phase relationships between the oscillators. Two standard mechanisms for phase adaptation, resulting in adaptive interlimb coordination for self-organized robot locomotion, are phase resetting (PR) [181] and continuous phase modulation (PM) [159,182] (Figure 6a, upper inset). PR uses discrete GRFs to intermittently reset CPG phases while PM uses continuous GRFs to modulate CPG phases. A recent comparative study of the two mechanisms can be seen at [192]. Based on the experimental setup of the comparative study, PM shows slower but more stable phase convergence while PR shows faster but less stable phase convergence. PM performs better than PR when the robot is subjected to symmetrical GRF distributions while PR performs better than PM when GRF distributions are asymmetrical.

In addition to the CPG-based control described above, reflex-based control (pure reflexes), relying on sensory feedback, has also been widely used for robot locomotion generation with adaptability to different situations [29, 59, 117, 193-195]. The most well-known reflex-based control is the Walknet control inspired by stick insect locomotion (see above and Figure 6a, middle inset). Walknet control is realized by using an artificial neural network (leg controller) for intralimb coordination and a coordination rule-based state machine for interlimb coordination. It has been implemented as a decentralized control architecture for hexapod robot locomotion [59,118] where six independent leg controllers are employed, one for each leg. Each leg controller requires position and velocity feedback of the joint angles of the leg as well as GRF and/or tactile contact feedback of the leg on the ground and consists of subnetworks for swing and stance leg movements. The subnetworks consist of a Stance-net, a Swing-net, and Target-nets. The Stance-net controls the movement during a stance phase. The Swing-net controls the movement during a swing phase. The Target-nets indicate the end position of the swing movement (AEP) during forward and backward walking. An extension of the original Walknet control has also been developed by adding motivation units for autonomously selecting between different locomotion behaviors [196]. Walknet control can generate
emergent gaits for forward or backward hexapod walking over uneven surfaces. It can also
allow for curve negotiation and leg amputations, and follow motion trajectories without
explicit pre-calculation. The most recent version of Walknet (neuroWalknet controller) can
be seen at [143]. Walknet control has also been modified (called Rollnet) for a ball rolling
gait of a dung beetle-like robot [195]. The gait combines backward walking of the front legs
and ball manipulation by the middle and hind legs.

While Walknet control shows impressive performance for versatile and adaptive robot
behavior generation (locomotion and object manipulation), it may lead to unstable robot
behavior or failure in cases of sensory failure. Therefore, a combination of CPG- and
reflex-based control has been actively investigated and various types of this combination
have been developed [142,197-203]. For instance, CPG-based control with a sensory event
mistiming detection method and reflexes was proposed [201]. The mistiming detection
method consists of a CPG for estimating the sensory phase, a radial basis function (RBF)
neuron for estimating the sensory event, and a leaky-integrate-and-fire neuron for detecting
the sensory mistiming and activating reflexes to avoid an obstacle and search for a foothold.
This control approach enables a hexapod robot to walk effectively over highly unstructured
terrain with cracks and a wet slippery surface. Adaptive neural locomotion control, consist-
ing of a CPG network with neuromodulation and local leg control mechanisms based on
sensory feedback (only foot contact) and adaptive neural forward models with efference
copies (copies of CPG signals), was developed [142]. This control approach enables a
hexapod robot to perform a multitude of different walking patterns, including insect-like
leg movements and gaits (Figure 8), with energy-efficient locomotion. It can deal with
changes in terrain, a loss of ground contact during the stance phase, stepping on or hitting
an obstacle during the swing phase, and leg amputations. This robot can still perform basic
locomotion even without the sensory feedback. Recently, an artificial hormone mechanism
(AHM) was applied to the adaptive neural locomotion control [202,203]. AHM, which
replicates the endocrine system, can continuously online adapt neural locomotion control
parameters (lifelong continuous adaptation) for walking on different complex terrains
(e.g., mesa terrain, ramp-up and -down terrains, rough terrain, terraced terrain, compliant
terrain with different softness levels, and loose terrain).

3.2.2. Engineering-Based Control

A conventional way of achieving intralimb coordination is to use inverse kinematics (IK) requiring a robot kinematic model [66,164] (Figure 6b). In this approach, the trajectory of the end of a foot (consisting of stance and swing phases) is designed and the IK translates the trajectory into the robot joint angle. For the trajectory design, one simple way is to use a straight or almost straight profile for the stance phase and an arch profile and swing phase [159]. An alternative way is to record an animal leg trajectory during locomotion and use it as the desired robot leg trajectory [66,126,204]. For instance, Cruse and Bartling [204] recorded and analyzed the swing trajectory in different stick insect walking situations and applied IK to calculate joint angles for robots. Ignasov et al. [66] introduced a complete framework for generating complex insect-like leg movements. The methodology consists of (1) tracking insect foot tip positions during walking; (2) simulating a bio-inspired robot with IK implementation; (3) transferring to a real robot; (4) validating robot foot tip positions. The insect foot tip positions and trajectories are recorded and analyzed frame by frame with a video tracking tool (e.g., Tracker). These trajectories are then used as the desired target positions which are converted into robot joint angles through IK for intralimb coordination. Ignasov et al. applied this framework to generate complex dung beetle-like leg movements during locomotion, dung ball manipulation, and dung ball transportation for a dung beetle robot prototype.
<figure><img src="image_11.png" /><figcaption>**Figure 8.** A variety of hexapod gaits with varying speeds generated by adaptive CPG-based neural locomotion control (modified from [142]). The frequency of the CPG outputs can be changed by modulating the synaptic connections of the CPG neurons with an extrinsic modulatory input *MI*. When the modulatory input *MI* is set to 0.0, each leg steps in a wave on each side with overlap. Stepping frequency increases as *MI* increases, and some legs step in pairs (see dashed enclosures). This results in insect-like gaits (Figure 5d) and various intermixed gaits. The caterpillar gait is characterized by the movement of two front, middle, or hind legs at the same time and the wave travels from back to front. Under this control approach, a transition gait as shown in Figure 5d is not found. For example, one can observe wave gaits with varying frequencies (*MI* = 0.01–0.04), tetrapod gaits with varying frequencies (*MI* = 0.05–0.06), caterpillar gaits with varying frequencies (*MI* = 0.07–0.10), and tripod gaits with varying frequencies (*MI* = 0.15–0.19). Legs are labeled as numbers 1–3 from front to back, and the left and right sides are L and R, respectively. It is worth noting that when *MI* is raised above 0.17, only two different gaits comparable to tripod gait (e.g., *MI* = 0.17) and caterpillar gait (e.g., *MI* = 0.10) appear.</figcaption></figure>

For interlimb coordination, this can be done by pre-defining the phase relationship between the legs for specific gaits. The basic setup will lead to certain locomotion patterns. To achieve dynamic locomotion or adaptation to uneven terrains, additional posture control mechanisms and environmental models are applied. For instance, the environmental model is employed to identify the roughness property of the terrain and obstacle information in such a way that the control system is able to plan and to adapt the foothold position, robot posture, and leg trajectory. This method requires exteroceptive feedback to perceive environmental information, such as that provided by a laser scanner sensor [205,206] and a depth camera sensor [164,207,208]. Recently, advanced engineering control methods like model predictive control and robot model abstraction-based control with planning, have been applied to achieve locomotion adaptation. For instance, Hu et al. [209] introduced a constrained model predictive controller for stabilization of non-periodic trajectories for walking, by a hexapod robot, over irregular terrain. Buchanan et al. [207] proposed a deformable bounding box abstraction of the hexapod robot model with mapping and planning strategies for hexapod locomotion control and body posture adaptation to navigate in confined spaces.

Insect-like muscle (compliant) properties can be achieved by a variable admittance/impedance control [210,211]. However, it is a challenge to simulate coordinated compliant
joint motions in adaptive insect-like locomotion due to high system redundancy [212]. These motions are achieved by integrating neural networks and muscle-like mechanisms [114,213]. To address this challenge, neuromechanical models have been developed to achieve coordinated compliant joint control on insect-like robots [214]. Szczecinski et al. proposed a novel way to tune a robot leg servomotor to exhibit insect muscle-like dynamics of equilibrium, perturbed responses, and active motions [215]. Ribak highlighted muscle compliance functions in insect and robot jumping [216]. He also provided potential insect-inspired solutions to solve small robot jump control and stability challenges. For instance, they bypass the power constraint of muscles by converting muscle work to elastic potential energy. Elastic elements act as springs storing the muscle work performed before jumping. Huerta et al. proposed an online muscle-like compliance adaptation control for robust 18-DOF insect robot walking [113].

### 3.2.3. Machine Learning-Based Control

In addition to the bio-inspired and engineering-based control approaches mentioned above, an alternative way to automatically generate robot gaits for walking on irregular terrains and dealing with complex situations (like leg damage) is to apply machine learning (Figure 6c). To date, different machine learning techniques have been actively explored for locomotion control. The techniques include (deep) reinforcement learning (RL) [165,217], imitation learning [218], intelligent trial and error [219], and evolutionary computation [166,220,221].

For instance, Hafner et al. [165] developed a reinforcement learning framework that can learn locomotion behavior for different types of legged robots including hexapods. This framework relies on a data-efficient, off-policy multi-task RL method and simple reward functions. The learning time is approximately 5 h to obtain basic locomotion skills by walking in different directions. Ting et al. [218] proposed an imitation learning method that can train a “student” hexapod to imitate the walking behavior of an “expert” hexapod by watching its leg movements. Cully et al. [219] proposed an intelligent trial-and-error algorithm that allows a hexapod robot to automatically learn a behavior map consisting of over 13,000 high-performing behaviors or gaits. The process required 20 million iterations (roughly 2 weeks) on one multi-core computer to obtain the map. The robot can later use the map to search for an appropriate locomotion behavior to compensate for unexpected damage (like damaged, broken, and missing legs). Parker [220] used a cyclic genetic algorithm (CGA) to evolve control programs for generating different gaits of fully capable and damaged hexapod robots. Cully and Mouret [221] introduced the transferability-based behavioral repertoire evolution algorithm (TBR-Evolution) that can find several hundreds of simple locomotion controllers (one controller is for one possible walking direction). The algorithm relies on novelty search with local competition (searching for high performing and diverse solutions), and the transferability approach (integrating simulation and real tests to develop the evolutionary controller of a real robot). This approach enables a hexapod robot to learn to walk in every direction with a single run of the evolutionary algorithm within 3000 iterations (2.5 h).

Several studies have used artificial neural networks with data-driven methods for learning locomotion [222]. For instance, Azayev et al. [222] proposed a scalable two-level architecture for hexapod locomotion on complex terrain by using joint angle and binary foot contact feedback and deep reinforcement learning. The control approach can enable a simulated hexapod robot to navigate over ground, stairs, or through narrow pipes, etc. For fast complex robot locomotion generation, some studies have demonstrated the use of a combination of multiple control approaches where bio-inspired control and/or engineering-based control is used to encode basic rhythmic pattern generation while machine learning is applied for optimizing locomotion control parameters to achieve complex patterns (Figure 6). For instance, Milicka et al. [223] combined a chaotic CPG with IK (Figures 6a,b). In this approach, the CPG generates desired trajectories (foot-tip positions) while IK translates the foot-tip positions into the joint angles to directly control the actuators of a hexapod robot. As a
result, the robot can perform a variety of movements, such as spot turning and walking with various gaits (e.g., tripod, ripple (a type of tetrapod), low gear (alternating between swinging two legs and one), and wave gaits). Chen et al. [224] used the same strategy combining a CPG with IK and additionally introduced force feedback to deal with irregularity in rough terrain (Figures 6a,b). While a combination of bio-inspired and engineering-based control can encode rhythmic patterns for a variety of hexapod gait generation and terrain adaptation through sensory feedback, one challenge of using such an approach is control parameter tuning or optimization. To address this issue, Fu et al. [225] proposed a combination of deep reinforcement learning with IK. The reinforcement learning method can automatically find motion planning policies for hexapod robots moving on uneven piles of plum-blossom (Figures 6b,c). Schilling et al. [217] introduced a biologically-inspired decentralized control architecture with deep reinforcement learning for adaptive locomotion in a hexapod robot. The architecture consists of six neural control modules, each of which controls one leg (Figures 6a,c). Thor et al. [188] presented a generic locomotion control approach, which combines bio-inspired CPG and RBF-based premotor neuron networks into a modular CPG-RBF neural control network (Figures 6a,c). This network uses a neural basis to produce complex rhythmic trajectories for the joints of walking robots. These trajectories are optimized using a probability-based black-box optimization (BBO) method. The framework was applied to teach and control three different simulated legged robots with varying morphologies including broken joints. Ouyang et al. [226] proposed an adaptive locomotion control approach which combines bio-inspired, machine learning, and engineering-based methods (Figures 6a-c). Specifically, they used a 3D two-layer CPG network where the first CPG layer for interlimb coordination generates basic robot locomotion patterns based on kinematics analysis. The second CPG layer is for intralimb coordination and controls the robot's limb movements to deal with environmental changes. An actor-critic reinforcement learning method with deep neural networks (DNNs) (known as Deep deterministic policy gradient (DDPG) [227]) was employed to optimize the CPG control parameters of the second layer for fast and stable hexapod locomotion with adaptability to different terrains (flat, sand paper, soft sand, and 10 degree up-slope).

## 4. From Locomotion to Cognition

In the previous sections of the manuscript, the relevance of locomotion generation and control aspects, mainly related to biomechanics (see Section 2) and neuron motor activity (see Section 3), has been reviewed. Locomotion has the objective of making the body flexible enough to face the environmental challenge of reaching specific destinations. In other words, locomotion involves physical interaction with the environment realized through the physical motion of the body towards satisfying specific needs. On the other hand, motion, in general, is not separated from mental activity. It could be stated that any movement has its origin from a mental state which, constrained by the environment, it's own possibilities, and by the internal motivational drives of the subject, designs, plans and provides the suitable commands for the execution of the corresponding action. Cognition (Figure 1a) involves non-physical (mental) interaction with the environment; once a certain mission has been defined, cognition refers to the “capability of planning ahead” i.e., of defining, from a high level, “what to do” and “how to do” in order to fulfill the mission [228]. This represents the main driver for the locomotion system, except for those actions purely driven by reflexes as also depicted in Figure 1. So, from this perspective, cognition is a brain function involved in planning body actions to reach complex goals. Centuries of research on mammals has tried to discover the details of how motion plans are conceived in the brain cortex to generate and plan limbs actions. Although this research has led to a number of important breakthroughs, especially in the neurorehabilitation field, we remain far from mastering the details of such a complex structure as the mammal brain cortex.

So, from a true engineering perspective, it will be more rewarding, at least in the near future, to work on those brain structures which are less difficult to study in detail, but which, at the same time, are able to generate complex behaviors that could boost the
capabilities of current robotic structures. Within the animal kingdom, insects are the class of living beings most present in nature, thanks to their adaptation capability to extreme environmental conditions and their brains whose structure efficiently manages to exploit all of the body's capabilities.

Only recently have insects been recognized to possess a “cognitive brain” structure [229]. In the interesting yet provocative work from Chittka [230], written in the last decade, it is argued that even simple brains possess all the ingredients necessary for most of the behaviors usually ascribed to larger and more evolved brains [231]. The idea started to arise that insects are not only reflex-based automata. In fact, even looking at simpler, non-social insects, they show, individually, such surprisingly complex behaviors as numerosity [232], attention and categorization-like processes [233], delayed matching to sample task (considered as the capability to learn the concept of sameness) [234], water maze solution [235], and other capabilities that can be defined as “proto-cognitive”. These are commonly considered as clear traits of high-level deliberative behaviors. The neural structures responsible for these capabilities, in some cases, can be investigated and reproduced in detail because they are so much simpler than those of mammals, and with a much lighter organization. To provide some examples, observational learning is among the functions likely to be important in future robotic research: insects, like wood crickets, are able to learn from the life-saving actions produced by other conspecifics when facing predators [236]. This clear example demonstrates how precious the result of this form of learning is, without it, survival would be impossible. It is well known that insects possess high level learning, called operant conditioning, which makes them able to learn on the basis of an expected outcome, under the guidance of a reward or punishment. This also means that insects can develop an expectation about the outcome of their actions, which can lead, in turn, to the planning of a specific behavior, or even a sequence of behaviors, even in the absence of the stimuli associated with the outcome. This form of expectation suggests lower and higher cognitive processes working together, and this has implications, for instance, in navigation and waggle dance communication in honeybees.

To include further key examples, bees learn a sequence of landmarks as cues for turns toward the feeder [237]. Thus, they develop expectations along a route associating specific landmarks in specific places within the sequence to the food source location. Route selection was also found to be time-dependent (e.g., morning or afternoon) [238]. Bees also perform novel shortcuts between different locations within a previously explored environment [239,240], but they also fly along shortcuts between a learned location and a location communicated by the waggle dance of a hive mate, without references to landmarks [241,242]. Observational learning even at a symbolic level is exemplified by dance communication in bees. Another important characteristic in insects is their capability to generalize, i.e., to flexibly respond when the animal is confronted with distorted versions of the learned associations, due to noise and developmental changes. Essentially this involves assessing the similarity between the presently perceived input and the previous experience. This generalization property was found in different sensory modalities such as olfaction [243], vision [244], and gustatory sense. Insects are also able to categorize, i.e., to group different objects or events based on common features [245]. Another important trait of insect cognitive capabilities, as introduced above, is “concept learning”, which relies on relations between objects, such as “same as”, “different from” [234], “above/below” and “to the left/right of” [246]. Insects are able to learn and also transfer the results of such learning also to unknown objects which may sometimes be completely different from those used for learning the relationships. More recently, bees were shown to process two concepts simultaneously and combine them in a rule for subsequent choices. This implies an even higher level of cognitive sophistication than dealing with one concept at a time.

The characteristics of functional/structural specialization of either side of the brain/body (i.e., lateralization) involve a fascinating research activity carried out in different insect species, such as locusts [247] and beetles [248]. In Romano et al. [248], a biomimetic animal replica of a larger grain borer was developed to modulate the behavioral responses in
insect. The results demonstrated that beetle pushing behavior is a complex communication
strategy where the information is mediated through the performed actions in terms of
number of acts and display duration. The desert locust is another well studied species
as demonstrated in [249] where the generation of adaptive responses, such as evasive
maneuvers was investigated in the presence of unpredictable events during a goal-oriented
behavior, followed by a reorientation and route correction. This behavior is strongly related
to selective attention in honeybees and flies where sensorial stimuli are opportunely gated
by the nervous system depending on their relevance [250]. Social learning is another
relevant characteristic ascribed to the cognitive repertoire of insect species such as ants,
bees and wasps. In [251], the capability of social wasps to learn the facial features of all
colony members was demonstrated with the aim of identifying the colony's hierarchy as it
relates to fighting skills. Some forms of social learning, commonly considered as high-level
cognitive processes, are also reported in solitary insects species, such as crickets [252] or
fruit flies [253] and recently have been shown also using robotic trainers [254]. This further
outlines the added value in inspecting such forms of complex interaction capabilities in
simple brains, from the engineering perspective aiming at building an insect brain model.
Unfortunately, although these astonishing capabilities have been found in such tiny brains
as those of insects, so far, the vast majority of them has not yet been specifically linked to
the exact brain area which gives rise to such behaviors. For this reason, in the following
section, we will refer primarily to those behaviors explicitly referring to the identified brain
areas. This is mostly found in the case of *Drosophila melanogaster*.

4.1. *The Fly Brain and Cognition*

Among insects, *Drosophila melanogaster*, also known as the fruit fly, is considered as a
primary model organism, widely studied to understand the details of the emergence of
specific behavioral phenomena and draw rules that can be extended to higher organisms.
The fruit fly is a perfect candidate thanks to the low number of neurons present in the
central brain (10⁵) as compared to other well-studied animals like the brown rat (2 × 10⁸)
and human (8.6 × 10¹⁰). Another important aspect is the possibility of creating mutants
using genetic tools based on the GAL4-UAS technique [255]: the acquired knowledge of
the *Drosophila* genome allows the manipulation of the nervous system of a fly to study
the structural and functional relationships between its neural structures and the different
behavioral capabilities.

To focus our work on the architecture where a clear picture of neural circuits, behavior
functionalities and experiments (e.g., either in simulation or using actual robotic systems)
was available, in the following section we will mainly deal with the *Drosophila* brain and
concentrate our attention on its behaviors related to walking.

Intact adult *Drosophila* are naturally able to fly, and do not demonstrate relevant
walking capabilities; but if their wings are clipped, a lot of experimental results have
testified to their capability of demonstrating adaptive locomotion skills. These have to be
learned from scratch, demonstrating the impressive flexibility of such a tiny brain to adapt
to a novel ecological niche.

In Section 4.2, we will introduce the insect brain architecture describing the neu-
roanatomical structure of two relevant neuropiles in the central brain. In Section 4.3, we
will introduce the state-of-the-art on biological experiments, functional models, and robotic
applications in relation to the neural structures involved. In Section 5.3, we will present
some future directions in insect-inspired robotic cognition.

4.2. *The Insect Brain Structure*

The *Drosophila* central brain includes two important neuropiles: the mushroom bodies (MBs) mainly devoted to the adaptive termination of behaviors and responsible for olfactory learning and multimodal integration, and the central complex (CX) responsible for the initiation of behaviors and mainly involved in processing visual stimuli.

MBs are a paired structure present with similar forms in all insect species.
The intrinsic neurons of the MBs are called Kenyon cells (there are about 2000 KCs per hemisphere in the fruit fly) which receive predominantly olfactory input in the calyx region. From there they project through the peduncle into different lobes (i.e., α -/β- lobes, α' -/β'-lobes and γ-lobe). The first identified function of the MBs corresponded to olfactory learning and memory. Olfactory receptors mainly reside within the antennae where olfactory receptor neurons (ORN) transfer the sensory information to the MBs to build associative memories for odors.

Odor perception and processing are fundamental because flies identify food sources and select sex partners through this sensory system. Although input from other sensory modalities is anatomically not evident in *Drosophila* MBs, there are several experiments showing the role of this center in tasks related to vision [256]. The bee brain, in turn, shows the presence of gustatory, and mechanosensory inputs in the MBs [257]. The MBs can therefore be ascribed as a multisensory integration system where mixed-modality signals are collected together with negative and positive reinforcement information conveyed by dopaminergic and octopaminergic neurons. The use of genetic tools has enabled the identification of other relevant functionalities associated with the MBs: it has been shown that mushroom-body-less flies have problems switching to a new type of behavior in the presence of changes in the environment; MBs are also devoted to sleep control and they form the basis of motor learning [258].

The CX neuropile resides in the middle of the MBs-paired structure and, in *Drosophila*, can be divided into the following substructures: the protocerebral bridge (PB), the fan-shaped body (FB), and the ellipsoid body (EB) (Figure 9) [259].

The PB is responsible for step-size control, selecting the proper speed, and direction of motion as demonstrated in walking and gap-climbing scenarios. Experiments performed using a flight simulator revealed that the FB in *Drosophila* is responsible for visual memory functions, extracting visual features that can subsequently be associated with specific behaviors. We can summarize the role of PB and FB as the centers responsible for answering the questions: “where?” and “what?”. Finally, the EB plays a relevant role in orientation memory and place learning. Its ring structure is able to identify the angular position of the object of interest and participate in the path integration process.

Even if, in *Drosophila*, there is no explicit involvement of the MBs in such tasks as visual learning, in higher complex structures, like ants and bees, MBs are definitely multimodal structures, hosting both olfactory and preeminent visual stimuli [260].

A block-size model of the different neural centers and their interaction with the sensing and actuation areas is reported in Figure 9. The sensory modalities taken into consideration are vision, olfaction and touch. The role of the MBs and CX is indicated in terms of processing the different sensory stimuli and enlightening the interactions among the relevant neuropiles we have considered. The motor command circuits are placed within the thoracic ganglia where local reflexes are also implemented. Rewarding and punishing signals mediated by octopaminergic and dopaminergic neurons are used as a driver for the learning processes.

4.3. *Insect Brain Functional Models, Implementations and Robotic Experiments*

From the analysis briefly outlined above, we can deduce that each of the two main
centers, MBs and CX, have individual functional roles, but there are complex tasks that
require the concurrent enrollment of both. In this subsection, these main functionalities
will be reviewed and shown via both computational models and experimental robotic
architectures. A summary of the state-of-the-art in the field is depicted in Table 3.
<figure><img src="image_12.png" /><figcaption>**Figure 9.** Block-size model of the relevant sensory modalities and neural processing centers in a *Drosophila* central brain. The insect's compound eyes photo-receptors (PR) acquire visual stimuli that are initially processed in the optical lobes, and transferred to the CX, and then the neuropile responsible for visual orientation. Chemical-receptors (CR) located in the antennae are responsive to olfactory stimuli whose neural response is transferred through the antennal lobes (AL), and subsequently to the projection neurons (PN), and finally to the MBs. The olfactory and visual inputs are here integrated with the other sensory modalities although their connecting paths are still not evident. The lateral horn (LH) is an inhibitory center that is activated by the PN and the ventrolateral protocerebrum (vlpr) to affect the MBs activity. Dopaminergic (DAN) and octopaminergic neurons (OAN) provide reinforcement learning signals used by learning and memory systems. Tactile stimuli, acquired by the mechanoreceptors (MR) located in the antennae, legs and halters, are locally processed in the antennal mechanosensory and motor center (AMMC) and in the thoracic ganglia for the generation of local reflexes where the CX is also concerned.</figcaption></figure>

<table><caption>**Table 3.** State-of-the-art of the different insect functionalities and behaviors ascribed to the MBs and CX. The subscript associated with each reference indicates the content of the corresponding work: analysis of insect neural circuits and associated behavioral experiments (E), modeling of neural circuits and simulations (S), neural models applied in robotic experiments (R).</caption><thead><tr><th>Insect Brain Areas</th><th>Functionalities</th><th>References</th></tr></thead><tbody><tr><td rowspan="7">Mushroom bodies</td><td>Olfactory learning</td><td>[261]<sub>S</sub>, [262]<sub>E</sub></td></tr><tr><td>Attention</td><td>[263]<sub>E</sub>, [264]<sub>S</sub></td></tr><tr><td>Expectation</td><td>[265]<sub>ES</sub>, [266]<sub>S</sub></td></tr><tr><td>Sameness</td><td>[264]<sub>E</sub>, [267]<sub>S</sub></td></tr><tr><td>Sequence learning</td><td>[268]<sub>SR</sub> [269]<sub>SR</sub></td></tr><tr><td>Navigation and visual sequential memory</td><td>[260]<sub>SR</sub></td></tr><tr><td>Classification and decision making</td><td>[270]<sub>S</sub>, [271]<sub>S</sub></td></tr><tr><td rowspan="4">Central complex</td><td>Navigation and detour paradigm</td><td>[272]<sub>E</sub>, [273]<sub>S</sub>, [274]<sub>S</sub></td></tr><tr><td>Goal-directed navigation</td><td>[275]<sub>E</sub> [276]<sub>S</sub></td></tr><tr><td>Spatial working memory in a water maze scenario</td><td>[235]<sub>E</sub></td></tr><tr><td>Body-size model</td><td>[277]<sub>E</sub>, [35]<sub>S</sub>, [278]<sub>SR</sub>, [279]<sub>S</sub></td></tr><tr><td rowspan="3">Mixed</td><td>Navigation and landmark targeting behavior</td><td>[280]<sub>S</sub>, [192]<sub>S</sub></td></tr><tr><td>Motor-skill learning</td><td>[231]<sub>E</sub>, [281]<sub>E</sub>, [282]<sub>S</sub></td></tr><tr><td>MBs and CX contribution to aversive visual learning</td><td>[283]<sub>E</sub></td></tr></tbody></table>
### 4.3.1. MB Models

Insects, with particular reference to bees and ants, are famous for homing behaviors: they are able to memorize a visual route that allows them to come back to the nest, even after navigation covering several hundred meters from home. In the study [260], the authors propose that this ability can arise from the interconnections among KCs in MBs, which could be in charge of encoding a spatiotemporal memory of visual motion while moving along a spatial route. Here a fairly reproducible computational model, demonstrating such behavior, is presented in detail. The model internally builds causal correlations among subsequent visual events, thus creating a short sequence learning structure. The model represents a simplified ant MB architecture employing a modified leaky integrate-and-fire neuron model and the Spike-timing-dependent plasticity (STDP) learning rule [284]. The neural response shown by the model matches the observed behavior in ant experiments. The model was finally demonstrated in action, performing indoor navigation control in the dual drive wheeled robot named TurtleBot Burger3 equipped with an event-based camera. The model presented above has a lot of similarities to the one developed some years before in [268,269], exploiting the deeper details known in the fly brain. In fact, the discovery of axo-axonal connections within the MB peduncle allowed the modeling of the phenomenon of sequence learning. It is a complex spatial temporal correlation task, where waves of consecutive stimuli, consolidated via reinforcement signals, led to the formation of complex sequences and sub-sequences of abstract stimuli. Different types of neuron structures and learning schemes were adopted in the overall architecture: Izhikevich's spiking neurons as well as Morris-Lecar models, or STDP and Hebbian-like learning for the different layers. The model was successfully tested on a differential drive robot trained in a reinforcement environment. One added value of the developed structure is the capability of internal simulation of the correlations learned during the experiments, which can explain the phenomenon of overnight memory consolidation. In fact, the network is able to reproduce the visual stimuli, simply as an effect of internal noise. These stimuli can create the internal dynamics whose effect is to reinforce the previously learned sequences. Part of this structure was subsequently found to act as a suitable neuromorphic model for classification and decision control [270]. Starting from the experimental evidence as well as from the known architecture of the fly brain, some additional behaviors were hypothesized to be hosted, which were already found in more complex insects like bees or ants. In particular, the role of extrinsic neurons connecting the lobe systems to the antennal lobe, paired with Hebbian-like learning within the antennal lobe system, was computationally found to be responsible for the elicitation of the attentional loop [264]. The model was tested in an experimental arena involving a robot that was able to maintain a targeting behavior while filtering out other distractors. As it developed, this model became able to learn what would be considered as an expectation, i.e., the capability to predict the appearance of a given (expected) input stimulus, if preceded by another (triggering) one [266].

Another relevant phenomenon that was found in flies is related to decision making when facing visual dilemmas: once the fly learns that a given association color-shape is rewarding, when faced with a dilemma caused by altering the color-shape arrangement of rewarding and punishing visual stimuli, the intact insect has a clear preference for the color. Once the color saturation goes below a certain threshold, flies suddenly decide to choose the shape learned as rewarding. This is a clear trait in decision making, which is experimentally ascribed to the MBs. A possible computational model was introduced in [271].

The MB structure, as in the case of attention [264], was also found able to host high-level behavior learning such as the so-called "delayed-match-to-sample task", considered as similar to the acquisition of the concept of sameness and difference [267].
### 4.3.2. CX Models

The CX of an insect participates in processing the temporal and spatial components of sensory cues, and utilizes those cues in creating an internal representation of orientation and context, while also directing motor control. In addition to the fruit fly, the role of the CX in controlling goal-directed navigation has been assessed in other insect species, such as cockroaches [275], finding a lot of similarities with navigation in higher animals (e.g., rats). In the study [275], an overview of current knowledge on the CX's role in insect navigation is performed. A deep study revealed that the apparent random path ending in the finding of a dark shelter was indeed a targeting behavior toward the shelter. To prove this, in the study [276], a computational model (called RAMBLER) was built after a series of experimental campaigns. It efficiently models the insect's motions, combining the classic wall following behavior with the visual verification of the dark shelter and the probability of adding other motions, such as turning back, whenever the shelter appeared behind the insect. Although the model was obtained exclusively by using insect experiments, several factors imply a direct responsibility of the CX in such tasks. For example, CX lesioned roaches often make wrong turns, and also speed changes while walking are preceded by an increased firing rate in CX neurons.

One of the main CX roles is to host visual and mechanosensory stimuli to learn such characteristics as body size, i.e., the awareness of the peripersonal space. Genetically identical flies, if fed abundantly, can grow up to 20% larger than normally fed ones. If differently sized flies are about to engage in a fight, the smaller fly gives up without fighting. Body size knowledge was discovered to have been learned during larval stages, if flies are grown in well-lit environments. They learn the association step-stride versus distance walked, creating a conceptual function describing "how big they are". This function was found to be resident in a sub part of the CX, known as the protocerebral bridge. A computational model of this behavior was designed and implemented in a simulated hexapod robot [35], and finally tested using a humanoid robotic platform [278]. This proves that computationally relevant models, drawn from insect structure, can also be adopted for controlling different robotic bodies, not necessarily related to an insect-like biomimetic structure.

To try to shed light on how the CX exploits sensory inputs to realize motor functions associated with spatial navigation, in [285], a neural model was developed, based on known connectivity within the CX with particular attention to the Ellipsoid Body (EB). The model was based on continuous-time differential equations called leaky integrators, used to simulate the mean-field activity of pools of neurons, and used the Monte Carlo method for parameter estimation. The model was successfully tested by means of a kinematic simulator. The same leaky integrator model was employed to generate robust ring attractor dynamics [274], i.e., neuron structures, which responded to the position of external stimuli and retained activity even in the absence of external stimuli. In that work the authors hypothesized that the emergence of ring attractor activity in the EB is due to reciprocal neuronal connections between EB and PB. Ring attractor networks were also modeled using spiking neuron structures. These were found to be able to show spatial memory formation exploiting the ring architecture of the EB [273].

### 4.3.3. Models Involving MB-CX Interaction

Some behaviors require, at least at the functional level, the participation of both MBs and the CX. This is the case, for example, in navigation tasks that are efficiently performed by insects using visual sensory feedback. Insects typically show innate curiosity for landmarks, especially moving ones, towards which they are attracted. Such landmarks can be used to elicit, after learning, specific directional steering maneuvers, which can subsequently be understood to be part of a complex navigation task. While the neuronal details of the concurrent existence of these two aspects of visual learning are not yet known, a potential computational model was proposed in [280] based on the neuroanatomy of the wood ant CX, typically involved in landmark targeting behavior. Here, a reward signal,
supposed to be provided either by the innate CX landmark attraction feature or by a long-
term visual memory, built into the ant MBs [260], was used to form a local vector memory
in the CX. The strategy was implemented using a simulated robot in a simple environment
endowed with a single visual cue. The architecture matches the experimental results
coming from unilateral MB lesions in ants, which restore the innate targeting behavior.

In the study [192], to address insect navigation in a cluttered environment, the authors speculated that specific strategies are needed to retrace familiar routes and return home. These would need both the CX and MBs working together: MBs check whether the current sensory stimulus is positive, while the CX steers the animal's heading towards the rewarding stimulus. The model was found to reproduce behavioral data in realistic environments.

Another experiment, called detour, was performed to demonstrate the presence of short-term visual memory in the fruit fly. The experiment verified that the fruit fly can show short term redirection towards invisible targets after the introduction of a second landmark, called a distractor [272]. This fact reinforces the idea that very similar behaviors, typically found in higher animals, can be shown in simpler brains, which, from the engineering perspective, can be more efficiently modeled.

As stated above, the CX is involved in spatial learning from visual cues as demonstrated in *Drosophila* [235]. Of course, larger and better CX structures in bees were ascribed possible roles in the spatial learning of color cues. Moreover, in the study [283], using a combination of color learning with electric shock as punishment, an interplay between CX and MBs was found by pharmacologically silencing these two centers. In particular, the CX was found to be necessary for mediating the goal-directed behavioral response to learned stimuli, while the MBs carried out the actual cue association.

As discussed above, wing-clipped flies have to learn from scratch how to efficiently interact with the environment. Therefore, they must adapt their locomotion capabilities to more complex situations. A joint study with neurogeneticians, supported by experimental evidence, led to the conclusion that the fly, responding to a difficult climbing scenario, started by making some attempts, and the most rewarding ones are retained and adopted to build the right sequence of leg motions for fulfilling the climbing action. A corresponding model, involving the concurrent activation of MBs and CX, was also developed [282].

**5. Lessons Learned from This Review**

5.1. *Future Directions in Biomechanics*

Emerging biomimetic leg design will be increasingly based on available micro-CT scans of real insects in order to improve their level of complexity, fidelity, and bio-inspiration in the 2020s. So far, none have been built with a structure similar to an ant exoskeleton. This kind of structure gives ants the ability to load and transport up to 15 times their own weight.

Many aspects of force sensing in insects could be applied to hexapod robotics, and these
will be particularly helpful in redesigning legs which incorporate force sensing. Moreover,
adding some compliancy to the leg structure at strategic locations could provide force
sensing, and significantly reduce the relative mass of legs in relation to the robot's overall
weight. In insects, force measurement is done at the level of the *campaniform sensilla* [286].
The *campaniform sensilla* are mechanoreceptors providing local control of legs; some current
mechanical solutions mimic them by adding a slider and a spring per joint, but at the
expense of compactness and increased mass.

5.2. *Future Directions in Locomotion Control*

The present review on locomotion control shows that significant efforts have been
made to understand the principles of insect locomotion control and translate those prin-
ciples to robot locomotion control using different technical control methods such as bio-
inspired, engineering-based, machine learning-based, as well as these methods in varying
combinations. However, current robot performance is still far from that of insects. This
is due to generated complex behavior that results from emerging processes derived from
biomechanics (see Section 2), locomotion control (covered in Section 3), and high-level cognitive control (see Section 4) as well as their dynamical interactions (Figure 1). Thus, focusing on particular aspects can only achieve partial solutions. From this point of view, future research directions might endeavor to integrate all these components and exploit their dynamical interactions. Such an integration can be viewed as a complete embodied closed-loop system [287]. This will also shed light on a key research question: *how do low-level motor/locomotion control and high-level cognitive control mechanisms (brain), a body (sensory-motor system), and an environment dynamically interact in real time to generate motion intelligence?*

While insects can intelligently exploit all their body parts (i.e., legs, thorax, and abdomen) for stable and versatile locomotion and object manipulation, most insect-like robots use only their legs for locomotion and object manipulation. During walking and climbing, a cockroach tries to keep its abdomen close to the ground (i.e., low center of mass) [288]. Ants can also exhibit a kind of "clutch" behavior with their legs and bodies, to try to avoid being blown away by a gust of wind [289]. During dung ball rolling, a dung beetle uses a part of its abdomen as another contact point to maintain stability [144]. From this point of view, such strategies, which can be considered as whole body locomotion and object manipulation, should also be realized for performance enhancement in insect-like robots [142,290]. As legged robots have been actively integrated into people's daily production activities and various service tasks (such as exploration, object transportation, inspection and maintenance, search and rescue, and construction), fast lifelong continuous adaptation for resilient and robust robot locomotion to deal with environments that change unexpectedly, and physical damage over the course of a mission, is becoming increasingly important.

### 5.3. *Future Directions in Insect-Inspired Robotic Cognition*

From the review just presented, it clearly emerges that, over the last decade a huge effort in the design of powerful targeted experimental setups has been able to describe interesting behaviors in insects. These ranged from the simple associative learning skills to surprisingly relevant behaviors that can definitely be considered proto-cognitive. Despite the relevance of these discoveries, engineers are still concentrating their efforts on the design and implementation of single experiment tailored models, both at the computational level and, sometimes at the robotic level. Although these steps are needed to address the necessity and sufficiency of specific neural assemblies' roles in the generation of corresponding behaviors, both at the insect experimental level and on the robotic side, the modeling strategy is stuck at a block sized level. As also reported in [291], the insect brain is a prime site for the detailed study of the link between neuroanatomical geometries, the corresponding function and the final behavioral outcomes, all the way from a wiring diagram to the level of insect intelligence. One of the main open research questions regards the possibility of studying the insect brain no longer as a block sized architecture, but viewing it from a holistic perspective, i.e., looking at the "whole", instead of considering simply a part of its composition. The pursuit of such a research line should result in greater benefits across a variety of fields such as neuroscience, bio-medicine, artificial intelligence, and robotics. Nowadays, a whole insect brain model is only partially achievable due to the limited knowledge both at the level of experiments on single behaviors and on the study of the brain as a whole. Indeed experimental data covers only a small part of the huge dynamical neural activity responsible for a plethora of concurrent sensory motor activities. The authors could justifiably claim that the constraints just cited, and the still inadequate computational resources constitute the current limit, but these are likely to be overcome by creating biologically realistic simulations of the entire insect nervous systems and this should be possible within the next 20 years [291]. Moreover, the authors believe that a possible "Rosetta Stone" would be to consider the role of emergence in such tiny brains. Huge dimensional nonlinear dynamics can emerge simply by wiring non trained neural assemblies together. This would be achieved through careful neural wiring developed by extracting the connecting rules from the insect connectome. In addition, trainable read-out
maps could be used to extract the specific sub-dynamics from this neural lattice to create ad-hoc associations with different behavioral needs. This could be realized concurrently. This new line of research, drawn from psychology, is known as "neural reuse": the same neural assembly can be exploited concurrently for different parallel tasks. Also, as outlined in [292], the smaller the brain, the larger the need for neural reuse. Insect brains are of a suitable size to try to investigate neural reuse in action, since in these small brains, the relatively low number of neurons and mainly short-distance connections are candidate elements for neural reuse, even if the circuits are composed of different brain areas. The authors believe that the neural reuse paradigm will be the best route to take to explain all the rich functionalities of insect brains, and could open the way to describing even more complex combinations of non trivial behaviors that characterize the efficiency of more sophisticated brain structures. In any case, even the replication of an insect brain on an insectoid body would provide an added value for attaining true autonomy in future advanced robotic structures.

Active research on insect cognitive capabilities could also be relevant to better understand the evolution of cognition in relation to the well-studied vertebrate field [293]. The next steps will include the definition of behavioral tests to measure cognition and the identification of the relevant factors that contribute to the evolution of cognition.

The authors are convinced that a complete understanding of the insect brain will give a considerable boost to both biological and robotic research, leading to the introduction of a new generation of adaptive, resilient, robust, and efficient bio-inspired machines.

## 6. Conclusions

Over the last ten years, the booming use of 3D printing has significantly boosted the development of brand new mechatronic designs for hexapod robots. So, what can we expect from the next 10 years in hexapod robot design?

New emerging printable materials combined with multi-material 3D printers will make it easy to design and to reproduce insect-based legs by using micro-CT scans of real insects in order to reach their level of complexity. This will, not only lead to a better understanding of how insects walk, but also to finding innovative leg structures incorporating smart force sensing by adding soft materials. Force sensing is a critical aspect in locomotion control for complex terrain prediction and adaptation, adding both compliance segments and actuators to the leg structure at strategic locations is the major issue for the next decade. In insects, force measurement is done at the level of the *campaniform sensilla* on their exoskeleton close to their joints, and not at the tip of their leg. Such an approach in force sensing could significantly improve the level of compactness of future hexapod robots. By means of such bio-inspired approaches to insect morphology coupled with 3D printing, leg structure based on exoskeleton design will be easy to prototype, to clone for academics, and to repair. Such a bio-inspired exoskeleton could permit a significant reduction in the legs' weight, and therefore enhance the capacity of load transportation in hexapod robots.

Locomotion control is also an important element in achieving intelligent behaviors (e.g., adaptive locomotion, load/object transportation) in hexapod robots. It is a basic mechanism to realize interlimb and intralimb coordination as well as joint compliance. Biological findings reveal that there are at least three components (CPGs, sensory feedback, and forward models) supporting the control. Thus, these components should be taken into account when developing locomotion control for hexapod robots. CPGs basically act as oscillators that produce basic rhythmic patterns for joint movements and gait generation without sensory feedback (open-loop control). Sensory feedback, however, is important for the frequency, shape, and phase adaptations of CPGs which result in adaptive locomotion (closed-loop control). Furthermore, it also drives reflexes to allow for walking on difficult terrains. Forward models complement the control in terms of sensory prediction or state estimation for highly adaptability. While hexapod robotics has been mainly focused on locomotion control, future research should be moved towards controlling a variety of robot
functions, approaching insect-like abilities such as object manipulation and transportation. This can be accomplished by considering how to make use of the entire robot body (i.e., legs and trunk) as insects do. Machine learning methods can also be used as part of the locomotion control method to create resilient and robust robot functions that can adapt to changing environments. Additionally, these methods, together with hexapod robots, can be used to decode insect movement intelligence, e.g., muscle multifunctionality in adaptive locomotion and object transportation.

As stated in the introduction, the details of the neural mechanisms in charge of each brain function, and how these neural mechanisms are coordinated for the implementation of complex tasks remain largely unknown in many cases. For this reason, a large number of operative hypotheses often need to be formulated to allow the design of efficient computational models. This task would definitely be facilitated by close collaboration between neurogenetics, computational neuroscience and robotics. The first, thanks to modern tools, is able to create mutants by targeting specific neural areas, addressing the behavioral role of specific neurons. The second would in turn design and build efficient computational, neurally grounded models, and the last would then implement these models in working robotic structures. Therefore, it is necessary to foster collaboration between these different scientific fields to extract from biology the necessary information to be translated into efficient deployable models and structures. At the other end, robotics offers the unique opportunity to see the computational insect brain models in action in mechatronic structures. This can generate new ideas for insect experiments. Moreover, experience suggests that the lack of suitable experimental setups has left a lot of insect behaviors undiscovered. In this sense too, a close collaboration with engineers would be invaluable.

Insect brains are much simpler than vertebrate brains, and so can be accurately modeled and reproduced. An insect brain could deliver, to an artificial body, capabilities of adaptation and autonomy, which would be called "insectoid": a robotic structure endowed with an insect-inspired brain.

The cognitive architecture modeling the insect brain has been proven to be independent of the particular robotic structure, although it co-evolves with the robot body by adapting to the behavioral outcome compatible with that specific structure.

**Author Contributions:** All authors wrote Section 1—"Introduction" and Section 6—"Conclusions". J.R.S., I.B., J.D., and S.V. wrote Section 2—"Biomechanics in hexapod robotics". P.M. and X.X. wrote Section 3—"From biomechanics to locomotion". L.P. and P.A. wrote Section 4—"From locomotion to cognition". All authors wrote the first draft of the paper, prepared, and then revised the manuscript. All authors have read and agreed to the published version of the manuscript

**Funding:** This work was funded with the help of the Centre National de la Recherche Scientifique (CNRS) in the framework of the MiMiC-ANT project subsidized by the PEPS INSIS 2020 "Ingénierie inspirée par la nature" program. P.M. acknowledges support from the Human Frontier Science Program under grant agreement no. RGP0002/2017, from Vidyasirimedhi Institute of Science and Technology (VISTEC) research funding on Bio-inspired Robotics. X.X. acknowledges support from the Brødrene Hartmanns and Thomas B. Thriges under grant agreements A36775 and 7648-2106.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Not applicable.

**Acknowledgments:** We thank Vincent Fourcassié and Prof. Pierre Moretto from CRCA UMR5163 (CNRS/University of Toulouse 3), France, for their fruitful discussions concerning biological inputs in ants. The authors would like to thank David Wood (http://www.eays.eu/) for revising the English of the manuscript. We are grateful to the two anonymous editors, whose suggestions helped us improve the manuscript.

**Conflicts of Interest:** The authors declare no conflict of interest.
## Abbreviations

The following abbreviations are used in this manuscript:

<table><tr><td>AEP</td><td>anterior extreme position</td></tr><tr><td>AFO</td><td>adaptive frequency oscillator</td></tr><tr><td>AFDC</td><td>adaptation through fast dynamical coupling</td></tr><tr><td>AHM</td><td>artificial hormone mechanism</td></tr><tr><td>CoT</td><td>cost of transport</td></tr><tr><td>CPG</td><td>central pattern generator</td></tr><tr><td>CX</td><td>central complex</td></tr><tr><td>DOF</td><td>degree of freedom</td></tr><tr><td>EB</td><td>ellipsoid body</td></tr><tr><td>FB</td><td>fan-shaped body</td></tr><tr><td>GRF</td><td>ground reaction force</td></tr><tr><td>KCs</td><td>Kenyon cells</td></tr><tr><td>IK</td><td>inverse kinematics</td></tr><tr><td>MBs</td><td>mushroom bodies</td></tr><tr><td>PB</td><td>protocerebral bridge</td></tr><tr><td>PEP</td><td>posterior extreme position</td></tr><tr><td>PM</td><td>phase modulation</td></tr><tr><td>PR</td><td>phase resetting</td></tr><tr><td>STDP</td><td>spike-timing-dependent plasticity</td></tr></table>

## References

1. Ijspeert, A.J. Biorobotics: Using robots to emulate and investigate agile locomotion. *Science* **2014**, *346*, 196–203.

2. Aguilar, J.; Zhang, T.; Qian, F.; Kingsbury, M.; McInroe, B.; Mazouchova, N.; Li, C.; Maladen, R.; Gong, C.; Travers, M. A review on locomotion robophysics: The study of movement at the intersection of robotics, soft matter and dynamical systems. *Rep. Prog. Phys.* **2016**, *79*, 110001.

3. Dupeyroux, J.; Serres, J.R.; Viollet, S. AntBot: A six-legged walking robot able to home like desert ants in outdoor environments. *Sci. Robot.* **2019**, *4*, doi:10.1126/scirobotics.aau0307.

4. Leung, B.; Thor, M.; Manoonpong, P. Modular neural control for bio-inspired walking and ball rolling of a dung beetle-like robot. In *ALIFE 2018: The 2018 Conference on Artificial Life*; MIT Press: Cambridge, MA, USA, 2018; pp. 335–342.

5. Xiong, X.; Manoonpong, P. No Need for Landmarks: An Embodied Neural Controller for Robust Insect-like Navigation Behaviors. *IEEE Trans. Cybern.* **2021**, *1–12*, doi:10.1109/TCYB.2021.3091127.

6. Beer, R.D.; Ritzmann, R.E.; McKenna, T.M. *Biological Neural Networks in Invertebrate Neuroethology and Robotics*; Academic Press: Cambridge, MA, USA, 1993.

7. Mocci, U.; Petternella, M.; Salinari, S. *Experiments with Six Legged Walking Machines with Fixed Gait*; Technical Report, Report 2–12; Institute of Automation, University of Rome: Rome, Italy, 1972.

8. Schneider, A.Y.; Gurfinkel, E.; Kanaev, E.; Ostapchuk, V. *A System for Controlling the Extremities of an Artificial Walking Apparatus*; Technical Report, Report No. 5, General and Molecular Physics Series; Physio-Technical Institute: Moscow, Russian, 1974.

9. Gurfinkel, V.; Gurfinkel, E.; Shneider, A.; Devjanin, E.; Lensky, A.; Shtilman, L. Walking robot with supervisory control. *Mech. Mach. Theory* **1981**, *16*, 31–36.

10. McGhee, R.B.; Iswandhi, G.I. Adaptive locomotion of a multilegged robot over rough terrain. *IEEE Trans. Syst. Man, Cybern.* **1979**, *9*, 176–182.

11. Kessis, J.; Penne, J.; Rambant, J. Six legged walking robot has brains in its legs. *Sens. Rev.* **1982**, *2*, 30–32, doi:10.1108/eb007564.

12. Carbone, G.; Ceccarelli, M. Legged robotic systems. In *Cutting Edge Robotics*; IntechOpen: London, UK, 2005.

13. Tedeschi, F.; Carbone, G. Design issues for hexapod walking robots. *Robotics* **2014**, *3*, 181–206.

14. Čížek, P.; Zoula, M.; Faigl, J. Design, Construction, and Rough-Terrain Locomotion Control of Novel Hexapod Walking Robot With Four Degrees of Freedom Per Leg. *IEEE Access* **2021**, *9*, 17866–17881.

15. Thor, M. MORF—Modular Robot Framework. Master’s Thesis, Maersk Mc-Kinney Moller Inst., Univ. Southern Denmark, Odense, Denmark, 2019.

16. Thor, M.; Manoonpong, P. A fast online frequency adaptation mechanism for CPG-based robot motion control. *IEEE Robot. Autom. Lett.* **2019**, *4*, 3324–3331.

17. HEBI Robotics. Daisy Hexapod Robot Kit (18-DoF), Ref. A-2049-01. 2019. Available online: https://www.hebirobotics.com/; https://docs.hebi.us/resources/kits/outOfTheBox/A-2049-01，请Manual.pdf (accessed on 19 November 2019).

18. Goldsmith, C.; Szczecinski, N.; Quinn, R. Drosophibot: A fruit fly inspired bio-robot. In *Conference on Biomimetic and Biohybrid Systems*; Springer International Publishing: Cham, Switzerland, 2019; pp. 146–157.

19. Goldsmith, C.A.; Szczecinski, N.S.; Quinn, R.D. Neurodynamic modeling of the fruit fly Drosophila melanogaster. *Bioinspir. Biomim.* **2020**, *15*, 065003, doi:10.1088/1748-3190/ab9e52.
20. Dupeyroux, J.; Viollet, S.; Serres, J.R. An ant-inspired celestial compass applied to autonomous outdoor robot navigation. *Robot. Auton. Syst.* **2019**, *117*, 40–56.

21. Cheah, W.; Khalili, H.H.; Arvin, F.; Green, P.; Watson, S.; Lennox, B. Advanced motions for hexapods. *Int. J. Adv. Robot. Syst.* **2019**, *16*, 1729881419841537.

22. Zhong, B.; Zhang, S.; Xu, M.; Zhou, Y.; Fang, T.; Li, W. On a CPG-based hexapod robot: AmphiHex-II with variable stiffness legs. *IEEE/ASME Trans. Mechatron.* **2018**, *23*, 542–551.

23. Silva, O.A.; Sigel, P.; Eaton, W.; Osorio, C.; Valdivia, E.; Frois, N.; Vera, F. CRABOT: A Six-Legged Platform for Environmental Exploration and Object Manipulation. In Proceedings of the CRoNe2018: 4th Congress on Robotics and Neuroscience, Valparaíso, Chile, 8–10 November 2018; pp. 46–51.

24. Dupeyroux, J.; Passault, G.; Ruffier, F.; Viollet, S.; Serres, J. Hexabot: A Small 3D-Printed Six-Legged Walking Robot Designed for Desert Ant-Like Navigation Tasks. In Proceedings of the 20th IFAC Word Congress 2017, Toulouse, France, 9–14 July 2017; Volume 2017; pp. 1628–1631.

25. Trossen Robotics. PhantomX AX Metal Hexapod MK-III Kit, Ref. KIT-PXC-HEX-MK3-AX12. 2020. Available online: https://www.trossenrobotics.com/phantomx-ax-hexapod.aspx (accessed on 3 June 2021).

26. Bjelonic, M.; Kottege, N.; Beckerle, P. Proprioceptive control of an over-actuated hexapod robot in unstructured terrain. In Proceedings of the 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea, 9–14 October 2016; pp. 2042–2049.

27. Halvorsen, K. MX Phoenix, Zenta Robotic Creations. 2016. Available online: http://zentasrobots.com/ (accessed on 10 June 2021).

28. Barron-Zambrano, J.H.; Torres-Huitzil, C.; Girau, B. Perception-driven adaptive CPG-based locomotion for hexapod robots. *Neurocomputing* **2015**, *170*, 63–78.

29. Palankar, M.; Palmer, L. A force threshold-based position controller for legged locomotion. *Auton. Robot.* **2015**, *38*, 301–316.

30. Szczecinski, N.S.; Chrzanowski, D.M.; Cofer, D.W.; Moore, D.R.; Terrasi, A.S.; Martin, J.P.; Ritzmann, R.E.; Quinn, R.D. MantisBot: A platform for investigating mantis behavior via real-time neural control. In *Conference on Biomimetic and Biohybrid Systems*; Springer International Publishing: Cham, Switzerland, 2015; pp. 175–186.

31. Szczecinski, N.S.; Getsy, A.P.; Martin, J.P.; Ritzmann, R.E.; Quinn, R.D. Mantisbot is a robotic model of visually guided motion in the praying mantis. *Arthropod Struct. Dev.* **2017**, *46*, 736–751.

32. Kalouche, S.; Rollinson, D.; Choset, H. Modularity for maximum mobility and manipulation: Control of a reconfigurable legged robot with series-elastic actuators. In Proceedings of the 2015 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), West Lafayette, IN, USA, 18–20 October 2015; pp. 1–8.

33. Festo. BionicANT. 2015. Available online: https://www.festo.com/group/en/cms/10157.htm (accessed on 11 June 2021).

34. Schneider, A.; Paskarbeit, J.; Schilling, M.; Schmitz, J. HECTOR, a bio-inspired and compliant hexapod robot. In *Conference on Biomimetic and Biohybrid Systems*; Springer International Publishing: Cham, Switzerland, 2014; pp. 427–429.

35. Dürr, V.; Arena, P.P.; Cruse, H.; Dallmann, C.J.; Drimus, A.; Hoinville, T.; Krause, T.; Mátéfi-Tempfli, S.; Paskarbeit, J.; Patanè, L. Integrative biomimetics of autonomous hexapedal locomotion. *Front. Neurorobot.* **2019**, *13*, 88, doi:10.3389/fnbot.2019.00088.

36. Meyer, H.G.; Klimeck, D.; Paskarbeit, J.; Rückert, U.; Egelhaaf, M.; Porrmann, M.; Schneider, A. Resource-efficient bio-inspired visual processing on the hexapod walking robot HECTOR. *PLoS ONE* **2020**, *15*, e0230620.

37. Belter, D.; Walas, K. A compact walking robot-flexible research and development platform. In *Recent Advances in Automation, Robotics and Measuring Techniques*; Springer International Publishing: Cham, Switzerland, 2014; pp. 343–352.

38. Belter, D.; Skrzypczyński, P.; Walas, K.; Wlodkowic, D. Affordable multi-legged robots for research and STEM education: A case study of design and technological aspects. In *Progress in Automation, Robotics and Measuring Techniques*; Springer International Publishing: Cham, Switzerland, 2015; pp. 23–34.

39. Rönnau, A.; Heppner, G.; Nowicki, M.; Dillmann, R. LAURON V: A versatile six-legged walking robot with advanced maneuverability. In Proceedings of the 2014 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, Besançon, France, 8–11 July 2014; pp. 82–87.

40. Buettner, T.; Roennau, A.; Heppner, G.; Dillmann, R. Design of an exchangeable, compact and modular bio-inspired leg for six-legged walking robots. In Proceedings of the CLAWAR 2017: 20th International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines, Porto, Portugal, 11–13 September 2017; pp. 89–96, doi: 10.1142/10736.

41. Roehr, T.M.; Cordes, F.; Kirchner, F. Reconfigurable integrated multirobot exploration system (RIMRES): Heterogeneous modular reconfigurable robots for space exploration. *J. Field Robot.* **2014**, *31*, 3–34.

42. Von Twickel, A.; Hild, M.; Siedel, T.; Patel, V.; Pasemann, F. Neural control of a modular multi-legged walking machine: Simulation and hardware. *Robot. Auton. Syst.* **2012**, *60*, 227–241.

43. Asif, U.; Iqbal, J. Motion planning using an impact-based hybrid control for trajectory generation in adaptive walking. *Int. J. Adv. Robot. Syst.* **2011**, *8*, 53.

44. Galloway, K.C.; Clark, J.E.; Yim, M.; Koditschek, D.E. Experimental investigations into the role of passive variable compliant legs for dynamic robotic locomotion. In Proceedings of the 2011 IEEE International Conference on Robotics and Automation, Shanghai, China, 9–13 May 2011; pp. 1243–1249.

45. Galloway, K.C.; Clark, J.E.; Koditschek, D.E. Variable stiffness legs for robust, efficient, and stable dynamic running. *J. Mech. Robot.* **2013**, *5*, 011009.
46. Galloway, K.C.; Haynes, G.C.; Ilhan, B.D.; Johnson, A.M.; Knopf, R.; Lynch, G.A.; Plotnick, B.N.; White, M.; Koditschek, D.E. *X-RHex: A Highly Mobile Hexapedal Robot for Sensorimotor Tasks*; Technical Report; University of Pennsylvania, Department of Electrical & Systems Engineering: Philadelphia, PA, USA, 2010.

47. Gorner, M.; Wimbock, T.; Baumann, A.; Fuchs, M.; Bahls, T.; Grebenstein, M.; Borst, C.; Butterfass, J.; Hirzinger, G. The DLR-Crawler: A testbed for actively compliant hexapod walking based on the fingers of DLR-Hand II. In Proceedings of the 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, Nice, France, 22–26 September 2008; pp. 1525–1531.

48. Manoonpong, P. Neural Preprocessing and Control of Reactive Walking Machines. Master's Thesis, University of Siegen, Siegen, Germany, 2006.

49. Manoonpong, P.; Pasemann, F.; Roth, H. Modular reactive neurocontrol for biologically inspired walking machines. *Int. J. Robot. Res.* **2007**, *26*, 301–331.

50. Pavone, M.; Arena, P.; Patané, L. An innovative mechanical and control architecture for a biomimetic hexapod for planetary exploration. *Space Technol.-Abingdon 2006*, *26*, 13–24.

51. Arena, P.; Fortuna, L.; Frasca, M.; Patané, L.; Pavone, M. Realization of a CNN-driven cockroach-inspired robot. In Proceedings of the 2006 IEEE International Symposium on Circuits and Systems, Island of Kos, Greece, 21–24 May 2006.

52. William, A.L. Insect-Inspired, Actively Compliant Robotic Hexapod. Master's Thesis, Case Western Reserve University, Cleveland, OH, USA, 2005.

53. Lewinger, W.A.; Quinn, R.D. A hexapod walks over irregular terrain using a controller adapted from an insect's nervous system. In Proceedings of the 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, Taipei, Taiwan, 18–22 October 2010; pp. 3386–3391.

54. Saranli, U.; Buehler, M.; Koditschek, D.E. RHex: A simple and highly mobile hexapod robot. *Int. J. Robot. Res.* **2001**, *20*, 616–631.

55. Tavolieri, C.; Ottaviano, E.; Ceccarelli, M.; Di Rienzo, A. Analysis and design of a 1-DOF leg for walking machines. *Proc. RAAD 2006*, *6*, 63–71.

56. Li, M.; He, Z.; Zha, F. Design and analysis of a semi-circular flexible hexapod robot miniRHex. In Proceedings of the 2018 13th World Congress on Intelligent Control and Automation (WCICA), Changsha, China, 4–8 July 2018; pp. 418–423, doi:10.1109/WCICA.2018.8630613.

57. Haldane, D.W.; Fearing, R.S. Running beyond the bio-inspired regime. *IEEE Int. Conf. Robot. Autom.* **2015**, *2015*, 4539–4546, doi:10.1109/ICRA.2015.7139828.

58. Soltero, D.E.; Julian, B.J.; Onal, C.D.; Rus, D. A lightweight modular 12-DOF print-and-fold hexapod. In Proceedings of the IEEE International Conference on Intelligent Robots and Systems, Tokyo, Japan, 3–7 November 2013; pp. 1465–1471, doi:10.1109/IROS.2013.6696542.

59. Schilling, M.; Hoinville, T.; Schmitz, J.; Cruse, H. Walknet, a bio-inspired controller for hexapod walking. *Biol. Cybern.* **2013**, *107*, 397–419.

60. Bjelonic, M.; Kottege, N.; Homberger, T.; Borges, P.; Beckerle, P.; Chli, M. Weaver: Hexapod robot for autonomous navigation on unstructured terrain. *J. Field Robot.* **2018**, *35*, 1063–1079, doi:10.1002/rob.21795.

61. Žák, M.; Rozman, J.; Zbořil, F.V. Design and Control of 7-DOF Omni-directional Hexapod Robot. *Open Comput. Sci.* **2021**, *11*, 80–89.

62. Delcomyn, F.; Nelson, M.E.; Cocatre-Zilgien, J.H. Sense organs of insect legs and the selection of sensors for agile walking robots. *Int. J. Robot. Res.* **1996**, *15*, 113–127.

63. Roennau, A.; Kerscher, T.; Dillmann, R. Design and kinematics of a biologically-inspired leg for a six-legged walking machine. In Proceedings of the 2010 3rd IEEE RAS and EMBS International Conference on Biomedical Robotics and Biomechatronics, BioRob 2010, Tokyo, Japan, 26–29 September 2010; pp. 626–631, doi:10.1109/BIOROB.2010.5626328.

64. Wohlgemuth, S.; Ronacher, B.; Wehner, R. Ant odometry in the third dimension. *Nature* **2001**, *411*, 795–798.

65. Drapin, J.; Arroyave-Tobon, S.; Linares, J.M.; Fourcassié, V.; Moretto, P. 3D kinematic of the leg joints at the ant scale. *Comput. Methods Biomech. Biomed. Eng.* **2021**. (in press).

66. Ignasov, J.; Kapilavai, A.; Filonenko, K.; Larsen, J.C.; Baird, E.; Hallam, J.; Büsse, S.; Kovalev, A.; Gorb, S.N.; Duggen, L. Bio-inspired design and movement generation of dung beetle-like legs. *Artif. Life Robot.* **2018**, *23*, 555–563.

67. Tedeschi, F.; Carbone, G. Design of a novel leg-wheel hexapod walking robot. *Robotics* **2017**, *6*, 40.

68. Wilcox, B.H.; Litwin, T.; Biesiadecki, J.; Matthews, J.; Heverly, M.; Morrison, J.; Townsend, J.; Ahmad, N.; Sirota, A.; Cooper, B. ATHLETE: A cargo handling and manipulation robot for the moon. *J. Field Robot.* **2007**, *24*, 421–434.

69. Chen, J.; Liang, Z.; Zhu, Y.; Zhao, J. Improving Kinematic Flexibility and Walking Performance of a Six-legged Robot by Rationally Designing Leg Morphology. *J. Bionic Eng.* **2019**, *16*, 608–620, doi:10.1007/s42235-019-0049-9.

70. Weihmann, T. Survey of biomechanical aspects of arthropod terrestrialisation – Substrate bound legged locomotion. *Arthropod Struct. Dev.* **2020**, *59*, 100983, doi:10.1016/j.asd.2020.100983.

71. Bullock, J.M.; Drechsler, P.; Federle, W. Comparison of smooth and hairy attachment pads in insects: Friction, adhesion and mechanisms for direction-dependence. *J. Exp. Biol.* **2008**, *211*, 3333–3343.

72. Clemente, C.J.; Goetzke, H.H.; Bullock, J.M.; Sutton, G.P.; Burrows, M.; Federle, W. Jumping without slipping: Leafhoppers (Hemiptera: Cicadellidae) possess special tarsal structures for jumping from smooth surfaces. *J. R. Soc. Interface* **2017**, *14*, doi:10.1098/rsif.2017.0022.

73. Dixey, L.; Gardiner, P. LXIII.—Heterogony in Messor barbarus L. var. capitatus Latreille. *J. Nat. Hist.* **1934**, *13*, 619–627.
74. Canio, G.D.; Stoyanov, S.; Larsen, J.C.; Hallam, J.; Kovalev, A.; Kleinteich, T.; Gorb, S.N.; Manoonpong, P. A robot leg with compliant tarsus and its neural control for efficient and adaptive locomotion on complex terrains. *Artif. Life Robot.* **2016**, *21*, 274–281, doi:10.1007/s10015-016-0296-3.

75. DeMario, A.; Zhao, J. Development and analysis of a three-dimensional printed miniature walking robot with soft joints and links. *J. Mech. Robot.* **2018**, *10*, 1–10, doi:10.1115/1.4039773.

76. Sun, J.; Zhao, J. An Adaptive Walking Robot with Reconfigurable Mechanisms Using Shape Morphing Joints. *IEEE Robot. Autom. Lett.* **2019**, *4*, 724–731, doi:10.1109/LRA.2019.2893439.

77. Kram, R.; Wong, B.; Full, R.J. Three-dimensional kinematics and limb kinetic energy of running cockroaches. *J. Exp. Biol.* **1997**, *200*, 1919–1929, doi:10.1242/jeb.200.13.1919.

78. Fernandez, J.G.; Ingber, D.E. Unexpected strength and toughness in chitosan-fibroin laminates inspired by insect cuticle. *Adv. Mater.* **2012**, *24*, 480–484, doi:10.1002/adma.201104051.

79. Zhang, Z.Q.; Yang, Q.; Zhao, J.; Gui, S. Dynamic model and performance analysis of rigid-flexible coupling four-bar leg mechanism for small scale bio-inspired jumping robot. *Microsyst. Technol.* **2019**, *25*, 3269–3285, doi:10.1007/s00542-019-04546-5.

80. Li, F.; Liu, W.; Fu, X.; Bonsignori, G.; Scarfogliero, U.; Stefanini, C.; Dario, P. Jumping like an insect: Design and dynamic optimization of a jumping mini robot based on bio-mimetic inspiration. *Mechatronics* **2012**, *22*, 167–176, doi:10.1016/j.mechatronics.2012.01.001.

81. Kitano, S.; Hirose, S.; Horigome, A.; Endo, G. TITAN-XIII: Sprawling-type quadruped robot with ability of fast and energy-efficient walking. *ROBOMECH J.* **2016**, *3*, 1–16, doi:10.1186/s40648-016-0047-1.

82. Merienne, H. Biomécanique et Energétique de la Locomotion et du Transport de Charge Chez la Fourmi. Ph.D. Thesis, University of Toulouse 3 (Paul Sabatier), Doctoral School "Science Ecologiques, Vétérinaires, Agronomiques et Bioingénieries", Toulouse, France, 2019.

83. Billeschou, P.; Bijma, N.N.; Larsen, L.B.; Gorb, S.N.; Larsen, J.C.; Manoonpong, P. Framework for developing bio-inspired morphologies for walking robots. *Appl. Sci.* **2020**, *10*, 6986.

84. Ríos, V.L.; Özdil, P.G.; Ramalingasetty, S.T.; Arreguit, J.; Ijspeert, A.J.; Ramdya, P. NeuroMechFly, a neuromechanical model of adult Drosophila melanogaster. *bioRxiv* **2021**, doi:10.1101/2021.04.17.440214.

85. Nishii, J. An analytical estimation of the energy cost for legged locomotion. *J. Theor. Biol.* **2006**, *238*, 636–645.

86. Kottege, N.; Parkinson, C.; Moghadam, P.; Elfes, A.; Singh, S.P. Energetics-informed hexapod gait transitions across terrains. In *Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA)*, Seattle, WA, USA, 26–30 May 2015; pp. 5140–5147.

87. Pontzer, H. A unified theory for the energy cost of legged locomotion. *Biol. Lett.* **2016**, *12*, 20150935.

88. Tucker, V.A. The energetic cost of moving about: Walking and running are extremely inefficient forms of locomotion. Much greater efficiency is achieved by birds, fish—and bicyclists. *Am. Sci.* **1975**, *63*, 413–419.

89. Alexander, R.M. Models and the scaling of energy costs for locomotion. *J. Exp. Biol.* **2005**, *208*, 1645–1652.

90. Seok, S.; Wang, A.; Chuah, M.Y.; Otten, D.; Lang, J.; Kim, S. Design principles for highly efficient quadrupeds and implementation on the MIT Cheetah robot. In *Proceedings of the 2013 IEEE International Conference on Robotics and Automation, Karlsruhe, Germany*, 6–10 May 2013; pp. 3307–3312.

91. Vanderborght, B.; Albu-Schäffer, A.; Bicchi, A.; Burdet, E.; Caldwell, D.G.; Carloni, R.; Catalano, M.; Eiberger, O.; Friedl, W.; Ganesh, G. Variable impedance actuators: A review. *Robot. Auton. Syst.* **2013**, *61*, 1601–1614.

92. Hunter, I.W.; Lafontaine, S. A comparison of muscle with artificial actuators. In *Proceedings of the Technical Digest IEEE Solid-State Sensor and Actuator Workshop, Hilton Head, SC, USA*, 22–25 June 1992; pp. 178–185.

93. Liang, W.; Liu, H.; Wang, K.; Qian, Z.; Ren, L.; Ren, L. Comparative study of robotic artificial actuators and biological muscle. *Adv. Mech. Eng.* **2020**, *12*, 1687814020933409.

94. Rothemund, P.; Kellaris, N.; Mitchell, S.K.; Acome, E.; Keplinger, C. HASEL Artificial Muscles for a New Generation of Lifelike Robots—Recent Progress and Future Opportunities. *Adv. Mater.* **2021**, *33*, 2003375.

95. El-Atab, N.; Mishra, R.B.; Al-Modaf, F.; Joharji, L.; Alsharif, A.A.; Alamoudi, H.; Diaz, M.; Qaiser, N.; Hussain, M.M. Soft Actuators for Soft Robotic Applications: A Review. *Adv. Intell. Syst.* **2020**, *2*, 2000128.

96. Mitchell, S.K.; Wang, X.; Acome, E.; Martin, T.; Ly, K.; Kellaris, N.; Venkata, V.G.; Keplinger, C. An Easy-to-Implement Toolkit to Create Versatile and High-Performance HASEL Actuators for Untethered Soft Robots. *Adv. Sci.* **2019**, *6*, 1900178.

97. Taghavi, M.; Helps, T.; Rossiter, J. Electro-ribbon actuators and electro-origami robots. *Sci. Robot.* **2018**, *3*, doi:10.1126/scirobotics. aau9795.

98. Nguyen, C.T.; Phung, H.; Hoang, P.T.; Nguyen, T.D.; Jung, H.; Moon, H.; Koo, J.C.; Choi, H.R. A novel bioinspired hexapod robot developed by soft dielectric elastomer actuators. In *Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)*, Vancouver, BC, Canada, 24–28 September 2017; pp. 6233–6238.

99. Schneider, A.; Schmucker, U. Force Sensing for Multi-Legged Walking Robots: Theory and Experiments Part 1: Overview and Force Sensing. In *Mobile Robotics, Moving Intelligence; IntechOpen: London, UK*, 2006; pp. 447–470.

100. Billeschou, P.; Albertsen, C.; Larsen, J.C.; Manoonpong, P. A Low-Cost, Compact, Sealed, Three-Axis Force/Torque Sensor for Walking Robots. *IEEE Sens. J.* **2021**, *21*, 8916–8926.

101. Zhang, H.; Wu, R.; Li, C.; Zang, X.; Zhang, X.; Jin, H.; Zhao, J. A force-sensing system on legs for biomimetic hexapod robots interacting with unstructured terrain. *Sensors* **2017**, *17*, 1514, doi:10.3390/s17071514.
102. Bachega, R.P.; Pires, R.; Campo, A.B. Force sensing to control a bio-inspired walking robot. *IFAC Proc. Vol. IFAC-Pap.* **2013**, *46*, 105–109, doi:10.3182/20130522-3-BR-4036.00052.

103. Dallmann, C.J.; Hoinville, T.; Dürr, V.; Schmitz, J. A load-based mechanism for inter-leg coordination in insects. *Proc. R. Soc. B Biol. Sci.* **2017**, *284*, doi:10.1098/rspb.2017.1755.

104. Kaliyamoorthy, S.; Quinn, R.D.; Zill, S.N. Force sensors in hexapod locomotion. *Int. J. Robot. Res.* **2005**, *24*, 563–574, doi:10.1177/0278364905055381.

105. Yang, M.; Kang, R.; Chen, Y. A highly mobile crawling robot inspired by hexapod insects. In Proceedings of the 2019 IEEE International Conference on Robotics and Biomimetics, ROBIO 2019, Dali, China, 6–8 December 2019; pp. 1797–1802, doi:10.1109/ROBIO49542.2019.8961825.

106. Nonami, K.; Barai, R.K.; Irawan, A.; Daud, M.R. *Hydraulically Actuated Hexapod Robots: Design, Implementation and Control*; Springer: Tokyo, Japan, 2014; Volume 66, doi:10.1007/978-4-431-54349-7.

107. Clark, J.E.; Goldman, D.I.; Lin, P.C.; Lynch, G.; Chen, T.S.; Komsuoglu, H.; Full, R.J.; Koditschek, D. Design of a bio-inspired dynamical vertical climbing robot. *Robot. Sci. Syst.* **2008**, *3*, 9–16, doi:10.7551/mitpress/7830.003.0003.

108. Jayaram, K.; Shum, J.; Castellanos, S.; Helbling, E.F.; Wood, R.J. Scaling down an insect-size microrobot, HAMR-VI into HAMR-Jr. In Proceedings of the 2020 IEEE International Conference on Robotics and Automation, Paris, France, 31 May–31 August 2020; pp. 10305–10311, doi:10.1109/ICRA40945.2020.9197436.

109. Stopp, S.; Wolff, T.; Irlinger, F.; Lueth, T. A new method for printer calibration and contour accuracy manufacturing with 3D-print technology. *Rapid Prototyp. J.* **2008**, *14*, 167–172, doi:10.1108/13552540810878030.

110. Zhu, Z.; Kin Tam, T.; Sun, F.; You, C.; Percival Zhang, Y.H. A high-energy-density sugar biobattery based on a synthetic enzymatic pathway. *Nat. Commun.* **2014**, *5*, 3026, doi:10.1038/ncomms4026.

111. Yang, X.; Chang, L.; Pérez-Arancibia, N.O. An 88-milligram insect-scale autonomous crawling robot driven by a catalytic artificial muscle. *Sci. Robot.* **2020**, *5*, 1–14, doi:10.1126/SCIROBOTICS.ABA0015.

112. Aoi, S.; Manoonpong, P.; Ambe, Y.; Matsuno, F.; Wörgötter, F. Adaptive control strategies for interlimb coordination in legged robots: A review. *Front. Neurorobot.* **2017**, *11*, 39.

113. Huerta, C.V.; Xiong, X.; Billeschou, P.; Manoonpong, P. Adaptive Neuromechanical Control for Robust Behaviors of Bio-Inspired Walking Robots. In *International Conference on Neural Information Processing*; Springer: Cham, Switzerland, 2020; pp. 775–786.

114. Xiong, X.; Wörgötter, F.; Manoonpong, P. Adaptive and energy efficient walking in a hexapod robot under neuromechanical control and sensorimotor learning. *IEEE Trans. Cybern.* **2015**, *46*, 2521–2534.

115. Xiong, X.; Wörgötter, F.; Manoonpong, P. Virtual agonist-antagonist mechanisms produce biological muscle-like functions. *Ind. Robot. Int.* **J. 2014**, *41*, 340–346, doi:10.1108/IR-11-2013-421.

116. Cruse, H.; Kindermann, T.; Schumm, M.; Dean, J.; Schmitz, J. Walknet—A biologically inspired network to control six-legged walking. *Neural Netw.* **1998**, *11*, 1435–1447.

117. Pearson, K.; Franklin, R. Characteristics of leg movements and patterns of coordination in locusts walking on rough terrain. *Int. J. Robot. Res.* **1984**, *3*, 101–112.

118. Espenschied, K.S.; Quinn, R.D.; Beer, R.D.; Chiel, H.J. Biologically based distributed control and local reflexes improve rough terrain locomotion in a hexapod robot. *Robot. Auton. Syst.* **1996**, *18*, 59–64.

119. Wilson, D.M. Insect walking. *Annu. Rev. Entomol.* **1966**, *11*, 103–122.

120. Ferrell, C. A comparison of three insect-inspired locomotion controllers. *Robot. Auton. Syst.* **1995**, *16*, 135–159.

121. Graham, D. A behavioural analysis of the temporal organisation of walking movements in the 1st instar and adult stick insect (Carausius morosus). *J. Comp. Physiol.* **1972**, *81*, 23–52.

122. Cruse, H. What mechanisms coordinate leg movement in walking arthropods? *Trends Neurosci.* **1990**, *13*, 15–21.

123. Delcomyn, F. The locomotion of the cockroach Periplaneta americana. *J. Exp. Biol.* **1971**, *54*, 443–452.

124. Wosnitza, A.; Bockemühl, T.; Dübbert, M.; Scholz, H.; Büschges, A. Inter-leg coordination in the control of walking speed in Drosophila. *J. Exp. Biol.* **2013**, *216*, 480–491.

125. Ambe, Y.; Aoi, S.; Nachstedt, T.; Manoonpong, P.; Wörgötter, F.; Matsuno, F. Simple analytical model reveals the functional role of embodied sensorimotor interaction in hexapod gaits. *PLoS ONE* **2018**, *13*, e0192469.

126. Cruse, H.; Dürr, V.; Schilling, M.; Schmitz, J. Principles of insect locomotion. In *Spatial Temporal Patterns for Action-Oriented Perception in Roving Robots*; Springer: Berlin/Heidelberg, Germany, 2009; pp. 43–96.

127. Büschges, A.; Akay, T.; Gabriel, J.P.; Schmidt, J. Organizing network action for locomotion: Insights from studying insect walking. *Brain Res. Rev.* **2008**, *57*, 162–171.

128. Bässler, U.; Wegner, U. Motor output of the denervated thoracic ventral nerve cord in the stick insect Carausius morosus. *J. Exp. Biol.* **1983**, *105*, 127–145.

129. Mantziaris, C.; Bockemühl, T.; Büschges, A. Central pattern generating networks in insect locomotion. *Dev. Neurobiol.* **2020**, *80*, 16–30.

130. Bässler, U.; Büschges, A. Pattern generation for stick insect walking movements—multisensory control of a locomotor program. *Brain Res. Rev.* **1998**, *27*, 65–88.

131. Berg, E.M.; Hooper, S.L.; Schmidt, J.; Büschges, A. A leg-local neural mechanism mediates the decision to search in stick insects. *Curr. Biol.* **2015**, *25*, 2012–2017.
132. Bucher, D.; Akay, T.; DiCaprio, R.A.; Büschges, A. Interjoint coordination in the stick insect leg-control system: The role of positional signaling. *J. Neurophysiol.* **2003**, *89*, 1245–1255.

133. Akay, T.; Ludwar, B.C.; Göritz, M.L.; Schmitz, J.; Büschges, A. Segment specificity of load signal processing depends on walking direction in the stick insect leg muscle control system. *J. Neurosci.* **2007**, *27*, 3285–3294.

134. Daun-Gruhn, S.; Tóth, T.I. An inter-segmental network model and its use in elucidating gait-switches in the stick insect. *J. Comput. Neurosci.* **2011**, *31*, 43–60.

135. Szczecinski, N.S.; Quinn, R.D. MantisBot changes stepping speed by entraining CPGs to positive velocity feedback. In *Conference on Biomimetic and Biohybrid Systems*; Springer International Publishing: Cham, Switzerland, 2017; pp. 440–452.

136. Niemeier, M.; Jeschke, M.; Dürr, V. Effect of thoracic connective lesion on inter-leg coordination in freely walking stick insects. *Front. Bioeng. Biotechnol.* **2021**, *9*, 628998.

137. Webb, B. Neural mechanisms for prediction: Do insects have forward models? *Trends Neurosci.* **2004**, *27*, 278–282.

138. Bläsing, B.; Cruse, H. Mechanisms of stick insect locomotion in a gap-crossing paradigm. *J. Comp. Physiol. A* **2004**, *190*, 173–183.

139. Möhl, B. 'Biological noise' and plasticity of sensorimotor pathways in the locust flight system. *J. Comp. Physiol. A* **1989**, *166*, 75–82.

140. Heisenberg, M.; Wolf, R. Reafferent control of optomotor yaw torque in Drosophila melanogaster. *J. Comp. Physiol. A* **1988**, *163*, 373–388.

141. Steingrube, S.; Timme, M.; Wörgötter, F.; Manoonpong, P. Self-organized adaptation of a simple neural circuit enables complex robot behaviour. *Nat. Phys.* **2010**, *6*, 224–230.

142. Manoonpong, P.; Parlitz, U.; Wörgötter, F. Neural control and adaptive neural forward models for insect-like, energy-efficient, and adaptable locomotion of walking machines. *Front. Neural Circuits* **2013**, *7*, 12.

143. Schilling, M.; Cruse, H. Decentralized control of insect walking: A simple neural network explains a wide range of behavioral and neurophysiological results. *PLoS Comput. Biol.* **2020**, *16*, e1007804.

144. Leung, B.; Bijma, N.; Baird, E.; Dacke, M.; Gorb, S.; Manoonpong, P. Rules for the Leg Coordination of Dung Beetle Ball Rolling Behaviour. *Sci. Rep.* **2020**, *10*, 1–8.

145. Mizrahi, J. Review: Mechanical Impedance and Its Relations to Motor Control, Limb Dynamics, and Motion Biomechanics. *J. Med. Biol. Eng.* **2015**, *35*, 1–20.

146. Bressler, B.H.; Clinch, N.F. The compliance of contracting skeletal muscle. *J. Physiol.* **1974**, *237*, 477–493, doi:10.1113/jphysiol.1974.sp010493.

147. Cruse, H.; Kühn, S.; Park, S.; Schmitz, J. Adaptive control for insect leg position: Controller properties depend on substrate compliance. *J. Comp. Physiol. A* **2004**, *190*, 983–991, doi:10.1007/s00359-004-0555-y.

148. Knoll, A.; Röhrbein, F.; Kuhn, A.; Akl, M.; Sharma, K. Neurorobotics. *Informatik-Spektrum* **2017**, *40*, 161–164, doi:10.1007/s00287-017-1031-8.

149. Hill, A.V. The mechanics of active muscle. *Proc. R. Soc. London. Ser. B—Biol. Sci.* **1953**, *141*, 104–117, doi:10.1098/rspb.1953.0027.

150. Proctor, J.L.; Holmes, P. The effects of feedback on stability and maneuverability of a phase-reduced model for cockroach locomotion. *Biol. Cybern.* **2018**, *112*, 387–401, doi:10.1007/s00422-018-0762-1.

151. Guo, S.; Lin, J.; Wöhrl, T.; Liao, M. A Neuro-Musculo-Skeletal Model for Insects With Data-driven Optimization. *Sci. Rep.* **2018**, *8*, 2129, doi:10.1038/s41598-018-20093-x.

152. Naris, M.; Szczecinski, N.S.; Quinn, R.D. A neuromechanical model exploring the role of the common inhibitor motor neuron in insect locomotion. *Biol. Cybern.* **2020**, *114*, 23–41, doi:10.1007/s00422-019-00811-y.

153. Dickinson, M.H.; Farley, C.T.; Full, R.J.; Koehl, M.A.R.; Kram, R.; Lehman, S. How Animals Move: An Integrative View. *Science* **2000**, *288*, 100–106.

154. Ahn, A.N. How muscles function—The work loop technique. *J. Exp. Biol.* **2012**, *215*, 1051–1052, doi:10.1242/jeb.062752.

155. Xiong, X.; Manoonpong, P. Muscles Can be Brakes: The Work Loop Technique for Stable Muscle-like Control. In *Proceedings of the International Symposium on Adaptive Motion of Animals and Machines (AMAM 2019)*, Lausanne, Switzerland, 20–23 August 2019; doi:10.5075/epfl-BIOROB-AMAM2019-67.

156. Xiong, X.; Wörgötter, F.; Manoonpong, P. Neuromechanical control for hexapedal robot walking on challenging surfaces and surface classification. *Robot. Auton. Syst.* **2014**, *62*, 1777–1789.

157. Braun, J.M.; Manoonpong, P.; Xiong, X. Editorial: Biology-Inspired Engineering and Engineering-Inspired Biology. *Front. Neurorobot.* **2020**, *14*, 614683.

158. Owaki, D.; Goda, M.; Miyazawa, S.; Ishiguro, A. A minimal model describing hexapedal interlimb coordination: The Tegotae-based approach. *Front. Neurorobot.* **2017**, *11*, 29.

159. Miguel-Blanco, A.; Manoonpong, P. General distributed neural control and sensory adaptation for self-organized locomotion and fast adaptation to damage of walking robots. *Front. Neural Circuits* **2020**, *14*, 46.

160. Sun, T.; Xiong, X.; Dai, Z.; Owaki, D.; Manoonpong, P. A Comparative Study of Adaptive Interlimb Coordination Mechanisms for Self-Organized Robot Locomotion. *Front. Robot. AI* **2021**, *8*, 86.

161. Cruse, H.; Dürr, V.; Schmitz, J. Insect walking is based on a decentralized architecture revealing a simple and robust controller. *Philos. Trans. R. Soc. A Math. Phys. Eng. Sci.* **2007**, *365*, 221–250.

162. Ayali, A.; Borgmann, A.; Büschges, A.; Couzin-Fuchs, E.; Daun-Gruhn, S.; Holmes, P. The comparative investigation of the stick insect and cockroach models in the study of insect locomotion. *Curr. Opin. Insect Sci.* **2015**, *12*, 1–10.
163. Bidaye, S.S.; Bockemühl, T.; Büschges, A. Six-legged walking in insects: How CPGs, peripheral feedback, and descending signals generate coordinated and adaptive motor rhythms. *J. Neurophysiol.* **2018**, *119*, 459–475.

164. Bjelonic, M.; Homberger, T.; Kottege, N.; Borges, P.; Chli, M.; Beckerle, P. Autonomous navigation of hexapod robots with vision-based controller adaptation. In *Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA)*, Singapore, 29 May–3 June; 2017; pp. 5561–5568.

165. Hafner, R.; Hertweck, T.; Klöppner, P.; Bloesch, M.; Neunert, M.; Wulfmeier, M.; Tunyasuvunakool, S.; Heess, N.; Riedmiller, M. Towards General and Autonomous Learning of Core Skills: A Case Study in Locomotion. *arXiv* **2020**, arXiv:2008.12228.

166. Pretorius, C.J.; du Plessis, M.C.; Gonsalves, J.W. Evolutionary robotics applied to hexapod locomotion: A comparative study of simulation techniques. *J. Intell. Robot. Syst.* **2019**, *96*, 363–385.

167. Ijspeert, A.J. Central pattern generators for locomotion control in animals and robots: A review. *Neural Netw.* **2008**, *21*, 642–653.

168. Fischer, J.; Pasemann, F.; Manoonpong, P. Neuro-controllers for walking machines—An evolutionary approach to robust behavior. In *Climbing and Walking Robots*; Springer: Berlin/Heidelberg, Germany, 2005; pp. 97–102.

169. Bai, L.; Hu, H.; Chen, X.; Sun, Y.; Ma, C.; Zhong, Y. CPG-Based Gait Generation of the Curved-Leg Hexapod Robot with Smooth Gait Transition. *Sensors* **2019**, *19*, 3705.

170. Arena, P.; Fortuna, L.; Frasca, M.; Patane, L. A CNN-based chip for robot locomotion control. *IEEE Trans. Circuits Syst. I Regul. Pap.* **2005**, *52*, 1862–1871.

171. Ekeberg, Ö.; Blümel, M.; Büschges, A. Dynamic simulation of insect walking. *Arthropod Struct. Dev.* **2004**, *33*, 287–300.

172. Brown, T.G. On the nature of the fundamental activity of the nervous centres; together with an analysis of the conditioning of rhythmic activity in progression, and a theory of the evolution of function in the nervous system. *J. Physiol.* **1914**, *48*, 18–46.

173. Hellgren, J.; Grillner, S.; Lansner, A. Computer simulation of the segmental neural network generating locomotion in lamprey by using populations of network interneurons. *Biol. Cybern.* **1992**, *68*, 1–13.

174. Ekeberg, Ö. A combined neuronal and mechanical model of fish swimming. *Biol. Cybern.* **1993**, *69*, 363–374.

175. Arena, P. The central pattern generator: A paradigm for artificial locomotion. *Soft Comput.* **2000**, *4*, 251–266.

176. Pasemann, F.; Hild, M.; Zahedi, K. SO (2)-networks as neural oscillators. In *International Work-Conference on Artificial Neural Networks*; Springer: Berlin/Heidelberg, Germany, 2003; pp. 144–151.

177. Van der Pol, B. LXXXVIII. On "relaxation-oscillations". *Lond. Edinb. Dublin Philos. Mag. J. Sci.* **1926**, *2*, 978–992.

178. Matsuoka, K. Sustained oscillations generated by mutually inhibiting neurons with adaptation. *Biol. Cybern.* **1985**, *52*, 367–376.

179. Taga, G.; Yamaguchi, Y.; Shimizu, H. Self-organized control of bipedal locomotion by neural oscillators in unpredictable environment. *Biol. Cybern.* **1991**, *65*, 147–159.

180. Righetti, L.; Buchli, J.; Ijspeert, A.J. Dynamic hebbian learning in adaptive frequency oscillators. *Phys. D Nonlinear Phenom.* **2006**, *216*, 269–281.

181. Aoi, S.; Egi, Y.; Sugimoto, R.; Yamashita, T.; Fujiki, S.; Tsuchiya, K. Functional roles of phase resetting in the gait transition of a biped robot from quadrupedal to bipedal locomotion. *IEEE Trans. Robot.* **2012**, *28*, 1244–1259.

182. Owaki, D.; Kano, T.; Nagasawa, K.; Tero, A.; Ishiguro, A. Simple robot suggests physical interlimb communication is essential for quadruped walking. *J. R. Soc. Interface* **2013**, *10*, 20120669.

183. Nachstedt, T.; Tetzlaff, C.; Manoonpong, P. Fast dynamical coupling enhances frequency adaptation of oscillators for robotic locomotion control. *Front. Neurorobot.* **2017**, *11*, 14.

184. Thor, M.; Manoonpong, P. Error-based learning mechanism for fast online adaptation in robot motor control. *IEEE Trans. Neural Netw. Learn. Syst.* **2019**, *31*, 2042–2051.

185. Chuthong, T.; Leung, B.; Tiraborisute, K.; Ngamkajornwiwat, P.; Manoonpong, P.; Dilokthanakul, N. Dynamical State Forcing on Central Pattern Generators for Efficient Robot Locomotion Control. In *International Conference on Neural Information Processing*; Springer: Cham, Switzerland, 2020; pp. 799–810.

186. Buchli, J.; Righetti, L.; Ijspeert, A.J. Engineering entrainment and adaptation in limit cycle systems. *Biol. Cybern.* **2006**, *95*, 645–664.

187. Buschmann, T.; Ewald, A.; von Twickel, A.; Bueschges, A. Controlling legs for locomotion—Insights from robotics and neurobiology. *Bioinspir. Biomim.* **2015**, *10*, 041001.

188. Thor, M.; Kulvicius, T.; Manoonpong, P. Generic neural locomotion control framework for legged robots. *IEEE Trans. Neural Netw. Learn. Syst.* **2020**, *32*, 4013–4025.

189. Goldschmidt, D.; Wörgötter, F.; Manoonpong, P. Biologically-inspired adaptive obstacle negotiation behavior of hexapod robots. *Front. Neurorobot.* **2014**, *8*, 3.

190. Grinke, E.; Tetzlaff, C.; Wörgötter, F.; Manoonpong, P. Synaptic plasticity in a recurrent neural network for versatile and adaptive behaviors of a walking robot. *Front. Neurorobot.* **2015**, *9*, 11.

191. Dasgupta, S.; Goldschmidt, D.; Wörgötter, F.; Manoonpong, P. Distributed recurrent neural forward models with synaptic adaptation and CPG-based control for complex behaviors of walking robots. *Front. Neurorobot.* **2015**, *9*, 10.

192. Sun, X.; Yue, S.; Mangan, M. A decentralised neural model explaining optimal integration of navigational strategies in insects. *eLife* **2020**, *9*, e54026, doi:10.7554/eLife.54026.

193. Roennau, A.; Kerscher, T.; Ziegenmeyer, M.; Zöllner, J.M.; Dillmann, R. Adaptation of a six-legged walking robot to its local environment. In *Robot Motion and Control 2009*; Springer: London, UK, 2009; pp. 155–164.
194. Paskarbeit, J.; Schilling, M.; Schmitz, J.; Schneider, A. Obstacle crossing of a real, compliant robot based on local evasion movements and averaging of stance heights using singular value decomposition. In Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA, 26–30 May 2015; pp. 3140–3145.

195. Thor, M.; Strøm-Hansen, T.; Larsen, L.B.; Kovalev, A.; Gorb, S.N.; Baird, E.; Manoonpong, P. A dung beetle-inspired robotic model and its distributed sensor-driven control for walking and ball rolling. *Artif. Life Robot.* **2018**, *23*, 435–443.

196. Schilling, M.; Cruse, H. ReaCog, a minimal cognitive controller based on recruitment of reactive systems. *Front. Neurorobot.* **2017**, *11*, 3.

197. Tieck, J.C.V.; Rutschke, J.; Kaiser, J.; Schulze, M.; Buettner, T.; Reichard, D.; Roennau, A.; Dillmann, R. Combining spiking motor primitives with a behaviour-based architecture to model locomotion for six-legged robots. In Proceedings of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, SAR, China, 3–9 November 2019 pp. 4161–4168.

198. von Twickel, A.; Büschges, A.; Pasemann, F. Deriving neural network controllers from neuro-biological data: implementation of a single-leg stick insect controller. *Biol. Cybern.* **2011**, *104*, 95–119.

199. Lewinger, W.A.; Quinn, R.D. Neurobiologically-based control system for an adaptively walking hexapod. *Ind. Robot.* **2011**, *38*, 258–263.

200. Yu, H.; Gao, H.; Deng, Z. Enhancing adaptability with local reactive behaviors for hexapod walking robot via sensory feedback integrated central pattern generator. *Robot. Auton. Syst.* **2020**, *124*, 103401.

201. Szadkowski, R.; Prágr, M.; Faigl, J. Self-Learning Event Mistiming Detector Based on Central Pattern Generator. *Front. Neurorobot.* **2021**, *15*, 5.

202. Ngamkajornwiwat, P.; Homchanthanakul, J.; Teerakittikul, P.; Manoonpong, P. Bio-inspired adaptive locomotion control system for online adaptation of a walking robot on complex terrains. *IEEE Access* **2020**, *8*, 91587–91602.

203. Homchanthanakul, J.; Ngamkajornwiwat, P.; Teerakittikul, P.; Manoonpong, P. Neural control with an artificial hormone system for energy-efficient compliant terrain locomotion and adaptation of walking robots. In Proceedings of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, SAR, China, 3–8 November 2019; pp. 5475–5482.

204. Cruse, H.; Bartling, C. Movement of joint angles in the legs of a walking insect, Carausius morosus. *J. Insect Physiol.* **1995**, *41*, 761–771.

205. Belter, D.; Skrzypczyński, P. Rough terrain mapping and classification for foothold selection in a walking robot. *J. Field Robot.* **2011**, *28*, 497–528.

206. Liu, T.; Chen, W.; Wang, J.; Wu, X. Terrain analysis and locomotion control of a hexapod robot on uneven terrain. In Proceedings of the 2014 9th IEEE Conference on Industrial Electronics and Applications, Hangzhou, China, 9–11 June 2014; pp. 1959–1964.

207. Buchanan, R.; Bandyopadhyay, T.; Bjelonic, M.; Wellhausen, L.; Hutter, M.; Kottege, N. Walking posture adaptation for legged robot navigation in confined spaces. *IEEE Robot. Autom. Lett.* **2019**, *4*, 2148–2155.

208. Murata, Y.; Inagaki, S.; Suzuki, T. Development of an adaptive hexapod robot based on Follow-the-contact-point gait control and Timekeeper control. In Proceedings of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, SAR, China, 3–8 November 2019; pp. 3321–3327.

209. Hu, N.; Li, S.; Zhu, Y.; Gao, F. Constrained model predictive control for a hexapod robot walking on irregular terrain. *J. Intell. Robot. Syst.* **2019**, *94*, 179–201.

210. Xiong, X.; Wörgötter, F.; Manoonpong, P. A Simplified Variable Admittance Controller Based on a Virtual Agonist-Antagonist Mechanism for Robot Joint Control. In *Nature-Inspired Mobile Robotics*; World Scientific: Singapore, 2013; pp. 281–288.

211. Xiong, X.; Manoonpong, P. Adaptive Motor Control for Human-like Spatial-temporal Adaptation. In Proceedings of the 2018 IEEE International Conference on Robotics and Biomimetics (ROBIO), Kuala Lumpur, Malaysia, 12–15 December 2018; pp. 2107–2112.

212. Holmes, P.; Full, R.; Koditschek, D.; Guckenheimer, J. The Dynamics of Legged Locomotion: Models, Analyses, and Challenges. *SIAM Rev.* **2006**, *48*, 207–304.

213. Sun, T.; Xiong, X.; Dai, Z.; Manoonpong, P. Small-Sized Reconfigurable Quadruped Robot With Multiple Sensory Feedback for Studying Adaptive and Versatile Behaviors. *Front. Neurorobot.* **2020**, *14*, 14, doi:10.3389/fnbot.2020.00014.

214. Kukillaya, R.; Proctor, J.; Holmes, P. Neuromechanical models for insect locomotion: Stability, maneuverability, and proprioceptive feedback. *Chaos Interdiscip. J. Nonlinear Sci.* **2009**, *19*, doi:10.1063/1.3141306.

215. Szczecinski, N.S.; Goldsmith, C.A.; Young, F.R.; Quinn, R.D. Tuning a Robot Servomotor to Exhibit Muscle-Like Dynamics. In *Biomimetic and Biohybrid Systems, Living Machines* 2019; Springer: Cham, Switzerland, 2019; pp. 254–265.

216. Ribak, G. Insect-inspired jumping robots: Challenges and solutions to jump stability. *Curr. Opin. Insect Sci.* **2020**, *42*, 32–38, doi:10.1016/j.cois.2020.09.001.

217. Schilling, M.; Konen, K.; Ohl, F.W.; Korthals, T. Decentralized Deep Reinforcement Learning for a Distributed and Adaptive Locomotion Controller of a Hexapod Robot. In Proceedings of the 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA, 24–30 October 2020; pp. 5335–5342, doi:10.1109/IROS45743.2020.9341754.

218. Ting, J.; Fang, Y.; Lele, A.; Raychowdhury, A. Bio-inspired gait imitation of hexapod robot using event-based vision sensor and spiking neural network. In Proceedings of the 2020 International Joint Conference on Neural Networks (IJCNN), Glasgow, UK, 19–24 July 2020; pp. 1–7.

219. Cully, A.; Clune, J.; Tarapore, D.; Mouret, J.B. Robots that can adapt like animals. *Nature* **2015**, *521*, 503–507.

220. Parker, G.B. Evolving gaits for hexapod robots using cyclic genetic algorithms. *Int. J. Gen. Syst.* **2005**, *34*, 301–315.

221. Cully, A.; Mouret, J.B. Evolving a behavioral repertoire for a walking robot. *Evol. Comput.* **2016**, *24*, 59–88.
222. Azayev, T.; Zimmerman, K. Blind Hexapod Locomotion in Complex Terrain with Gait Adaptation Using Deep Reinforcement Learning and Classification. *J. Intell. Robot. Syst.* **2020**, *99*, 1–13.

223. Milicka, P.; Cizek, P.; Faigl, J. On Chaotic Oscillator-Based Central Pattern Generator for Motion Control of Hexapod Walking Robot. *ITAT 2016*, **1649**, 131–137.

224. Chen, W.; Ren, G.; Wang, J.; Liu, D. An adaptive locomotion controller for a hexapod robot: CPG, kinematics and force feedback. *Sci. China Inf. Sci.* **2014**, *57*, 1–18.

225. Fu, H.; Tang, K.; Li, P.; Zhang, W.; Wang, X.; Deng, G.; Wang, T.; Chen, C. Deep Reinforcement Learning for Multi-contact Motion Planning of Hexapod Robots. In *Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI)*, Montréal, QC, Canada, 21 August 2021; pp. 2381–2388.

226. Ouyang, W.; Chi, H.; Pang, J.; Liang, W.; Ren, Q. Adaptive Locomotion Control of a Hexapod Robot via Bio-Inspired Learning. *Front. Neurorobot.* **2021**, *15*, 1.

227. Lilicrap, T.P.; Hunt, J.J.; Pritzel, A.; Heess, N.; Erez, T.; Tassa, Y.; Silver, D.; Wierstra, D. Continuous control with deep reinforcement learning. *arXiv* **2015**, arXiv:1509.02971.

228. Cruse, H.; Schilling, M. Mental States as Emergent Properties From Walking to Consciousness. In *Open Mind*; MIND Group: Frankfurt am Main, Germany, 2015; pp. 335–373, doi:10.15502/9783958570436.

229. Perry, C.J.; Barron, A.B.; Chittka, L. The frontiers of insect cognition. *Curr. Opin. Behav. Sci.* **2017**, *16*, 111–118. doi:https://doi.org/10.1016/j.cobeha.2017.05.011.

230. Chittka, L.; Niven, J. Are Bigger Brains Better? *Curr. Biol.* **2009**, *19*, R995–R1008, doi:10.1016/j.cub.2009.08.023.

231. Wolf, R.; Voss, A.; Hein, S.; Heisenberg, M.; Sullivan, G.; Barlow, H.B.; Frisby, J.P.; Horridge, G.A.; Jeeves, M.A. Can a fly ride a bicycle? *Philos. Trans. R. Soc. London. Ser. B Biol. Sci.* **1992**, *337*, 261–269, doi:10.1098/rstb.1992.0104.

232. Dacke, M.; Srinivasan, M. Evidence for counting in insect. *Anim. Cogn.* **2008**, *11*, 683–689, doi:10.1007/s10071-008-0159-y.

233. Zhang, S.; Srinivasan, M.V.; Zhu, H.; Wong, J. Grouping of visual objects by honeybees. *J. Exp. Biol.* **2004**, *207*, 3289–3298, doi:10.1242/jeb.01155.

234. Giurfa, M.; Zhang, S.; Jenett, A.; Menzel, R.; Srinivasan, M. The concepts of ‘sameness’ and ‘difference’ in an insect. *Nature* **2001**, *410*, 930–933, doi:10.1038/35073582.

235. Ofstad, T.; Zuker, C.; Reiser, M. Visual Place Learning in Drosophila melanogaster. *Nature* **2011**, *474*, 204–207, doi:10.1038/nature10131.

236. Giurfa, M. Social learning in insects: A higher-order capacity? *Front. Behav. Neurosci.* **2012**, *6*, 57, doi:10.3389/fnbeh.2012.00057.

237. Collett, T.; Fry, S.; Fry, S. Sequence learning by honeybees. *J. Comp. Physiol. A* **1993**, *172*, 693–706, doi:doi.org/10.1007/BF00195395.

238. Menzel, R. Serial Position Learning in Honeybees. *PLoS ONE* **2009**, *4*, 4694, doi:10.1371/journal.pone.0004694.

239. Menzel, R.; Greggers, U.; Smith, A.; Berger, S.; Brandt, R.; Brunke, S.; Bundrock, G.; Hülse, S.; Plümpe, T.; Schaupp, F.; et al. Honey bees navigate according to a map-like spatial memory. *Proc. Natl. Acad. Sci. USA* **2005**, *102*, 3040–3045, doi:10.1073/pnas.0408550102.

240. Menzel, R.; Lehmann, K.; Manz, G.; Degen, J.; Koblofsky, M.; Greggers, U. Vector integration and novel shortcutting in honeybee navigation. *Apidologie* **2012**, *43*, 229–243, doi:10.1007/s13592-012-0127-z.

241. Menzel, R.; Kirbach, A.; Haass, W.D.; Fischer, B.; Degen, J.; Koblofsky, M.; Lehmann, K.; Reiter, L.; Meyer, H.; Nguyen, H.; et al. A Common Frame of Reference for Learned and Communicated Vectors in Honeybee Navigation. *Curr. Biol. CB* **2011**, *21*, 645–650, doi:10.1016/j.cub.2011.02.039.

242. Collett, T. Insect navigation en route to the goal: Multiple strategies for the use of landmarks. *J. Exp. Biol.* **1996**, *199*, 227–235.

243. Bos, N.; Dreier, S.; Jørgensen, C.; Nielsen, J.; Guerrieri, F. Learning and perceptual similarity among cuticular hydrocarbons in ants. *J. Insect Physiol.* **2011**, *58*, 138–146, doi:10.1016/j.jinsphys.2011.10.010.

244. Brembs, B.; Hempel de Ibarra, N. Different parameters support generalization and discrimination learning in Drosophila at the flight simulator. *Learn. Mem. (Cold Spring Harb. N.Y.)* **2006**, *13*, 629–637, doi:10.1101/lm.319406.

245. Zentall, T.; Galizio, M.; Critchfield, T. Categorization, concept learning, and behavior analysis: An introduction. *J. Exp. Anal. Behav.* **2002**, *78*, 237–248, doi:10.1901/jeab.2002.78-237.

246. Avarguès-Weber, A.; Dyer, A.G.; Giurfa, M. Conceptualization of above and below relationships by an insect. *Proc. R. Soc. B Biol. Sci.* **2010**, *278*, 898–905.

247. Bell, A.T.A.; Niven, J.E. Strength of forelimb lateralization predicts motor errors in an insect. *Biol. Lett.* **2016**, *12*, 20160547, doi:10.1098/rsbl.2016.0547.

248. Romano, D.; Benelli, G.; Kavallieratos, N.; Athanassiou, C.; Canale, A.; Stefanini, C. Beetle-robot hybrid interaction: Sex, lateralization and mating experience modulate behavioural responses to robotic cues in the larger grain borer Prostephanus truncatus (Horn). *Biol. Cybern.* **2020**, *114*, doi:10.1007/s00422-020-00839-5.

249. Bockhorst, T.; Homberg, U. Interaction of compass sensing and object-motion detection in the locust central complex. *J. Neurophysiol.* **2017**, *118*, 496–506.

250. de Bivort, B.L.; van Swinderen, B. Evidence for selective attention in the insect brain. *Curr. Opin. Insect Sci.* **2016**, *15*, 9–15.

251. Sheehan, M.; Tibbetts, E. Specialized Face Learning Is Associated with Individual Recognition in Paper Wasps. *Science* **2011**, *334*, 1272–1275, doi:10.1126/science.1211334.

252. Coolen, I.; Dangles, O.; Casas, J. Social Learning in Noncolonial Insects? *Curr. Biol. CB* **2005**, *15*, 1931–1935, doi:10.1016/j.cub.2005.09.015.
253. Sachin, S.; Reuven, D. Social learning about egg-laying substrates in fruitflies. *Proc. R. Soc. B Biol. Sci.* **2009**, *276*, 4323–4328, doi:10.1098/rspb.2009.1294.

254. Romano, D.; Benelli, G.; Stefanini, C. Opposite valence social information provided by bio-robotic demonstrators shapes selection processes in the green bottle fly. *J. R. Soc. Interface* **2021**, *18*, 20210056, doi:10.1098/rsif.2021.0056.

255. Pfeiffer, B.D.; Jenett, A.; Hammonds, A.S.; Ngo, T.T.B.; Misra, S.; Murphy, C.; Scully, A.; Carlson, J.W.; Wan, K.H.; Laverty, T.R.; et al. Tools for neuroanatomy and neurogenetics in Drosophila. *Proc. Natl. Acad. Sci. USA* **2008**, *105*, 9715–9720, doi:10.1073/pnas.0803697105.

256. Tang, S.; Guo, A. Choice Behavior of Drosophila Facing Contradictory Visual Cues. *Science* **2001**, *294*, 1543–1547, doi:10.1126/science.1058237.

257. Schröter, U.; Menzel, R. A new ascending sensory tract to the calyces of the honeybee mushroom body, the subesophageal-calycal tract. *J. Comp. Neurol.* **2003**, *465*, 168–178, doi:10.1002/cne.10843.

258. Patanè, L.; Strauss, R.; Arena, P. Biological Investigation of Neural Circuits in the Insect Brain. In *Nonlinear Circuits and Systems for Neuro-inspired Robot Control*; Springer: Cham, Switzerland, 2018; pp. 1–20,

259. Turner-Evans, D.B.; Jayaraman, V. The insect central complex. *Curr. Biol.* **2016**, *26*, R453–R457, doi:10.1016/j.cub.2016.04.006.

260. Zhu, L.; Mangan, M.; Webb, B. Spatio-Temporal Memory for Navigation in a Mushroom Body Model. In *Biomimetic and Biohybrid Systems*; Vouloutsi, V., Mura, A., Tauber, F., Speck, T., Prescott, T.J., Verschure, P.F.M.J., Eds.; Springer International Publishing: Cham, Switzerland, 2020; pp. 415–426.

261. Rapp, H.; Nawrot, M. A spiking neural program for sensorimotor control during foraging in flying insects. *Proc. Natl. Acad. Sci. USA* **2020**, *117*, 28412–28421, doi:10.1073/pnas.2009821117.

262. Gerber, B.; Stocker, R. The Drosophila larva as a model for studying chemosensation and chemosensory learning: A review. *Chem. Senses* **2007**, *32*, 65–89, doi:10.1093/chemse/bjl030.

263. Koenig, S.; Wolf, R.; Heisenberg, M. Visual Attention in Flies-Dopamine in the Mushroom Bodies Mediates the After-Effect of Cueing. *PLoS ONE* **2016**, *11*, 1412, doi:10.1371/journal.pone.0161412.

264. Arena, P.; Patanè, L.; Termini, P.S. Modeling attentional loop in the insect Mushroom Bodies. In *Proceedings of the 2012 International Joint Conference on Neural Networks (IJCNN)*, Brisbane, QLD, Australia, 10–15 June 2012; pp. 1–7, doi:10.1109/IJCNN.2012.6252833.

265. Zhao, C.; Widmer, Y.; Diegelmann, S.; Petrovici, M.; Sprecher, S.; Senn, W. Predictive olfactory learning in Drosophila. *Sci. Rep.* **2021**, *11*, 6795, doi:10.1038/s41598-021-85841-y.

266. Arena, P.; Patanè, L.; Termini, P.S. Learning expectation in insects: A recurrent spiking neural model for spatio-temporal representation. *Neural Netw.* **2012**, *32*, 35–45. https://doi.org/10.1016/j.neunet.2012.02.034.

267. Arena, P.; Patané, L.; Stornanti, V.; Termini, P.S.; Zäpf, B.; Strauss, R. Modeling the insect mushroom bodies: Application to a delayed match-to-sample task. Special Issue on Autonomous Learning. *Neural Netw.* **2013**, *41*, 202, https://doi.org/10.1016/j.neunet.2012.11.013.

268. Arena, P.; Calí, M.; Patanè, L.; Portera, A.; Strauss, R. A Fly-Inspired Mushroom Bodies Model for Sensory-Motor Control Through Sequence and Subsequence Learning. *Int. J. Neural Syst.* **2016**, *26*, 1650035. doi:10.1142/S0129065716500350.

269. Arena, P.; Calí, M.; Patané, L.; Portera, A.; Strauss, R. Modelling the insect Mushroom Bodies: Application to sequence learning. *Neural Netw.* **2015**, *67*, 37–53, doi:10.1016/j.neunet.2015.03.006.

270. Arena, P.; Calí, M.; Patané, L.; Portera, A.; Spinosa, A.G. A CNN-based neuromorphic model for classification and decision control. *Nonlinear Dyn.* **2019**, *95*, 1999–2017.

271. Arena, P.; Patanè, L.; Termini, P. Decision making processes in the fruit fly: A computational model. In *Frontiers in Artificial Intelligence and Applications, Neural Nets WIRN11*; IOS Press: Amsterdam, The Netherlands, 2011; Volume 234; pp. 284–291, doi:10.3233/978-1-60750-972-1-275.

272. Kuntz, S.; Poeck, B.; Sokolowski, M.; Strauss, R. The visual orientation memory of Drosophila requires Foraging (PKG) upstream of Ignorant (RSK2) in ring neurons of the central complex. *Learn. Mem.* **2012**, *19*, 337–340, doi:10.1101/lm.026369.112.

273. Arena, P.; Maceo, S.; Patanè, L.; Strauss, R. A spiking network for spatial memory formation: Towards a fly-inspired ellipsoid body model. In *Proceedings of the 2013 International Joint Conference on Neural Networks (IJCNN)*, Dallas, TX, USA, 4–9 August 2013; pp. 1–6, doi:10.1109/IJCNN.2013.6706882.

274. Kakaria, K.S.; de Bivort, B.L. Ring attractor dynamics emerge from a spiking model of the entire protocerebral bridge. *Front. Behav. Neurosci.* **2017**, *11*, 8.

275. Varga, A.G.; Kathman, N.D.; Martin, J.P.; Guo, P.; Ritzmann, R.E. Spatial Navigation and the Central Complex: Sensory Acquisition, Orientation, and Motor Control. *Front. Behav. Neurosci.* **2017**, *11*, 4, doi:10.3389/fnbeh.2017.00004.

276. Daltorio, K.; Mirletz, B.; Bender, J.; Webster-Wood, V.; Szczecinski, N.; Branicky, M.; Ritzmann, R.; Quinn, R. A model of exploration and goal-searching in the cockroach, Blaberus discoidalis. *Adapt. Behav.* **2013**, *21*, 404–420, doi:10.1177/1059712313491615.

277. Krause, T.; Spindler, L.; Poeck, B.; Strauss, R. Drosophila Acquires a Long-Lasting Body-Size Memory from Visual Feedback. *Curr. Biol.* **2019**, *29*, 1833–1841.e3, doi:10.1016/j.cub.2019.04.037.

278. Arena, P.; Patané, L.; Sanalitro, D.; Vitanza, A. Insect-Inspired Body Size Learning Model on a Humanoid Robot. In *Proceedings of the 2018 7th IEEE International Conference on Biomedical Robotics and Biomechatronics (Biorob)*, Enschede, The Netherlands, 26–29 August 2018; pp. 1127–1132, doi:10.1109/BIOROB.2018.8487635.
279. Arena, P.; Mauro, G.D.; Krause, T.; Patanè, L.; Strauss, R. A spiking network for body size learning inspired by the fruit fly. In Proceedings of the 2013 International Joint Conference on Neural Networks (IJCNN), Dallas, TX, USA, 4–9 August 2013; pp. 1–7, doi:10.1109/IJCNN.2013.6706883.

280. Goulard, R.; Buehlmann, C.; Niven, J.E.; Graham, P.; Webb, B. A unified mechanism for innate and learned visual landmark guidance in the insect central complex. *PLoS Comput. Biol.* **2021**, *15*, 1473–1478 doi:10.1101/2021.01.28.428620.

281. Pick, S.; Strauss, R. Goal-Driven Behavioral Adaptations in Gap-Climbing Drosophila. *Curr. Biol.* **2005**, *15*, 1473–1478, doi:10.1016/j.cub.2005.07.022.

282. Arena, E.; Arena, P.; Strauss, R.; Patanè, L. Motor-Skill Learning in an Insect Inspired Neuro-Computational Control System. *Front. Neurorobot.* **2017**, *11*, 12, doi:10.3389/fnbot.2017.00012.

283. Plath, J.A.; Entler, B.V.; Kirkerud, N.H.; Schlegel, U.; Galizia, C.G.; Barron, A.B. Different roles for honey bee mushroom bodies and central complex in visual learning of colored lights in an aversive conditioning assay. *Front. Behav. Neurosci.* **2017**, *11*, 98.

284. Izhikevich, E.; Desai, N. Relating stdp to bcm. *Neural Comput.* **2003**, *15*, 1511–1523, doi:10.1162/089976603321891783.

285. Fiore, V.G.; Kottler, B.; Gu, X.; Hirth, F. In silico Interrogation of Insect Central Complex Suggests Computational Roles for the Ellipsoid Body in Spatial Navigation. *Front. Behav. Neurosci.* **2017**, *11*, 142, doi:10.3389/fnbeh.2017.00142.

286. Zill, S.; Schmitz, J.; Büschges, A. Load sensing and control of posture and locomotion. *Arthropod Struct. Dev.* **2004**, *33*, 273–286.

287. Manoonpong, P.; Tetzlaff, C. Neural computation in embodied closed-loop systems for the generation of complex behavior: From biology to technology. *Front. Neurorobot.* **2018**, *12*, 53.

288. Ritzmann, R.E.; Quinn, R.D.; Fischer, M.S. Convergent evolution and locomotion through complex terrain by insects, vertebrates and robots. *Arthropod Struct. Dev.* **2004**, *33*, 361–379.

289. Wystrach, A.; Schwarz, S. Ants use a predictive mechanism to compensate for passive displacements by wind. *Curr. Biol.* **2013**, *23*, R1083–R1085.

290. Manoonpong, P.; Petersen, D.; Kovalev, A.; Wörgötter, F.; Gorb, S.N.; Spinner, M.; Heepe, L. Enhanced locomotion efficiency of a bio-inspired walking robot using contact surfaces with frictional anisotropy. *Sci. Rep.* **2016**, *6*, 1–11.

291. Collins, L. The case for emulating insect brains using anatomical “wiring diagrams” equipped with biophysical models of neuronal activity. *Biol. Cybern.* **2019**, *113*, 465–474.

292. Niven, J.E.; Chittka, L. Reuse of identified neurons in multiple neural circuits. *Behav. Brain Sci.* **2010**, *33*, 285–285, doi:10.1017/S0140525X10001068.

293. Simons, M.; Tibbetts, E. Insects as models for studying the evolution of animal cognition. *Curr. Opin. Insect Sci.* **2019**, *34*, 117–122. doi:10.1016/j.cois.2019.05.009.