# Wankelmut: A Simple Benchmark for the Evolvability of Behavioral Complexity
## 2 Our Benchmark Task  
For the sake of simplicity, we restrict ourselves to a very simple task that is hard to evolve in an open-ended, uninformed, and unguided way. As shown in Figure 1, we assume an agent that moves in an environment expressing one single quality factor (e.g., height, water or air pressure, luminance, temperature) and that has to evolve a behavior that first makes the agent move uphill in this environmental gradient. In the \"uphill-walk\" phase of the experiment, the agent always should move in the direction of the sensor that reports the higher value of the focal quality factor. As soon as the agent reaches an area of sufficiently high quality (above a threshold \(\Theta_{max}\)), the agent should switch its behavioral mode and start to seek areas of low environmental quality. In this \"downhill-walk\" phase, the agent should always move towards the side where its sensor reports the lowest environmental quality value. After the agent has reached a sufficiently low quality area (below a threshold \(\Theta_{min}\)), the agent should switch back to the \"uphill-walk\" behavior again. We call the behavior that we aim to evolve \"Wankelmut\", a term that expresses in German a character trait in which one always switches between two different goals as soon as one of the goals is reached. A \"Wankelmut\" agent is never satisfied and thus does not decide one thing and does not stick with it. It is a variant of \"the grass is always greener,\" a commonly found personality feature in natural agents (from humans to animals), and despite its negative perception in many cultural moral systems, it has its benefits. It keeps the agent going, being explorative, curious, and never satisfied. That is exactly the desired behavior of an autonomous probe on a distant planet that needs to be explored. In summary, the task is to follow an environmental gradient up and down in an alternating way, hence maximizing the coverage (monitoring, observation, patrolling) of  
Figure 1: (**A**) Schematic drawing of the problem posed in our focal benchmark task. The evolved software controller should make a vehicle cycle between the high and the low region. (**B**) The focal task is broken down to a 1D cellular simulation with discrete space and time to achieve the maximum simplification without losing the difficulty of the benchmark. the areas between \(\Theta_{min}\) and \(\Theta_{max}\). There are many examples that have been evolved by natural selection [27]. In social insects (ants, termites, wasps, honeybees), foragers have first to go out of the nest, and after they encounter food, they switch their behavior and go back to the nest. After they have unloaded the food to other nest workers, they go outwards to forage again [28]. Often, environmental cues and gradients are involved in the homing and in the foraging behaviors (sun compass, nest scent, pheromone marks) and are exploited differently by the workers in the outbound behavioral state compared to the inbound behavioral state [29]. Other biological sources of inspiration are animals following a diurnal rhythm (day-walkers and night-walkers). In an engineering context, the rhythm might be imposed by energy recharging cycles, water depths, or aerial heights in transportation tasks by underwater vehicles or aerial drones. Figure 1B shows that we de-complexified the benchmark task to a one-dimensional cellular space of \(N\) cells in which always one cell of index \(P(t)\) is occupied by the agent that has state \(S(t)\). The environmental gradient is produced by the Gauss error function:  
\[\text{erf}(x)=\frac{2}{\sqrt{\pi}}\int_{0}^{x}e^{-t^{2}}\,\text{dt}\, \tag{1}\]  
whereby the quality of every cell \(i\) is modeled as:  
\[\text{quality}[i]=\text{erf}\Bigg{(}\frac{4(i-\frac{N}{2})}{N}\Bigg{)}. \tag{2}\]  
In every time step \(t\in[0,t_{max}]\), the agent has access to two lateral sensor readings, which are modeled as:  
\[s_{l}(t)= \begin{cases}\text{quality}[P(t)-1]&\text{if }(P(t)>0)\\ \text{quality}[P(t)]&\text{otherwise}\end{cases}