G OAL -G UIDED N EURAL C ELLULAR A UTOMATA : L EARNING TO C ONTROL S ELF -O RGANISING S YSTEMS
A BSTRACT
Inspired by cellular growth and self-organization, Neural Cellular Automata (NCAs) have been capable of ”growing” artificial cells into images, 3D structures, and even functional machines.
NCAs are flexible and robust computational systems but – similarly to many other self-organizing systems — inherently uncontrollable during and after their growth process.
We present an approach to control these type of systems called Goal-Guided Neural Cellular Automata (GoalNCA), which leverages goal encodings to control cell behavior dynamically at every step of cellular growth.
This approach enables the NCA to continually change behavior, and in some cases, generalize its behavior to unseen scenarios.
We also demonstrate the robustness of the NCA with its ability to preserve task performance, even when only a portion of cells receive goal information.
I NTRODUCTION
Cellular growth and morphogenesis has resulted in an incredibly diverse collection of multi-cellular organisms on Earth.
A single cell starts off a chain of replication and local interactions that eventually result in a fully grown organism.
Some of these organisms have learned to control and manipulate characteristics of their cells dynamically, resulting in diverse behaviors, such as the ability to change skin color, texture, and even shape (Figure 1).
Inspired by the flexibility and adaptability of these biological systems, we present Goal-Guided Neural Cellular Automata (GoalNCA), an artificial self-organizing system that is able to not only grow complex shapes through the local interaction of its cells, but also guide their behaviors dynamically.
rainfrog, can change its skin texture from smooth to spiky. Images from Gentle (2015).
1. C OMPUTATIONAL M ODEL : N EURAL C ELLULAR A UTOMATA
Cellular Automata (CAs), inspired by biological cellular growth and self-organization, are systems that apply local rules repeatedly on artificial cells that are typically represented as a 1 or a 0 in a grid.
Conway’s Game of Life, a simple CA system with only four discrete local rules, is Turing
Complete and able to represent many complex behaviors.
Neural Cellular Automata (NCAs) attempt to learn the local rules using compact neural networks and cell states, guided by an objective function (Mordvintsev et al., 2020; Nichele et al., 2017; Wulff & Hertz, 1992).
Each cell, now represented as a vector, is composed of target channels, a living channel that determines whether a cell should be alive or dead, and hidden channels which serve as a sort of cell memory, similar to the hidden state in an RNN (Sherstinsky, 2018).
This formulation has been shown to be effective at learning a variety of tasks, such as self-classifying 2D images (Randazzo et al., 2020) and physical 2D shapes (Walker et al., 2022), game levels (Earle et al., 2022), and regenerative functional 3D machines (Sudhakaran et al., 2021).
Typically, an NCA starts with a small set of initial “live” cells, or “seeds”, that eventually grow into a full artificial organism over time.
This seed can be a static vector (with the hidden channels being set to 1), or they can be learned in order to grow a diverse set of objects from a single NCA network (Palm et al., 2022; Frans, 2021).
However, controlling a self-organizing system composed of many independent components can be challenging.
As Kelly (2009) writes: “Guiding a swarm system can only be done as a shepherd would drive a herd: by applying force at crucial leverage points, and by subverting the natural tendencies of the system to new ends.” .
For example, NCAs can lack control over cell behaviour because the system’s stability and behaviors rely on repeated operations on initial seeds.
These cells are not trained to have dynamic behaviors during and after growth, so naively nudging a cell in a direction can cause a chain reaction that results in the collapse of the system.
Our proposed variation, GoalNCA, uses encoded goal vectors to perturb and guide these cells throughout growth, giving the ability to change cell behaviors dynamically.
The insights gained on NCAs (e.g. the system can be guided also if only a portion of the cells receive the goal information) could be useful to making self-organizing systems in general more controllable, opening interesting future research directions.
R ELATED W ORK : G UIDING S ELF -O RGANIZING S YSTEMS
Recently, ideas from collective intelligence and self-organizing systems have started to capture the interest of the larger machine learning community (Risi, 2021;
Ha & Tang, 2021).
However, selforganizing systems in general have shown to be inherently difficult to control (Kelly, 2009; Gershenson, 2007).
In this section we review the two most related approaches to the one introduced in this paper, which both deal with the control of self-organization in cellular automata.
The first one is Lenia , an extension of classical CA, moving a discrete grid-like space to a continuous one (Chan, 2018).
A computational system classified as artificial life —or virtual creature—, Lenia exhibits a series of remarkable properties reminiscent of life forms.
It thus constitutes an interesting abstract model to study the emergence of complex morphogenetic patterns in-silico .
Lenia has recently been extended for goal-guided locomotion (Hamon et al., 2022), using NCAs and curriculum learning with sampled goals to grow creatures that can traverse through obstacles (Forestier et al., 2020).
Our approach is similar in that we also grow creatures that are able to perform a variety of goals, but we explicitly incorporate goal encodings directly into the cells, and perturb the cell states to change behaviors dynamically.
The second one is Adversarial Perturbations in Neural Cellular Automata .
The idea of changing and guiding behaviors of NCA systems has been explored before in Randazzo et al.
(2021), where different methods of adversarial attacks on NCA states were evaluated.
One particularly effective method was multiplying trainable symmetric perturbation matrices with NCA cell states.
The authors showed that this method can change the cells’ behaviors, guided by a particular target (such as changing the colors of the cells).
To train these perturbation matrices, the authors first freeze a pretrained NCA’s weights, and then “fine tune” the perturbation matrices towards goals.
This setup has the advantage of training smaller parameters efficiently and re-using the existing trained NCA.
In our work, instead of freezing the NCA weights and training perturbation matrices, we train a goal encoder concurrently with the NCA in order to change cell behavior throughout the growth process.
We also opted for a simpler, more computationally efficient approach, where we instead add goal encoded perturbation vectors.
G OAL -G UIDED N EURAL C ELLULAR A UTOMATATA
Our system and network architecture is similar to that in Mordvintsev et al.
The main contribution of the presented approach is a goal encoder which adds updates to every living cell’s hidden states at each step in the growth cycle (Figure 2).
Surprisingly, this simple addition is effective in guiding cell behavior towards tasks such as morphing images and learning locomotion dynamics.
3. M ODEL D ETAILS AND T RAINING P ROCEDURE
Our GoalNCA model consists of two major components: A simple 3-layer MLP goal encoder and a typical NCA, similar to the one proposed in Mordvintsev et al.
Our NCA consists of a learnable 3 × 3 single convolution, followed by 3 1 × 1 convolutional blocks.
A single update step of our NCA is described in Figure 3.
We first take the cell states at step n , perturb all live cells’ hidden states by a encoded goal vector (Figure 2) and calculate updates via forward pass through our convolutional layers.
Before adding each update, we multiply it by a boolean stochastic mask, where each cell has a probability of being updated (which is referred to as cell fire rate ).
We then multiply an “alive mask”, which utilizes the cells’ living channels (3rd channel in the cell vector), where if all neighborhood cells have values < 0.1, the cell is considered dead and multiplied by 0.0.
initialize sample pool S with N initial seeds initial collection of goals G initialize NCA f θ with parameters θ initialize goal encoder e ϕ with parameters ϕ initialize min steps , max steps for training iterations t = 1 ...
T do sample NCA states from sample pool b t sample goals and targets g t , target t encode e t = e ϕ ( g t )
o t = b t for num steps ∼ [ min steps, max steps ] do o t =
f θ ( o t , e t ) end for θ ← θ - ∇ θ L ( o t , target t )
ϕ ← ϕ - ∇ ϕ L ( o t , target t ) replace b t with o t in S end for
Our training procedure, detailed in Algorithm 1, is also similar to that in Mordvintsev et al.
(2020), with the addition of goal sampling.
At the start of training, we populate the sample pool with initial seeds (single alive cell in the center with ones for its hidden values and living channel), which acts as a replay memory to ensure that the NCA is able to preserve the stability of its cells over many iterations.
At every training step, we uniformly sample and encode goals and NCA states from the sample pool, replacing 2 of the states with an initial seed.
The NCA states are updated for a random number of steps in the range [ min steps, max steps ] , guided by the goal encodings.
Next, we update the NCA and goal encoder via MSE loss between the new NCA states’ RGBA channels and the corresponding sampled goals’ target images.
Finally, we replace the older NCA states in the sample pool with the new NCA states.
E XPERIMENTS
For all experiments, we use the Adam Optimizer (Kingma & Ba, 2017) with 1e-3 learning rate.
We also normalize the gradients and clip NCA states between [-10, 10].
We use 64 channels for our 1 × 1 convolutional layers and for our 3 × 3 convolutional layer we have num cell channels ∗output channels.
We also trained for 100k training iterations on an NVIDIA 2080ti GPU.
Additional hyperparameter details shown in Table 1.
Code for both experiments can be found at https://github.com/shyamsn97/controllable-ncas
[Table]
[Figure]
Inspired by organisms that can change their shape and colors, we aim to learn a GoalNCA that is able to manipulate cells to morph into different emojis, shown in Figure 4.
For our goal encoder, we chose a simple 3-layer MLP that takes in a one-hot encoded vector, corresponding to a particular emoji.
We also experimented with a convolutional goal encoder that takes in the actual emoji, which resulted in more detailed grown emojis.
However, this also introduced over 100k additional parameters.
We train using min steps = 96 and max steps = 192 , but because of GPU limitations, we instead grow samples for a random number of steps between [48 , 96] twice and perform NCA updates for each.
We also set “cell fire rate” to 0.5, denoting that a cell has a 50% chance of receiving an update at every growth step.
We found that this stabilizes training and adds robustness to the system.
The GoalNCA learns to grow cells that are able to transform from any emoji in the dataset to another while keeping the cells stable, essentially learning a cell mapping between emojis.
The system can also interpolate between emojis, revealing a mixture of characteristics when cells are morphing between them (Figure 5).
4. C ONTROLLABLE L OCOMOTION
Cellular Automata systems, such as Lenia (Hamon et al., 2022; Chan, 2018) and Game of Life (Conway et al., 1970), can create diverse artificial creatures that can move while also keeping consistent shape.
In this experiment, we aim to extend this ability by learning a system that can not only grow and move artificial creatures, but control and learn general locomotion dynamics.
We choose the salamander emoji as our target and use 5 different directions (stay in place, up, down, left, right) as our goals.
We represent the directions as one-hot encoded vectors following a linear goal encoder.
In our implementation, we use the Pytorch Embeddings module (Paszke et al.
for simplicity.
The target for each direction is the salamander shifted by num steps/ 8 number of pixels in the particular direction, meaning that 8 NCA growth steps should result in a shift of one pixel.
In order to achieve smooth movement, at every training step we sample random directions and use a series of 4 targets that are gradually farther away.
This setup allows for the NCA to learn natural movement while also being able to retain the salamander’s shape (Figure 6).
The GoalNCA learns the fundamentals of movement and is able to guide the cells to move freely in diverse trajectories that had not been observed during training (Figure 7).
We observe that it is able to do this remarkably well, as it’s able to move in an erratic fashion for 1000s of steps without collapsing the system.
Also, because NCAs operate solely on local information, we can expand the grid size infinitely and thus discover an infinite number of trajectories.
4. R OBUSTNESS TO G OAL D ROPOUT
grid Self-organizing systems, leveraging local communication, are often inherently robust to adversarial modifications, such as observation dropout (Tang & Ha, 2021) and damage (Mordvintsev et al., 2020; Palm et al., 2022).
We find that in our controllable locomotion experiment, in which all cells received updates during training (cell fire rate = 1.0), the system is robust even if only a portion of cells receive an update during evaluation (Figure 8).
In the controllable locomotion experiment, we found that if cells only have a 75% chance of receiving updates per timestep, the grown salamander is still able to move and retain its rough shape.
At 25% - 50%, even though the salamander shape isn’t completely preserved, the cells still manage to move slightly in the goal directions.
This strengthens the case for developing dynamic behaviors in self-organizing systems, which can enable them to thrive in imperfect environments with missing information.
D ISCUSSION AND F UTURE WORK
NCAs and self-organizing systems are incredibly powerful and can be robust to damage, but they lack flexibility and control over their cells after the growth process is complete.
Here we show that by simply adding goal encoded perturbations to hidden cell states, our GoalNCA approach is able to guide cells towards diverse and robust behaviors throughout the growth cycle.
We also notice that, as a byproduct of the training procedure, the GoalNCA ends up learning a goal embedding space, able to group similar goals closer together and keep others farther apart (Figure 9).
The GoalNCA was able to effectively guide cells towards desired behaviors while only requiring an additional small linear goal encoder ( < 2k params) for both experiments.
In the morphing image experiment, we also tested a convolutional encoder with over 100k params, which was able to grow and morph sharper images with the tradeoff of a large increase in parameters.
One limitation with the goal encoders presented is that they are not generative models, in that we cannot learn a goal distribution.
This approach can be incredibly powerful, as we would be able to sample goals and diverse behaviors.
This would be similar to the Variational Neural Cellular Automata (VNCA) introduced in Palm et al.
(2022), which samples seed states in order to grow diverse images.
Our goal sampling method, essential in order for the GoalNCA to learn behaviors associated with goals, is naive and can be optimized.
At every training step, we sample goals uniformly with replacement.
Because we used a small goal space, our system was exposed to enough samples of each goal during training.
However, in cases of large goal spaces, it is not guaranteed that goals will be observed equally and some may even be overlooked entirely.
In the future, we hope to improve our goal sampling methods with ideas from curriculum learning.
One possibility could be Intrinsically
Motivated Goal Exploration Processes (IMGEP)
(Forestier et al., 2020), where goals are dynamically chosen based on an exploration / exploitation procedure, allowing for efficient exploration and learning with large goal spaces.
In the future, we plan to extend our approach to more complex tasks that require incorporating dynamic feedback in self-organizing systems.
We are particularly interested in guiding cells that actually react to their environment by leveraging goals that represent environmental states.
This could prove useful when designing self-organizing controllers, which need to be able to incorporate environment information.
A CKNOWLEDGEMENTS