GLSO: Grammar-guided Latent Space Optimization for Sample-efficient Robot Design Automation
Authors: Jiaheng Hu
Julian Whitman
Howie Choset
Venue: CoRL 2022 Poster (2022)
Keywords: Robot Design Automation
Latent Optimization
Graph Grammar

Abstract: Robots have been used in all sorts of automation, and yet the design of robots remains mainly a manual task. We seek to provide design tools to automate the design of robots themselves. An important challenge in robot design automation is the large and complex design search space which grows exponentially with the number of components, making optimization difficult and sample inefficient. In this work, we present Grammar-guided Latent Space Optimization (GLSO), a framework that transforms design automation into a low-dimensional continuous optimization problem by training a graph variational autoencoder (VAE) to learn a mapping between the graph-structured design space and a continuous latent space. This transformation allows optimization to be conducted in a continuous latent space, where sample efficiency can be significantly boosted by applying algorithms such as Bayesian Optimization. GLSO guides training of the VAE using graph grammar rules and robot world space features, such that the learned latent space focus on valid robots and is easier for the optimization algorithm to explore. Importantly, the trained VAE can be reused to search for designs specialized to multiple different tasks without retraining. We evaluate GLSO by designing robots for a set of locomotion tasks in simulation, and demonstrate that our method outperforms related state-of-the-art robot design automation methods.

GLSO: Grammar-guided Latent Space Optimization for Sample-efficient Robot Design Automation
Jiaheng Hu Roboties Institute Carnegie Mellon University j jiahengh@nddrew. cmu. cmu. edu
Julian Whitman
Department of Mechanical Engineering Carnegie Mellon University j jwhi tman@andrew, cmu. edu
Howie Choset
Robotics Institute
Carnegie Mellon University
cm.. edu
A Abstract: Robots have been used all sorts of automation, and yet the design of robots remains mainly a manual task. We seek to provide design tools to automate the design robots themselves. An important challenge in robot design automa- tion is the large and complex design search space which grows exponentially with the number of components, making optimization difficult and sample inefficient. In this work, present Srammar-guidee Latent Space Optimization (GLSO), a framework that transforms design automation into a cow-dimensional continu- optimization problem by training a graph variational autoencoder (VAE) learn a mapping between the graph-structured design space and a continuous la- tent space. This transformation allows optimization be conducted in continu- ous latent space, where sample efficiency can be significantly boosted by applying algorithms such Bayesian Optimization. GLSO guides training of the VAE us- ing graph grammar rules and robot world space features, such that the learned latent space focus on valid robots and easier for the optimization algorithm explore. Importantly, the trained VAE can be reused to search for designs special- ized multiple different tasks without retraining. We evaluate GLSO by design- ing robots for a set of locomotion tasks in simulation, and demonstrate that our method outperforms related state-of-the-art robot design automation methods.
Keywords: Robot Design Automation, Latent Optimization, Graph Grammar
1
Introduction
Robot design automation aims to discover robot body structures that optimize a given objective. While this subject has been long-studied [1. 3. 4, 5], the problem is difficult due the large search space and the computational expense involved in evaluating candidate designs. Classic de- sign automation approaches resort to discrete black-box optimization techniques such as Genetic Algorithms (G,)[1], Genetic Programming (GP) [3], and Random Graph Search (RGS)[4). While these methods typically work well when the objective function is inexpensive to evaluate (e.g., ob- taining the locomotion speed of simple robot in simulation with pre-defined controller), they require a large number of objective function evaluations and are not suitable for situations where evaluation is expensive (e.g., through real-world evaluation, or creating customized dynamic motion plans for each new design). Recent works utilize graph grammar rules to confine the search space [5, 1]. such that the number of evaluations be reduced. However, they still operate in high- dimensional combinatorial search space and require a considerable number of sample evaluations.
In this work, we introduce Hlammar-guided Latent Space Optimization (GLSO), framework for sample-efficient robot design automation. Given a robot design space defined by the possible combi- nations of a set of discrete primitive components, GLSO first learns low-dimensional, continuous
6th Conference on Robot Learning (CoRI. 2022), Auckland, New Zealand.

---

@-G-O
Graph Grammar Rules Dara Generation
@
G
Geraer
Robot World Spuce Features
1?"
appoorg
Doods
Input Design
Low Dimensional
Reconstructed Design
Graph Representation
Latent Representation
Graph Representation
Our framework begins by collecting dataset robot designs based on a set of graph grammar rules, shown the top left. This process of enumerating designs is computationally inexpensive, as no controller is needed. The collected data (example on bottom left) is subsequently used train graph variational autoencoder (VAE), which defines mapping between low dimen- sional continuous latent space and the combinatorial design space. A property predictor (shown as the green trapezoid) simultaneously trained to predict the world space features grounding of the robots from the latent vector, in order to encourage physically similar robots to be grouped together in the latent space. After the VAE is trained, optimization be performed in the latent space in search of high-perforning designs. This VAE can further be used for multiple distinct tasks without the need for retraining.
Figure 1:
representation of the robot design space through unsupervised learning. The learned representation allows us then convert the combinatorial design automation search into a continuous optimization problem, where we apply sample-efficient Bayesian Optimization (BO) [6] to search in the latent space for tigh-performing designs. GLSO uses graph variational auto-encoder (VAE) learn mappings between the design space and latent space. Importantly, the learned latent space can be used optimize designs for multiple different tasks without the need for retraining.
GLSO is inspired by recent advancements molecule synthesis [7, 8]. However, unlike molecule optimization, where existing large-scale datasets such ZINC [9] are readily available supervise training, the domain of robot design has no such dataset available. Instead, GLSO generates training data by leveraging graph grammar rules [5], which confine the search space and implicitly inject prior into our learned latent representation.
An additional challenge associated with robot design automation is the potentially "chaotic" objec- tive function, i.e. that two structurally similar robots may have very different performance for a given task. For example, consider the designs shown in Fig. 3b: they have nearly the same design graph, but they all have drastically different locomotive performance. We address this problem by including additional objective in VAE training, which ensures that neighboring designs in the latent space not only have similar graphs, but also share similar world space features, such as the bounding box of the robot or the end-effector workspace. That is, addition the conventional VAE encoder/decoder reconstruction loss, GLSO simultaneously trains a neural network to learn world space feature grounding the robot designs, which we called the property prediction network. The complete pipeline can be seen in Fig. 1!.
We evaluate GLSO by designing robots for set of locomotion tasks, with component library that includes different joints and links with various rotational angles and axes, sizes, and weights. Our method outperforms state-of-the-ar robot design automation methods, consistently identifying higher-performing designs when given the same number of sample evaluations?.
1 Uppeer portion the figure adopted from Xu et al. [10] with author's 's approval "The code used in this work is available attps::/gittie httpaaa...ooomaaaoeeaeeeee Jias//gl/nnng/./.......

---

(a) Frozen Lake Task
(b) Flat Terrain Task
(c) Ridged Temain Task
Wall Terrain Task
Figure 2: We evaluate our framework on set of locomotion tasks shown in the figures. The above images showcase the best design identified by our method for four tasks, A video showing the motion of these robots is provided in the supplement.
Related Work
2.1 Evolutionary Algorithms
A de facto choice for robot design automation is population-baset stochastic optimization algo- rithms, such evolutionary algorithms (EAs), due to their ability search combinatorial design spaces [1, 2, 4, 11, 12, 13, 14, 15]. Specifically, EAs maintain and update population of candidate solutions through set of operations (e.g., mutations, cross-over, evaluation, and elite selection) to search for high-quality designs. However, EAs require a large number of objective function evalua- tions, and can quickly become computationally infeasible when the objective function evaluation is costly, such when the determining the optimal controller can be computationally burdensome, or when the evaluation needs take place in real world instead of in simulation.
2.2 End-to-end Design Generation
Another class of design automation methods uses learning-basse based approaches to train design gen- erator (typically the form of a neural network) to predict suitable designs for given task [16, 17, 18]. Learning-based approaches are particularly promising for field deployment due to their near real-time execution speed. However, they require computationally expensive training pro- cedures in which many different combinations of tasks and designs must be tested. During this training, they apply strong assumptions about the set of tasks for which robot will be special- ized. Furthermore, recent works also assume restrictive design space (e.g.. fixed-topologe graph, binary 3D voxel grid), and is unclear how end-to-end approaches can extended to more complex designs.
2.3 RoboGrammar
Graph grammar rules provide an effective means to restrict a combinatorial design space, which improve optimization efficiency, especially when the design space is high-dimensiooal [5, 1]. In robotics, graph grammars have been applied to model both physical structure as well as control laws [19, 20, 21]. More recently, RoboGrammar [5, 10] introduced mar rules for robot design automation. RoboGrammar operates on graphs composed of terminal and non-terminal symbols, where non-terminal symbols temporary nodes and terminal symbols correspond to physical robot components, Starting from nonterminal symbol "S", RoboGrammar iteratively applies available rules until reaching acyclic graph of terminal symbols. The final graph corresponds to robot design, where the hardware components are represented as nodes and the physical links are represented as edges. The upper left corner of Fig. demonstrates the set of rules proposed by RoboGrammar. To optimize the robot designs based on the grammar rules, RoboGram- mar proposed Graph Heuristic Search (GHS), which operates on search grammar rules. The search is conducted in an A like manner, where a heuristic function in the form of graph neural network [22] trained during the search process. For more details about the RoboGrammar method, see Zhao et al. [5].
of recursive graph gram-
defined by the graph
2.4 Latent Space Optimization
Latent space optimization (LSSO) is a framework designed to extend continuous optimization tech- niques to combinatorial search spaces [23, 24). LSO first learns a generative model (typically

---

variational autoencoder) [25] 9: 2 - X' which maps data points from a continuous latent space Z the combinatorial search space X. Continuous optimization can subsequently be performed in the latent space 2 using standard continuous optimization algorithms such as BO. LSO has been shown to an effective framework for domains involving discrete data, including natural language [26], arithmetic expressions [7], programs generation [27] and molecules synthesis [28, 8]. In the domain of robotics, Spielberg et al. [29, 30] facilitated design and control co-optimization for soft robots by learning low-dimensional latent representation. For graph- representation of robot designs, Kim et al. [31] made initial progress in learning a latent representation of robot morphology. but only with small set of serial-chate topology designs, and did not perform design optimization within that latent space.
2.5 Graph Neural Networks for Robotics
Graph Neural Networks (GNN) [22] are a neural network architecture which operates on graph- structured inputs. For robots with graph-based representation, GNNs have been used to control different designs by abstracting the robots as graphs [32, 33, 35]. In our work, we utilize GNNs a premutation-invaree process to learn compressed continuous representation of graph-structured robot designs.
3 GLSO: Learning Latent Design Representation
GSLO trains a generative model that defines a mapping between a continuous latent space and the discrete design space. Similar previous LSO works, method model. The VAE consists of an encoder (section 3.2) and a decoder (section 3.3). In addition to the encoder and decoder, co-train property prediction network (section 3.4) to predict world space features of the robots. The goal of co-training the property predictor is to encourage designs with similar physical properties to be close together in the latent space; we find that in turn, this makes downstream design optimization in the latent space more efficient. The training data for the VAE generated through recursively applying set of graph grammar rules (section 3.1). The grammar ensures that the training designs are valid, which implicitly bias the mapping from latent space to design space towards promising robots. Importantly, the trained VAE can be reused to search for designs specialized to multiple different tasks without retraining.
VAE as generative
Notation. Each design is represented as acyelic graph g = (V, E) where V corresponds to the hardware components (nodes) and E the connectivity between them (edges). Note that an acyclic design graph can also be viewed as tree, where the root node corresponds to the one component on the body, and the each of the leaf nodes correspond to the robot's end-effectors. We use j, & to refer to nodes indices, and define (i) the neighbor of node i. We denote the sigmoid function as a (.) and the ReLU function as r-), and trainable weights in the models as W and U.
3.1 Data Collection via Graph Grammar
For given set of robot hardware components, GSLO begins by collecting a dataset of robots com- posed of these hardware components. This dataset will then be used train the graph VAE. We desire dataset that covers much of the design space, while containing few invalid designs.
One key idea behind GSLO is that graph grammar rules serve as a guiding tool to generate large dataset of designs, due to their ability to filter out invalid designs and to produce a tractable and meaningful subspace of designs. Specifically, we adopt the set of grammar rules proposed in RoboGrammar [5]. To generate a random design using RoboGrammar, we start from the start symbol "S" and randomly apply valid grammar rules until the design graph consists of only terminal nodes. Each terminal node corresponds to specific hardware component, and therefore each design graph has one-to-one correspondence with a physical robot. We additionally collect each design's contact locations when that design is placed at rest on flat ground in simulation. These contact locations will be used to train the property predictor in section 3.4. Since the data collection process does not involve creating controllers, it can be readily applied to a new set of grammar rules / hardware components. In our experiments, the data collection process produces around 5c.5 designs in approximately four hours, and could be parallelized for additional speedup.

---

(a) Latent Space with PPN
(b) Latent Space without PPN
Figure 3: Visualization two different latent spaces (a) when the VAE co-trained with the prop- erty predictor, and (b) without the property predictor. Here we project the latent points into a two- dimensional space, for visualization purposes, by taking the first two principal components. The color of the points correspond to its performance for given task (here, traversing flat terrain), where a darker color denotes worse performance. We additionally show designs neighboring the same robot (with red border) in the different latent spaces. We see that the addition of the property predictor implicitly encourages designs with similar performance to be grouped together, which we find results in more efficient optimization.
3.2
Robot Encoder
Since the robots are represented graphs, we use a graph autoencoder to encode the robots via graph message passing neural network (MPNN)(36, 8]. MPNNs operate graph-structure inputs, where hidden representations of the graph nodes are updated iteratively messages sent along graph edges [22]. Each node V; contains one-hot feature vector z; indicating its type. We denote a message from a node U; to Uy as m,,j, and mj.; vice versa.
At each message passing iteration, messages are updated as:
m,j = G-U (z,, (mai)keN((/s)
(I)
where GRU refers Gated Recurrent Unit [37] adapted for graph message passing. For detailed
formula of the GRU update, please refer to the supplementary material.
After iterations of message passing, obtain the latent representation of each node by aggregating its inward messages,
h, (WW
Σ
U'ma))
kEN()
The robot graph representation is calculated as the sum of the latent representation of the leaf nodes,
hgg Σ Meaf-
Finally, the mean jgg and variance og of the variational posterior approximation are computed from he by applying two separate affine layers, where the latent vector 2g is sampled from Gaussian distribution 2g ~ NN(g, og)
3.3
Robot Decoder
The robot decoder maps the continuous latent vector zg back to robot design tree G by sequentially adding nodes to a partial design tree depth-first order, starting from the root. For every visited node, the decoder first predicts whether it has children to be generated. If so, and attached to the current node, with its node type predicted by the decoder. The messages of existing nodes will not be cleared during decoding. This procedure is recursively applied to the newly created nodes until the current node has no more children to generate, where the decoder backtracks its parent node and repeat this process.
node is created
The decoder makes predictions based on message propagation in the current partial tree at each time step. The messages are propagated using the same GRU structure as in the encoder. The decision
5

---

of whether to create new node at each time step is predicted as probability Pe based on the inward messages ha latent vector 2g and the node features 2;,
Pt ou (Wif,, + W5 + W Σ h).,)))
(4)
k
For a new node j created from parent i, the label oj is predicted with,
fj (ettmax(U">TNAAA + Wahhs))-
(5)
The decoder is trained maximize the reconstruction likelihood p(g]=g). Let Pr. and dj be the ground truth values of pr and q₁ respectively, the decoder loss is:
(a(g)= = - (Pt Σ Ciag,
(6)
where Le and L are cross-entropy losses. Similar to language generation, we apply teacher forcing [38] during training, where ground truth topology and labels are used at each step for prediction.
3.4 Property Predictor
We hypothesize that learned latent space in which designs with similar capabilities are close to- gether, and which contains few invalid designs, will allow for more efficient optimization. To obtain such a latent space, we co-train property prediction network (PPN) with the VAE. The PPN maps the mean of the variational encoding distribution fág to the corresponding robot's world space fea- tures, which implicitly encourages robots with similar world space features to have similar latent representations.
this work, since we are primarily experimenting with locomotion tasks, we de- fine the world space features vector Ucontctt consisting of the robot's 2D contact locations flat ground. The contact locations are sorted based on their x values and zero padded to a fixed length to form Vountact- Voomiact is collected together with the robot data described in section 3.1. The PPN is trained to minimize the mean square error between the predicted contact vector Ocontact and the ground truth contact vector Ucontact, i.c. LPPN Training:
Ocontact
Vcontact
The final loss of the graph VAE is weighted sum of the decoder loss La, the PPN loss CPPN, and the KL-divergence LKL between the variational posterior and the prior latent distribution A (0, I):
Loee La + LKL + ALPPN
(7)
We minimize Leae using gradient descent. Training took around five hours on NVIDIA GTX 1070 graphics card. We provide the detailed training hyperparameters, as well as visualization of interpolation in the learned latent space, in the supplementary materials.
4 GLSO: Optimization in the Learned Latent Space
The trained VAE associates each robot design with a latent vector, given by the mean of the varia- tional encoding distribution fag. We can use the VAE to transform robot design automation from combinatorial optimization problem into a continuous one, We then apply Bayesian Optimization to search for the latent vector where the associated robot design has the highest performance.
4.1 Controller & Evaluation
To evaluate the performance of given robot structure, need derive its controls. This reveals one challenge in robot design optimization- hidden inside each evaluation of the design optimization objective function trajectory or policy optimization problem, which itself is often computation- ally expensive. In this work, we utilized model predictive control (MPC) [39] to create controllers for different robot structures across different terrains. MPC predicts future behaviour using a model of the system, optimizes controls for finite horizon, executes small number of the optimized control, and then replans. Specifically, we used model predictive path integral control (MPPI) [40], sampling based MPC method for controlling the generated robot designs. Our implementation fol- lows [5]. Each robot controlled a simulation implemented using the Bullet Physics library [41]. We adopt the same objective function as in [5), where robots are rewarded for forward progression initial orientation. Computing the trajectory of each robot takes 30 60 sec, and is the primary computational bottleneck of the design optimization.
while maintaining

---

GLSO
GHS
MCTS
Random
GA
3.25
3.45
5.45
5.00
5,00
5.00
4.75
4.50
4.25
4.00
4.0
(a) Frozen Lake
(b) Flat Terrain
(c) Ridged Terrain
(d) Wall Terrain
Figure 4: Comparison against related methods. The solid line shows average over different random seeds, and the error band represents the maximum and minimum. In each task, GLSO produces
higher-reward designs within 500 steps (design samples).
4.2 Bayesian Optimization in Latent Space
We use Bayesian Optimization (BO) [6], a sample-efficient black-box optimizer, perform opti- mization in the latent space. Specifically, our BO implementation uses Gaussian Processes (GP) [42] build model of the objective function, which includes both the current estimation of the function and the uncertainty around the estimation. We then use expected improvement (EI) as the acquisition function to determine the optimal point sample next. The sampled point is subsequently evaluated and used to update the GP model. This procedure is repeated until we reach the computation limit, defined as the maximum number of objective function evaluations. We additionally apply domain reduction [43], where the bounds of the latent space are contracted during the optimization to reduce oscillation. The hyperparameters of the BO are provided in the supplementary material.
Experimental Results
We evaluate our method by generating designs for the following four locomotion tasks, each defined by its corresponding terrain:
Flat Terrain: A flat plane with a friction coefficient of 0.9.
Frozen Lake Terrain: A flat plane with low friction coefficient of 0.05. Ridged Terrain: Includes hurdles that the robots must jump or crawl over.
Wall Terrain:
Includes high walls placed in slalom-like manner.
We report comparisons to previous approaches well as ablated versions of our approach. For each each task, allow each method a maximum of 500 objective function evaluations, i.c., control and trajectory optimization for up to 500 designs. Images of the tasks and optimized designs are shown in Fig. 2. Videos of their motions are the supplementary material.
The reward of each robot is measured as described section 4.1.
5.1 Comparisons and Baselines
To demonstrate the effectiveness of GLSO, we benchmark our method against the following:
Random Search: Random designs are generated using the graph grammar rules.
Monte Carlo Tree Search (MCTS): baseline adopted from [5], performing Monte Carlo Tree Search (MCTS) [44] on search tree defined by the graph grammar rules.
Graph Heuristic Search (GHS): A design automation method proposed in [5]. GHS performs search on the same graph grammar search tree as does MCTS. Our implementation follows [5].
Genetic Algorithm (GA): Our implementation of Genetic Algorithm (GA) follows [4], where graph mutation with uncertainty is used to mutate the population each iteration. used a population size of 50 and evolved them until the number of evaluation limit was reached. Note that unlike the other comparison methods, GA does not operate in the grammar space, as crossover and mutation operations not clearly defined for the graph grammar proposed by Zhao et al. [5].
7

---

GLSO
GLSO _L5O_nopred
GLSO nogram
3.25
6.0
5.25
3.00
5.00
5,00
4.75
Rowerdd
4.75
4.75
4.50
5.0
4.50
4.50
4.25
4.25
4.25
4.00
4,00
100
200
300
400
500
100
200
100
400
500
100
200
300
400
500
100
200
300
400
500
3top
Stop
30ep
Step
(a) Frozen Lake
(b) Flat Terrain
(c) Ridged Terrain
(d) Wall Terrain
Figure 5: Comparison of GLSO with ablated variants. The solid line shows average over 3 different random seeds, and the error band represents the maximum and minimum. These results show that both the graph grammar and the property predictor are important to GLSO.
The optimization curves for each of the tested algorithms is shown Fig. 4. We found our method outperform all comparison methods and baselines.
5.2 Ablation Studies.
We performed ablation studies to investigate the effects of the graph grammar rules and the property prediction network. Results shown in Fig. 5. In the "GLSO_no _nogram" test, removed the graph grammar rules during data collection, and the VAE training set is created through random generation of topology and node labels. In the "GLSO nopred" test, removed the property prediction net- work during VAE training. We found that both the graph grammar and property predictor are crucial the success of GLSO. Additionally, network influences the latent space. Fig. 3 presents points in the latent space created by training the VAE with and without the property prediction.
created visualization of how the property prediction
Limitations & Conclusions
In this work, we proposed GLSO: sample-efficient approach for optimizing robot designs based on latent encoding. We believe that our approach general to the domain of robot design automa- tion, and represents step towards achieving efficient robot design automation, in particular when evaluation of each design is computationally expensive.
An important limitation our approach the requirement for a pre-defined set of graph grammar rules well world space features. Extending this work to different of hardware components and tasks will likely take some expert knowledge or domain intuition. Secondly, while we ultimately aim extend design automation to real-world tasks, the designs in our current experiments do correspond to existing physical hardware. An important next step will be to demonstrate GLSO on set of modular robotic hardware components. Furthermore, for the evaluation steps to take place in real life, it will be necessary to further improve the sample efficiency of our methods, as 500 sample evaluations will likely prove to too many designs prototype with real-worth hardware.
Besides the limitations above, there are also a few potential extensions of this work. Firstly, the learned latent representation of the designs may have other besides design automation. For example, down-stream learning tasks that require outputting a rohot design [18, 16) may benefit from learning to output continuous latent vector instead of discrete graph. Secondly, recent advancement on latent space optimization, such as leveraging decoder uncertainty [45] and weighted retraining [23], may be beneficial our optimization scheme. A future direction would be to explore how these techniques can be adapted to improve the performance of GLSO.

---

Acknowledgments
We thank the anonymous reviewers for their helpful comments on improving the paper. We thank the member of Biorobotics lab for their valuable feedback on idea formulation and manuscript.
References
[1] S. Homby, H. Lipson, and B. Pollack. Evolution of generative design systems for modular physical robots. In Proceedings 2001 ICRA. IEEE Int. Conf. on Robotics and Automation (Cat. No. 01C1137144), volume 4, pages 4146-4151, IEEE, 2001.
[2] N. Cheney, MacCurdy, J. Clune, and Lipson. Unshackling evolution: evolving soft robots with multiple materials and a powerful generative encoding, ACM SIGEVOlution, 7(1):11-23, 2014.
[3] H. Lipson. How draw straight line using a gp: Benchmarking evolutionary design against 19th century kinematic synthesis. In Late Breaking Papers at the 2004 Genetic and Evolution- ary Computatioon Conference, Seattle, Washington, USA, volume 26, 2004.
[4] Wang, Zhou, Fidler, and J. Ba. Neural graph evolution: Towards efficient automatic robot design. In Int. Conf. Learning Reprexentations, 2018.
[5] A. Zhao, J. Xu, M. Conaković-Lukovia J. Hughes, A. Spielberg, D. Rus, and W. Matusik. Robogrammar: graph grammar for terrain-optimized robot design. ACM Trunsactions on Graphics (TOG), 39(6):1-16, 2020.
[6] M. Pelikan, D. E. Goldberg, E. Cantú-Paz, et al. Boa: The bayesian optimization algorithm. In Proceedings of the genetic and evolutionary computation conference GECCO-99, volume 1, pages 525-532. Citeseer, 1999.
[7] M. J. Kusner, B. Paige, and J. M. Hernández-Lobato. Grammar variational autoencoder. In Int. Conf. on machine learning. pages 1945-1954. PMLR, 2017.
[8] W. Jin, R. Barzilay, and T. Jaakkola. Junetion tree variational autoencoder for molecular graph generation. In Int. Conf. on machine learning, pages 2323-2332. PMLR, 2018
[9] J. J. Irwin, T. Sterling, M. M. Mysinger, E. S. Bolstad, and R. G. Coleman. Zine: free tool to discover chemistry for biology. Journal of chemical information and modeling, 52(7): 1757-1768, 2012.
[10] J. Xu, A. Spielberg, A. Zhao, D. Rus, and W. Matusik. Multi-objective graph heuristic search for terrestrial robot design. In 2021 IEEE Int. Conf. on Robotics and Automation (ICRA), pages 9863-9869. IEEE, 2021.
[11] O. Chocron and P. Bidaud. Evolutionary algorithms in kinematic design of robotic systems. In Proc. of the 1997 IEEE/RSJ Int'l I Conf.on Intelligent Robots and Systems, volume 2, pages 1111-1117, 1997. 1 69956655148.
[12] C. Leger. Darwin2K: An evolutionary approach to automated design robotics, volume 574. Springer Science & Business Media, 2012.
[13] A. Gupta, S. Savarese, S. Ganguli, and L. Fei-Fei. Embodied intelligence via learning and evolution. Nature communications, 12(1):1-12, 2021.
[14] Bhatia, Jackson, Y. Tian, J. Xu, and W. Matusik. Evolution gym: large-scale benchmark for evolving soft robots. Advances in Neural Information Processing Systems, 14:2201-22111 2021.
[15] K. Sims. Evolving virtual creatures. In Proceedings of the 2 annual conference on Computer graphics and interactive techniques, pages 15-22, 1994.
[16] J. Hu, J. Whitman, M. Travers, and H. Choset. Modular robot design optimization with gen- erative adversarial networks. Proceedings 2022 ICRA. IEEE Int. Conf. on Robotics and Automation, 2022.

---

[17] H. Ha, S. Agrawal, and S. Song. Fit2For:: 3D generative model for robot gripper form design. In Conf. on Robotic Learning (CoRL), 2020.
[18] Whitman, R. Bhirangi, M. Travers, and Choset. Modular robot design synthesis with deep reinforcement learning, In Proc. of the AAAI Conf. on Artificial Intelligence, 2020.
[19] M. Lau, A. Ohgawara, J. Mitani, and T. Igarashi. Converting 3d furniture models to fabricatable parts and connectors. ACM Transactions on Graphics (TOG), 30(4):1-6, 2011.
[20] B. Smith, A. Howard, J.-M. McNew, J. Wang, and M. Egerstedt. Multi-robot deployment and coordination with embedded graph grammars. Autonomous Robots, 26(1):79-9)) 2009.
[21] E. Klavins, R. Ghrist, and D. Lipsky. Graph grammars for self assembling robotic systems. In IEEE Int. Conf. on Robotics and Automation, 2004. Proceedings. ICRA '04 2004, volume 5, pages
IEEE, 2004.
[22] Z. Wu, S. Pan, F. Chen, G. LLong C. Zhang, and S. Y. Philip. A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems, 32(1): 2-24, 2020.
[23] A. Tripp, E. Daxberger, and J. M. Hernández-Looato. Sample-efficient optimization in the la- tent space of deep generative models via weighted retraining. Advances Neural Information P Processing Systems, 33:1 1259-11272 2020.
[24] X. Pan, A. Garg, A. Anandkumar, and Y. Zhu. Emergent hand morphology and control from optimizing robust grasps of diverse objects. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 7540-7547. IEEE, 2021.
[25] D. P. Kingma and M. Welling. Auto-encoding variational bayes. arXiv:1312.6114 14, 2013.
arXiv preprint
[26] Bowman, L. Vilnis, O. Vinyals, A. M. Dai, R. Jozefowiez, and S. Bengio. Generating sentences from continuous space. arXiv preprint arXiv: arXiv:/511.06349, 2015.
[27] H. Dai, Y. Tian, B. Dai, S. Skiena, and L. Song. Syntax-directed variational autoencoder for structured data, arXiv preprint arXiv: /00::08088888 2018
[28] R.
N. Wei, D. Duvenaud, J. M. Hernández-Lobato, Sánchez- T. D. Hirzel, R. P. Adams, and A. Aspuru- Guzík. Automatic chemical design using a data-driven continuous representation of molecules. ACS central science, 4(2):268-277) 2018.
Lengeling, D. Sheberla, J.
[29] A. Spielberg, A. Zhao, Y. Hu, T. Du, W. Matusik, and D. Rus. Learning-in-the--ooy optimiza- tion: End-to-end control and co-design soft robots through learned deep latent representa- tions. Advances in Neural Information Processing Systems, 32, 2019.
[30] A. Spielberg, A. Amini, L. Chin, W. Matusik, and D. Rus. Co-learning of task and sensor placement for soft robotics. IEEE Robotics and Automation Letters, 6(2):1208- 2021.
[31] Kim, J. Park, Choi, and Ha. Learning robot structure and motion embeddings using graph neural networks. arXiv preprint arXiv:2 2021.
[32] Y. Yuan, Y. Song, Z. Luo, W. Sun, and K. Kitani. Transform2act: Learning transform-and- control policy for efficient agent design. arXiv preprint arXiv:21 2021.
[33] D. Pathak, C. Lu, T. Darrell, Isola, and A. A. Efros. Learning to control self assebassemlling morphologies: study of generalization via modularity, Advances in Neural Information Pro- cessing Systems, 32, 2019.
[34] Wang, R. Liao, J. Ba, and S. Fidler. Nervenet: Learning structured policy with graph neural networks. In International conference on learning representations, 2018.
[35] W. Huang, I. Mordatch, and D. Pathak. One policy to control them all: Shared modular policies for agent-agnostic control. International Conference on Machine Learning, pages 4455 4464. PMLR, 2020.
10

---

[36] H. Dai, B. Dai, and L. Song. Discriminative embeddings of latent variable models for struc- tured data, In Int. Conf. on machine learning. pages 2702-2711. PMLR, 2016.
[37] Chung, C. Gulcehre, K. Cho, and Bengio. Empirical evaluation of gated recurrent neural networks sequence modeling. arXiv preprint urXiv: 2014.
[38] R. Williams and D. Zipser. A learning algorithm for continually running fully recurrent neural networks. Neural computation, 1(2):270-280, 1989.
[39] C. E. Garcia, D. M. Prett, and M. Morari. Model predictive control: Theory and practice a survey, Automatica, 25(3):335-348, 1989.
[40] G. Williams, Drews, B. Goldfain, J. M. Rehg, and E. Theodorou. Aggressive driving with model predictive path integral control. In 2016 Int. Conf. on Robotics and Automation, 2016.
[41] E. Coumans. Bullet physics simulation. In ACM SIGGRAPHI 2015 Courses, page 1. 2015.
[42] E. Schulz, M. Speckenbrink, and A. Krause. A tutorial on gaussian process regression: Mod- elling, exploring, and exploiting functions. Journal of Mathematical Psychology, 2018.
[43] N. Stander and K. Craig. On the robustness of a simple domain reduction scheme for simulation-based optimization. Engineering Computations, 2002.
[44] C. B. Browne, E. Powley, D. Whitehouse, S. M. Lucas, P. Cowling, Rohlfshagen, S. Tavener, D. Perez, S. Samothrakis, and S. Colton. A survey of monte carlo tree search methods. IEEE Transactions on Computational Intelligence and Al in games, 4(1): 4-3, 2012.
[45] Notin, J. M. Hemández-Lobato, and Y. Gal. Improving black-box optimization in vae latent space using decoder uncertainty. Advances in Neural Information Processing Systems, 2021.


Reviews:
[{"review_id": "xVq-1dS8ZR", "reviewer": "Reviewer_9cM8", "summary": "The paper describes an approach to improving the computational efficiency of optimization over discrete, grammar-constrained robot design spaces. Fundamental to this approach is using a variational autoencoder to learn a continuous, low-dimensional representation of the design space that can be optimized using Bayesian optimization. Experimental results show advantages over existing baselines along with the value of training the VAE with an additional objective that encourages the learned representations of similar designs to be nearby in the latent space.\n\nThe idea of using a learned latent representation to improve the efficiency of optimization is interesting and offers potential advantages over standard methods that involve combinatorial search. As such, this work may prove useful to others in the robot learning community who are interested in design-control optimization. However, the advantages of this approach in terms of computational efficiency are not fully clear.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper considers the problem of learning the physical design of (legged) robots optimized according to a given (locomotion) task. Building off of recent work, the paper adopts a graph-based representation of the design space that is governed by a grammar. While the grammar has been shown enable heuristic search strategies, the *discrete* design space remains combinatorially large.\n\nThe key contribution of this paper is the proposal to learn a low-dimensional, *continuous* design representation that affords efficient optimization using standard optimization methods (i.e., Bayesian optimization). This representation is learned using a graph neural network-based variational autoencoder (VAE) that seeks to reconstruct a collection of input graphs that model designs randomly generated according to the grammar. In addition to the decoder, the network includes a \"property predictor\" head that takes as input the learned latent representation and outputs the 2D coordinates of a design's ground contact points. The VAE is trained using a loss that combines the standard VAE objective with the MSE of the predicted coordinates. Having learned the latent representation, the authors use Bayesian optimization to reason over candidate designs and use MPC to control each design.\n\nThe method is evaluated in simulation on a series of locomotion tasks and compared to several baselines, including the graph heuristic search proposed in the RoboGrammar paper that this work builds on. The paper then ablates the contributions of the VAE prediction objective as well as the grammar.\n\n\nSTRENGTHS\n\n+ The paper considers an important problem that has attracted a fair bit of attention in the robot learning community of-late.\n+ The idea of learning a low-dimensional continuous representation of the otherwise discrete design space as a means of improving efficiency is interesting.\n+ The paper is well written and easy to follow.\n\nWEAKNESSES\n\n- The core contribution of the paper is the use of a VAE to learn a continuous design representation as a means of improving optimization efficiency. In its current form, however, the paper lacks a concrete demonstration of these gains in efficiency over baseline methods.\n- It is not clear what logic was used to choose the maximum steps for training (500). Comparing the results presented here to those in the RoboGrammar paper [5 (in this paper], Guided Heuristic Search (GHS) reaches ~5.5 reward at ~1200 iterations for Frozen Lake, ~6.15 reward at 2000 iterations (and ~6.0 at ~1000 iterations) for Flat Terrain, ~5.25 reward by 2000 iterations for Ridged Terrain, and ~5.6 reward by 2000 iterations (with a rapid increase around 500 iterations). This may make one wonder whether the choice of 500 was cherry picked. What does the reward for GLSO look like farther into training?\n- Given the motivation of improving computational efficiency, it would be helpful to see a visualization of performance as a function of computation time (ignoring the fact that methods like the random baseline could be parallelized).\n- The means by which contact points are used to define the additional prediction objective is ad hoc and seems brittle (e.g., it relies on a default configuration of each design). It's also not clear how it would generalize to other domains, while the paper claims that the overall method generalizes across domains.\n- The paper omits a discussion of related work in design-control optimization [1,2,3]. This includes existing work that uses a VAE to learn a low-dimensional representation that facilitates the optimization of the design and control of soft robots (i.e., by making the full pipeline differentiable) [1]  and another that employs a variational objective to learn a latent representation suited to control and estimation [2]. Also relevant is work that uses graph neural networks to reason over graph-based representations of robot design spaces [3, 4, 5, 6].\n- The limitations discussion is weak. The inability to handle acyclic graphs is listed as a limitation without a discussion of the advantages of being able to handle cycles. A more obvious limitation, which the text only partially alludes to, is the challenge of generalizing this approach to produce physically realizable designs that can be fabricated and controlled in the real world.\n\nQUESTIONS/COMMENTS:\n\n* How was the computational cost of the random baseline balanced with the computation afforded the other methods (e.g., since the random baseline can be parallelized)?\n* Lines 47–53: I find this discussion a bit contradictory. The paper states that structurally similar robots may have very different performance and uses this as motivation for using contact point prediction as an auxiliary objective. Wouldn't the same structurally similar designs also have similar contact points?\n* Can the authors elaborate on the \"computationally expensive training procedures\" required of end-to-end methods.\n* The VAE is trained on a random set of designs generated according to the grammar. Instead, it would be interesting if this optimization could be focused on higher-performing designs.\n\nMINOR:\n\n* Line 29 (and elsewhere): \"e.g.\" should be followed by a comma\n* Line 56: Erroneous space before footnote.\n* Footnote 1* (and elsewhere): Numbered citations shouldn't be used as words.\n* Line 73: \"... when the design space are ...\" --> \"... when the design **spaces** are ...\"\n* Line 84: \"... in a A* manner, ...\" --> \"... in **an** A* manner, ...\"\n* Line 153: $\\sigma$ is overloaded.\n* Line 186: \"KL\" should be capitalized.\n* Line 209: \"... uses Gaussian Process\" --> \"... uses Gaussian **Processes**\"\n\n\nREFERENCES\n\n[1] Andrew Spielberg, Allan Zhao, Tao Du, Yuanming Hu, Daniela Rus, and Wojciech Matusik, \"Learning-In-The-Loop Optimization: End-To-End Control And Co-Design of Soft Robots Through Learned Deep Latent Representations,\" NeurIPS 2019\n\n[2] Andrew Spielberg, Alexander Amini, Lillian Chin, Wojciech Matusik, and Daniela Rus, \"Co-learning of task and sensor placement for soft robotics.\" IEEE Robotics and Automation Letters 6, no. 2 (2021): 1208-1215.\n\n[3] Ye Yuan, Yuda Song, Zhengyi Luo, Wen Sun, and Kris Kitani. \"Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Design,\" arXiv preprint arXiv:2110.03659 (2021).\n\n[4] Deepak Pathak, Christopher Lu, Trevor Darrell, Phillip Isola, and Alexei A. Efros, \"Learning to control self-assembling morphologies: a study of generalization via modularity,\" NeurIPS 2019.\n\n[5] Tingwu Wang, Renjie Liao, Jimmy Ba, and Sanja Fidler, \"Nervenet: Learning structured policy with graph neural networks.\" ICLR 2018.\n\n[6] Wenlong, Huang Igor Mordatch, and Deepak Pathak, \"One policy to control them all: Shared modular policies for agent-agnostic control.\" ICML 2020.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The paper describes an approach to improving the computational efficiency of optimization over discrete, grammar-constrained robot design spaces. Fundamental to this approach is using a variational autoencoder to learn a continuous, low-dimensional representation of the design space that can be optimized using Bayesian optimization. Experimental results show advantages over existing baselines along with the value of training the VAE with an additional objective that encourages the learned representations of similar designs to be nearby in the latent space.\n\nThe idea of using a learned latent representation to improve the efficiency of optimization is interesting and offers potential advantages over standard methods that involve combinatorial search. As such, this work may prove useful to others in the robot learning community who are interested in design-control optimization. However, the advantages of this approach in terms of computational efficiency are not fully clear.\n\nUPDATE AFTER AUTHOR RESPONSE/DISCUSSION:\n\nI appreciate the authors' response, which addresses some of my initial questions and concerns.\n\nI remain unconvinced by the justification for the choice to stop at 500 iterations as well as the argument against parallelization. I appreciate that running GLSO for a greater number of iterations and for multiple seeds would take more time, but we aren't talking about a lot of time here, particularly if GLSO was run on a machine more powerful than a. desktop, particularly given that the baseline results already exist. Real-world fabrication is not a focus here---the algorithm is not designed with the specific goal of enabling sim-to-real transfer and there is little discussion about how the challenge of realizing designs that are effective in the real-world would be addressed. This is not a criticism of the work, since it is consistent with the large majority of recent approaches to co-optimization, but calls into question the arguments for stopping at 500 iterations and for not considering parallelization.\n\nAs a result, I am keeping my score.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Relevant but unlikely to deploy to hardware in near future", "quality_of_limitations": "Limitations are not well addressed", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "issues": "See issues discussed above.", "reviewer_expertise": "5: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature"}, "extra_text": {}}, {"review_id": "wlac8VMzfE_", "reviewer": "Reviewer_tfFP", "summary": "This paper presents a robot co-design algorithm that co-optimizes the shape and control of terrestrial robots. This paper is based upon the RoboGrammar paper [5] and uses grammar to constrain the robot topology search space to be compact. The key idea of this paper is to train an autoencoder and decoder for the robot structure, which maps the discrete robot topology into a continuous latent space representation. This encoder-decoder framework enables to optimize in the continuous latent space with bayesian optimization instead of optimizing in the discrete and constrained robot topology space. The authors demonstrate the advantages of the proposed method for the tasks defined in [5] and properly design the ablation study to validate each technical component of the method.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The idea of mapping a discrete topology space into a continuous latent space to simplify the robot design optimization problem is interesting. The method to decode the continuous latent space into the robot structure with a depth-first approach and using teacher-force style training is inspiring. The most appealing part of the algorithm to me is the reusability of the trained autoencoder, which has the potential to apply the pre-trained autoencoder to other tasks/problems.\n\nHowever, there are a few concerns and questions I have for the authors:\n1. To train the autoencoder, the authors collect a large dataset of designs by applying random rules in the RoboGrammar. However, randomly picking the rules to apply will result in a super unbalanced dataset, where the number of simple designs with short rule sequences will dominate the whole dataset. The autoencoder trained with such an unbalanced dataset will probably lead to a biased encoder that only generates simple designs. Does the author do any data balance technique to address this problem? \n\n2. The robot decoder part (section 3.3) is the most important part of the whole pipeline since it requires a clever way to convert a continuous latent space to valid discrete structure space. More details about the decoder will be necessary for a better understanding and reproducibility of the approach. Specifically, I have the following questions regarding the decoder part:\n     1. How does the recorder message passing work? Since the partial graph keeps changing during the decoding phase, do the messages stored at each node get cleared and propagate from scratch every round?\n\n     2. One of the advantages of using grammar is to constrain the robot designs to be valid and manufacturable designs. However from the paper, during the decoding phase, it seems that the algorithm just iteratively adds a node with a certain type to the current partial graph without any verification on whether the design is actually spanned by the grammar. If that is the case, the produced designs can be invalid for real manufacturing thus the applicability of the proposed method to a real robot system is questionable.\n\n     3. In RoboGrammar, the grammar rules are constructed to provide the symmetry of the robot structure. The designs in the associated video also have such symmetry property. From the paper, I didn’t find such symmetry constraints in the decoder part. Could you clarify more on this part?\n\n     4. While it is appealing to convert a discrete parameter space (i.e. robot topology space) into a continuous space (i.e. latent space), it is counter-intuitive. While there are gradients in the continuous space, the gradients are not defined in the discrete space. As an example, if we slightly perturb the latent representation, how will the robot structure produced by the decoder change? \n\n     5. It is good to visualize the learned latent space. You can take two latent vectors and get intermediate latent vectors by interpolating between these two vectors, and visualize what the corresponding robot structures after decoding look like.\n\n     6. How is the ground truth label is constructed? For a node, if it has multiple other nodes connected to it (e.g. the root node), then there can be multiple choices for the label of the first node starting from this node, which means the ground truth label q is not unique.\n\n     7. The training takes place in a teacher-force style, it will produce aggregation errors during testing. How do the authors resolve this problem?\n\n3. The wall terrain task looks different from the one in RoboGrammar [5] (the wall is narrower in this work), any reasons for that?\n\n4. The comparison to GHS looks questionable. In [5], GHS can achieve much higher rewards after training for 2000 iterations (e.g. over 6 in Flat Terrain Task), which means both the GHS and GLSO have not reached the best design yet after 500 iterations. It would be helpful to train the GLSO and GHS for a long enough time to compare their convergence speed and final performance.\n\n5. How long does the algorithm run? The major advantage of training an autoencoder is that it can generalize to different new tasks without re-training the encoder and saves time. So it would be more supportive if the robot optimization time can be reported.\n\n6. Missing references:\n    1. Evolutionary Algorithms:\n        > Embodied intelligence via learning and evolution, Gupta et al. 2021\n   \n        > Evolution Gym: A Large-Scale Benchmark for Evolving Soft Robots, Bhatia et al. 2021\n       \n        > Evolving virtual creatures, Sims 1994\n     \n    2. RoboGrammar: citation [10] should be included in section 2.2.\n\n    3. Latent Space Optimization: \n        > Emergent hand morphology and control from optimizing robust grasps of diverse objects, Pan et al. 2020\n      \n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "Given the novelty of the method and the potentially promising results and applications, I would suggest weak acceptance for now and will change my score accordingly based on whether the authors address my concerns properly.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Relevant but unlikely to deploy to hardware in near future", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "issues": "All the issues have been mentioned in the above.", "reviewer_expertise": "5: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature"}, "extra_text": {}}, {"review_id": "rv8ndwTc4Ir", "reviewer": "Reviewer_t5Qe", "summary": "Design optimization of kinematic robots (e.g. walking robots) is challenging because the design space is highly combinatorial; it is effectively a search over graphs representing kinematic chains. This combinatorial structure has (in prior work) precluded the use of efficient optimization methods over continuous spaces. To solve this challenge, the authors propose to use a graph variational auto-encoder to learn a continuous latent representation of the graph grammar defining a walking robot, then apply Gaussian-process Bayesian optimization to optimize in this continuous latent space. They additionally use a \"performance predictor\" to regularize the learned latent space to group physically similar robots together. The paper is well-written and includes sufficient experimental evidence to justify the authors' design choices and show an improvement over the state of the art.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The paper is well-written.\n- The challenge of optimizing kinematic robots is clear, and the authors' approach is well-motivated and novel.\n- The paper contains sufficient experimental evidence to show that the proposed method outperforms the state of the art, and the ablation studies justify the design choices made.\n\nWeaknesses:\n- There is little theoretical justification for the effect of performance prediction on the structure of the latent space. I am curious about questions like, \"how does the choice of predicted metric impact the structure of the latent space?\" and \"is the contact configuration the best metric for this task?\". The paper provides sufficient empirical results to justify their heuristic, but I am still curious.\n- As the authors note in the paper, the walking robots they study are not terribly realistic and might be difficult to construct and deploy in hardware.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "This paper was fun to read and contains a very interesting core idea (using a learned latent space to enforce a continuous structure on a combinatorial problem) that I hope to see future works build upon.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Excellent: Enjoyable to read.", "potential_impact": "", "robotics_focus": "Relevant but unlikely to deploy to hardware in near future", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "reviewer_expertise": "3: The reviewer is fairly confident that the evaluation is correct"}, "extra_text": {"issues": "The biggest issue with this paper is that the details of the encoder and decoder architecture are not very clear, particularly to a reader unfamiliar with graph neural networks. It would be helpful to either reference a clear explanation of this message passing process (perhaps a survey or tutorial) or include a more thorough explanation in the supplementary materials."}}, {"review_id": "VrdTZ5_nJyn", "reviewer": "Reviewer_4uZ5", "summary": "The paper describes a method for optimizing a robot design given a task. \n\nThe robot topology is defined as a graph which, in turn, is generated according to a grammar that guarantees only valid robot topologies. \n\nThe grammar is also used to generate a synthetic Dataset of 500k robot topologies, used to train a VAE mapping the graph structure of the robots to a continuous latent space. An additional constraint is imposed that similar points in the latent space produce robots with a similar “shape”.\n\nBayesian Optimization is used to search over the latent space to generate the robot with the highest performance.\nGenerated robots are evaluated in Sim against existing baselines.\n\nThe controller is computed using a variation of MPC which is the most computationally expensive part of the algorithm.\n\nAn ablation study is performed to show the effectiveness of the auxiliary parameter (the Property Predictor) in conjunction with the VAE to locate similar design in the latent space.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths:\n\n* The paper is very well written, It’s clear, builds on solid foundations yet performs quite well over other benchmarks. \n\n* The idea of training a continuous latent space on a variety of robots and being able to use it to optimize for other tasks or other domains altogether (manipulation and so forth) seems particularly powerful and promising.\n\n## Weakness:\n\n* No hardware experimentation", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The paper is very well written, It’s clear, builds on solid foundations yet provides a meaningful increase over the performance metric. The Authors supply the Code to run the experiments and allow the community to keep building upon this work.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "3: The reviewer is fairly confident that the evaluation is correct"}, "extra_text": {"issues": "The paper lacks an “on robot” experiment; albeit not strictly required, a validation on simple robots made with cheap servo motors and 3D printed limbs could provide stronger evidence on the quality of the algorithm.\n\nOther than the above mentioned issue the Reviewer could not find any meaningful shortcoming to the proposed work.\n"}}]

Author rebuttal:

Decision: Accept (poster)