Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: I thank the authors for their rebuttal replies and for further improving the quality of this paper. I think my score is still accurate and I look forward to seeing this work published.

The paper proposes a differentiable model for learning meta material layouts and neural network encoder that can produce actuations to control these materials. They show through a number of simulations that this works with simple translations, rotations and complex shapes. They also build this material and show that it works. Finally, they show how this combined framework can be trained to translate MNIST digits into 7 slits representations.
strength_and_weaknesses: Strength: The paper is very well written, easy to follow and fun to read. Even for a reader unfamiliar with meta materials and the associated physics, they manage to write an engaging scientific exploration. The proposed model is very creative and works at an exciting intersection of material and data science.

Weaknesses: It would have been great to include a tiny bit more detail about technical things, for instance, how is the adjoins method implemented, which PDE solver is used etc. However, I do realise that this might come at the expense of making the paper less accessible.
clarity,_quality,_novelty_and_reproducibility: Clarity: Extremely high! Section 2.1 is a little difficult to digest at first read.

2.4 2nd paragraph, 2nd brackets ‘wel’?

I am a little surprised at the spiky shapes emerging in many of the shown optimised geometries. Is this an optimisation issue? Or is this some repeating, near optimal, solution (maybe one that also appears in nature after evolution)?

The MNIST experiments are really cool, although I am thinking that the encoder network probably just does the classification and then translates this into the actuation for the 7 slit representation. But even if this is a bit of a gimmick, it is still a very cool demonstration!

Quality: Very high, as far as I can tell. It might be nice to include a section with limitations and optimisation pitfalls, i.e., all the things you tried on the way that did not work – that would be helpful for other researchers to build on this work.

Novelty, I cannot really judge because I don’t work in this area of meta materials. However, for what it is worth, I have never seen a demonstration like this and think that it is really creative and novel.
summary_of_the_review: The authors propose a model combining a neural network controller and a learned geometry. This is a very creative solution to a problem in material sciences (I suppose) and it is implemented with some cool technical methods based on spline representations, adjoins computation and extended efforts to build a differentiable system that works under a number of physical and geometric constraint. I think that this is a really project and would be happy to see it published – supposing that other reviewers which are more familiar with this research area agree on its novelty and soundness.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This works presents a co-design of metamaterials based mechanical/morphological actuation and neural network-based control. The control tasks are allowed to be image, coordinate on scalar parameter based on the end goal, which are mapped on to a latent dimension (using an encoder) matching the number of linear actuations. The decoder is a differentiable simulation that solves for equilibrium of non-linear elastic materials under mechanical load that takes the geometric design parameters and actuations (output from encoder). The solver is made differentiable wrt the geometry parameters (represented as a porous shape with quadrilateral boundary) by adopting isogeometic/FEM analysis and B-spline basis to solve the PDE. The results show that the geometry and control co-design is able to achieve translation, rotation and shape matching efficiently and better than geometry-only scenario.  It is interesting to see the digital MNIST benchmark and the application of this approach to that case
strength_and_weaknesses: Strengths:

* The use of neural networks to converts tasks to actuations and that used along with the simulation-based decoder in a single differentiable framework shows great promise in achieving nontrivial displacements
* The adoption of cellular solids and B-spline patches into the mechanical simulation/decoder seems to work well to produce stable solids.

Weaknesses:
* No new algorithmic/machine learning approaches have been proposed
* The work is just coupling the a simple multi-layer perceptron to a well-designed simulator (PDE solver) to achieve the joint optimization of geometry and boundary conditions
* Time complexity of training/learning in this approach is not discussed
* What is the motivation for using L2 loss in one case and L1 in the other?
* How expensive is the generation of training data; is this approach scalable to train with larger datasets for more complex deformations?
* What is the reason to choose the Neo-Hooken material model?

clarity,_quality,_novelty_and_reproducibility: The paper is well written and a good demonstration of machine learning application. Novelty in the machine learning algorithms is not present. Code is shared for reproducibility
summary_of_the_review: The geometry and control co-design using differentiable simulator and neural network combination shows good results, but no algorithmic innovations in machine learning are proposed in this work. More details need to be provided about the approach to better assess its applicability for other general scenarios.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The authors introduce neuromechanical autoencoders (NMA) -- neural encoder combined with a differentiable mechanical decoder -- to learn to perform morphological computation using gradient descent. The authors show how it is feasible for the decoder to perform translation, rotation and matching shapes specified to the encoder via actuations output by the encoder. They also demonstrate the performance of NMA on a material they manufactured to verify how the model works in the real-world. 
strength_and_weaknesses: Strengths:
- The paper presents a highly novel and technically sound integration of neural network-based encoders with differentiable mechanical decoders. This combination potentially has several applications in the real-world such as robotics, material design, 3D printing, etc. 
- The digital MNIST task that the authors develop -- in which the neural encoder receives an RGB image of a digit as input and the mechanical decoder produces deformations to the metamaterial that classifies the digit -- is very intriguing and a great demonstration of learning (with gradient descent) a solution to the common and accessible MNIST task through morphological computation.
- The authors share their code as part of the submission, this should is helpful in more thoroughly understanding the proposed work.

Weaknesses:
- I found it hard to grasp Methods subsections 2.3 and 2.5. I am not an expert in this domain and I'm not surprised that I find it hard to read these sections, but it would be great if the authors could please see if they can further improve readability of these sections for the broader audience.
clarity,_quality,_novelty_and_reproducibility: I believe that the presented work is technically sound and makes highly novel contributions with potential applications in interdisciplinary areas. The work has been written clearly, yet, I think it will be hard for the broader audience to grasp some important parts of the paper as mentioned in my review. In terms of reproducibility, the authors have shared their code used in the paper to reproduce results.
summary_of_the_review: I find the presented work to be highly novel and potentially impactful for the intersection of machine learning and multiple basic sciences. I believe this line of work is important and it looks like the domain is very new and understudied. I find this work to be very relevant to representation learning and the ICLR audience, and hence recommend acceptance. That said, I am not an expert in this area and am willing to read other reviews and update my score based on potential weaknesses I may have missed to identify.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: = Update after rebuttal =

I thank the authors for their response to my comments. I maintain my recommendation for acceptance.

= Original review =

The authors show how to optimize the shape of an elastic material, as well as neural networks to control its actuators, imn order  to make the material perform various mechanical tasks. 

Both the material's  shape and the controlling neural network  are optimized by gradient descent for various task domains. Given a target action (which,  depending on the  domain, might be a location for  a point, a shape,  or even an MNIST digit image to be classified), the neural network translates a specification of  the action into adequate actuations, through which the optimized shape performs the  specified action.

The authors apply their method to translation/rotation of a point on the material, controlled deformation of a  shape, and -  remarkably - translating MNIST images into seven-segment digit displays through deformation of a colored shape  seen through slits.

The authors also show a working real-life implementation of the system, demonstrating practical applicability.
strength_and_weaknesses: Strength:

- The method is novel (to my knowledge) and interesting. While there has been much work on optimizing morphologies and controllers jointly (e.g. Karl Sims and other "virtual creatures" models - these should be cited!), as well as optimizing elastic shapes (e.g.  Sodarace), the system described here  looks like nothing I'm aware of.

Weaknesses: 

- The only minor defect I can see is that some points in the explanation could be clarified - see below
clarity,_quality,_novelty_and_reproducibility: The paper is well written throughout. Just a  few clarifications:

- What exactly are the "actuations" mentioned in the figures? We just see blue vertical or horizontal brackets on each side of the square, but it's not clear what the actuation actually does - I initially thought  it would be squeezing along the direction of the bracket but that doesn't seem to be the case.

- The purpose of section 2.3 might be explained a little  bit more - what exactly are we computing here and how is it used in the system?

- Caption of Figure 8 a): should the first "optimized" be "unoptimized"?

- Typo in the last paragraph of p. 4, "e.g.,wel"

summary_of_the_review: It's new, interesting, and seems to work well. Easy accept.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: All four reviewers recommend accepting the paper (6-8-8-8). They find the paper to be highly novel and creative, and praise its clear and engaging writing. The reviewers note that the paper is accessible to readers who are unfamiliar with meta materials and associated physics. They highlight the creative neuromechanical model proposed in the paper, which works at the intersection of materials and neural networks. Experiments include both simulation and real-world (hardware) results, which is quite unusual for an ML paper. Overall, this contribution is a beautiful example of an applied machine learning paper.
justification_for_why_not_higher_score: None of the reviewers selected the highest scores of 9 or 10 for the paper. Additionally, the scope of the paper may not be broad or central enough to warrant an oral presentation. The reviewers describe the paper as a solid and enjoyable applied machine learning paper, but it may not be of interest to a large portion of the community.
justification_for_why_not_lower_score: This is a very well executed application paper.
Decision: Accept: notable-top-25%