Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: In this work, neural networks with sinusoidal activation functions are used as continuous function approximators for both the density and displacement fields in structural topology optimization problems. This allows for the development of a mesh-free topology optimization procedure, which is claimed to be on par with existing traditional FEM-based approaches, while also allowing for the learning of continuous solution spaces over certain parameters.
main_review: This work is clearly presented.

The authors method seems well motivated, and draws from recent advances in physics informed neural networks and the utilization of periodic activation functions (SIRENs). However, supposedly given the complexity of the optimization tasks being studied, a number of additional tweaks are necessary to ensure good behavior of the optimization algorithm. These are studied in ablation studies in the experimental section.

The empirical results are the core of the paper. Below I comment on these, and raise some questions which I hope the authors will address.

 First, the authors demonstrate that the proposed method is able to achieve results on par (or slightly better) than traditional FEM-based approaches. These are good results, yet it would be important for the authors to also have made available the comparison along other axes that make the tradeoffs between methods more evident. For example, what are the computational costs and runtime of each, both for training and evaluation (for the machine learning-based approach). Similarly for the "learning continuous solution spaces", how would the run time of a traditional solver compare here? (Maybe for solving each of the evaluated positions). It would be important to have these values so that the reader can have a clearer picture of what are the pros and cons of each method.

The 3D evaluation seems qualitatively good, yet quantitative results are not presented, and a comparison against the traditional methods is also absent. (An analysis of computational cost tradeoffs, as mentioned in the previous paragraph would be interesting in this case too.) Given this, the claim in the conclusion that the "can generalize well to 3D" (line 303) seems a bit strong, given the brevity (and qualitative nature) of the experimental results in this area.

The ablation studies seem to support the claims of the authors. However, to make the results more clearly interpretable, it would probably be a good idea to combine the values reported into lines 283 and 284 into Table 1 (or a similar table), where the reader could easily see the comparison across methods with similar units. Additionally, it is not clear to me why results on these baselines were only reported on one experiment. As mentioned previously, it would probably make sense to have a filled in Table 1 (or similar table) with the values for baselines for all experiments, so that a more comprehensive comparison can be performed, not only between the proposed methods and traditional FEM approaches, but also to the baselines.

Despite the issues raised in the previous paragraphs, I believe the methods presented in this paper are interesting and the results look promising. As such, I am willing to improve my evaluation if the authors address these concerns. 

-------------

Update after authors' response:

The authors have addressed the concerns I raised above, and as such I am updating my rating of the paper.
limitations_and_societal_impact: Societal impact was not discussed and claimed to be N/A.

The work is lacking some discussion of possible limitations, these are discussed in the main comments above (with respect to comparison to other methods). The authors should address these.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper proposes a method for topology optimization (TO) in solid mechanics by modeling displacement and material densities as implicit neural networks. TO is a very important field in mechanics and it is exciting to see successful ML applications in the area. The paper shows that their method can solve for topologies under various load and displacement constraints in both 2D and 3D.

main_review: ## Strengths

- The results appear to be very strong on a variety of cases compared to state-of-the-art methods.

## Weaknesses

- The paper seems to be targeted to topology optimization audience and as such lacks background on a lot of concepts. Specifically I think the multi-batch optimality criterion lacks details and should perhaps be alluded to before (4). A figure on how sensitivity filtering helps might make it more intuitive for a reader. 

## Questions

- In Appendix 2, how does the Lagrange multiplier in (4) become the strain in (5)? Also is there a reason why not accounting for u's dependence on $\rho$ is okay in (9). Some additional clarity here would be useful as this defines the density space gradients which is key to the paper.

- How sensitive are the methods to initialization of the network?  

- The sensitivities are first introduced in Algorithm 1 and then in (8); can you elaborate on what these are? 

## Originality

The key idea seems novel and solid.

## Clarity

A lot of the sections in the paper were not clear to me (see weaknesses). I think the paper will do well if written with a broader audience at NeurIPS in mind. 
limitations_and_societal_impact: Limitations were discussed but potential negative societal impact needs to be added.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This work introduces a method for topology optimization based on implicit neural representations. 
The goal of topology optimization generally is to find optimal material distribution that produces the structure which is as stiff as possible. The key idea presented in this work is to use two neural networks: the density network, which implicitly represents optimal material density field, and the displacement network, which implicitly represents displacement field. 
The optimization algorithm then proceeds by alternatively training these two networks, while applying additional filtering and ad-hoc supervision to steer the process to better optima. Evaluation is conducted on a set of standard 2D topology optimization tasks. 
main_review: 
### Paper writing / clarity
**-** Paper is often hard to follow, especially for people who are not closely familiar with topology optimization. See the list of minor issues below which could help improve readability / clarity. 

### Motivation / novelty
**+** Using neural nets to approximate solutions to complex physical systems is in general a very promising direction, and it seems like implicit neural representations quite naturally fit the topology optimization domain. 

**-** However, it appears that the idea of using a neural network to implicitly model optimal density field and use those in an optimization loop has been explored before [53]. It is not clear if simply using SIREN in place of a relu MLP is a sufficiently large technical contribution, even if it does improve performance. It would be great if authors discuss other key differences between their proposed framework and "TOuNN".

### Evaluation 
**+** Numerical evaluation indicates that proposed method outperforms a strong baseline - SIMP by a small margin. However, the resulting solutions seem to be _very_ similar qualitatively to the baseline. It would be great to have some context for non-domain experts to understand how large an improvement is reported in Table 1.

**-** Discussion of efficiency of the proposed approach with respect to the baseline (SIMP) is not present. 

**-** Evaluation in 3D settings is not very convincing: authors report an example without any numerical comparisons to any baselines. At very least this should be addressed by changing the claims.

**+** I liked the examples of continuously changing volume constraints in the supplementary: if those can be obtained in real-time, it might make sense to elaborate on this in the main paper (and also would be great to know if competing approaches such as [53] and SIMP could get to similar results).

### Minor 
* Might be a good idea to adapt notation a little bit to indicate which quantities are vectors vs scalars (e.g. use bold).
* L29: domain of interest
* L78: "use fully-connected layers": you mean MLPs?
* L90: notation ":" confused me until I reached 4.1.
* Notations in "Algorithm 1" are a bit too dense and confusing. Maybe makes sense to unroll the "opt" notation for better readability. 
* L146-147: what is meant by "standard MLP with residual links"? if my understanding is correct, SIRENs do not use residual connections. 
* L170: why 5 ? mb makes sense to introduce a notion for this scalar? 

limitations_and_societal_impact: Authors did discuss minor limitations of their approach.
review_title: null

Meta-review: This paper got three 6 ratings and is therefore a borderline accept.

The authors carefully answered reviewers comments and the reviewers were satisfied by the answers.

Moreover, the meta review of the previous ICML submission has been taken into account.

I would therefore like to accept the paper.

The authors should make sure to improve the clarity of the paper for readers without background in topology optimization.
Decision: Accept (Poster)