Paper:
Rethinking physics in the era of deep learning is an increasingly important topic. This topic is special because, in addition to data, one can leverage a vast library of physical prior models (e.g. kinematics, fluid flow, etc) to perform more robust inference. The nascent sub-field of physics-based learning (PBL) studies this problem of blending neural networks with physical priors. While previous PBL algorithms have been applied successfully to specific tasks, it is hard to generalize existing PBL methods to a wide range of physics-based problems. Such generalization would require an architecture that can adapt to variations in the correctness of the physics, or in the quality of training data. No such architecture exists. In this paper, we aim to generalize PBL, by making a first attempt to bring neural architecture search (NAS) to the realm of PBL. We introduce a new method known as physics-based neural architecture search (PhysicsNAS) that is a top-performer across a diverse range of quality in the physical model and the dataset.

Review:
=== Overall comments ===
This paper proposes to generalize approaches to physics-based learning (PBL) by performing network architecture search (NAS) over elements from PBL models found in the literature. This entails including physical inputs to the network and the incorporation of new operations to the NAS. I think the idea has merit and rather like it. However, there are several aspects of the work that could be improved. The technical novelty is small, as the extension of the  existing NAS models to handle physical inputs and a few new operators is relatively straightforward. The experiments, while well designed, only explore uninteresting toy problems. While I appreciate the necessity to explore the methods performance in a more controlled setting, a more impactful testbed would be more convincing. Another drawback of the evaluation is the lack of a proper statistical analysis of the results, given the small data and model sizes.
=== Relevance & Prior Work ===
+ The related work gives a good summary and categorization of prior work in physics-based learning
+ The problem (physics-based learning) is interesting and relevant to the community
=== Novelty & Approach===
+ application of NAS to physics based learning
+ incorporation of physics solutions as inputs into differentiable NAS
+ creation of physics-informed operation sets to merge physical models into network
- technical steps to merge NAS and PBL are relatively straightforward
=== Evaluation ===
Two representative physical simulations were chosen for evaluation, where elements of the physics model are intentionally omitted,  1) estimating trajectory of a ball in presence of wind and air resistance, and 2) a collision speed simulation where two objects collide, where sliding friction is not accounted for in the physics model.
The baselines consist of: a 3-layer MLP (data-driven), a 3-layer MLP with Physical Regularization, a 3-layer MLP with residual connection to the physics prediction, an MLP with two input branches, on for the data and one for the physics predictions (Physical Fusion), and the Embedded Physics model which estimates parameters for the physics modelu using a 3-layer MLP.
PhysicsNAS can combine elements of the baseline models, but the total number of nodes is limited to 5.
+ Experiments testing the dependence of the model on the numbers of samples and the strength of the physical inconsistencies were conducted. In both cases, PhysicsNAS outperformed the best specialized physics models.
- The chosen testbed tasks are toy problems. While these types of experiments are necessary to understand the performance of the model, it would have been interesting to see PhysicsNAS applied to a more impactful task
- Given the size of the networks and the training data, there is no reason why a more sophisticated statistical analysis of the results wasn’t performed (confidence intervals, t-test, p-value). Similarly, a more complete set of experiments with more sample amounts could be provided with little effort.
=== Clarity & Other Comments ===
- “precious nodes” -> previous nodes