Title: A Deep Learning Surrogate Framework for High-Dimensional Regression Problems in Mechanical Engineering
Venue: NeurIPS 2025
Accepted: no
Keywords: Deep learning

Surrogate model

High-dimensional regression

Mechanical engineering
Abstract: This paper introduces the first large-scale deep learning-based surrogate model for high-dimensional regression tasks in real-world mechanical engineering contexts. The model, comprising 43 million parameters, is trained on a custom in-house dataset, containing 2.8 billion data points from 31 million samples that are generated entirely through easy-to-evaluate, physics-based simulations. Each sample consists of 26 scalar features and 64 scalar targets. This large-scale synthetic dataset enables the training of deep neural networks over exhaustive and realistic mechanical design spaces. It exhibits complex statistical characteristics, including zero inflation, mutually exclusive features, strong multicollinearity, and a mix of real- and integer-valued data. Despite the scale and complexity of the dataset, the model is trained using entry-level consumer-grade graphics cards, thereby demonstrating the practical viability of deep learning for regression tasks in mechanical engineering applications.


Peer reviews:
comments: SUMMARY:
This paper presents a method of using data-driven surrogates for predicting the static transmission error (STE) of a gear system, given the gear configuration. The paper curates a large dataset of these configurations and carefully studies the representation and compression of 'design space' and 'performance space'.

STRENGTHS AND WEAKNESSES:
Strengths: 
- The paper is technically thorough with a lot of details. 

- The paper's exposition is quite good and easy to understand. 

Weaknesses: 
- The paper research contributions are limited. It is not clear what exactly is the new insight one can learn from this paper. Scaling up the deep learning systems is something quite common in practice. What is so difficult for this application? 

- The paper use of large datasets and large networks resembles an effort for a 'foundation model' for mechanical engineering but the key, which is missing, is to use this foundation model for other applications in the same domain. 

- The paper raises the expectations by it's too broad title. The paper can and should change the title as it only shows one application and not a general mechanical engineering solution. 

- How the dataset it created and how the ground truths labels are created is not quite clear. 

- A cost analysis of why surrogation is a good idea here is missing. Why not using the ground truth simulation?

QUESTIONS:
- What is the fast-to-evaluate physics base model for building the large dataset? 
- If the model is fast-to-evaluate, why do we need a surrogate? 
- I believe a figure describing the gear system would make the problem more tangible. 
- I believe the work is more appreciated in a mechanical engineering venue.

LIMITATIONS:
Good.
TITLE: Official Review
CLARITY: 3
IMPACT: 1
ORIGINALITY: 1
DATE: 2025-06-18

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a dataset of physics simulations (gear modeling) with a very large number of samples: 31 million. The dataset is tabular: inputs are of dimension 26 and outputs of dimension 64; a statistical analysis of the datasets is carried out. Then, experiments are described: hardware is limited to consumer grade, dimensionality reduction is explored (PCA, autoencoders). It is decided that no dimensionality reduction is used for the inputs, while an autoencoder with 5 latent variables is used for the outputs. The regressor is presented in the form of a classical MLP, whose parameters are carefully chosen using optuna. The performance of the regressor is a mean adjusted R2 of 0.9975. Even if the reference model is low fidelity (hence fast to compute), inference throughput of the regressor is 95 samples per second, leading to a speedup of 100k.\nThe annex contains a detailed description of the mechanical problem, additional statistical indicators of the datasets, and instructions for retrieving the trained model, the training datasets and computing scores and plots.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- Very large dataset constructed and trained model made available\n- Involved descriptions of the physics setting, dataset and statistical analysis\n- Careful optimization of the model hyperparameters\n- Clear motivation and good results\n\n\nWeaknesses:\n\n- Extensive geometrical description without any scheme, which renders the information extremely hard to understand\n- The authors compare to a linear baseline, which do not help positioning the proposed regressor with respect to state of art methods; this is a strong weakness. The problem is in a low output dimension (5 with autoencoder, or 6-8 with PCA), which can deserve comparison with faster to train and smaller models. Physics problem can have regular outputs with respect to initial conditions (well-posed in the sense of Hadamard), in this context, maybe MMD subsampling + efficient Gaussian-process regression can be suitable. \nGradient boosting methods are known to be very competitive (fast training, accuracy) on tabular problems [1] and regularly top machine learning competitions. I did a quick experiment with a PCA on the outputs with 6 components and catboost on 1M rows randomly chosen in the dataset. Train/test split on these 1M rows to get a R2 of 0.61 on the test, using the authors scripts (only 5 minutes to compute the PCA and train on a consumer 3900X 12-core CPU), without any hyperparameter optimization. The linear baseline provided by the authors has a R2 of 0.35. \n\n[1] A Comprehensive Benchmark of Machine and Deep Learning Across Diverse Tabular Datasets (https://arxiv.org/pdf/2408.14817v1)\n\nQUESTIONS:\n- Can you include a fair comparison with respect to state of the art methods and reassess the conclusion regarding the accuracy and throughput performance of the proposed regressor ? The result appears of very good quality, but a strong baseline is expected to be able to confirm it. I’ll increase my rating provided baselines with R2 much larger than my quick test are included\n- The architecture is simple, yes effective here. Can you comment on other possible architecture that may reach the same level of accuracy, but maybe lighter and faster to train ?\n- How is the design of experiment computed ? Comments are made regarding the statistical complexity of the input, but since you control them, did you choose one with a priori good properties for learning ? (space-filling, low-discrepancy, uniform coverage in lower-dimensional subspaces, minimize multicollinearity)\n\nLIMITATIONS:\n- Low fidelity model used to compute the dataset\n- Demonstration limited to gear mechanics\n- Consumer grade hardware used\n- Single dataset and single experiment (no state of the art comparison)\n\nIn my opinion, the first three limitations do not weaken the conclusions of the paper.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-21"}

comments: SUMMARY:
The paper presents a 43 M-parameter autoencoder-MLP surrogate (“Presto”) that maps 26 gear-design scalars to a 64-point static-transmission-error (STE) curve. They generate 31 M synthetic samples (≈2.8 B scalars) with a fast low-fidelity model, compress the output to five latent coordinates, and fit a 7-layer MLP. On a  M-sample test split, the system reports good accuracy metrics and compute time (even though a single RTX-4060 is used).

STRENGTHS AND WEAKNESSES:
This seems to be good dataset for a particular engineering application, and the model seems to perform well for this particular dataset.
Large data set and practical target; impressive that the full pipeline runs on consumer GPUs.
There is no basis to judge whether the same task can be accomplished better (and with fewer training samples). The comparison is only with partial least squares (which performs poorly, as one might expect). Absent are Gaussian Processes, Gradient-Boosted Trees, random-feature kernels, or Fourier-basis surrogates that directly exploit periodicity. One cannot conclude the deep model is superior beyond “one non-linear method > one linear method”.

A single random split; no k-folds, no uncertainty estimates, no out-of-distribution stress tests (manufacturing noise, tooth-breakage scenarios). Reported errors are relative to the mean absolute value (MAE_rel ≈ 2.96 %) 

No novelty in method either. Perhaps better suited for a mechanical engineering conference.

QUESTIONS:
Given the linear-looking latent structure, how much worse is Ridge on PCA targets?

What about experiments with: (i) sparse Fourier regressors on the 64-point curve; (ii) GP with deep kernel?

Can this be finetuned for specific (but new) cases?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-06-29

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents deep learning-based surrogate modeling techniques for high-dimensional regression tasks in mechanical engineering, with a focus on predicting Static Transmission Error (STE) in gear systems.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n- **Engineering Relevance.** Predicting static transmission error (STE) in gears is an industrially significant problem, and the paper offers detailed physical modeling and constraints, indicating domain knowledge.\n\n### Weakness\n- **Out of Scope for NeurIPS.** NeurIPS is a premier venue for foundational contributions to machine learning. This submission is more aligned with applied ML and data-driven surrogate modeling. It would be more appropriately submitted to venues such as KDD, CIKM, or perhaps mechanical engineering venues focused on computational modeling.\n\n- **Limited Novelty.** Previous studies, such as Sakaridis et al. (2023) [1] and Willecke et al. (2023) [2], have already explored deep learning-based surrogate modeling for STE prediction. While the scale here is larger, the core modeling technique, an MLP with standard components,  is not novel.\n\n- **Missing Related Work Section.** The paper does not include a dedicated related work section, which makes it difficult for readers to contextualize the contribution with respect to existing literature. As a result, it is unclear how this work advances the state of the art or how it compares to previous efforts in surrogate modeling, particularly in mechanical engineering or gear design. Incorporating and critically analyzing prior works such as those by Sakaridis et al. and Willecke et al. would significantly strengthen the clarity and positioning of the paper.\n\n- **Conventional Techniques Used.** The modeling pipeline comprises standard components: MLPs with batch normalization and PReLU, AdamW optimizer, OneCycleLR, PCA, and gradient clipping. These are common best practices, not novel contributions. The authors present these as part of their design decisions, but they do not introduce any methodological innovations in architecture, loss functions, or optimization strategy.\n\n- **Model Size is Not “Large” by Modern Standards.** The model size (43 million parameters) is not large in the context of modern deep learning. Claims about “large-scale deep learning” should ideally refer to billion-scale parameters where scaling laws [3] are relevant. Without this, the argument for scale lacks impact.\n\n- **Lack of Comparative Baselines.** The paper does not benchmark the MLP against other contemporary models such as gradient-boosted trees, Gaussian processes, or neural operators that have shown strong performance in scientific regression tasks.\n\n- **Presentation Framing.** While the paper emphasizes the use of consumer-grade GPUs, this detail, while practically relevant, does not directly contribute to the scientific merit of the proposed methodology. It reads more as an engineering anecdote than a scientific insight.\n\n[1] Sakaridis, Emmanouil, et al. \"Symmetry preserving neural network models for spur gear static transmission error curves.\" Mechanism and Machine Theory 187 (2023): 105369.\n\n[2] Willecke, Marius, Jens Brimmers, and Christian Brecher. \"Surrogate model based prediction of transmission error characteristics based on generalized topography deviations.\" Forschung im Ingenieurwesen 87.1 (2023): 431-440.\n\n[3] Kaplan, Jared, et al. \"Scaling laws for neural language models.\" arXiv preprint arXiv:2001.08361 (2020).\n\nQUESTIONS:\nGiven that the final model has approximately 43 million parameters—which is not particularly large by modern standards—what is the motivation behind aggressively reducing the dimensionality of the inputs and outputs? Was this choice primarily driven by hardware constraints, or were there other considerations such as overfitting, noise robustness, or inference latency?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 5, "CLARITY": 1, "IMPACT": 1, "ORIGINALITY": 2, "DATE": "2025-07-01"}