# Rule-Based DSL for Continuous Features and ML Models Selection in Multiple Sclerosis Research
## 3 Dsl Design
### Meta Model Design
By leveraging the pipeline recorded from AutoML, the system preserves the features post-preprocessing and feature engineering. This stage facilitates the provision of additional models for users to define and experiment with. Through the establishment of rules, the system can be directed to train various models and features at different times. For demonstration, we chose two prevalent models in MS research: random forests (RFs) and support vector machines (SVMs). Following the initial feature importance evaluation from AutoML, we removed less-relevant features. This stage also encompasses continuing the steps from the AutoML-derived pipeline and setting rules for training multiple models if needed. In Figures 5 and 6, we depict the effects of different hyperparameters on the main test scores and feature importance for process and walk speed time by SVM. And in Figures 7 and 8, we show different results training by the random forest. The optimal parameters for SVM are C = 1000 and gamma = 1.0. However, the metrics indicate that the SVM model struggles to make accurate predictions on this dataset, with metrics such as MAE = 0.628, MSE = 0.753, R2_score = 0.475, and RMSE = 0.868. n contrast, the random forest model shows improved performance, with metrics of MAE = 0.39, MSE = 0.269, R2_score = 0.813, and RMSE = 0.519, with the best parameters being n_estimators = 100 and random_state = 20. This model outperforms both the SVM and the KNeighborsRegressor obtained from AutoML. All these metrics are automatically logged by our system. Users can view them by defining visualizations in our DSL or through the webpage provided by MLflow. Additionally, these two algorithms exhibit different feature importance rankings in Figures 5a and 7a. For walk speed time, the random forest model performs better than the SVM, but not as well as the decision tree model obtained from AutoML, as shown in Figure 8.

After two rounds of training, we identified the random forest model as the best ML model for predicting process speed time, with specific hyperparameters. The optimal model for predicting walk speed was the decision tree, also with specific hyperparameters. To ensure the robustness and relevance of our selected models, we aim to validate them on a dataset collected in the last three years. This validation will help us assess the models' performance and verify if the feature importance remains consistent with our initial findings, as shown in Figure 9. On the new dataset, the logged random forest model for process speed time achieved MAE = 0.363, MSE = 0.263, R2_score = 0.8



Figure 5: SVM training results for process speed time. (**a**) Parameters in SVM for process speed time. (**b**) Feature importance in SVM for process speed time. Figure 6: SVM training results for walk speed time. (**a**) Parameters in SVM for walk speed time; (**b**) feature importance in SVM for walk speed time. Figure 7: Random Forest training results for process speed time. (**a**) Parameters comparisons in random forest for process speed time; (**b**) feature importance in random forest for process speed time. ## 5 Related Work

ML is pivotal in MS research, facilitating disease progression prediction, identifying effective treatments, and personalizing patient care, thereby significantly enhancing our comprehension and management of MS. Research in this field often focuses on diagnostic classification and disease subtyping, utilizing SVMs [30] and KNNs [31] to accurately classify MS patients versus healthy controls and differentiate between MS subtypes. Another critical application of ML in MS is predicting disease progression and treatment response, with studies employing ML models to forecast the transition from relapsing-remitting MS to secondary progressive MS and to predict disability progression using data from motor-evoked potentials and other biomarkers [21; 32]. These studies often integrate various data sources, including clinical [23], genetic [33], and imaging data [34], to develop multimodal and accurate predictive models.