Title: Development of a Machine Learning-Based Model for Predicting Positive Margins in
high-grade squamous intraepithelial lesion(HSIL) Treatment by Cold Knife Conization(CKC):
A Single-Center Retrospective Study
Version: 1 Date: 29 Feb 2024
Reviewer:  Minh-Khang le
Reviewer's report:
Thank you for the opportunity to review this paper. Zhang et al. conducted a study to investigate
the risk factors associated with a positive margin following cervical cold knife conization. They
also developed a machine-learning model to predict the risk of a positive surgical margin. The
authors have appropriately adhered to the general guidelines for validating classification models.
I hope that my comments will be beneficial in enhancing the quality of the study:
Major comments:
1. In the multivariate analysis (Table 2), there are supposed to be nine variables (with significant
p-values in the univariate analysis). If there are reasons to exclude the other three variables, these
should be clarified in the section. Conversely, if these three variables did not exhibit statistical
significance in the multivariate analysis, this should also be transparently reported.
2. The inclusion of a confusion matrix for the model prediction in the validation set is necessary.
3. This comment is optional, but it is generally recommended to perform cross-validation of the
training set to select the best classifier among various classification models, including Support
Vector Machine, K-Nearest Neighbor, Decision Tree, Random Forest, XGBoost, and Naive
Bayes Classifiers.
The authors can still perform calibration and ROC analysis by extracting the predictions of these
models and conducting logistic regression of these predictions on the positive margin.
4. Given that the model is trained and internally validated by a relatively homogeneous cohort,
the audience may question the robustness of the model. Therefore, I suggest performing a
sensitivity analysis (SA). There are several ways to conduct an SA.
One approach is to create random masks of missing values in the test set (e.g., 10%, 20%, and
30%) and then impute the missing values using the Multiple Imputation by Chained Equation
(MICE) algorithm. Finally, we re-evaluate our model using ROC, calibration, the Hosmer-
Lemeshow test, as well as the decision curve.
5. In the Discussion section, including a richer and longer paragraph on how the model can be
integrated into the clinical workflow will significantly enhance the study’s significance.
For instance, what would the authors suggest to a clinician if the model returns “positive” results?
A closer follow-up could be a good approach. Suggesting potential clinical contexts where the
model could be applied would make the Discussion more engaging.
Minor comments:
1. In the abstract, TZ3 is not previously described in full terms, similar to CKC, OR, and HPV.
2. What method did the authors use to determine the cut-off? I presume it is the default in the
pROC package, which is based on the sum of sensitivity and specificity. It is crucial to clearly
state this in the main text because the method for finding the cut-off is critical.
3. The manuscript could benefit from refinement in English writing.