Original: While Figure 2 presents the best performing certified proportion, it is important to understand therelative performance of the Single Transitivity, Double Transitivity, and Boundary treatments, in asimilar fashion to Figures 3 and 4.
Revised: While Figure 2 presents the best performing certified proportion, it is important to understand therelative performance of the Single Transitivity, Double Transitivity, and Boundary treatments, in asimilar fashion to Figures 3 and 4.

Original: In the case of MNIST, while the percentage increases exhibitedin Figure 6 as r 1 → 1 are broadly similar to their Tiny-Imagenet counterpart for the Approximatesolver, the difference between those results and the Full derivative treatment is significantly smaller,especially at σ = 1 .
Revised: In the case of MNIST, while the percentage increases exhibitedin Figure 6 as r 1 → 1 are broadly similar to their Tiny-Imagenet counterpart for the Approximatesolver, the difference between those results and the Full derivative treatment is significantly smaller,especially at σ = 1 .

Original: This may, in part, be driven by the 1500 samples employed when using MNISTand CIFAR- 10 , in contrast to 1000 for Tiny-Imagenet, which should decrease the uncertainty of thegradient estimation steps.
Revised: This may, in part, be driven by the 1500 samples employed when using MNISTand CIFAR- 10 , in contrast to 1000 for Tiny-Imagenet, which should decrease the uncertainty of thegradient estimation steps.

Original: However, the fact that this decreased difference in performance holds for CIFAR- 10 at σ = 1 .
Revised: However, the fact that this decreased difference in performance holds for CIFAR- 10 at σ = 1 .

Original: 0but not σ = 0 .
Revised: 0but not σ = 0 .

Original: 5 suggests that the performance difference between the techniques is also dependentupon the semantic complexity of the prediction task.
Revised: 5 suggests that the performance difference between the techniques is also dependentupon the semantic complexity of the prediction task.

Original: While CIFAR- 10 is a more complex predictiveenvironment than MNIST, which should increase the complexity of the gradient based search routineemployed within this work, the increased level of noise at σ = 1 . 0 has a smoothing influence thatdecreases the complexity of the search task, and it would appear that this is the primary driver of the
Revised: While CIFAR- 10 is a more complex predictiveenvironment than MNIST, which should increase the complexity of the gradient based search routineemployed within this work, the increased level of noise at σ = 1 . 0 has a smoothing influence thatdecreases the complexity of the search task, and it would appear that this is the primary driver of the

Original: r± 0 .
Revised: r± 0 .

Original: A v e r ag e
Revised: A v e r ag e

Original: P
Revised: P

Original: e r ce n t ag e I m
Revised: e r ce n t ag e I m

Original: p
Revised: p

Original: r
Revised: r

Original: o v e
Revised: o v e

Original: m e n t a) σ =
Revised: m e n t a) σ =

Original: [Figure]
Revised: [Figure]

Original: Single (Appox)
Revised: Single (Appox)

Original: Double (Appox)
Revised: Double (Appox)

Original: Boundary (Appox)
Revised: Boundary (Appox)

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: Boundary
Revised: Boundary

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: ) σ
Revised: ) σ

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: current (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58) value, (cid:58)(cid:58) to (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58) explore (cid:58)(cid:58)(cid:58) the (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58) influence (cid:58)(cid:58) of (cid:58)(cid:58)(cid:58) the (cid:58)(cid:58)(cid:58)(cid:58) size (cid:58)(cid:58) of (cid:58)(cid:58)(cid:58) the (cid:58)(cid:58)(cid:58)(cid:58) radii (cid:58)(cid:58)(cid:58) on (cid:58)(cid:58)(cid:58) the (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58) average (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58) improvement.
Revised: current (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58) value, (cid:58)(cid:58) to (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58) explore (cid:58)(cid:58)(cid:58) the (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58) influence (cid:58)(cid:58) of (cid:58)(cid:58)(cid:58) the (cid:58)(cid:58)(cid:58)(cid:58) size (cid:58)(cid:58) of (cid:58)(cid:58)(cid:58) the (cid:58)(cid:58)(cid:58)(cid:58) radii (cid:58)(cid:58)(cid:58) on (cid:58)(cid:58)(cid:58) the (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58) average (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58) improvement.

Original: Equivalent figures for Tiny-Imagenet can be seen in Figure
Revised: Equivalent figures for Tiny-Imagenet can be seen in Figure

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Table]
Revised: [Table]

Original: improvement.
Revised: improvement.

Original: Equivalent figures for Tiny-Imagenet can be seen in Figurerelative under performance of the Approximate derivatives in both Tiny-Imagenet and CIFAR-
Revised: Equivalent figures for Tiny-Imagenet can be seen in Figurerelative under performance of the Approximate derivatives in both Tiny-Imagenet and CIFAR-

Original: [Table]
Revised: [Table]

Original: when σ = 0 .
Revised: when σ = 0 .

Original: When considering the median percentage improvement (relative to Cohen et al. ) of these techniques,
Revised: When considering the median percentage improvement (relative to Cohen et al. ) of these techniques,

Original: MNIST again reveals interesting properties when we consider Figure 10.
Revised: MNIST again reveals interesting properties when we consider Figure 10.

Original: When compared to CIFAR10 and Tiny-Imagenet (in Figures 11 and 5) it becomes apparent that the Approximate approachonly produces consistently larger certifications in MNIST.
Revised: When compared to CIFAR10 and Tiny-Imagenet (in Figures 11 and 5) it becomes apparent that the Approximate approachonly produces consistently larger certifications in MNIST.

Original: Given the increased uncertainty in thederivatives calculated by the Approximate technique, this would suggest that the approximate solvermay be improved by considering common improvements to gradient descent methods like momentumor the addition of calibrated noise.
Revised: Given the increased uncertainty in thederivatives calculated by the Approximate technique, this would suggest that the approximate solvermay be improved by considering common improvements to gradient descent methods like momentumor the addition of calibrated noise.

Original: While MNIST may be the simplest of all the prediction tasks, Figure 8 demonstrates that at lowσ < 0 .
Revised: While MNIST may be the simplest of all the prediction tasks, Figure 8 demonstrates that at lowσ < 0 .

Original: 5 the majority of samples cannot be improved upon by any of the certification enhancementsdeveloped within this paper.
Revised: 5 the majority of samples cannot be improved upon by any of the certification enhancementsdeveloped within this paper.

Original: Given that this does not hold for CIFAR- 10 nor Tiny-Imagenet (in
Revised: Given that this does not hold for CIFAR- 10 nor Tiny-Imagenet (in

Original: Figure 9 and 4 respectively) this would suggest that the potential for Cohen et al.
Revised: Figure 9 and 4 respectively) this would suggest that the potential for Cohen et al.

Original: to be improvedupon in low semantic complexity datasets
Revised: to be improvedupon in low semantic complexity datasets

Original: 3.2 σ 0 .0 .0 .0 .0 .1 .
Revised: 3.2 σ 0 .0 .0 .0 .0 .1 .

Original: P r o p o rt i o n a) Approximate 0.
Revised: P r o p o rt i o n a) Approximate 0.

Original: 3.2 σ b) Full
Revised: 3.2 σ b) Full

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)
Revised: (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)

Original: [Figure]
Revised: [Figure]

Original: (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)
Revised: (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)

Original: [Figure]
Revised: [Figure]

Original: upon (cid:58)(cid:58)(cid:58) the (cid:58)(cid:58)(cid:58)(cid:58) base (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58) certification, (cid:58)(cid:58)(cid:58) we (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58) assign (cid:58)(cid:58)(cid:58) the (cid:58)(cid:58)(cid:58)(cid:58)(cid:58) largest (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58) certification (cid:58)(cid:58)(cid:58) as (cid:58)(cid:58)(cid:58)(cid:58)(cid:58) having (cid:58)(cid:58)(cid:58)(cid:58) been (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58) calculated (cid:58)(cid:58)(cid:58) by (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)
Revised: upon (cid:58)(cid:58)(cid:58) the (cid:58)(cid:58)(cid:58)(cid:58) base (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58) certification, (cid:58)(cid:58)(cid:58) we (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58) assign (cid:58)(cid:58)(cid:58) the (cid:58)(cid:58)(cid:58)(cid:58)(cid:58) largest (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58) certification (cid:58)(cid:58)(cid:58) as (cid:58)(cid:58)(cid:58)(cid:58)(cid:58) having (cid:58)(cid:58)(cid:58)(cid:58) been (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58) calculated (cid:58)(cid:58)(cid:58) by (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)

Original: [Figure]
Revised: [Figure]

Original: et al.
Revised: et al.

Original: (cid:58)
Revised: (cid:58)

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: we (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58) assign (cid:58)(cid:58)(cid:58) the (cid:58)(cid:58)(cid:58)(cid:58)(cid:58) largest (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58) certification (cid:58)(cid:58)(cid:58) as (cid:58)(cid:58)(cid:58)(cid:58)(cid:58) having (cid:58)(cid:58)(cid:58)(cid:58) been (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58) calculated (cid:58)(cid:58)(cid:58) by (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)
Revised: we (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58) assign (cid:58)(cid:58)(cid:58) the (cid:58)(cid:58)(cid:58)(cid:58)(cid:58) largest (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58) certification (cid:58)(cid:58)(cid:58) as (cid:58)(cid:58)(cid:58)(cid:58)(cid:58) having (cid:58)(cid:58)(cid:58)(cid:58) been (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58) calculated (cid:58)(cid:58)(cid:58) by (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)

Original: Cohen et al.
Revised: Cohen et al.

Original: (cid:58) .
Revised: (cid:58) .

Original: A.3 Influence of the starting step size
Revised: A.3 Influence of the starting step size

Original: The one heretofore un-considered feature is the influence of the initial step-size γ within Algorithm 1.
Revised: The one heretofore un-considered feature is the influence of the initial step-size γ within Algorithm 1.

Original: As is shown in Figures 12 and 13, while the Full solver only exhibits sensitivity to γ when γ > 0 .
Revised: As is shown in Figures 12 and 13, while the Full solver only exhibits sensitivity to γ when γ > 0 .

Original: 1 , theapproximate solvers are far more sensitive, with deleterious performance being observed for γ > 0 .
Revised: 1 , theapproximate solvers are far more sensitive, with deleterious performance being observed for γ > 0 .

Original: 01in CIFAR- 10 , and even earlier for Tiny-Imagenet.
Revised: 01in CIFAR- 10 , and even earlier for Tiny-Imagenet.

Original: This is likely due to the added uncertainty in the
Revised: This is likely due to the added uncertainty in the

Original: Approximate derivatives leading to convergence upon local sub-optima in the more semanticallycomplex datasets.
Revised: Approximate derivatives leading to convergence upon local sub-optima in the more semanticallycomplex datasets.

Original: Based upon these results, the starting step size was uniformly set to 0 .
Revised: Based upon these results, the starting step size was uniformly set to 0 .

Original: 01 for allexperiments.
Revised: 01 for allexperiments.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: ce n t ag e I m p
Revised: ce n t ag e I m p

Original: r
Revised: r

Original: o v e
Revised: o v e

Original: m e n t
Revised: m e n t

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: by
Revised: by

Original: [Figure]
Revised: [Figure]

Original: (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58) approaches (cid:58) relative to Cohen et across
Revised: (cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58)(cid:58) approaches (cid:58) relative to Cohen et across

Original: [Figure] [Figure]
Revised: [Figure] [Figure]