Improving Certified Robustness via Statistical Learning with Logical Reasoning
Anonymous Author(s)
AffiliationAddress
Abstract
Intensive algorithmic efforts have been made to enable the rapid improvementsof certificated robustness for complex ML models recently.
However, currentrobustness certification methods are only able to certify under a limited perturbation radius.
Given that existing pure data-driven statistical approaches havereached a bottleneck, in this paper, we propose to integrate statistical ML models with knowledge (expressed as logical rules) as a reasoning component using
Markov logic networks (MLN), so as to further improve the overall certified robustness.
This opens new research questions about certifying the robustness ofsuch a paradigm, especially the reasoning component (e.g., MLN).
As the first steptowards understanding these questions, we first prove that the computational complexity of certifying the robustness of MLN is #P -hard.
Guided by this hardnessresult, we then derive the first certified robustness bound for MLN by carefullyanalyzing different model regimes.
Finally, we conduct extensive experiments onfive datasets including both high-dimensional images and natural language texts,and we show that the certified robustness with knowledge-based logical reasoningindeed significantly outperforms that of the state-of-the-arts.
Introduction
Given extensive studies on adversarial attacks against ML models recently [3, 13, 38, 24, 59, 23, 51]
building models that are robust against such attacks is an important and emerging topic.
Thus, aplethora of empirical defenses have been proposed to improve the ML robustness [29, 56, 22, 42];however, most of these are attacked again by stronger adaptive attacks [3, 1, 45].
To end such repeatedsecurity cat-and-mouse games there is a line of research focusing on developing certified defensesfor DNNs under certain adversarial constraints [8, 26, 25, 52, 27].
Though promising, existing certified defenses are restricted to certifying the model robustness withina limited ℓ p norm bounded perturbation radius [54, 8].
One potential reason for such limitationsfor existing robust learning approaches is inherent in the fact that most of them have been treatingmachine learning as a “pure data-driven" technique that solely depends on a given training set, withoutinteracting with the rich exogenous information such as domain knowledge (e.g., a stop sign shouldbe of the octagon shape ); while we know human, who has knowledge and inference abilities, isresilient to such attacks.
Indeed, a recent seminal work [17] illustrates that integrating knowledgerules can significantly improve the empirical robustness of ML models, while leaving the certifiedrobustness completely unexplored.
In this paper, we follow this promising Learning+Reasoning paradigm [17] and conduct, to ourbest knowledge, the first study on certified robustness for it.
Actually, such a Learning+Reasoningparadigm has enabled a diverse range of applications [37, 57, 2, 36, 31, 53, 17] including the ECCV’14best paper [10] that encodes label relationships as a probabilistic graphical model and improves theempirical performance of deep neural networks on ImageNet.
In this work, we first provide a concrete
Sensing-reasoning pipeline following such paradigm to integrate statistical learning with logicalreasoning as illustrated in Figure 1.
In particular, the Sensing Component contains a set of statistical
ML models such as deep neural networks (DNNs) that output their predictions as a set of Booleanrandom variables; and the Reasoning Component takes this set of Boolean random variables as inputsfor logical inference models such as Markov logic networks (MLN) [39] or Bayesian networks(BN)
[35] to produce the final output.
We then prove the hardness of certifying the robustness of sucha pipeline with MLN for reasoning.
Finally, we provide an algorithm to certify the robustness ofsensing-reasoning pipeline and we evaluate it on five datasets including both image and text data.
[Figure]
However, certifying the robustness of sensingreasoning pipeline is challenging, especiallygiven the inference complexity of the reasoningcomponent.
Our goal is to take the first step intackling this challenge.
In particular, the robustness certification of sensing-reasoning pipelinecan be expressed as the confidence interval ofthe marginal probability for the final output ofreasoning component .
That is to say, we canuse existing state-of-the-art methods to certifythe robustness of the sensing component thatcontains DNNs or ensembles [8, 41, 55].
Thus,to provide the end-to-end certification for thewhole pipeline, what is left is to understand howto certify the reasoning component , which is the focus of this work.
Compared with previous efforts focusing on certified robustness of neural networks, the reasoningcomponent brings its own challenges and opportunities.
Different from a neural network whoseinference can be executed in polynomial time, many reasoning models such as MLN can be #Pcomplete for inference.
However, as many reasoning models define a probability distribution in theexponential family, we have more functional structures that could potentially make the robustnessoptimization (which essentially solves a min-max problem) easier.
In this paper, we provide the firsttreatment to this problem characterized by these unique challenges and opportunities.
We focus on MLN as the reasoning component , and explored three technical questions, each of whichcorresponds to a technical contribution of this work.1.
Is certifying robustness for the reasoning component feasible when the inference of the reasoningcomponent is #P -hard?
(Section 3)
Before any concrete algorithm can be proposed, it is importantto understand the computational complexity of the robustness certification.
We first prove that thefamous problem of counting in statistical inference [48] can be reduced to the problem of checkingthe certified robustness of general reasoning components and MLN.
Therefore, checking certifiedrobustness is no easier than counting on the same family of distribution.
In other words, when thereasoning component is a graphical model such as MLN, checking certified robustness is no easierthan calculating the partition function of the underlying graphical model, which is #P -hard.2.
Can we efficiently reason about the certified robustness for the reasoning component when given anoracle for statistical inference?
(Section 4.2) Given the above hardness result, we focus on certifyingthe robustness given an inference oracle.
However, even when statistical inference can be done by agiven oracle [21, 18], it is still challenging to certify the robustness of MLN.
Our second technicalcontribution is to develop such an algorithm for MLN as the reasoning component.
We prove thatproviding certified robustness for MLN is possible because of the structure inherent in the probabilisticgraphical models and distributions in the exponential family, which could lead to monotonicity andconvexity properties under certain conditions for solving the certification optimization.3.
Can a reasoning component improve the certified robustness compared with the state-of-the-artcertification methods?
(Section 5)
We test our algorithms on multiple sensing-reasoning pipelines,in which the sensing components contain the state-of-the-art deep neural networks .
We constructthese pipelines to cover a range of applications including image classification and natural languageprocessing tasks.
We show that based on our certification method on the reasoning component, theknowledge-enriched sensing-reasoning pipelines achieves significantly higher certified robustnessthan the state-of-the-art certification methods for DNNs.
The rest of the paper is organized as follows.
We will first introduce the design of the sensingreasoning pipeline in Section 2.1, followed by concrete illustrations taking the Markov Logic
Networks as an example of the reasoning component in Section 2.2.
Next, to certify the robustness ofthe sensing-reasoning pipeline, especially for the reasoning component, we first prove that certifyingthe robustness of the reasoning component itself is #P -complete
(Section 3), and therefore we proposea certification algorithm to upper/lower bound the certification in Section 4, We provide the evaluationof our robustness certification considering different tasks in Section 5.
Robust Statistical Learning with Logical Reasoning
[Figure]
In this section, we first provide a sensing-reasoning pipeline and then formally defined its certified [Figure] robustness, and particularly links it to certifying the robustness for the reasoning component. [Figure]
2.1 Sensing-Reasoning Pipeline
[Figure]
A sensing-reasoning pipeline contains a set of nsensors { S i } i ∈ [ n ] and a reasoning component R .
[Figure] Each sensor is a binary classifier (for multi-classclassifier it corresponds to a group of sensors) —
[Figure] given an input data example X , each of the sensor S i outputs a probability p
i ( X ) (i.e., if S i is aneural network, p i ( X ) represents its output afterthe final softmax layer).
The reasoning component takes the outputs of all sensing models asits inputs, and outputs a new Boolean randomvariable R ( { p i ( X ) } i ∈ [ n ] ) .
One natural choice of the reasoning component is to use a probabilistic graphical model (PGM).
Inthe following subsection, we will make the reasoning component R more concrete by instantiatingit as a Markov logic network (MLN).
The output of a sensing-reasoning pipeline on the input dataexample
X is the expectation of the output of reasoning component R : E
[ R ( { p i ( X ) } i ∈ [ n ] )] .
Example.
A sensing-reasoning pipeline provides a generic, principled way of integrating domainknowledge with the output of statistical predictive models such as neural networks.
One such exampleis [10] the task of ImageNet classification.
Here each sensing model corresponds to the classifierfor one specific class in ImageNet, e.g., S dog ( X ) and S animal ( X ) .
The reasoning component thenencodes domain knowledge such that “ If an image is classified as a dog then it must also be classifiedas an animal ” using a PGM.
There is no prior work considering the certified robustness of such aknowledge-enabled ML pipeline.
Figure 2 illustrates a concrete sensing-reasoning pipeline, in whichthe reasoning component is implemented as an MLN.
2.2 Reasoning Component as Markov Logic Networks
Given the generic definition of a sensing-reasoning pipeline, one can use different models to implement the reasoning components.
In this paper, we focus on Markov logic networks (MLN), which isa popular way to define a probabilistic graphical model using first-order logic [40].
Concretely, wedefine the reasoning component implemented as an MLN, which contains a set of weighted first-orderlogic rules, as illustrated in Figure 2(b).
After grounding, an MLN defines a joint probabilisticdistribution among a collection of random variables, as illustrated in Figure 2(c).
We adapt thestandard MLN semantics to a sensing-reasoning pipeline and use a slightly more general variantcompared with the original MLN [40].
Each MLN program corresponds to a factor graph — Due tothe space limitation, we will not discuss the grounding part and point the readers to [40].
We focuson defining the result after grounding, i.e., the factor graph.
Specifically, a grounded MLN is a factor graph G =
( V , F ) , where V is a set of Boolean randomvariables.
Specific to a sensing-reasoning pipeline, there are two types of random variables V
= X ∪Y :
Interface Variables X
= { x i } i ∈ [ n ] : Each sensing model S i corresponds to one interface
variable x
i in the grounded factor graph;
Interior variables Y =
{ y i } i ∈ [ m ] are other variables introduced by the MLN model.
Each factor F ∈ F contains a weight w F and a factor function f F defined over a subset of variables¯ v F ⊆ V that returns { 0 , 1 } .
There are two sets of factors F = G ∪ H :
Interface Factors G : For each interface variable x
i , we create one interface factor G i with
weight w G i = log[ p i ( X ) / (1 − p
i ( X ))] and factor function f
G i ( a ) =
I [ a = 1] defined
over ¯ v f Gi = { x i } .
Interior Factors H are other factors introduced by the MLN program.
Remarks: MLN-specific Structure.
Our result applies to a more general family of factor graphs andare not necessarily specific to those grounded by MLN.
Moreover, MLN provides an intuitive way ofgrounding such a factor graph with domain knowledge, and factor graphs grounded by MLN havecertain properties that we will use later, e.g., all factors only return non-negative values, and there areno unusual weight sharing structures.
The above factor graph defines a joint probability distribution among all variables V .
We define apossible world as a function σ : V (cid:55)→ { 0 , 1 } that corresponds to one possible assignment of values toeach random variable.
Let Σ denote the set of all (exponentially many) possible worlds.
The statistical inference process of a reasoning component implemented using MLNs [40] computesthe marginal probability of a given variable v ∈ V :
[Equation]
where the partition functions Z 1 and Z 2 are defined as
[Equation]
i σ ( x i ) + (cid:88)
H ∈H w H f H ( σ (¯ v H ))
 
Why w G i = log[ p i ( X ) / (1 − p
i ( X ))] ?
When the MLN does not introduce any interior variables andinterior factors, it is easy to see that setting w G i = log
[ p i ( X ) / (1 − p
i ( X ))] ensures that the marginalprobability of each interface variable equals to the output of the original sensing model p i ( X ) .
Thismeans that if we do not have additional knowledge in the reasoning component, the pipeline outputsthe same distribution as the original sensing component.
Learning Weights for Interior Factors?
In this paper, we view all weights for interior factors ashyperparameters.
These weights can be learned by maximizing the likelihood with weight learningalgorithms for MLNs [28].
Beyond Marginal Probability for a Single Variable.
We have assumed that the output of a sensingreasoning pipeline is the marginal probability distribution of a given random variable in the groundedfactor graph.
However, our result can be more general — given a function over possible worlds andoutputs { 0 , 1 } , the output of a pipeline can be the marginal probability of such a function.
This willnot change the algorithm that we propose later.
Hardness of Certifying Reasoning Robustness
Given a reasoning component R , how hard is it to reason about its robustness?
In this section, weaim at understanding this fundamental question.
In order to provide the certified robustness of thereasoning component, which is defined as the lower bound of model predictions for inputs consideringan adversarial perturbation with bounded magnitude [8], we need to analyze the hardness of thiscertification problem first.
Specifically, we present the hardness results of determining the robustnessof the reasoning component defined above, before we can provide our certification algorithm in Section 4.2.
We start by defining the counting [48] and robustness problems on general distribution.
We prove that counting can be reduced to checking for reasoning robustness, and hence the latter is atleast as hard; We then prove the complexities of reasoning with MLN.
3.1 Harness of Certifying General Reasoning Model
Let X = { x 1 , x 2 , . . .
, x n } be a set of variables.
Let π α be a distribution over D [ n ] defined by a setof parameters α ∈ P [ m ] , where D is the domain of variables, either discrete or continuous, and Pis the domain of parameters.
We call π accessible if for any σ ∈ D [ n ] , π α ( σ ) ∝ w ( σ ; α ) , wherew : D [ n ] × P [ m ] → R ≥ 0 is a polynomial-time computable function.
We will restrict our attentionto accessible distributions only.
We use Q : D [ n ] → { 0 , 1 } to denote a Boolean query, which is apolynomial-time computable function.
We define the following two oracles:
Definition 1 (C OUNTING ) .
Given input polynomial-time computable weight function w ( · ) and query 190 function Q ( · ) , parameters α , a real number ϵ > 0 , a C OUNTING oracle outputs a real number Z that
[Equation]
Definition 2 (R OBUSTNESS ) .
Given input polynomial-time computable weight function w ( · ) andquery function Q ( · ) , parameters α , two real numbers ϵ > 0 and δ > 0 , a R OBUSTNESS oracledecides, for any α ′ ∈ P [ m ] such that ∥ α − α ′ ∥ ∞ ≤ ϵ , whether the following is true:
[Equation]
We can prove that R OBUSTNESS is at least as hard as C OUNTING by a reduction argument.
Theorem 1 (C OUNTING ≤ t R OBUSTNESS ) .
Given polynomial-time computable weight functionw ( · ) and query function Q ( · ) , parameters α and real number ϵ > 0 , the instance of C OUNTING ,( w, Q, α, ϵ ) can be determined by up to O (1 /ε 2 c ) queries of the R OBUSTNESS oracle with inputperturbation ϵ
= O ( ε c ) .
Proof-sketch.
We define the partition function Z i :=
(cid:80) σ : Q ( σ )=
i w ( σ ; α ) and E [ σ ∼ π α ] Q ( σ ) =
Z 1 / ( Z 0 + Z 1 ) .
We then construct a new weight function t ( σ ; α ) :
= w ( σ ; α ) exp( βQ ( σ )) by introducing an additional parameter β , such that τ β ( σ ) ∝ t ( σ ; β ) , and E [ σ ∼ τ β ] Q ( σ ) =
[Equation]
[Equation]
+
[Equation]
β [Equation] [Equation]
Then we consider the perturbation β ′ =
β ± ϵ , with ϵ =
O ( ε c ) and query the R OBUSTNESS oracle with input ( t, Q, β, ϵ, δ ) multiple times to perform a binary search in δ to estimate | E [ σ ∼ π β ] Q ( σ ) − E [ σ ∼ π β ′ ]
Q ( σ ) | .
Perform a further “outer" binary search to find theβ which maximizes the perturbation.
This yields a good estimator for log Z 0 Z 1 which in turn gives
E [ σ ∼ π α ] Q ( σ ) with ε c multiplicative error.
We leave detailed proof to Appendix A.
3.2 Hardness of Certifying Markov Logic Networks
Given Theorem 1, we can now state the following result specifically for MLNs:
Theorem 2 (MLN Hardness) .
Given an MLN whose grounded factor graph is G =
( V , F ) inwhich the weights for interface factors are w G i
= log p
i ( X ) / (1 − p
i ( X )) and constant thresholdsδ, { C i } i ∈ [ n ] , deciding whether
[Equation]
is as hard as estimating E R MLN ( { p i ( X ) } i ∈ [ n ] ) up to ε c multiplicative error, with ϵ i
= O ( ε c ) .
Proof.
Let α =
[ p i ( X )] , query function Q ( . )
=
R MLN ( . ) and π α defined by the marginal distribution over interior variables of MLN.
Theorem 1 directly implies that O (1 /ε 2 c ) queries of a
R OBUSTNESS oracle can be used to efficiently estimate E R MLN ( { p i ( X ) } i ∈ [ n ] ) .
In general, statistical inference in MLNs is #P -complete, and checking robustness for general MLNsis also #P -hard.
Certifying the Robustness of Sensing-Reasoning Pipeline
Given a sensing-reasoning pipeline with n sensors { S i } i ∈ [ n ] and a reasoning component R , we willfirst formally define its end-to-end certified robustness and then its connection to the robustnessof each component.
In particular, based on the above hardness result for certifying the robustnessof the reasoning component in Section 3, we will provide an effective certification method toupper/lower bound the certification, taking any oracle for the inference of the reasoning componentinto account. With the certification of the reasoning component, we will finally provide the robustnesscertification for the sensing-reasoning pipeline by combining the certification of sensing and reasoningcomponents.
Definition 3 ( ( C I , C E , p ) -robustness) .
A sensing-reasoning pipeline with n sensors { S i } i ∈ [ n ] and areasoning component R is ( C I , C E , p ) -robust
on the input X , if for input perturbation η, ||
η || p ≤ C
I
[Equation]
I.e., a perturbation ||
η || p <
C I on the input only changes the final pipeline output by at most C E .
Sensing Robustness and Reasoning Robustness.
We decompose the end-to-end certified robustnessof the pipeline into two components.
The first component, which we call the sensing robustness , hasbeen studied by the research community recently [20, 44, 8] — given a perturbation ||
η || p <
C I onthe input X , we say each sensor S
i is ( C I , C ( i ) S , p ) -robust
if
[Equation]
The robustness of the reasoning component R is defined as: Given a perturbation | ϵ i | < C ( i ) S on theoutput of each sensor
S i ( X ) , we say the reasoning component R is (cid:16) { C ( i ) S } i ∈ [ n ] , C E (cid:17) -robust if
[Equation]
[Equation]
It is easy to see that when the sensing component is (cid:16) C I , { C ( i ) S } i ∈ [ n ] , p (cid:17) -robust
and the reasoningcomponent is (cid:16) { C ( i ) S } i ∈
[ n ] , C E (cid:17) -robust on X , the sensing-reasoning pipeline is ( C I , C E , p ) -robust.
Since the sensing robustness has been intensively studied by previous work, in this paper, we mainlyfocus on the reasoning robustness and therefore analyze the robustness of the pipeline.
4.1 Certifying Sensing Robustness
There are several existing ways to certify the robustness of sensing models, such as Interval Bound
Propagation (IBP) [16], Randomized Smoothing [8], and others [58, 50].
Here we will leveragerandomized smoothing to provide an example for certifying the robustness of sensing components.
Corollary 1.
Given a sensing model S i , we construct a smoothed sensing model
g i ( X ; ˆ σ ) =
246 E ξ ∼N (0 , ˆ σ 2 ) p i ( X + ξ ) .
With input perturbation
||
η || 2 ≤ C
I , the smoothed sensing model satisfiesΦ(Φ
[Equation]
( g i ( X ; ˆ σ ))
− C I / ˆ σ ) ≤ g i ( X + η ; ˆ σ ) ≤ Φ(Φ − 1 ( g i ( X ; ˆ σ ))
+ C I / ˆ σ ) where Φ is the Gaussian CDF and Φ − 1 as its inverse.
is input : Oracles calculating (cid:102)
Z 1 and (cid:102) Z 2 ; maximal pertur- bations { C i } i ∈ [ n ] .
output : An upper bound for input R MLN ( { p i ( X ) + ϵ i } ) 1: R min ← 1 2: initialize λ 3: for b ∈ search budgets do 4: λ → update ( { λ } ; λ i ∈ ( −∞ , − 1] ∪ [0 , + ∞ )) 5: for i = 1 to n do 6: if λ i ≥ 0 then 7: ϵ i =
C i , ϵ ′ i = − C
i 8: else if λ i ≤ − 1 then 9: ϵ i =
− C i , ϵ ′ i
= C i 10: end if 11:
R ← (cid:102) Z 1 ( { ϵ i } i ∈ [ n ] ) − (cid:102)
Z 2 ( { ϵ ′ i } i ∈ [ n ] ) 12:
R min ← min( R min , R ) 13: end for 14: end for 15: return R min
Thus, the output probability of smoothed sensing model can be bounded given input perturbations.
Note that the specific ways of certifyingsensing robustness is orthogonal to certifyingreasoning robustness, and one can plug in different sensing certification strategies.
4.2 Certifying Reasoning Robustness
Given the hardness results for certifying reasoning robustness in Section 3.2, in this paper,we assume that we have access to an oracle forstatistical inference, and provide a novel algorithm to certify the reasoning robustness.
I.e.,we assume that we are able to calculate thetwo partition functions Z 1 ( { p i ( X ) } i ∈ [ n ] ) and
Z 2 ( { p i ( X ) } i ∈ [ n ] ) .
Lemma 4.1 (MLN Robustness) .
Given access 264 to partition functions Z 1 ( { p i ( X ) } i ∈ [ n ] ) and
Z 2 ( { p i ( X ) } i ∈ [ n ] ) , and maximum perturbations{ C i } i ∈ [ n ] , ∀ ϵ 1 , ..., ϵ n , if ∀ i. | ϵ i | < C i , we havethat ∀ λ 1 , ..., λ n ∈ R ,
[Equation] [Equation]
where
[Equation]
i
[Equation]
We leave the proof to the Appendix B.
The high-level proof idea is to decouple Z 1 /Z 2 into twosub-problems via a collection of Lagrangian multipliers, i.e., { λ i } .
For any assignment of { λ i } , weobtain a valid upper/lower bound, which reduces the certification process to the process of searchingfor an assignment of these multipliers that minimize the upper bound (maximize the lower bound).
To efficiently search for the optimal assignment of { λ i } , it is crucial to consider the interactionsbetween these { λ i } and the corresponding solution of (cid:102) Z r , which hinges on the structure of MLN.
Inparticular, we can prove the following (Detailed proofs and discussions in Appendix C):
Proposition 1 (Monotonicity) .
When λ i ≥ 0 , (cid:102) Z r ( { ϵ i } i ∈ [ n ] ) monotonically increases w.r.t.
ϵ i ; Whenλ i ≤ − 1 , (cid:102)
Z r ( { ϵ i } i ∈ [ n ] ) monotonically decreases w.r.t.
ϵ i .
Proposition 2 (Convexity) .
(cid:102)
Z r ( { ˜ ϵ i } i ∈ [ n ] ) is a convex function in ˜ ϵ
i , ∀ i with ˜ ϵ
i = log (cid:20) (1 − p
i ( X ))
( p i
[Equation]
X
[Equation]
i ) p
[Equation]
Implication.
Given the monotonicity region, the maximal and minimal of (cid:102)
Z r are achieved ateither ϵ i = − C
i or ϵ i
= C i respectively.
Given the convexity region, the maximal is achievedat ϵ i ∈ {− C
i , C i } , and the minimal is achieved at ϵ i ∈ {− C i , C i } or at the zero gradient of(cid:102)
Z r ( { ˜ ϵ i } i ∈ [ n ] ) .
As a result, our analysis leads to the following certification algorithm.
Algorithm of Certifying Reasoning Robustness.
Algorithm 1 illustrates the detailed algorithmbased on the above result to upper bound the robustness of MLN.
The main step is to explore differentregimes of the { λ i } .
In this paper, we only explore regimes where λ ∈ ( −∞ , − 1] ∪ [0 , + ∞ ) asthis already provides reasonable solutions in our experiments.
The function update ( { λ i } ) definesthe exploration strategy — Depending on the scale of the problem, one can explore { λ i } using gridsearch, random sampling, or even gradient-based methods.
For experiments in this paper, we useeither grid search or random sampling.
It is an exciting future direction to understand other efficientexploration and search strategies.
We leave the detailed explanation of the algorithm to Appendix C.
Experiments
We conduct intensive experiments on five datasets to evaluate the certified robustness of the sensingreasoning pipeline.
We focus on two tasks with different modalities: image classification task on
Road Sign dataset created based on GTSRB dataset [43] following the standard setting as [17];and information extraction task with stocks news on text data.
We also report additional results ontwo other image classification tasks (Word50 [6] and PrimateNet, which is a subset of ImageNet
ILSVRC2012 [9]) with natural knowledge rules in Appendix G and Appendix F. We also reportresults on standard image benchmarks (MNIST and CIFAR10) with manually constructed knowledge rules in Appendix H.
The code is provided at https://github.com/Sensing-Reasoning/
Sensing-Reasoning-Pipeline .
5.1 Experimental Setup
Datasets and Tasks.
For the road sign classification task, we follow [17] and use the same dataset
GTSRB [43], which contains 12 types of German road signs {"Stop”, "Priority Road”, "Yield”,"Construction Area”, "Keep Right”, "Turn Left”, "Do not Enter”, "No Vihicles”, "Speed Limit 20”,"Speed Limit 50”, "Speed Limit 120”, "End of Previous Limitation”}.
It consists of 14880 trainingsamples, 972 validation samples, and 3888 testing samples.
We also include 13 additional detectorsfor knowledge integration, detecting attributes such as whether the border has an octagon shape (See
Appendix D for a full list).
For the information extraction task, we use the HighTech dataset which consists of both daily closingasset price and financial news from 2006 to 2013 [12].
We choose 9 companies with the most news,resulting in 4810 articles related to 9 stocks filtered by company name.
We split the dataset intotraining and testing days chronologically.
We define three information extraction tasks as oursensing models: StockPrice(Day, Company, Price) , StockPriceChange(Day, Company,
Percent) , StockPriceGain(Day, Company) .
The domain knowledge that we integrate depictsthe relationships between these relations (See Appendix E for more details).
Knowledge Rules.
We integrate different types of knowledge rules for these two applications.
Weprovide the full list of knowledge rules in the Appendix D.
For road sign classification , we follow [17], which includes two different types of knowledge rules —
Indication rules (road sign class u indicates attribute v ) and Exclusion rules (attribute classes u and vwith the same general type such as "Shape”, "Color”, "Digit” or "Content” are naturally exclusive).
For information extraction , we integrate knowledge about the relationships between the sensing models (e.g., StockPrice , StockPriceChange , StockPriceGain ).
For example, the stock pricesof two consecutive days, StockPrice ( d 1 , Company, p 1 ) and StockPrice ( d 2 , Company, p 2 ) ,should be consistent with StockPriceChange ( d 2 , Company, p ) , i.e., p = ( p 2 − p 1 ) /p 1 .
Implementation Details.
Throughout the road sign classification experiment, we implement allsensing models using the GTSRB-CNN [13] architecture.
During training, we train all sensors with
[Table] Isotropic Gaussian ϵ ∼ N (0 , ˆ σ 2 I d ) augmented data with 50000 training iterations until converge andtune the training parameters on the validation set, following [8].
We use the SGD-momentum withthe initial learning rate as 0 . 01 and the weight decay parameter as 10 − 4 to train all the sensors for50000 iterations with 200 as the batch size, following [17].
During certification, we adopt the samesmoothing parameter for training to construct the smoothed model based on Monte-Carlo sampling.
For information extraction, we use BERT as our model architecture.
During training, we use thefinal hidden state of the first token
[CLS] from BERT as the representation of the whole input andapply dropout with probability p = 0 .
5 on this final hidden state.
Additionally, there is a fullyconnected layer added on top of BERT for classification.
To fine-tune the BERT classifiers for threeinformation tasks, we use the Adam optimizer with the initial learning rate as 10 − 5 and the weightdecay parameter as 10 − 4 .
We train all the sensors for 30 epochs, and the batch size 32 .
Evaluation Metrics.
We adopt the standard certified accuracy as our evaluation metric, definedby the percentage of instances that can be certified under certain ℓ p -norm bounded perturbations.
Specifically, given the input x with ground-truth label y , once we can certify the bound of themodel’s output confidence on predicting label y under the norm-bounded perturbation as [ L , U ] ,the certified accuracy can be defined by: 1 N (cid:80) Ni
=
I ([ L i > 0 . 5]) where I ( · ) denotes the indicatorfunction.
Since each sensing component’s certification is performed by randomized smoothing, whichyields the failure probability characterized by ζ 0 , we will control the failure probability ζ for thewhole sensing-reasoning pipeline pipeline with n sensing models as ζ 0 = 1 − (1 − ζ ) 1 /n by applyingthe union bound.
Throughout all the experiments, ζ is kept to 0 .
so our end-to-end certification isguaranteed to be correct with at least 99 . 9% confidence.
5.2 Results of Road Sign Classification
In this section, we evaluate the certified robustness of our sensing-reasoning pipeline under theℓ 2 -norm bounded perturbation.
We first report the ℓ 2 certified accuracy of our sensing-reasoningpipeline and compare it to a strong baseline as a vanilla randomized smoothing trained model (withoutknowledge).
Note that it is flexible to replace the sensing component with other robust trainingalgorithms.
We conduct our evaluation under different smoothing parameters ˆ σ = { 0 . 12 , 0 .
and various ℓ 2 perturbation magnitudes on the input image
C I = { 0 .
(Table 1).
During certification, we evaluate our certification time per sample with 25 sensors as 5.39s, whichshows that the overall certification time is generally acceptable.
As shown in Table 1, we can see that with knowledge integration, sensing-reasoning pipelineachieves consistently higher certified accuracy compared to the baseline smoothed ML model withoutknowledge under all the perturbation magnitudes C I and smoothing parameter ˆ σ settings.
Underthe small perturbation magnitude cases, our improvement is very significant (around 5% ).
Moreinterestingly, given large C I but small smoothing parameter ˆ σ , vanilla randomized smoothing-basedcertification directly fails ( 0% certified accuracy) due to the looseness of the hypothesis testing bound,while the sensing-reasoning pipeline could still achieve reasonable certified robustness (over 71% on
C I = 0 .
50 , 49% on C I = 1 . 00 )
under the same ( C I , ˆ σ ) settings.
This indicates a very realistic case:we always under-estimate the attacker’s ability easily under the real-world setting – in this case, thesensing-reasoning pipeline could remain robust even provide reasonable certified accuracy with aconservative smoothing parameter.
5.3 Results of Information Extraction
In this section, we conduct the certified robustness evaluation on the information extraction task ontext data.
Since there is no good certification method on discrete NLP data for sensing models, wedirectly assume the maximal perturbation on the output of sensors ( C S ).
Table 2 shows the certified [Figure] Correct [Figure]
knowledge without knowledge (a) 𝑪 𝒔 ≤ 𝟎.
(b) 𝑪 𝒔 ≤ 𝟎.(c) 𝑪 𝒔 ≤ 𝟎.(d) 𝑪 𝒔 ≤ 𝟎. 11 -  11 -
[Figure]
P[
[Figure]
] –P[
[Figure]
P [Figure] ] –P [Figure]
P [Figure] [Figure]
[Figure]
F r e qu e n cy accuracy on the final outputs of the reasoning component.
We see that the sensing-reasoning pipelineprovides significantly higher certified robustness, and even under a high perturbation magnitudeon all sensing models’ output confidence ( C S = 0 . 5 ), which means the sensing-reasoning pipelinecan still leverage the knowledge to help enhance the robustness given strong attacker.
To furtherillustrate intuitively why such knowledge-based reasoning helps, Figure 3 shows the “margin” — theprobability of the ground truth class minus the probability of the wrong class — with or withoutknowledge integration.
We see that, with knowledge integration, we can significantly increasethe number of examples with a large “margin” under adversarial perturbations.
This explains theimprovement of certified robustness, which highly relies on such prediction confident margin.
[Figure]
[Table] We also conduct experiments on PrimateNet, Word50, MNIST, CIFAR10 datasets for the imageclassification tasks in Appendix F- Appendix H. Weobserve similar results that knowledge integrationsignificantly boosts the certified robustness.
Related Work
Robustness for Single ML model and Ensemble Lots of efforts have been made to improve therobustness of single ML or ensemble models.
Adversarial training [15], and its variations [46, 30, 51]have generally been more successful in practice, but usually come at the cost of accuracy and increasedtraining time [47, 51].
To further provide certifiable robustness guarantees for ML models, variouscertifiable defenses and robustness verification approaches have been proposed [20, 44, 8].
Amongthese strategies, randomized smoothing [8] has achieved scalable performance.
With improvementsin training, including pretraining and adversarial training, the certified robustness bound can befurther improved [4, 41].
In addition to the single ML model, some work proposed to promote thediversity of classifiers and therefore develop a robust ensemble [33, 56, 55].
Although promising,these defense approaches, either empirical or theoretical, can only improve the robustness of a single
ML model.
Certifying or improving the robustness of such single or pure ensemble models is verychallenging, given that there is no additional information that can be utilized.
In practice, the MLlearning process usually favors a pipeline that is able to incorporate different sensing components aswell as domain knowledge.
Thus, certifying the robustness of such pipelines is of great importance.
Robustness of End-to-end Systems There have been intensive studies on joint inference betweenmultiple models and predictions within an end-to-end pipeline can help to improve their accuracy [53,10, 37, 32, 7, 5] and have been applied to a range of real-world applications [2, 36, 31].
Often,these approaches use different statistical inference models such as factor graphs [49], Markov logicnetworks [40], Bayesian networks [34] as a way to integrate knowledge.
In this paper, we takea different perspective on this problem — instead of treating joint inference as a way to improvethe application accuracy , we explore the possibility of using it as a way to improve the end-toend certified robustness .
A recent work [17] explores the empirical robustness improvement viaknowledge integration, while there is no guarantee provided.
As we show in this paper, by integratingdomain knowledge, we can improve the certified robustness of the ML pipeline significantly.
Conclusions
We provide the first certified robustness analysis for the sensing-reasoning pipeline with knowledgebased logical reasoning.
Our theoretical results provide the certified robustness of such ML pipelinestogether with its complexity analysis.
Our empirical results demonstrate the robustness of sensingreasoning pipeline, and we believe our work would shed light on future research towards improvingand certifying robustness for general ML frameworks as well as different ways to integrate logicalreasoning with statistical learning.