Title: Learning with Explanation Constraints
Venue: NeurIPS 2023
Accepted: yes
Keywords: Interpretable ML

Semi-supervised learning

Learning theory
Abstract: As larger deep learning models are hard to interpret, there has been a recent focus on generating explanations of these black-box models. 
In contrast, we may have apriori explanations of how models should behave. In this paper, we formalize this notion as learning from explanation constraints and provide a learning theoretic framework to analyze how such explanations can improve the learning of our models.  One may naturally ask, "When would these explanations be helpful?"
Our first key contribution addresses this question via a class of models that satisfies these explanation constraints in expectation over new data. We provide a characterization of the benefits of these models (in terms of the reduction of their Rademacher complexities) for a canonical class of explanations given by gradient information in the settings of both linear models and two layer neural networks. In addition, we provide an algorithmic solution for our framework, via a variational approximation that achieves better performance and satisfies these constraints more frequently, when compared to simpler augmented Lagrangian methods to incorporate these explanations. We demonstrate the benefits of our approach over a large array of synthetic and real-world experiments.

Learning with Explanation Constraints

Abstract

As larger deep learning models are hard to interpret, there has been a recent focus on generating explanations of these black-box models. In contrast, we may have apriori explanations of how models should behave. In this paper, we formalize this notion as learning from explanation constraints and provide a learning theoretic framework to analyze how such explanations can improve the learning of our models. One may naturally ask, 'When would these explanations be helpful?' Our first key contribution addresses this question via a class of models that satisfies these explanation constraints in expectation over new data. We provide a characterization of the benefits of these models (in terms of the reduction of their Rademacher complexities) for a canonical class of explanations given by gradient information in the settings of both linear models and two layer neural networks. In addition, we provide an algorithmic solution for our framework, via a variational approximation that achieves better performance and satisfies these constraints more frequently, when compared to simpler augmented Lagrangian methods to incorporate these explanations. We demonstrate the benefits of our approach over a large array of synthetic and real-world experiments.

1 Introduction

There has been a considerable recent focus on generating explanations of complex black-box models so that humans may better understand their decisions. These can take the form of feature importance [31, 35], counterfactuals [31, 35], influential training samples [18, 43], etc. But what if humans were able to provide explanations for how these models should behave? We are interested in the question of how to learn models given such apriori explanations. A recent line of work incorporates explanations as a regularizer, penalizing models that do not exhibit apriori given explanations [33, 32, 15, 36]. For example, Rieger et al. [32] penalize the feature importance of spurious patches on a skin-cancer classification task. These methods lead to models that inherently satisfy 'desirable' properties and, thus, are more trustworthy. In addition, some of these empirical results suggest that constraining models via explanations also leads to higher accuracy and robustness to changing test environments. However, there is no theoretical analysis to explain this phenomenon.

We note that such explanations can arise from domain experts and domain knowledge, but also other large 'teacher' models that might have been developed for related tasks. An attractive facet of the latter is that we can automatically generate model-based explanations given unlabeled data points. For instance, we can use segmentation models to select the background pixels of images solely on

low empirical

explanation risk

Unlabeled

data

Explanation

Constraints

Satisfy

Explanation

Constraints

Minimize

Empirical

Risk

Match Unlabeled

data

Labeled

data

Teacher model

Student model

unlabeled data, which we can use in our model training. We thus view incorporating explanation constraints from such teacher models as a form of knowledge distillation into our student models [13].

In this paper, we provide an analytical framework for learning from explanations to reason when and how explanations can improve the model performance. We first provide a mathematical framework for model constraints given explanations. Casting explanations as functionals g that take in a model h and input x (as is standard in explainable AI), we can represent domain knowledge of how models should behave as constraints on the values of such explanations. We can leverage these to then solve a constrained ERM problem where we additionally constrain the model to satisfy these explanation constraints. Since the explanations and constraints are provided on randomly sampled inputs, these constraints are random. Nevertheless, via standard statistical learning theoretic arguments [38], any model that satisfies the set of explanation constraints on the finite sample can be shown to satisfy the constraints in expectation up to some slack with high probability. In our work, we term a model that satisfies explanations constraints in expectation, an CE model (see Definition 1). Then, we can capture the benefit of learning with explanation constraints by analyzing the generalization capabilities of this class of CE models (Theorem 3.2). This analysis builds off of a learning theoretic framework for semi-supervised learning of Balcan and Blum [1, 2]. We remark that if the explanation constraints are arbitrary, it is not possible to reason if a model satisfies the constraints in expectation based on a finite sample. We provide a detailed discussion on when this argument is possible in Appendix B,D. In addition, we note that our work also has a connection with classical approaches in stochastic programming [16, 4] and is worth investigating this relationship further.

Another key contribution of our work is concretely analyzing this framework for a canonical class of explanation constraints given by gradient information for linear models (Theorem 4.1) and two layer neural networks (Theorem 4.2). We focus on gradient constraints as we can represent many different notions of explanations, such as feature importance and ignoring spurious features. These corollaries clearly illustrate that restricting the hypothesis class via explanation constraints can lead to fewer required labeled data. Our results also provide a quantitative measure of the benefits of the explanation constraints in terms of the number of labeled data. We also discuss when learning these explanation constraints makes sense or is possible (i.e., with a finite generalization bound). We note that our framework allows for the explanations to be noisy, and not fully satisfied by even the Bayes optimal classifier. Why then would incorporating explanation constraints help? As our analysis shows, this is by reducing the estimation error (variance) by constraining the hypothesis class, at the expense of approximation error (bias). We defer the question of how to explicitly denoise noisy explanations to future work.

Now that we have provided a learning theoretic framework for these explanation constraints, we next consider the algorithmic question: how do we solve for these explanation-constrained models? In general, these constraints are not necessarily well-behaved and are difficult to optimize. One can use augmented Lagrangian approaches [33, 7], or simply regularized versions of our constrained problems [32] (which however do not in general solve the constrained problems for non-convex parameterizations but is more computationally tractable). We draw from seminal work in posterior regularization [9], which has also been studied in the capacity of model distillation [14], to provide a variational objective. Our objective is composed of two terms; supervised empirical risk and the discrepancy between the current model and the class of CE models. The optimal solution of our objective is also the optimal solution of the constrained problem which is consistent with our theoretical analysis. Our objective naturally incorporates unlabeled data and provides a simple way to

control the trade-off between explanation constraints and the supervised loss (Section 5). We propose a tractable algorithm that iteratively trains a model on the supervised data, and then approximately projects this learnt model onto the class of CE models. Finally, we provide an extensive array of experiments that capture the benefits of learning from explanation constraints. These experiments also demonstrate that the variational approach improves over simpler augmented Lagrangian approaches and can lead to models that indeed satisfy explanations more frequently.

2 Related Work

Explainable AI. Recent advances in deep learning have led to models that achieve high performance but which are also highly complex [20, 11]. Understanding these complex models is crucial for safe and reliable deployments of these systems in the real-world. One approach to improve our understanding of a model is through explanations. This can take many forms such as feature importance [31, 35, 23, 37], high level concepts [17, 44], counterfactual examples [39, 12, 25], robustness of gradients [41], or influential training samples [18, 43].

In contrast to generating post-hoc explanations of a given model, we aim to learn models given apriori explanations. There has been some recent work along such lines. Koh et al. [19], Zarlenga et al. [45] incorporates explanations within the model architecture by requiring a conceptual bottleneck layer. Ross et al. [33], Rieger et al. [32], Ismail et al. [15], Stacey et al. [36] use explanations to modify the learning procedure for any class of models: they incorporate explanations as a regularizer, penalizing models that do not exhibit apriori given explanations; Ross et al. [33] penalize input gradients, while Rieger et al. [32] penalize a Contextual Decomposition score [26]. Some of these suggest that constraining models via explanations leads to higher accuracies and more robustness to spurious correlation, but do not provide analytical guarantees. On the theoretical front, Li et al. [22] show that models that are easier to explain locally also generalize well. However, Bilodeau et al. [3] show that common feature attribution methods without additional assumptions on the learning algorithm or data distribution do no better than random guessing at inferring counterfactual model behavior.

Learning Theory. Our contribution is to provide an analytical framework for learning from explanations that quantify the benefits of explanation constraints. Our analysis is closely related to the framework of learning with side information. Balcan and Blum [2] shows how unlabeled data can help in semi-supervised learning through a notion of compatibility between the data and the target model. This work studies classical notions of side information (e.g., margin, smoothness, and cotraining). Subsequent papers have adapted this learning theoretic framework to study the benefits of representation learning [10] and transformation invariance [34]. On the contrary, our paper focuses on the more recent notion of explanations. Rather than focus on the benefits of unlabeled data, we characterize the quality of different explanations. We highlight that constraints here are stochastic, as they depend on data points which differs from deterministic constraints that have been considered in existing literature, such as constraints on the norm of weights (i.e., L2 regularization).

Self-Training. Our work can also be connected to the self-training literature [5, 42, 40, 8], where we could view our variational objective as comprising a regularized (potentially simpler) teacher model that encodes these explanation constraints into a student model. Our variational objective (where we use simpler teacher models) is also related to distillation, which has also been studied in terms of gradients [6].

3 Learning from Explanation Constraints

̸

Let X be the instance space and Y be the label space. We focus on binary classification where Y = {-1 , 1 } , but which can be naturally generalized. Let D be the joint data distribution over ( X , Y ) and D X the marginal distribution over X . For any classifier h : X → Y , we are interested in its classification error err( h ) := Pr ( x,y ) ∼ D ( h ( x ) = y ) , though one could also use other losses to define classification error. Our goal is to learn a classifier with small error from a family of functions H . In this work, we use the words model and classifier interchangeably. Now, we formalize local explanations as functionals that take in a model and a test input, and output a vector:

Definition 1 (Explanations) . Given an instance space X , model hypothesis class H , and an explanation functional g : H×X → R r , we say g ( h, x ) is an explanation of h on point x induced by g .

For simplicity, we consider the setting when g takes a single data point and model as input, but this can be naturally extended to multiple data points and models. We can combine these explanations with prior knowledge on how explanations should look like at sample points in term of constraints.

Definition 2 (Explanation Constraint Set) . For any instance space X , hypothesis class H , an explanation functional g : H×X → R r , and a family of constraint sets { C ( x ) ⊆ R r | x ∈ X} , we say that h ∈ H satisfies the explanation constraints with respect to C iff:

In our definition, C ( x ) represents values that we believe our explanations should take at a point x . For example, 'an input gradient of a feature 1 must be larger than feature 2' can be represented by g ( h, x ) = ∇ x h ( x ) and C ( x ) = { ( x 1 , . . . , x d ) ∈ R d | x 1 > x 2 } . In practice, human annotators will be able to provide the constraint set C ( x ′ ) for a random sample k data points S E = { x ′ 1 , . . . , x ′ k } drawn i.i.d. from D X . We then say that any h ∈ H S E -satisfies the explanation constraints with respect to C iff g ( h, x ) ∈ C ( x ) , ∀ x ∈ S E . We note that the constraints depends on random samples x ′ i and therefore are random . To tackle this challenge, we can draw from the standard learning theoretic arguments to reason about probably approximately satisfying the constraints in expectation. Before doing so, we first consider the notion of explanation surrogate losses, which will allow us to generalize the setup above to a form that is amenable to practical estimators.

Definition 3. (Explanation surrogate loss) An explanation surrogate loss ϕ : H×X → R quantifies how well a model h satisfies the explanation constraint g ( h, x ) ∈ C ( x ) . For any h ∈ H , x ∈ X :

ϕ ( h, x ) ≥ 0 .

For example, we could define ϕ ( h, x ) = 1 { g ( h, x ) ∈ C ( x ) } . Given such a surrogate loss, we can substitute the explanation constraint that g ( h, x ) ∈ C ( x ) with the surrogate ϕ ( h, x ) ≤ 0 . We now have the machinery to formalize how to reason about the random explanation constraints given a random set of inputs. First, denote the expected explanation loss as ϕ ( h, D ) := E x ∼D [ ϕ ( h, x )] . We are interested in models that satisfy the explanation constraints up to some slack τ (i.e. approximately) in expectation. We define a learnability condition of this explanation surrogate loss as EPAC (Explanation Probably Approximately Correct ) learnability.

Definition 4 (EPAC learnability) . For any δ ∈ (0 , 1) , τ > 0 , the sample complexity of ( δ, τ ) - EPAC learning of H with respect to a surrogate loss ϕ , denoted m ( τ, δ ; H , ϕ ) is defined as the smallest m ∈ N for which there exists a learning rule A such that every data distribution D X over X , with probability at least 1 -δ over S ∼ D m ,

If no such m exists, define m ( τ, δ ; H , ϕ ) = ∞ . We say that H is EPAC learnable in the agnostic setting with respect to a surrogate loss ϕ if ∀ δ ∈ (0 , 1) , τ > 0 , m ( τ, δ ; H , ϕ ) is finite.

Furthermore, for a constant τ , we denote any model h ∈ H with τ -Approximately Correct Explanation where ϕ ( h, D ) ≤ τ , with a τ - CE models. We define the class of τ - CE models as

We simply use H ϕ,τ to denote this class of CE models. From natural statistical learning theoretic arguments, a model that satisfies the random constraints in S E might also be a CE model.

Proposition 3.1. Suppose a model h S E -satisfies the explanation constraints then

with probability at least 1 -δ , when k = | S E | and G = { ϕ ( h, · ) | h ∈ H} .

We use R k ( · ) to denote Rademacher complexity; please see Appendix A where we review this and related concepts. Note that even when h satisfies the constraints exactly on S , we can only guarantee a bound on the expected surrogate loss ϕ ( h, D X ) .We can achieve a bound similar to that in Proposition 3.1 via a single and simpler constraint on the empirical expectation ϕ ( h, S E ) =

1 | S E | ∑ x ∈ S E ϕ ( h, x ) . We can then extend the above proposition to show that if ϕ ( h, S E ) ≤ τ , then ϕ ( h, D X ) ≤ τ +2 R k ( G ) + √ ln(4 /δ ) 2 k , with probability at least 1 -δ . Another advantage of such a constraint is that the explanation constraints could be noisy, or it may be difficult to satisfy them exactly, so τ also serves as a slack. The class G contains all surrogate losses of any h ∈ H . Depending on the explanation constraints, G can be extremely large. We remark that the surrogate loss ϕ allows us to reason about satisfying an explanation constraint on a new data point and in expectation. However, for many constraints, ϕ does not have a closed-form or is unknown on an unseen data point. The question of which types of explanation constraints are generalizable may be of independent interest, and we further discuss this in Appendix B and provide further examples of learnable constraints in Appendix D.

EPAC-ERM Objective. Let us next discuss combining the two sources of information: the explanation constraints that we set up in the previous section, together with the usual set of labeled training samples S = { ( x 1 , y 1 ) , . . . , ( x n , y n ) } drawn i.i.d. from D that informs the empirical risk. Combining these, we get what we call EPAC-ERM objective:

We provide a learnability condition for a model that achieve both low average error and surrogate loss in Appendix F.

3.1 Generalization Bound

We assume that we are in a doubly agnostic setting. Firstly, we are agnostic in the usual sense that there need be no classifier in the hypothesis class H that perfectly labels ( x, y ) ; instead, we hope to achieve the best error rate in the hypothesis class, h ∗ = arg min h ∈H err D ( h ) . Secondly, we are also agnostic with respect to the explanations, so that the optimal classifier h ∗ may not satisfy the explanation constraints exactly, so that it incurs nonzero surrogate explanation loss ϕ ( h ∗ , D ) > 0 .

Theorem 3.2 (Generalization Bound for Agnostic Setting) . Consider a hypothesis class H , distribution D , and explanation loss ϕ . Let S = { ( x 1 , y 1 ) , . . . , ( x n , y n ) } be drawn i.i.d. from D and S E = { x ′ 1 , . . . , x ′ k } drawn i.i.d. from D X . With probability at least 1 -δ , for h ∈ H that minimizes empirical risk err S ( h ) and has ϕ ( h, S E ) ≤ τ , we have

when G = { ϕ ( h, x ) | h ∈ H , x ∈ X} and h ∗ τ = arg min h ∈H ϕ,τ err D ( h ) .

Proof. The proof largely follows the arguments in Balcan and Blum [2], but we use Rademacher complexity-based deviation bounds instead of VC-entropy. We defer the full proof to Appendix E.

Our bound suggests that these constraints help with our learning by shrinking the hypothesis class H to H ϕ,τ + ε k , reducing the required sample complexity. However, there is also a trade-off between reduction and accuracy. In our bound, we compare against the best classifier h ∗ τ -ε k ∈ H ϕ,τ -ε k instead of h ∗ . Since we may have ϕ ( h ∗ , D ) > 0 , if τ is too small, we may reduce H to a hypothesis class that does not contain any good classifiers. Recall that the generalization bound for standard supervised learning - in the absence of explanation constraints - is given by

We can see the difference between this upper bound and the upper bound in Theorem 3.2 here as a possible notion of the goodness of an explanation constraint. We further discuss this in Appendix C.

4 Gradient Explanations for Particular Hypothesis Classes

In this section, we further quantify the usefulness of explanation constraints on different concrete examples and characterize the Rademacher complexity of the restricted hypothesis classes. In particular, we consider an explanation constraint of a constraint on the input gradient. For example, we may want our model's gradient to be close to that of some h ′ ∈ H . This translates to g ( h, x ) = ∇ x h ( x ) and C ( x ) = { x ∈ R d | ∥ x -∇ x h ′ ( x ) ∥ ≤ τ } for some τ > 0 .

4.1 Gradient Explanations for Linear Models

We now consider the case of a uniform distribution on a sphere, and we use the symmetry of this distribution to derive an upper bound on the Rademacher complexity (full proof to Appendix H).

Theorem 4.1 (Rademacher complexity of linear models with a gradient constraint, uniform distribution on a sphere) . Let D X be a uniform distribution on a unit sphere in R d , let H = { h : x ↦→ ⟨ w h , x ⟩ | w h ∈ R d , || w h || 2 ≤ B } be a class of linear models with weights bounded by a constant B . Let ϕ ( h, x ) = θ ( w h , w h ′ ) be a surrogate loss where θ ( u, v ) is an angle between u, v . We have

where p = erf ( √ d sin( τ ) √ 2 ) and erf( x ) = 2 √ π ∫ x 0 e -t 2 dt is the standard error function.

The standard upper bound on the Rademacher complexity of linear models is B √ n . Our bound has a nice interpretation; we shrink our bound by a factor of ( 1 -p 2 +sin( τ ) p ) . We remark that d increases, we observe that p → 1 , so the term sin( τ ) p dominates this factor. As a consequence, we get that our bound is now scaled by sin( τ ) ≈ τ and the the Rademacher complexity scales down by a factor of τ . This implies that given n labeled data, to achieve a fast rate O ( 1 n ) , we need τ to be as good as O ( 1 √ n ) .

4.2 Gradient Explanations for Two Layer Neural Networks

Theorem 4.2 (Rademacher complexity of two layer neural networks ( m hidden nodes) with a gradient constraint) . Let X be an instance space and D X be a distribution over X with a large enough support. Let H = { h : x ↦→ ∑ m j =1 w j σ ( u ⊤ j x ) | w j ∈ R , u j ∈ R d , ∑ m j =1 | w j | ≤ B, ∥ u j ∥ 2 = 1 } be a class of two layer neural networks with a ReLU activation function and bounded weight. Assume that there exists some constant C > 0 such that E x ∼D X [ ∥ x ∥ 2 2 ] ≤ C 2 . Consider an explanation loss given by

for some τ > 0 . Then, we have that R n ( H ϕ,τ ) ≤ 3 τmC √ n .

Proof. (Sketch) The key ingredient is to identify the impact of the gradient constraint and the form of class H ϕ,τ . We provide an idea when we have m = 1 node. We write h ( x ) = wσ ( u ⊤ x ) and h ′ ( x ) = w ′ σ ( u ′⊤ x ) . Note that ∇ x h ( x ) - ∇ x h ′ ( x ) = wu 1 { u ⊤ x > 0 } -w ′ u ′ 1 { ( u ′ ) ⊤ x > 0 } is

a piecewise constant function (Figure 2). Assume that the probability mass of each region is nonnegative, our gradient constraint implies that the norm of each region cannot be larger than τ .

If u, u ′ have different directions, we have 4 regions in ∇ x h ( x ) -∇ x h ′ ( x ) and can conclude that | w | < τ, | w ′ | < τ .

If u = u ′ have the same direction, we only have 2 regions in ∇ x h ( x ) -∇ x h ′ ( x ) and can conclude that ∥ wu -w ′ u ′ ∥ = | w -w ′ | < τ .

The gradient constraint enforces a model to have the same node boundary ( u = u ′ ) with a small weight difference | w -w ′ | < τ or that node would have a small weight | w | < τ . This finding allows us to determine the restricted class H ϕ,τ , and we can use this to bound the Rademacher complexity accordingly. For full details, see Appendix I.

We compare this with the standard Rademacher complexity of a two layer neural network [24],

We can do better than this standard bound if τ < 2 B 3 m . One interpretation for this is that we have a budget at most τ to change the weight of each node and for total m nodes, we can change the weight by at most τm . We compare this to B which is an upper bound on the total weight ∑ m j =1 | w j | ≤ B . Therefore, we can do better than a standard bound when we can change the weight by at most two thirds of the average weight 2 B 3 m for each node. We would like to point out that our bound does not depend on the distribution D because we choose a specific explanation loss that guarantees that the gradient constraint holds almost everywhere. Extending to a weaker loss such as ϕ ( h, x ) = ∥∇ x h ( x ) -∇ x h ′ ( x ) ∥ is a future research direction. In contrast, our result for linear models uses a weaker explanation loss and depends on D (Theorem H.1). We also assume that there exists x with a positive probability density at any partition created by ∇ x h ( x ) . This is not a strong assumption, and it holds for any distribution where the support is the R d , e.g., Gaussian distributions.

5 Algorithms for Learning from Explanation Constraints

Although we have analyzed learning with explanation constraints, algorithms to solve this constrained optimization problem are non-trivial. In this setting, we assume that we have access to n labeled data { ( x i , y i ) } n i =1 , m unlabeled data { x n + i } i m =1 , and k data with explanations { ( x n + m + i , ϕ ( · , x n + m + i )) } k i =1 . We argue that in many cases, n labeled data are the most expensive to annotate. The k data points with explanations also have non-trivial cost; they require an expert to provide the annotated explanation or provide a surrogate loss ϕ . If the surrogate loss is specified then we can evaluate it on any unlabeled data, otherwise these data points with explanations could be expensive. On the other hand, the m data points can cheaply be obtained as they are completely unlabeled. We now consider existing approaches to incorporate this explanation information.

EPAC-ERM: Recall our EPAC-ERM objective from (2):

̸

for some constant τ . This constraint in general requires more complex optimization techniques (e.g., running multiple iterations and comparing values of τ ) to solve algorithmically. We could also consider the case where τ = 0 , which would entail the hypotheses satisfy the explanation constraints exactly, which however is in general too strong a constraint with noisy explanations.

Augmented Lagrangian objectives:

̸

As is done in prior work [32], we can consider an augmented Lagrangian objective. A crucial caveat with this approach is that the explanation surrogate loss is in general a much more complicated

functional of the hypothesis than the empirical risk. For instance, it might involve the gradient of the hypothesis when we use gradient-based explanations. Computing the gradients of such a surrogate loss can be more expensive compared to the gradients of the empirical risk. For instance, in our experiments, computing the gradients of the surrogate loss that involves input gradients is 2.5 times slower than that of the empirical risk. With the above objective, however, we need to compute the same number of gradients of both the explanation surrogate loss and the empirical risk. These computational difficulties have arguably made incorporating explanation constraints not as popular as they could be.

5.1 Variational Method

To alleviate these aforementioned computational difficulties, we propose a new variational objective

where ℓ is some loss function and t ≥ 0 is some threshold. The first term is the standard expected risk of h while the second term can be viewed as a projection distance between h and τ -CE models. It can be seen that the optimal solution of EPAC-ERM would also be an optimal solution of our proposed variational objective. The advantage of this formulation however is that it decouples the standard expected risk component from the surrogate risk component. This allows us to solve this objective with the following iterative technique, drawing inspiration from prior work in posterior regularization [9, 14]. More specifically, let h t be the learned model at time t and at each timestep t ,

We project h t to the class of τ -CE models.

The first term is the difference between h t and f on unlabeled data. The second term is the surrogate loss, which we want to be smaller than t . η is a regularization hyperparameter.

Wecalculate h t +1 that minimizes the empirical risk of labeled data and matches pseudolabels from f t +1 ,ϕ

Here, the discrepancy between h and f t +1 ,ϕ is evaluated on the unlabeled data { x j } n + m j = n +1 .

The advantage of this decoupling is that we could use a differing number of gradient steps and learning rates for the projection step that involves the complicated surrogate loss when compared to the empirical risk minimization step. Secondly, we can simplify the projection iterate computation by replacing H ϕ,τ with a simpler class of teacher models F ϕ,τ for greater efficiency. Thus, the decoupled approach to solving the EPAC-ERM objective is in general more computationally convenient.

We initialize this procedure with some model h 0 . We remark that could see this as a constraint regularized self-training where h t is a student model and f t is a teacher model. At each timestep, we project a student model to the closest teacher model that satisfies the constraint. The next student model then learns from both labeled data and pseudo labels from the teacher model. In the standard self-training, we do not have any constraint and we have f t = h t .

6 Experiments

We provide both synthetic and real-world experiments to support our theoretical results and clearly illustrate interesting tradeoffs of incorporating explanations. In our experiments, we compare our method against 3 baselines: (1) a standard supervised learning approach, (2) a simple Lagrangianregularized method (that directly penalizes the surrogate loss ϕ ), and (3) self-training, which propagates the predictions of (1) and matches them on unlabeled data. We remark that (2) captures the essence of the method in Ross et al. [33], except there is no ℓ 2 regularization term.

Our experiments demonstrate that the proposed variational approach is preferable to simple Lagrangian methods and other supervised methods in many cases. In particular, the variational approach leads to a higher accuracy under limited labeled data settings. In addition, our method leads

10

20

30

40

50

Number of Labeled Data (n)

3

2

1

0

1

MSE (log scale)

Supervised

Lagrangian

Lagrangian + Self-train

Variational

10

20

30

40

50

Number of Labeled Data (n)

2

1

0

1

2

MSE (log scale)

Supervised

Lagrangian

Lagrangian + Self-train

Variational

10

20

30

40

50

Number of Labeled Data (n)

2.0

2.5

3.0

3.5

4.0

Gradient Distance (log scale)

Supervised

Lagrangian

Lagrangian + Self-train

Variational

to models that satisfy the explanation constraints much more frequently than other baselines. We also compare to a Lagrangian-regularized + self-training baseline (first, we use the model (2) to generate pseudolabels for unlabeled data and then train a new model on both labeled and unlabeled data) in Appendix L. We remark that this baseline isn't a standard method in practice and does not fit nicely into a theoretical framework, although it seems to be the most natural approach to using unlabeled data in this procedure. More extensive ablations are deferred to Appendix N, and code to replicate our experiments will be released with the full paper.

6.1 Regression Task with Exact Gradient Information

In our synthetic experiments, we focus on a regression task where we try to learn some model contained in our hypothesis class. Our data is given by X = R d , and we try to learn a target function h ∗ : X → R . Our data distribution is given by X ∼ N (0 , σ 2 I ) , where I is a d × d identity matrix. We generate h ∗ by randomly initializing a model in the specific hypothesis class H . We assume that we have n labeled data, m unlabeled data, and k data with explanations.

We first present a synthetic experiment for learning with a perfect explanation, meaning that ϕ ( h ∗ , S ) = 0 . We consider the case where we have the exact gradient of h ∗ . Here, let H be a linear classifier and note that the exact gradient gives us the slope of the linear model, and we only need to learn the bias term. Incorporating these explanation indeed helps as both methods that include explanation constraints (Lagrangian and ours) perform much better (Figure 3).

We also demonstrate incorporating this information for two layer neural networks. We observe a clear difference between the simpler Lagrangian approach and our variational objective (Figure 4 - left). Our method is clearly the best in the setting with limited labeled data and matches the performance of the strong self-training baseline with sufficient labeled data. We note that this is somewhat expected, as these constraints primarily help in the setting with limited labeled data; with enough labeled data, standard PAC bounds suffice for strong performance.

We also analyze how strongly the approaches enforce these explanation constraints on new data points that are seen at test time (Figure 4 - right) for two layer NNs. We observe that our variational objective approaches have input gradients that more closely match the ground-truth target network's

10

15

20

25

30

35

40

45

50

Number of Labeled Data (n)

0.675

0.700

0.725

0.750

0.775

0.800

0.825

0.850

Supervised

Lagrangian

Lagrangian + Self-train

Variational

10

15

20

25

30

35

40

45

50

Number of Labeled Data (n)

0.56

0.58

0.60

0.62

0.64

0.66

0.68

0.70

0.72

Supervised

Lagrangian

Lagrangian + Self-train

Variational

input gradients. This demonstrates that, in the case of two layer NNs with gradient explanations, our approach best achieves both good performance and satisfying the constraints. Standard selftraining achieves similar performance in terms of MSE but has no notion of satisfying the explanation constraints. The Lagrangian method does not achieve the same level of satisfying these explanations as it is unable to generalize and satisfy these constraints on new data.

6.2 Tasks with Imperfect Explanations

Assuming access to perfect explanations may be unrealistic in practice, so we present experiments when our explanations are imperfect. We present classification tasks (Figure 5) from a weak supervision benchmark [46]. In this setting, we obtain explanations through the approximate gradients of a single weak labeler, as is done in [ ? ]. More explicitly, weak labelers are heuristics designed by domain experts; one example is functions that check for the presence of particular words in a sentence (e.g., checking for the word 'delicious' in a Yelp comment, which would indicate positive sentiment). We can then access gradient information from such weak labelers, which gives us a notion of feature importance about particular features in our data. We note that these examples of gradient information are rather easy to obtain, as we only need domain experts to specify simple heuristic functions for a particular task. Once given these functions, we can apply them easily over unlabeled data without requiring any example-level annotations.

We observe that our variational objective achieves better performance than all other baseline approaches on the majority of settings defined by the number of labeled data. We remark that the explanation in this dataset is a noisy gradient explanation along two feature dimensions, yet this still improves upon methods that do not incorporate this explanation constraint. Indeed, our method outperforms the Lagrangian approach, showing the benefits of iterative rounds of self-training over the unlabeled data. In addition to our real-world experiments, we present synthetic experiments with noisy gradients in Appendix K.1.

7 Discussion

Our work proposes a new learning theoretic framework that provides insight into how apriori explanations of desired model behavior can benefit the standard machine learning pipeline. The statistical benefits of explanations arise from constraining the hypothesis class: explanation samples serve to better estimate the population explanation constraint, which constrains the hypothesis class. This is to be contrasted with the statistical benefit of labeled samples, which serve to get a better estimate of the population risk. We provide instantiations of our analysis for the canonical class of gradient explanations, which captures many explanations in terms of feature importance. It would be of interest to provide corollaries for other types of explanations in future work. As mentioned before, the generality of our framework has larger implications towards incorporating constraints that are not considered as 'standard' explanations. For example, this work can be leveraged to incorporate more general notions of side information and inductive biases. We also discuss the societal impacts of our approach in Appendix O. As a whole, our paper supports using further information (e.g., explanation constraints) in the standard learning setting.

References

M.-F. Balcan and A. Blum. A pac-style model for learning from labeled and unlabeled data. In International Conference on Computational Learning Theory , pages 111-126. Springer, 2005. 2

M.-F. Balcan and A. Blum. A discriminative model for semi-supervised learning. Journal of the ACM (JACM) , 57(3):1-46, 2010. 2, 3, 5, 20

B. Bilodeau, N. Jaques, P. W. Koh, and B. Kim. Impossibility theorems for feature attribution. arXiv preprint arXiv:2212.11870 , 2022. 3

J. R. Birge and F. Louveaux. Introduction to stochastic programming . Springer Science & Business Media, 2011. 2

O. Chapelle, B. Scholkopf, and A. Zien. Semi-supervised learning (chapelle, o. et al., eds.; 2006)[book reviews]. IEEE Transactions on Neural Networks , 20(3):542-542, 2009. 3

W. M. Czarnecki, S. Osindero, M. Jaderberg, G. Swirszcz, and R. Pascanu. Sobolev training for neural networks. Advances in neural information processing systems , 30, 2017. 3

F. Fioretto, P. Van Hentenryck, T. W. Mak, C. Tran, F. Baldo, and M. Lombardi. Lagrangian duality for constrained deep learning. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases , pages 118-135. Springer, 2021. 2

S. Frei, D. Zou, Z. Chen, and Q. Gu. Self-training converts weak learners to strong learners in mixture models. In International Conference on Artificial Intelligence and Statistics , pages 8003-8021. PMLR, 2022. 3

K. Ganchev, J. Grac ¸a, J. Gillenwater, and B. Taskar. Posterior regularization for structured latent variable models. The Journal of Machine Learning Research , 11:2001-2049, 2010. 2, 8

S. Garg and Y. Liang. Functional regularization for representation learning: A unified theoretical perspective. Advances in Neural Information Processing Systems , 33:17187-17199, 2020. 3

I. Goodfellow, Y. Bengio, and A. Courville. Deep learning . MIT press, 2016. 3

Y. Goyal, Z. Wu, J. Ernst, D. Batra, D. Parikh, and S. Lee. Counterfactual visual explanations. In International Conference on Machine Learning , pages 2376-2384. PMLR, 2019. 3

G. Hinton, O. Vinyals, and J. Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 , 2015. 2

Z. Hu, X. Ma, Z. Liu, E. Hovy, and E. Xing. Harnessing deep neural networks with logic rules. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 2410-2420, 2016. 2, 8

A. A. Ismail, H. Corrada Bravo, and S. Feizi. Improving deep learning interpretability by saliency guided training. Advances in Neural Information Processing Systems , 34:26726-26739, 2021. 1, 3

P. Kall, S. W. Wallace, and P. Kall. Stochastic programming , volume 6. Springer, 1994. 2

B. Kim, M. Wattenberg, J. Gilmer, C. Cai, J. Wexler, F. Viegas, et al. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In International conference on machine learning , pages 2668-2677. PMLR, 2018. 3

P. W. Koh and P. Liang. Understanding black-box predictions via influence functions. In International conference on machine learning , pages 1885-1894. PMLR, 2017. 1, 3

P. W. Koh, T. Nguyen, Y. S. Tang, S. Mussmann, E. Pierson, B. Kim, and P. Liang. Concept bottleneck models. In International Conference on Machine Learning , pages 5338-5348. PMLR, 2020. 3

Y. LeCun, Y. Bengio, and G. Hinton. Deep learning. nature , 521(7553):436-444, 2015. 3

M. Ledoux and M. Talagrand. Probability in Banach Spaces: isoperimetry and processes , volume 23. Springer Science & Business Media, 1991. 15

J. Li, V. Nagarajan, G. Plumb, and A. Talwalkar. A learning theoretic perspective on local explainability. In International Conference on Learning Representations , 2020. 3

S. M. Lundberg and S.-I. Lee. A unified approach to interpreting model predictions. Advances in neural information processing systems , 30, 2017. 3

T. Ma. Lecture notes from machine learning theory, 2022. URL http://web.stanford.edu/ class/stats214/ . 7, 14, 26

R. K. Mothilal, A. Sharma, and C. Tan. Explaining machine learning classifiers through diverse counterfactual explanations. In Proceedings of the 2020 conference on fairness, accountability, and transparency , pages 607-617, 2020. 3

W. J. Murdoch, P. J. Liu, and B. Yu. Beyond word importance: Contextual decomposition to extract interactions from lstms. In International Conference on Learning Representations , 2018. 3

N. Natarajan, I. S. Dhillon, P. K. Ravikumar, and A. Tewari. Learning with noisy labels. Advances in neural information processing systems , 26, 2013. 17

R. Pukdee, D. Sam, P. K. Ravikumar, and N. Balcan. Label propagation with weak supervision. In The Eleventh International Conference on Learning Representations , 2022. 17

A. Ratner, S. H. Bach, H. Ehrenberg, J. Fries, S. Wu, and C. R´ e. Snorkel: Rapid training data creation with weak supervision. In Proceedings of the VLDB Endowment. International Conference on Very Large Data Bases , volume 11, page 269. NIH Public Access, 2017. 17

A. J. Ratner, C. M. De Sa, S. Wu, D. Selsam, and C. R´ e. Data programming: Creating large training sets, quickly. Advances in neural information processing systems , 29, 2016. 17, 35

M. T. Ribeiro, S. Singh, and C. Guestrin. ' why should i trust you?' explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining , pages 1135-1144, 2016. 1, 3

L. Rieger, C. Singh, W. Murdoch, and B. Yu. Interpretations are useful: penalizing explanations to align neural networks with prior knowledge. In International conference on machine learning , pages 8116-8126. PMLR, 2020. 1, 2, 3, 7

A. S. Ross, M. C. Hughes, and F. Doshi-Velez. Right for the right reasons: training differentiable models by constraining their explanations. In Proceedings of the 26th International Joint Conference on Artificial Intelligence , pages 2662-2670, 2017. 1, 2, 3, 8, 16

H. Shao, O. Montasser, and A. Blum. A theory of PAC learnability under transformation invariances. In A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, editors, Advances in Neural Information Processing Systems , 2022. URL https://openreview.net/forum?id=l1WlfNaRkKw . 3

D. Smilkov, N. Thorat, B. Kim, F. Vi´ egas, and M. Wattenberg. Smoothgrad: removing noise by adding noise. ICML Workshop on Visualization for Deep Learning, 2017 , 2017. 1, 3

J. Stacey, Y. Belinkov, and M. Rei. Supervising model attention with human explanations for robust natural language inference. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 11349-11357, 2022. 1, 3

M. Sundararajan, A. Taly, and Q. Yan. Axiomatic attribution for deep networks. In International conference on machine learning , pages 3319-3328. PMLR, 2017. 3

L. G. Valiant. A theory of the learnable. Communications of the ACM , 27(11):1134-1142, 1984. 2

S. Wachter, B. Mittelstadt, and C. Russell. Counterfactual explanations without opening the black box: Automated decisions and the gdpr. Harv. JL & Tech. , 31:841, 2017. 3

C. Wei, K. Shen, Y. Chen, and T. Ma. Theoretical analysis of self-training with deep networks on unlabeled data. In International Conference on Learning Representations , 2020. 3

M. R. Wicker, J. Heo, L. Costabello, and A. Weller. Robust explanation constraints for neural networks. In The Eleventh International Conference on Learning Representations , 2022. 3

Q. Xie, M.-T. Luong, E. Hovy, and Q. V. Le. Self-training with noisy student improves imagenet classification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 10687-10698, 2020. 3

C.-K. Yeh, J. Kim, I. E.-H. Yen, and P. K. Ravikumar. Representer point selection for explaining deep neural networks. Advances in neural information processing systems , 31, 2018. 1, 3

C.-K. Yeh, B. Kim, S. Arik, C.-L. Li, T. Pfister, and P. Ravikumar. On completeness-aware concept-based explanations in deep neural networks. Advances in Neural Information Processing Systems , 33:20554-20565, 2020. 3

M. E. Zarlenga, P. Barbiero, G. Ciravegna, G. Marra, F. Giannini, M. Diligenti, F. Precioso, S. Melacci, A. Weller, P. Lio, et al. Concept embedding models. In NeurIPS 2022-36th Conference on Neural Information Processing Systems , 2022. 3

J. Zhang, Y. Yu, Y. Li, Y. Wang, Y. Yang, M. Yang, and A. Ratner. Wrench: A comprehensive benchmark for weak supervision. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2) , 2021. 10, 38

A Uniform Convergence via Rademacher Complexity

A standard tool for providing performance guarantees of supervised learning problems is a generalization bound via uniform convergence. We will first define the Rademacher complexity and its corresponding generalization bound.

Definition 5. Let F be a family of functions mapping X → R . Let S = { x 1 , . . . , x m } be a set of examples drawn i.i.d. from a distribution D X . Then, the empirical Rademacher complexity of F is defined as

where σ 1 , . . . , σ m are independent random variables uniformly chosen from {-1 , 1 } .

Definition 6. Let F be a family of functions mapping X → R . Then, the Rademacher complexity of F is defined as

The Rademacher complexity is the expectation of the empirical Rademacher complexity, over n samples drawn i.i.d. from the distribution D X .

Theorem A.1 (Rademacher-based uniform convergence) . Let D X be a distribution over X , and F a family of functions mapping X → [0 , 1] . Let S = { x 1 , . . . , x n } be a set of samples drawn i.i.d. from D X , then with probability at least 1 -δ over our draw S ,

This holds for every function f ∈ F , and ˆ E S [ f ( x )] is expectation over a uniform distribution over S .

This bound on the empirical Rademacher complexity leads to the standard generalization bound for supervised learning.

Theorem A.2. For a binary classification setting when y ∈ {± 1 } with a zero-one loss, for H ⊂ { h : X → {1 , 1 }} be a family of binary classifiers, let S = { ( x 1 , y 1 ) , . . . , ( x n , y n ) } is drawn i.i.d. from D then with probability at least 1 -δ , we have

̸

and is the empirical error on S .

For a linear model with a bounded weights in ℓ 2 norm, the Rademacher complexity is O ( 1 √ n ) . We refer to the proof from Ma [24] for this result.

Theorem A.3 (Rademacher complexity of a linear model ([24])) . Let X be an instance space in R d , let D X be a distribution on X , let H = { h : x →⟨ w h , x ⟩ | w h ∈ R d , || w h || 2 ≤ B } be a class of linear model with weights bounded by some constant B > 0 in ℓ 2 norm. Assume that there exists a constant C > 0 such that E x ∼D X [ || x || 2 2 ] ≤ C 2 . For any S = { x 1 , . . . , x n } is drawn i.i.d. from D X , we have and

for every h ∈ H when

̸

Many of our proofs require the usage of Talgrand's lemma, which we now present.

Lemma A.4 (Talgrand's Lemma [21]) . Let ϕ : R → R be a k -Lipschitz function. Then for a hypothesis class H = { h : R d → R } , we have that

where ϕ ◦ H = { f : z ↦→ ϕ ( h ( z )) | h ∈ H} .

B Generalizable Constraints

We know that constraints C ( x ) capture human knowledge about how explanations at a point x should behave. For any constraints C ( x ) that are known apriori for all x ∈ X , we can evaluate whether a model satisfies the constraints at a point x ∈ X . This motivates us to discuss the ability of models to generalize from any finite samples S E to satisfy these constraints over X with high probability. Having access to C ( x ) is equivalent to knowing how models should behave over all possible data points in terms of explanations, which may be too strong of an assumption. Nevertheless, many forms of human knowledge can be represented by a closed-form function C ( x ) . For example,

An explanation has to take value in a fixed range can be represented by C ( x ) = Π r i =1 [ a i , b i ] , ∀ x ∈ X .

An explanation has to stay in a ball around x can be represented by C ( x ) = { u ∈ R d | || u -x || 2 ≤ r } .

An explanation has to stay in a rectangle around x 3 can be represented by C ( x ) = { u ∈ R d | x i 3 -a i ≤ u i ≤ x i 3 + b i , i = 1 , . . . , d } .

ball around

fixed interval

rectangle around

In this case, there always exists a surrogate loss that represents the explanation constraints C ( x ) ; for example, we can set ϕ ( h, x ) = 1 { g ( h, x ) ∈ C ( x ) } . On the other hand, directly specifying explanation constraints through a surrogate loss would also imply that C ( x ) is known apriori for all x ∈ X . The task of generalization to satisfy the constraint on unseen data is well-defined in this setting. Furthermore, if a surrogate loss ϕ is specified, then we can evaluate ϕ ( h, x ) on any unlabeled data point without the need for human annotators which is a desirable property.

On the other hand, we usually do not have knowledge over all data points x ∈ X ; rather, we may only know these explanation constraints over a random sample of k data points S E = { x ′ 1 , . . . , x ′ k } . If we do not know the constraint set C ( x ) , it is unclear what satisfying the constraint at an unseen data point x means. Indeed, without additional assumptions, it may not make sense to think about generalization. For example, if there is no relationship between C ( x ) for different values of x , then it is not possible to infer about C ( x ) from C ( x ′ i ) for i = 1 , . . . , k . In this case, we could define

where we are only interested in satisfying these explanation constraints over the finite sample S E . For other data points, we have ϕ ( h, x ) = 0 . This guarantees that any model with low empirical explanation loss would also achieve loss expected explanation loss, although this does not have any particular implication on any notion of generalization to new constraints. Regardless, we note that our explanation constraints still reduce the size of the hypothesis class from H to H ϕ,τ , leading to an improvement in sample complexity.

The more interesting setting, however, is when we make an additional assumption that the true (unknown) surrogate loss ϕ exists and, during training, we only have access to instances of this surrogate loss evaluated on the sample ϕ ( · , x ′ i ) . We can apply a uniform convergence argument to achieve

with probability at least 1 -δ over S E , drawn i.i.d. from D X and G = { ϕ ( h, · ) | h ∈ H} , k = | S E | . Although the complexity term R k ( G ) is unknown (since ϕ is unknown), we can upper bound this by the complexity of a class of functions Φ (e.g., neural networks) that is large enough to wellapproximate any ϕ ( h, · ) ∈ G , meaning that R k ( G ) ≤ R k (Φ) . Comparing to the former case when C ( x ) is known for all x ∈ X apriori, the generalization bound has a term that increases from R k ( G ) to R k (Φ) , which may require more explanation-annotated data to guarantee generalization to new data points. We note that the simpler constraints lead to a simpler surrogate loss, which in turn implies a less complex upper bound Φ . This means that simpler constraints are easier to learn.

Nonetheless, this is a more realistic setting when explanation constraints are hard to acquire and we do not have the constraints for all data points in X . For example, Ross et al. [33] considers an image classification task on MNIST, and imposes an explanation constraint in terms of penalizing the input gradient of the background of images. In essence, the idea is that the background should be less important than the foreground for the classification task. In general, this constraint does not have a closed-form expression, and we do not even have access to the constraint for unseen data points. However, if we assume that a surrogate loss ϕ ( h, · ) can be well-approximated by two layer neural networks, then our generalization bound allows us to reason about the ability of model to generalize and ignore background features on new data.

C Goodness of an explanation constraint

Definition 7 (Goodness of an explanation constraint) . For a hypothesis class H , a distribution D and an explanation loss ϕ , the goodness of ϕ with respect to a threshold τ and n labeled examples is:

Here, we assume access to infinite explanation data so that ε k → 0 . The goodness depends on the number of labeled examples n and a threshold t . In our definition, a good explanation constraint leads to a reduction in the complexity of H while still containing a classifier with low error. This suggests that the benefits from explanation constraints exhibit diminishing returns as n becomes large. In fact, as n →∞ , we have R n ( H ) → 0 , R n ( H ϕ,τ ) → 0 which implies G n ( ϕ, H ) → err D ( h ∗ ) -err D ( h ∗ τ ) ≤ 0 . On the other hand, explanation constraints help when n is small. For t large enough, we expect err D ( h ∗ ) -err D ( h ∗ τ ) to be small, so that our notion of goodness is dominated by the first term: R n ( H ) -R n ( H ϕ,τ ) , which has the simple interpretation of reduction in model complexity.

D Examples for Generalizable constraints

In this section, we look at the Rademacher complexity of G for different explanation constraints to characterize how many samples with explanation constraints are required in order to generalize to satisfying the explanation constraints on unseen data. We remark that this is a different notion of sample complexity; these unlabeled data require annotations of explanation constraints, not standard labels. In practice, this can be easier and less expertise might be necessary if define the surrogate loss ϕ directly. First, we analyze the case where our explanation is given by the gradient of a linear model.

Proposition D.1 (Learning a gradient constraint for linear models) . Let D be a distribution over R d . Let H = { h : x ↦→⟨ w h , x ⟩ | w h ∈ R d , ∥ w h ∥ 2 ≤ B } be a class of linear models that pass through the origin. Let ϕ ( h, x ) = θ ( w h , w h ′ ) be a surrogate explanation loss. Let G = { ϕ ( h, · ) | h ∈ H} , then we have

Proof. For a linear separator, ϕ ( h, · ) is a constant function over X . The Rademacher complexity is given by

We compare this with the Rademacher complexity of linear models which is given by R m ( H ) ≤ B √ m . The upper bound does not depend on the upper bound on the weight B . In practice, we know that the gradient of a linear model is constant for any data point. This implies that knowing a gradient of a single point is enough to identify the gradient of the linear model.

We consider another type of explanation constraint that is given by a noisy model. Here, we could observe either a noisy classifier and noisy regressor, and the constraint could be given by having similar outputs to this noisy model. This is reminiscent of learning with noisy labels [27] or weak supervision [30, 29, 28]. In this case, our explanation g is simply the hypothesis element h itself, and our constraint is on the values that h ( x ) can take. We first analyze this in the classification setting.

̸

Proposition D.2 (Learning a constraint given by a noisy classifier) . Let D be a distribution over R d . Consider a binary classification task with Y = {-1 , 1 } . Let H be a hypothesis class. Let ϕ ( h, x ) = 1[ h ( x ) = h ′ ( x )] be a surrogate explanation loss. Let G = { ϕ ( h, · ) | h ∈ H} , then we have

Proof.

Here, to learn the restriction of G is on the same order of R n ( H ) . For a given noisy regressor, we observe slightly different upper bound.

Proposition D.3 (Learning a constraint given by a noisy regressor) . Let D be a distribution over R d . Consider a regression task with Y = R . Let H be a hypothesis class that ∀ h ∈ H , -h ∈ H . Let ϕ ( h, x ) = | h ( x ) -h ′ ( x ) | be a surrogate explanation loss. Let G = { ϕ ( h, · ) | h ∈ H} , then we have

Proof.

where in the last line, we apply Talgrand's lemma A.4 and note that the max function max(0 , h ( x )) is 1-Lipschitz; in the third line, we note that we break up the supremum as both terms by definition of the max function are non-negative. Then, noting that we do not optimize over h ′ ( x ) , we further simplify this as

As mentioned before, knowing apriori surrogate loss ϕ might be too strong. In practice, we may only have access to the instances ϕ ( · , x i ) on a set of samples S = { x 1 , . . . , x k } . We also consider the case when ϕ ( h, x ) = | h ( x ) -h ′ ( x ) | when h ′ is unknown and h ′ belongs to a learnable class C .

Proposition D.4 (Learning a constraint given by a noisy regressor from some learnable class C ) . Assume D is a distribution over R d . Let H and D be hypothesis classes. Let ϕ h ′ ( h, x ) = | h ( x ) -h ′ ( x ) | be a surrogate explanation loss of a constraint corresponding to h ′ . Let G C = { ϕ h ′ ( h, · ) | h ∈ H , h ′ ∈ C} , then we have

Proof.

where the lasts line again holds by an application of Talgrand's lemma. In this case, we indeed are optimizing over h ′ , so we get that

We remark that while this value is much larger than that of R n ( H ) , we only need information about ϕ ( h, x ) and not the true label. Therefore, in many cases, this is preferable and not as expensive to learn.

E Proof of Theorem 3.2

We consider the agnostic setting of Theorem 3.2. Here, we have two notions of deviations; one is deviation in a model's ability to satisfy explanations, and the other is a model's ability to generalize to correctly produce the target function.

Proof. From Rademacher-based uniform convergence, for any h ∈ H , with probability at least 1 -δ/ 2 over S E

Therefore, with probability at least 1 -δ/ 2 , for any h ∈ H ϕ,t -ε k we also have ϕ ( h, S E ) ≤ t and for any h with ϕ ( h, S E ) ≤ t , we have h ∈ H ϕ,t + ε k . In addition, by a uniform convergence bound, with probability at least 1 -δ/ 2 , for any h ∈ H ϕ,t + ε k

Now, let h ′ be the minimizer of err S ( h ) given that ϕ ( h, S E ) ≤ t . By previous results, with probability 1 -δ , we have h ′ ∈ H ϕ,t + ε k and

F EPAC-PAC learnability

We note that in our definition of EPAC learnability, we are only concerned with whether a model achieves a lower surrogate loss than τ . However, the objective of minimizing the EPAC-ERM objective is to achieve both low average error and low surrogate loss. We characterize this property as EPAC-PAC learnability.

Definition 8 (EPAC-PAC learnability) . For any δ ∈ (0 , 1) , τ > 0 , the sample complexity of ( δ, τ, γ ) - EPAC learning of H with respect to a surrogate loss ϕ , denoted m ( δ, τ, γ ; H , ϕ ) is defined as the smallest m ∈ N for which there exists a learning rule A such that every data distribution D X over X , with probability at least 1 -δ over S ∼ D m ,

If no such m exists, define m ( δ, τ, γ ; H , ϕ ) = ∞ . We say that H is EPAC-PAC learnable in the agnostic setting with respect to a surrogate loss ϕ if ∀ δ ∈ (0 , 1) , τ > 0 , m ( δ, τ, γ ; H , ϕ ) is finite.

G A Generalization Bound in the Realizable Setting

In this section, we assume that we are in the doubly realizable [2] setting where there exists h ∗ ∈ H such that err D ( h ∗ ) = 0 and ϕ ( h ∗ , D ) = 0 . The optimal classifier h ∗ lies in H and also achieve zero expected explanation loss. In this case, we want to output a hypothesis h that achieve both zero empirical risk and empirical explanation risk.

Theorem G.1 (Generalization bound for the doubly realizable setting) . For a hypothesis class H , a distribution D and an explanation loss ϕ . Assume that there exists h ∗ ∈ H that err D ( h ∗ ) = 0 and ϕ ( h ∗ , D ) = 0 . Let S = { ( x 1 , y 1 ) , . . . , ( x n , y n ) } is drawn i.i.d. from D and S E = { x ′ 1 , . . . , x ′ k } drawn i.i.d. from D X . With probability at least 1 -δ , for any h ∈ H that err S ( h ) = 0 and ϕ ( h, S E ) = 0 , we have

when

Proof. We first consider only classifiers than has low empirical explanation loss and then perform standard supervised learning. From Rademacher-based uniform convergence, for any h ∈ H , with probability at least 1 -δ/ 2 over S E

and

when G = { ϕ ( h, x ) | h ∈ H , x ∈ X} . Therefore, for any h ∈ H with ϕ ( h, S E ) = 0 , we have h ∈ H ϕ,ε k with probability at least 1 -δ/ 2 . Now, we can apply the uniform convergence on H ϕ,ε k . For any h ∈ H ϕ,ε k with err S ( h ) = 0 , with probability at least 1 -δ/ 2 , we have

Therefore, for h ∈ H that ϕ ( h, S E ) = 0 , err S ( h ) = 0 , we have our desired guarantee.

We remark that, since our result relies on the underlying techniques of the Rademacher complexity, our result is on the order of O ( 1 √ n ) . In the (doubly) realizable setting, this is somewhat loose, and more complicated techniques are required to produce tighter bounds. We leave this as an interesting direction for future work.

H Rademacher Complexity of Linear Models with a Gradient Constraint

We calculate the empirical Rademacher complexity of a linear model under a gradient constraint.

Theorem H.1 (Empirical Rademacher complexity of linear models with a gradient constraint) . Let X be an instance space in R d , let D X be a distribution on X , let H = { h : x → ⟨ w h , x ⟩ | w h ∈ R d , || w h || 2 ≤ B } be a class of linear model with weights bounded by some constant B > 0 in ℓ 2 norm. Assume that there exists a constant C > 0 such that E x ∼D X [ || x || 2 2 ] ≤ C 2 . Assume that we have an explanation constraint in terms of gradient constraint; we want the gradient of our linear model to be close to the gradient of some linear model h ′ . Let ϕ ( h, x ) = θ ( w h , w h ′ ) be an explanation surrogate loss when θ ( u, v ) is an angle between u, v . For any S = { x 1 , . . . , x n } is drawn i.i.d. from D X , we have

when v = ∑ n i =1 x i σ i and

For the proof, we refer to Appendix H for full proof. We compare this with the standard bound on linear models which is given by

.

The benefits of the explanation constraints depend on the underlying data distribution; in the case of linear models with a gradient constraint, this depends on an angle between v = ∑ n i =1 x i σ i and w ′ . The explanation constraint reduces the term inside the expectation by a factor of f ( v ) depending on θ ( v, w ′ ) . When θ ( v, w ′ ) ≤ τ then f ( v ) = 1 which implies that there is no reduction. The value of f ( v ) decreases as the angle between θ ( v, w ′ ) increases and reaches f ( v ) = 0 when θ ( v, w ′ ) ≥ π 2 + τ . When the data is concentrated around the area of w ′ , the possible regions for v would be close to w ′ or -w ′ (Figure 8 (Top)). The value of f ( v ) in this region would be either 1 or 0 and the reduction would be 1 2 on average. In essence, this means that the gradient constraint of being close to w ′ does not actually tell us much information beyond the information from the data distribution. On the other hand, when the data points are nearly orthogonal to w ′ , the possible regions for v would lead to a small f ( v ) (Figure 8 (Bottom)). This can lead to a large reduction in complexity. Intuitively, when the data is nearly orthogonal to w ′ , there are many valid linear models including those not close in angle to w ′ . The constraints allows us to effectively shrink down the class of linear models that are close to w ′ .

Proof. (Proof of Theorem H.1) Recall that H ϕ,τ = { h : x → ⟨ w h , x ⟩ | w h ∈ R d , || w h || 2 ≤ B,θ ( w h , w h ′ ) ≤ τ } . For a set of sample S , the empirical Rademacher complexity of H ϕ,τ is given by

For a vector w ′ ∈ R d with ∥ w ′ ∥ 2 = 1 , and a vector v ∈ R d , we will claim the following,

If θ ( v, w ′ ) ≤ τ , we have

If π 2 + τ ≤ θ ( v, w ′ ) ≤ π , we have

If τ ≤ θ ( v, w ′ ) ≤ π 2 + τ , we have

For the first claim, we can see that if θ ( v, w ′ ) ≤ τ , we can pick w = Bv ∥ v ∥ and achieve the optimum value. For the second claim, we use the fact that θ ( · , · ) satisfies a triangle inequality and for any w that θ ( w,w ′ ) ≤ τ , we have

This implies that for any w that θ ( w,w ′ ) ≤ τ , we have ⟨ w,v ⟩ = ∥ w ∥∥ v ∥ cos( θ ( v, w )) ≤ 0 and the supremum is given by 0 where we can set ∥ w ∥ = 0 . For the third claim, we know that ⟨ w,v ⟩ is maximum when the angle between v, w is the smallest. From the triangle inequality above, we must have θ ( w,w ′ ) = τ to be the largest possible value so that we have the smallest lower bound θ ( v, w ) ≥ θ ( v, w ′ ) -θ ( w,w ′ ) . In addition, the inequality holds when v, w ′ , w lie on the same plane. Since we do not have further restrictions on w , there exists such w and we have

as required. One can calculate a closed form formula for w by solving a quadratic equation. Let w = B ˜ w ∥ ˜ w ∥ when ˜ w = v + λw ′ for some constant λ > 0 such that θ ( w,w ′ ) = τ . With this we have an equation

Let µ = ⟨ v, w ′ ⟩ , solving for λ , we have

Solve this quadratic equation, we have

Since λ > 0 , we have λ = -µ +cot( τ ) √ ∥ v ∥ 2 -µ 2 . We have

With these claims, we have

Proof. From Theorem H.1, we have that

when v = ∑ n i =1 x i σ i . Because x i is drawn uniformly from a unit sphere, in expectation θ ( v, w ′ ) has a uniform distribution over [0 , π ] and the distribution ∥ v ∥ for a fixed value of θ ( v, w ′ ) are the same for all θ ( v, w ′ ) ∈ [0 , π ] . From Trigonometry, we note that

where

By the symmetry property and the uniformity of the distribution of θ ( v, w ′ ) and ∥ v ∥ .

when ˜ θ ( v, w ′ ) = π 2 -θ ( v, w ′ ) . We have

The last equation follows from the symmetry and uniformity properties. We can bound the first expectation

Next, we can simply note that, since our data is distributed over a unit sphere, each data has norm no greater than 1. Therefore, we know that C = 1 is indeed an upper bound on E x ∼D X [ || x || 2 2 ] . For the probability term, we note that in expectation v has the same distribution as a random vector u drawn uniformly from a unit sphere. We let this be some probability p :

We know that the projection ⟨ u, w ′ ⟩ ∼ N (0 , 1 d ) . Then, we have that |⟨ u, w ′ ⟩| is given by a Folded Normal Distribution, which has a CDF given by

We then observe that

Plugging this in yields the following bound

where

I Rademacher Complexity for Two Layer Neural Networks with a Gradient Constraint

Here, we present the full proof of the generalization bound for two layer neural networks with gradient explanations. In our proof, we use two results from Ma [24]. One result is a technical lemma, and the other is a bound on the Rademacher complexity of two layer neural networks.

Lemma I.1. Consider a set S = { x 1 , ..., x n } and a hypothesis class F ⊂ { f : R d → R } . If

then, we have that

Theorem I.2 (Rademacher complexity for two layer neural networks [24]) . Let X be an instance space and D X be a distribution over X . Let H = { h : x ↦→ ∑ m i =1 w i σ ( u ⊤ i x ) | w i ∈ R , u i ∈ R d , ∑ m i =1 | w i |∥ u i ∥ 2 ≤ B } be a class of two layer neural networks with m hidden nodes with a ReLU activation function σ ( x ) = max(0 , x ) . Assume that there exists some constant C > 0 such that E x ∼D X [ ∥ x ∥ 2 2 ] ≤ C 2 . Then, for any S = { x 1 , . . . , x n } is drawn i.i.d. from D X , we have that

We defer interested readers to [24] for the full proof of this result. Here, the only requirement of the data distribution is that E x ∼D X [ ∥ x ∥ 2 2 ] ≤ C 2 . We now present our result in the setting of two layer neural networks with one hidden node m = 1 to provide clearer intuition for the overall proof.

Theorem I.3 (Rademacher complexity for two layer neural networks ( m = 1 ) with gradient constraints) . Let X be an instance space and D X be a distribution over X . Let H = { h : x ↦→ wσ ( u ⊤ x ) | w ∈ R , u ∈ R d , | w | ≤ B, ∥ u ∥ = 1 } . Without loss of generality, we assume that ∥ u ∥ = 1 . Assume that there exists some constant C > 0 such that E x ∼D X [ ∥ x ∥ 2 2 ] ≤ C 2 . Our explanation constraint is given by a constraint on the gradient of our models, where we want the gradient of our learnt model to be close to a particular target function h ′ ∈ H . Let this be represented by an explanation loss given by

for some τ > 0 . Let h ′ ( x ) = w ′ σ (( u ′ ) ⊤ x ) the target function, then we have

and

Proof. Our choice of ϕ ( h, x ) guarantees that, for any h ∈ H ϕ,τ , we have that ∥∇ x h ( x ) -∇ x h ′ ( x ) ∥ ≤ τ almost everywhere. We note that for h ( x ) = wσ ( u ⊤ x ) , the gradient is given by ∇ x h ( x ) = wu 1 { u ⊤ x > 0 } , which is a piecewise constant function over two regions (i.e., u ⊤ x > 0 , u ⊤ x ≤ 0) , captured by Figure I.

We now consider ∇ x h ( x ) -∇ x h ′ ( x ) , and we have 3 possible cases.

Case 1: θ ( u, u ′ ) > 0

This implies that the boundaries of ∇ x ( h ) and ∇ x h ′ ( x ) are different. Then, we have that ∇ x h ( x ) -∇ x h ′ ( x ) is a piecewise constant function with 4 regions, taking on values

If we assume that each region has probability mass greater than 0 then our constraint ∥∇ x h ( x ) -∇ x h ′ ( x ) ∥ 2 ≤ τ implies that | w | = | w |∥ u ∥ ≤ τ, | w ′ | = | w ′ |∥ u ′ ∥ ≤ τ, ∥ wu -w ′ u ′ ∥ ≤ τ .

Case 2: θ ( u, u ′ ) = 0

This implies that the boundary of ∇ x h ( x ) and ∇ x h ′ ( x ) are the same. Then, ∇ x h ( x ) -∇ x h ′ ( x ) is a piecewise constant over two regions

This gives us that | w -w ′ | = ∥ wu -w ′ u ′ ∥ ≤ τ .

Case 3: θ ( u, u ′ ) = π

Here, we have that the decision boundaries of ∇ x h ( x ) and ∇ x h ′ ( x ) are the same but the gradients are non-zero on different sides. Then, ∇ x h ( x ) -∇ x h ′ ( x ) is a piecewise constant on two regions

This gives us that | w | ≤ τ and | w ′ | ≤ τ .

These different cases tell us that the constraint ∥∇ x h ( x ) -∇ x h ′ ( x ) ∥ ≤ τ reduces H into a class of models follows either

u = u ′ and | w -w ′ | < τ .

̸

u = u ′ and | w | < τ . However, this case only possible when | w ′ | < τ .

If | w ′ | > τ , we know that we must only have the first case. Now, we can calculate the Rademacher complexity of our restricted class H ϕ,τ . We will again do this in separate cases.

′

Case 1: | w | > τ For any h ∈ H ϕ,τ , we have that u = u ′ and | w -w ′ | < τ . For a sample S = { x 1 , ..., x n } ,

Since, | w -w ′ | < τ ,

Therefore, we have

Now, we can calculate the Rademacher complexity as

Then, we can compute the supremum over w as

)

Combining this with the fact that E [ ∥ x ∥ 2 ] ≤ C 2 , we have

Case 2: | w ′ |∥ u ′ ∥ < τ .

We know that H ϕ,τ = H (1) ϕ,τ ⋃ H (2) ϕ,τ when

We have

The second line holds as sup x ∈ A ∪ B f ( x ) ≤ sup x ∈ A f ( x ) + sup x ∈ B f ( x ) when sup x ∈ A f ( x ) ≥ 0 and sup x ∈ B f ( x ) ≥ 0 . We know that both of these supremums be greater than zero, as we can recover the value of 0 with w = 0 . From Case 1 , we know that

We also note that H (2) ϕ,τ is a class of two layer neural networks with weights with norms bounded by τ . From Theorem I.2, we have that

Therefore, in Case 2 , as required.

Now, we consider in the general setting (i.e., no restriction on m ).

Theorem I.4 (Rademacher complexity for two layer neural networks with gradient constraints ) . Let X be an instance space and D X be a distribution over X with a large enough support. Let H = { h : x ↦→ ∑ m j =1 w j σ ( u ⊤ j x ) | w j ∈ R , u j ∈ R d , ∥ u j ∥ 2 = 1 , ∑ m j =1 | w j | ≤ B } . Assume that there exists some constant C > 0 such that E x ∼D X [ ∥ x ∥ 2 2 ] ≤ C 2 . Our explanation constraint is given by a constraint on the gradient of our models, where we want the gradient of our learnt model to be close to a particular target function h ′ ∈ H . Let this be represented by an explanation loss given by

̸

for some τ > 0 . Then, we have that

To be precise,

when q is the number of node j of h ′ such that | w ′ j | < τ .

We note that this result indeed depends on the number of hidden dimensions m ; however, we note that in the general case (Theorem I.2), the value of B is O ( m ) as it is a sum over the values of each hidden node. We now present the proof for the more general version of our theorem.

Proof. For simplicity, we first assume that any h ∈ H has that ∥ u j ∥ = 1 , ∀ j . Consider h ∈ H , we write h = ∑ m j =1 w ′ j σ (( u ′ j ) ⊤ x ) and let h ′ ( x ) = ∑ m j =1 w ′ j σ (( u ′ j ) ⊤ x ) be a function for our gradient constraint. The gradient of a hypothesis h is given by

which is a piecewise constant function over at most 2 m regions. Then, we consider that

which is a piecewise constant function over at most 2 2 m regions. We again make an assumption that each of these regions has a non-zero probability mass. Our choice of ϕ ( h, x ) guarantees that the norm of the gradient in each region is less than τ . Similar to the case with m = 1 , we will show that the gradient constraint leads to a class of functions with the same decision boundary or neural networks that have weights with a small norm.

Assume that among u 1 , ..., u m there are k vectors that have the same direction as u ′ 1 , ..., u ′ m . Without loss of generality, let u j = u ′ j for j = 1 , ..., k . In this case, we have that ∇ x h ( x ) - ∇ x h ′ ( x ) is a piecewise function over 2 2 m -k regions. As each region has non-zero probability mass, for each j ∈ { 1 , ..., k } , we know that ∃ x such that

̸

̸

In other words, we can observe a data point from each region that uniquely defines the value of a particular w j , u j . In this case, we have that

From our gradient constraint, we know that ||∇ x h ( x ) - ∇ x h ′ ( x ) || ≤ τ, ∀ x , which implies that | w j -w ′ j | ≤ τ for j = 1 , ..., k .

On the other hand, for the remaining j = k +1 , ..., m , we know that there exists x such that

̸

Then, we have that ∇ x h ( x ) = w j u j , and our constraint implies that | w j |∥ u j ∥ = | w j | ≤ τ . Similarly, we have that | w ′ j |∥ u ′ j ∥ = | w ′ j | < τ, for j = k +1 , ..., m . We can conclude that H ϕ,τ is a class of two layer neural networks with m hidden nodes (assuming ∥ u i ∥ = 1) that for each node w j σ ( u ⊤ j x ) satisfies

There exists l ∈ [ m ] that u j = u ′ l and | w j -w ′ l | < τ .

We further note that for a node w ′ l σ (( u ′ l ) ⊤ x ) in h ′ ( x ) that has that a high weight | w ′ l | > τ , there must be a node w j σ ( u ⊤ j x ) in h with the same boundary u j = u l . Otherwise, there is a contradiction with | w ′ l | < τ for all nodes in h ′ without a node in h with the same boundary. We can utilize this characterization of the restricted class H ϕ,τ to bound the Rademacher complexity of the class. Let

This is a class of two layer neural networks with at most m nodes such that each node is from h ′ . We also have a condition that if the weight of the j -th node in h ′ is greater than τ , the j -th node must be present in any member of this class. Let

be a class of two layer neural networks with m nodes such that the weight of each node is at most τ . We claim that for any h ∈ H ϕ,τ there exists h 1 ∈ H ′ , h 2 ∈ H ( τ ) that h = h 1 + h 2 . For any h ∈ H ϕ,τ , let p h : [ m ] → [ m ] ∪ { 0 } be a function that match a node in h with the node with the same boundary in h ′ . Formally,

The function p h maps j to 0 if there is no node in h ′ with the same boundary. Let w ′ 0 = 0 , u ′ 0 = [0 , . . . , 0] , we can write

̸

̸

The first term is a member of H ( τ ) because we know that | w j -w ′ p ( j ) | < τ or | w j | < τ . The second term is also a member of H ′ since for any l that | w ′ l | > τ , there exists j that p h ( j ) = l . Therefore, we can write h in terms of a sum between a member of H ′ and H ( τ ) . This implies that

From Theorem I.2, we have that

Now, we will calculate the Rademacher complexity of H ′ . For S = { x 1 , . . . , x n } ,

To achieve the supremum, if w ′ j ∑ n i =1 σ (( u ′ j ) ⊤ x i ) σ i > 0 we need to set a j = 1 , otherwise, we need to set a j = 0 . Therefore,

From Theorem 4.1, we can conclude that

when q is the number of nodes j of h ′ such that | w ′ j | < τ . Therefore,

A tighter bound is given by (2 m + q ) τC √ n when q is the number of w ′ j that | w ′ j | < τ . As τ → 0 , we also have q → 0 . This implies that we have an upper bound of 2 mτC √ n if τ is small enough. When comparing this to the original bound 2 BC √ n , we can do much better if τ ≪ B m . We would like to point out that our bound does not depend on the distribution D because we choose a strong explanation loss

which guarantees that ∥∇ x h ( x ) -∇ x h ′ ( x ) ∥ 2 ≤ τ almost everywhere. We also assume that we are in a high-dimensional setting d ≫ m and there exists x with a positive probability density at any partition created by ∇ x h ( x ) .

J Algorithmic Results for Two Layer Neural Networks with a Gradient Constraint

Now that we have provided generalization bounds for the restricted class of two layer neural networks, we also present an algorithm that can identify the parameters of a two layer neural network (up to a permutation of the weights). In practice, we might solve this via our variational objective or other simpler regularized techniques. However, we also provide a theoretical result for the required amount of data (given some assumptions about the data distribution) and runtime for an algorithm to exactly recover the parameters of these networks under gradient constraints.

We again know that the gradient of two layer neural networks with ReLU activations can be written as

where we consider || u i || = 1 . Therefore, an exact gradient constraint given of the form of pairs ( x, ∇ x f ( x )) produces a system of equations.

Proposition J.1. If the values of u i 's are known, we can identify the parameters w i with exactly m fixed samples.

Proof. We can select m datapoints, which each achieve value 1 for the indicator value in the gradient of the two layer neural network. This would give us m equations, which each are of the form

Therefore, we can easily solve for the values of w i , given that u i is known.

To make this more general, we now consider the case where u i 's are not known but are at least linearly independent.

Proposition J.2. Let the u i 's be linearly independent. Assume that each region of the data (when partitioned by the values of u i ) has non-trivial support > p . Then, with probability 1 -δ , we can identify the parameters w i , u i with O ( 2 m + m +log( 1 δ ) log( 1 1 -p ) ) data points and in O (2 2 m ) time.

Proof. Let us partition X into regions satisfying unique values of the binary vector (1 { u T 1 x > 0 } , ..., 1 { u T m x > 0 } ) , which by our assumption each have at least some probability mass p . First, we calculate the probability that we observe one data point with an explanation from each region in this partition. This is equivalent to sampling from a multinomial distribution with probabilities ( p 1 , ..., p 2 m ) , where p i ≥ p, ∀ i . Then,

Pr( observe all regions in n draws ) = 1 -Pr( ∃ i s.t. we do not observe region i )

Algorithm 1 Algorithm for identifying parameters of a two layer neural network, given exact gradient constraints

Setting this as no less than 1 -δ leads to that n ≥ m +log( 1 δ ) log( 1 1 -p ) .

Given O (2 m + m +log( 1 δ ) log( 1 1 -p ) ) pairs of data and gradients, we will observe at least one pair from each region of the partition. Then, identifying the values of u i 's and w i 's is equivalent to identifying the datapoints that correspond to a value of the binary vector where only one indicator value is 1. These values can be identified in O (2 3 m ) time; the algorithm is given in Algorithm J.1. These results demonstrate that we can indeed learn the parameters (up to a permutation) of a two layer neural network given exact gradient information.

J.1 Algorithm for Identifying Regions

We first note that identifying the parameters u i 's and w i 's of a two layer neural network is equivalent to identifying the values { x 1 , ..., x m } from the set { ∑ x ∈ C x | C ∈ P ( { x 1 , ..., x m } ) } , where P denotes the power set. We also assume that x 1 , ..., x m are linearly independent, so we cannot create x i from any linear combination of x j 's with j = i . Then, we can identify the set { x 1 , ..., x m } as in Algorithm 1. This algorithm runs in O (2 3 m ) time as it iterates through each point in M and computes the overlapping set O and resulting updated sum S , which takes O (2 2 m ) time. From the resulting set B , we can exactly compute values u i and w i up to a permutation.

̸

K Additional Synthetic Experiments

We now present additional synthetic experiments that demonstrate the performance of our approach under settings with imperfect explanations and compare the benefits of using different types of explanations.

K.1 Variational Objective is Better with Noisy Gradient Explanations

Here, we present the remainder of the results from the synthetic regression task of ?? , under more settings of noise ϵ added to the gradient explanation.

Again, we observe that our method does better than that of the Lagrangian approach and the selftraining method. Under high levels of noise, the Lagrangian method does poorly. On the contrary,

20

40

60

80

Number of Labeled Data (n)

2.5

2.0

1.5

1.0

0.5

0.0

0.5

MSE (log scale)

Epsilon : 0.0001

20

40

60

80

Number of Labeled Data (n)

2.5

2.0

1.5

1.0

0.5

0.0

0.5

MSE (log scale)

Epsilon : 0.1

20

40

60

80

Number of Labeled Data (n)

2.5

2.0

1.5

1.0

0.5

0.0

0.5

MSE (log scale)

Epsilon : 0.5

20

40

60

80

Number of Labeled Data (n)

2

1

0

1

MSE (log scale)

Epsilon : 1

20

40

60

80

Number of Labeled Data (n)

3

2

1

0

1

2

3

MSE (log scale)

Epsilon : 2

Supervised

Lagrangian

Lagrangian + Self-train

Variational

our method is resistant to this noise and also outperforms self-training significantly in settings with limited labeled data.

K.2 Comparing Different Types of Explanations

Here, we present synthetic results to compare using different types of explanation constraints. We focus on comparing noisy gradients as before, as well as noisy classifiers, which are used in the setting of weak supervision [30]. Here, we generate our noisy classifiers as h ∗ ( x ) + ϵ , where ϵ ∼ N (0 , σ 2 ) . We omit the results of self-training as it does not use any explanations, and we keep the supervised method as a baseline. Here, t = 0 . 25 .

We observe different trends in performance as we vary the amount of noise in the noisy gradient or noisy classifier explanations. With any amount of noise and sufficient regularization ( λ ), this influences the overall performance of the methods that incorporate constraints. With few labeled data, using noisy classifiers helps outperform standard supervised learning. With a larger amount

20

40

60

80

Number of Labeled Data (n)

3.0

2.5

2.0

1.5

1.0

0.5

0.0

0.5

1.0

MSE (log scale)

Epsilon : 0.0001

20

40

60

80

Number of Labeled Data (n)

3.0

2.5

2.0

1.5

1.0

0.5

0.0

0.5

1.0

MSE (log scale)

Epsilon : 0.1

20

40

60

80

Number of Labeled Data (n)

3.0

2.5

2.0

1.5

1.0

0.5

0.0

0.5

1.0

MSE (log scale)

Epsilon : 0.5

20

40

60

80

Number of Labeled Data (n)

3.0

2.5

2.0

1.5

1.0

0.5

0.0

0.5

1.0

MSE (log scale)

Epsilon : 0.0001

20

40

60

80

Number of Labeled Data (n)

3.0

2.5

2.0

1.5

1.0

0.5

0.0

0.5

1.0

MSE (log scale)

Epsilon : 0.1

20

40

60

80

Number of Labeled Data (n)

3.0

2.5

2.0

1.5

1.0

0.5

0.0

0.5

1.0

MSE (log scale)

Epsilon : 0.5

Supervised

Lagrangian

Lagrangian + Self-train

Variational

of labeled data, this leads to no benefits (if not worse performance of the Lagrangian approach). However, with the noisy gradient, under small amounts of noise, the restricted class of hypothesis will still capture solutions with low error. Therefore, in this case, we observe that the Lagrangian approach outperforms standard supervised learning in the case with few labeled data and matches it with sufficient labeled data. Our method outperforms or matches both methods across all settings.

We consider another noisy setting, where noise has been added to the weights of a copy of the target two layer neural network. Here, we compare how this information impacts learning from the direct outputs (noisy classifier) or the gradients (noisy gradients) of that noisy copy (Figure 12).

20

40

60

80

Number of Labeled Data (n)

2

1

0

1

2

MSE (log scale)

Epsilon : 0.3

20

40

60

80

Number of Labeled Data (n)

2

1

0

1

2

3

4

5

MSE (log scale)

Epsilon : 0.5

20

40

60

80

Number of Labeled Data (n)

2

0

2

4

6

MSE (log scale)

Epsilon : 0.7

20

40

60

80

Number of Labeled Data (n)

3

2

1

0

1

2

MSE (log scale)

Epsilon : 0.3

20

40

60

80

Number of Labeled Data (n)

3

2

1

0

1

2

MSE (log scale)

Epsilon : 0.5

20

40

60

80

Number of Labeled Data (n)

3

2

1

0

1

2

3

MSE (log scale)

Epsilon : 0.7

Supervised

Lagrangian

Lagrangian + Self-train

Variational

L Additional Baselines

We compare against an additional baseline of a Lagrangian-regularized model + self-training on unlabeled data. We again note that this is not a standard method in practice and does not naturally fit into a theoretical framework, although we present it to compare against a method that uses both explanations and unlabeled data.

10

20

30

40

50

Number of Labeled Data (n)

3

2

1

0

1

MSE (log scale)

Supervised

Lagrangian

Lagrangian + Self-train

Variational

10

20

30

40

50

Number of Labeled Data (n)

2

1

0

1

2

MSE (log scale)

Supervised

Lagrangian

Lagrangian + Self-train

Variational

10

15

20

25

30

35

40

45

50

Number of Labeled Data (n)

0.675

0.700

0.725

0.750

0.775

0.800

0.825

0.850

Supervised

Lagrangian

Lagrangian + Self-train

Variational

10

15

20

25

30

35

40

45

50

Number of Labeled Data (n)

0.56

0.58

0.60

0.62

0.64

0.66

0.68

0.70

0.72

Supervised

Lagrangian

Lagrangian + Self-train

Variational

We observe that our method outperforms this baseline (Figure 13), again especially in the settings with limited labeled data. We observe that although this new method indeed satisfies constraints, when performing only a single round of self-training, it no longer satisfies these constraints as much. Thus, this supports the use of our method to perform multiple rounds of projections onto a set of EPAC models.

M Experimental Details

For all of our synthetic and real-world experiments, we use values of m = 1000 , k = 20 , T = 3 , τ = 0 , λ = 1 , unless otherwise noted. For our synthetic experiments, we use d = 100 , σ 2 = 5 . Our two layer neural networks have hidden dimensions of size 10. They are trained with a learning rate of 0.01 for 50 epochs. We evaluate all networks on a (synthetic) test set of size 2000.

For our real-world data, our two layer neural networks have a hidden dimension of size 10 and are trained with a learning rate of 0.1 (YouTube) and 0.1 (Yelp) for 10 epochs. λ = 0 . 01 and gradient values computed by the smoothed approximation in [ ? ] has c = 1 . Test splits are used as follows from the YouTube and Yelp datasets in the WRENCH benchmark [46].

We choose the initialization of our variational algorithm h 0 as the standard supervised model, trained using gradient descent.

N Ablations

We also perform ablation studies in the same regression setting as Section 6. We vary parameters that determine either the experimental setting or hyperparameters of our algorithms.

N.1 Number of Explanation-annotated Data

First, we vary the value of k to illustrate the benefits of our method over the existing baselines.

20

40

60

80

Number of Labeled Data (n)

2

1

0

1

MSE (log scale)

k : 20

20

40

60

80

Number of Labeled Data (n)

2

1

0

1

MSE (log scale)

k : 40

20

40

60

80

Number of Labeled Data (n)

3

2

1

0

1

MSE (log scale)

k : 60

20

40

60

80

Number of Labeled Data (n)

3

2

1

0

1

MSE (log scale)

k : 80

20

40

60

80

Number of Labeled Data (n)

2

1

0

1

MSE (log scale)

k : 100

20

40

60

80

Number of Labeled Data (n)

2

1

0

1

MSE (log scale)

k : 200

Supervised

Lagrangian

Lagrangian + Self-train

Variational

We observe that our variational approach performs much better than a simple augmented Lagrangian method, which in turn does better than supervised learning with sufficiently large values of k . Our approach is always better than the standard supervised approach.

We also provide results for how well these methods satisfy these explanations over varying values of k .

N.2 Simpler Teacher Models Can Maintain Good Performance

As noted before, we can use simpler teacher models to be regularized into the explanation-constrained subspace. This can lead to overall easier optimization problems, and we synthetically verify the impacts on the overall performance. In this experimental setup, we are regressing a two layer neural network with a hidden dimension size of 100, which is much larger than in our other synthetic experiments. Here, we vary over simpler teacher models by changing their hidden dimension size.

20

40

60

80

Number of Labeled Data (n)

2.5

2.0

1.5

1.0

0.5

MSE (log scale)

Variational Hidden Dimension: 2

Variational Hidden Dimension: 4

Variational Hidden Dimension: 6

Variational Hidden Dimension: 8

Variational Hidden Dimension: 10

simpler teachers can no longer fit the approximate projection onto our class of EPAC models accurately. However, slightly smaller networks (e.g., 6, 8) can fit this projection as well, if not better in some cases. and get comparable results, showing that this simpler teacher can help with scalability without much or any drop in performance.

N.3 Number of Unlabeled Data

As a main benefit of our approach is the ability to incorporate large amounts of unlabeled data, we provide a study as we vary the amount of unlabeled data m that is available. When varying the amount of unlabeled data, we observe that the performance of self-training and our variational objective improves at similar rates.

20

40

60

80

Number of Labeled Data (n)

2

1

0

1

2

MSE (log scale)

m : 100

20

40

60

80

Number of Labeled Data (n)

2

1

0

1

2

MSE (log scale)

m : 250

20

40

60

80

Number of Labeled Data (n)

2

1

0

1

2

MSE (log scale)

m : 500

20

40

60

80

Number of Labeled Data (n)

2

1

0

1

2

MSE (log scale)

m : 1000

Supervised

Lagrangian

Lagrangian + Self-train

Variational

N.4 Data Dimension

We also provide ablations as we vary the underlying data dimension d . As we increase the dimension d , we observe that the methods seem to achieve similar performance, due to the difficulty in modeling the high-dimensional data. Also, here gradient information is much harder to incorporate, as the input gradient itself is d -dimensional, so we do not see as much of a benefit of our approach as d grows.

20

40

60

80

Number of Labeled Data (n)

2.5

2.0

1.5

1.0

0.5

0.0

0.5

1.0

MSE (log scale)

d : 25

20

40

60

80

Number of Labeled Data (n)

2

1

0

1

2

MSE (log scale)

d : 50

20

40

60

80

Number of Labeled Data (n)

2

1

0

1

2

MSE (log scale)

d : 75

20

40

60

80

Number of Labeled Data (n)

2

1

0

1

MSE (log scale)

d : 100

Supervised

Lagrangian

Lagrangian + Self-train

Variational

N.5 Hyperparameters

First, we compare the different approaches over different values of regularization ( λ ) towards satisfying the explanation constraints. Here, we compare the augmented Lagrangian approach, the self-training approach, and our variational approach.

20

40

60

80

Number of Labeled Data (n)

3

2

1

0

1

MSE (log scale)

lambda : 0.1

20

40

60

80

Number of Labeled Data (n)

3

2

1

0

1

MSE (log scale)

lambda : 1

20

40

60

80

Number of Labeled Data (n)

3

2

1

0

1

MSE (log scale)

lambda : 10

20

40

60

80

Number of Labeled Data (n)

3

2

1

0

1

MSE (log scale)

lambda : 100

Supervised

Lagrangian

Lagrangian + Self-train

Variational

We observe that there is not a significant trend as we change the value of λ across the different methods. Since we know that our explanation is perfect (our restricted EPAC class contains the target classifier), increasing the value of λ should help, until this constraint is met.

20

40

60

80

Number of Labeled Data (n)

3.25

3.00

2.75

2.50

2.25

2.00

1.75

1.50

MSE (log scale)

Variational, T = 3

Variational, T = 5

Variational, T = 7

Variational, T = 10

20

40

60

80

Number of Labeled Data (n)

2.6

2.4

2.2

2.0

1.8

1.6

1.4

MSE (log scale)

Variational, t = 0

Variational, t = 0.1

Variational, t = 0.2

Variational, t = 0.3

Variational, t = 0.4

Next, we compare different hyperparameter settings for our variational approach. Here, we analyze trends as we vary the values of T (number of iterations) and τ (threshold before adding hinge penalty).

We note that the value of τ does not significantly impact the performance of our method while increasing values of T seems to generally benefit performance on this task.

O Social Impacts

While our proposed method has the potential to improve performance and efficiency in a variety of applications, our method could introduce new biases or reinforce existing biases in the data used to train the model. For example, if our explanations constraints are poorly specified and reflect biased behavior, this could lead to inaccurate or discriminatory predictions, which could have negative impacts on individuals or groups that are already marginalized. Therefore, it is important to note that these explanation constraints must be properly analyzed and specified to exhibit the desired behavior of our model.

Peer reviews:
comments: SUMMARY:
This paper tries to answer an important and fundamental question: why can interpretability constraints on model parameters improve model performance. The authors developed an analytic framework. They first define constraints as a functional on a model function and a data point, and the model satisfying the constraint is hence represented as the functional value lies in a set. Following this setup, they defined the surrogate loss based on the constraint violation, and showed the generalization bound on this constraint violation loss. They showed that the constraints helped shrink the hypothesis class. They also showed that with gradient constraints, linear and 2-layer NN models could benefit in requiring fewer labeled data. They also discussed optimization algorithms through projections that could make the model satisfy the constraints.

STRENGTHS:
This paper tries to answer an important and fundamental question: why can interpretability constraints on model parameters improve model performance. The presentation of this paper is clear, techniques sound, and contributions good.

WEAKNESSES:
The paper lacks a discussion on the stringency of their conditions in theorems, which makes it more difficult to judge the significance of the contribution.
The paper focus on gradient constraints and lacks a discussion and related work on model shape constraints, such as monotonicity and convexity.

QUESTIONS:
Please address the comments in the weakness section.

LIMITATIONS:
This paper doesn't have potential negative societal impact. The authors are encouraged to discuss the plausibility of the conditions of their theorems.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-05

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nTraditionally, learning from labelled examples has been well studied. However, with the advent of large models (and few-shot learning) and risk of misspecifications (spurious correlations), need for augmenting supervision such as with example explanations is gaining wider interest.\nThis paper deals with this timely topic of analysing when and how learning from explanations can improve sample efficiency.\nAlthough learning from explanations is also promising in reducing systematic biases, this paper focusses on sample efficiency.\nThey also propose an interesting learning algorithm to accommodate explanation constraints.\n\nI believe the topic and analysis is timely but I have a few concenrns regarding evaluation and analysis.\n\nSTRENGTHS:\n- Important practical problem.\n- Well written and easy to follow.\n- The proposed variational method is smart, which can exploit both labelled, unlabelled data along with explanations.\n\nWEAKNESSES:\nDisclaimer: I have not checked the proofs. But the statements seem correct.\n\nConsider the following also as questions.\n\n*How are explanations different?* Fundamentally, I did not understand why explanation constraints are given a status different from example labels. Can we not simply consider them to be additional labels and minimize the joint risk such as in the Lagrangian method or Rieger et.al. and handle the error bound using the classical analysis as shown between L161 and L162?\nEspecially since the explanations are imposed only approximately through objective functions. \n\n*Clarification regarding Theorem 2.2.* From L 156: \"... help with our learning by shrinking the hypothesis class $H$ to $H_{\\phi,\\tau+\\epsilon_k}$\". Although this sounds intuitively correct, from the definition of $H$, isn't $H_{\\phi, \\tau+\\epsilon_k}$ larger than $H$?\nMoreover, I do not see how constraints help improve the sample complexity since both $R_n(H_{\\phi,\\tau+\\epsilon_k})$ and $err_D(h^*_{\\tau-\\epsilon_k})$ increase for a non-zero value of $\\epsilon_k$?\n\n*Motivation for variational method.* The need for variational method is motivated to avoid evaluating gradients of a potentially complicated surrogate loss: $\\phi$, however even the variational method needs to evaluate the same when optimizing for $f$ in the first step.\n\n*Evaluation concern*. Lagrangian method may have performed poorly because it is not exposed to unlabelled data unlike variational. What happens if we add an additional term to lagrangian objective on unlabelled data that is similar to self-training?\nOtherwise, I do not see why lagrangian may poorly perform when compared with variational at least when we can compute gradient of gradients well (which is not much of a problem these days). \n\n*Real world performance gains are unimpressive*. (Section 5.2 and Figure 5). I cannot clearly see the observation noted in L312-L313: \"We observe that our variational objective achieves better performance than all other baseline approaches, across varying amounts of labeled data.\"\nThe weak labellers and the task should be better defined. How are the gradients (of $\\phi$) computed (for say largrangian method) when weak learners are used for imposing constraints?\n\nQUESTIONS:\nPlease see weaknesses above.\n\nLIMITATIONS:\nAddressed.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this work, the authors theoretically investigate learning with constraints on the gradient of the model. Like other works in a similar vein, this paper focuses on gradient-based explanations and on neural networks. Unlike  related works, however, this paper formulates explanation constraints as being over the data distribution rather than locally to a single input. In this setting, the authors key contributions come from their focus on a learning theoretic analysis of models that satisfy their constraints (called EPAC models) which I find to be significant contribution.\n\nThis paper is timely given the current focus on trustworthy AI and an emphasis on explainable AI in proposed legislation. I feel the analysis is simplistic in some places. Additionally, it is unused in the experimental section, and a few key prior works are not mentioned. I do believe this work provides a solid learning theoretic foundation for the budding sub-field of machine learning from explanations (MLX) and for works studying different kinds of learning constraints (e.g., adversarial certification). \n\nI will note that though I have given the paper a borderline score, I will happily update my score to an accept if the authors can substantially address my comments during the rebuttal period.\n\nSTRENGTHS:\nThe paper tackles a considerably interesting problem that will certainly have impact on future works and can serve as an important basis for understanding a budding sub-field of machine learning research.\n\nThe formulations all appear sound and correct to the best of my knowledge. \n\nThe paper is very easy to follow as it is well written. I raise a few minor points about the writing below, but in general I appreciate the high-quality of the paper’s presentation.\n\nWEAKNESSES:\nThe main drawback of the paper is that the theory and experiments are almost wholly disjointed. The first portion of the paper focuses specifically on theoretical guarantees using standard learning theoretic tools; however, I do note see where any of these tools are experimentally evaluated even in the toy experiments. Moreover, machine learning from explanation (MLX) papers and even robust explanation constraint papers focus on datasets and models that are considerably more sophisticated than what is proposed in this paper.  \n\n* EPAC-ERM objective is exactly the same as what is proposed in prior machine learning from explanation (MLX) works, but no discussion is given. \n* For completeness the work should cite and at least discuss if not experimentally compare to papers [1][2] and [3] There are several scientifically vague sentences which I find inappropriate without further discussion of formalization. Three in particular come to mind: \n\nAnother part of the paper I find lacking is the existence of several vague scientific statements that are not backed up by formal argument or good intuition. I give three examples and my issues below: \n\n“We argue that in many cases, n labeled data are the most expensive to annotate. The k data points with explanations also have non-trivial cost; they require an expert to provide the annotated explanation or provide a surrogate loss.” - Page 6. It seems that for images at least, eliciting explanation annotations from humans is considerably more costly. Moreover, the authors claim that they “argue” the case of having a loss function, the k data points with explanations would be less costly, but I see no such argument. It is not obvious to me that eliciting a high-quality loss function $\\phi$ would be any less expensive than eliciting annotations themselves. Happy to hear an argument to the contrary, but on its own this statement feels unscientific and needs to be buttressed with an actual argument. To this end, I would find it very interesting, and I think it would also be interesting to the community working on these problems if a query/sample complexity was explicitly modeled and compared (i.e., the sample complexity of learning a loss function versus the sample complexity of learning a EPAC model from concretely provided labels) \n\n“Computing the gradients of this surrogate loss  in turn is much more expensive compared to gradients of the empirical risk.“ - Page 7. Again, I feel this is a vague and unjustified statement. The gradient of the surrogate loss for explanations is of course more expensive than the standard classification loss, but in previous papers regularization of explanations based on gradients has been done for even ResNet-50 sized models without any hint of computational strain. So, while the general case of the loss might be considerably more difficult to differentiate, in the case of studying gradient-based explanations and their losses dont seem “much more expensive” \n\n“Thus, the decoupled approach to solving the EPAC-ERM objective is in general more computationally convenient. “ - Page 7. I feel this statement would be easy enough to back up experimentally in the experimental setting put forward in this work. One can simply provide the numbers showing this. However, I am curious if this remains the case when the method is scaled up.  \n\n[1] Ross et. Al., 2017 https://arxiv.org/abs/1703.03717 (this one is cited but not compared experimentally which I think it ought to be)\n[2] Wicker et. Al. 2022 https://arxiv.org/abs/2212.08507 (this ought to be cited, but is in a different vein so no experimental comparison needed) \n[3] Czarnecki et. Al. 2017 https://arxiv.org/pdf/1706.04859.pdf (Similar aim, ought to be discussed again perhaps not experimentally compared)\n\nQUESTIONS:\nSee the above section.\n\nLIMITATIONS:\nSee the weaknesses section", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nDisclaimer: I am not an expert in Learning Theory, but I come from a background of interpretability and explanation model’s decision making. So I am familiar with the methods that this work is trying to describe and some related works. I will review this paper modestly, and hope authors and AC can understand.  \n\n\nIn the papers, the authors studied recent methods in regularization of prediction models using explanations. An example of explanation constraints would be enforcing certain properties of the gradients of the objective w.r.t. to the data. Another would be requiring a separate teacher model that takes in the prediction model and data and require the teacher model to have specific desired outputs. Authors analyze these explanation constraints from a statistical learning perspective, such as presenting two different explanation constraints and the analysis of their learnability and generalization bounds. Then, the authors proposed the algorithm for learning with explanation constraints, and a variational method for solving the objective.  Experiments performed on synthetic data and simple real-world data demonstrates that their variational approach achieves better performance than standard approach.\n\nSTRENGTHS:\nThe paper is overall well-organized and well-written. The notation seems clear and not cluttered. The formulation and definitions are also not difficult to follow, and the writing describes most of the motivation of formulations well. Since I am not an expert, I cannot comment on the novelty of the work. Nonetheless theoretical analysis of explanation methods or methods that incorporate explanations into training are rare. So this work could potentially be a start to introducing theory in explanation methods. The authors have cited sufficient related works in the appendix. The experiments in this work are solid, where authors compared with multiple baselines and sufficiently discussed the results. The setup of the experiments also seem fair and complete. Appendix also supplies a range of ablation study such as hyperparameters.\n\nWEAKNESSES:\nI do not observe any major weaknesses in this work. Although, in terms of writing, the transition into section 4.1 was a bit of a jump. It was not immediately clear to me why there is a need for variational method. Although in experiments it shows to obtain better performance that non-variational approaches, it wasn’t immediately obvious what prompted the authors to develop a variational approach, hence there seems to be a gap between going from Line 234 to line 236.\n\nQUESTIONS:\n1. The authors claim that in Line 29 “An attractive facet of the latter is that we can automatically generate model-based explanations given unlabeled data points.” Can the authors cite works that generate model-based explanations given unlabeled data or clarify what this means?\n\nLIMITATIONS:\nThe authors did not address any limitation of their work, but addressed its social impact. One limitation could be that in this work, authors seem to focus on restricting gradients as a explanation constraints. It will be great if future work can also address other versions of explanation constraints, and potential to extensions such as feature selection methods as explanation methods. Analysis of models beyond two-layer neural network regime could potentially be discussed.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents the concept of learning from explanations and theoretically shows that learning from explanations can improve the standard excess risk bounds in the agnostic setting. They treat explanations as constraints in the ERM procedure and introduce an extension of PAC model referred to as EPAC model. The paper introduces formal notions of explanations, EPAC learnability, explanation constraint sets etc. The study analyzes the impact of explanations on model learning using standard learning theoretic tools and characterizes the constraints for linear models and two-layer neural networks. The statistical advantages of explanations stem from their role in constraining the hypothesis class: explanation samples improve the estimation of the population explanation constraint, thereby further restricting the hypothesis class. Further, the paper provides an algorithmic solution based on variational approximation to solve the problem of learning with explanation constraints. The proposed solution works better than other methods like augmented Lagrangian.\n\nSTRENGTHS:\nStrengths:\n-------------\n\n1. The paper provides a theoretical framework for studying learning with explanations by formalizing various key notions like explanation functionals, explanation constraint set, EPAC model and EPAC learnability etc. \n\n\n2. Their analysis provides insights into why learning with explanations could improve the overall excess error of the ERM solution. The key insight is that having explanation constraints in the ERM procedure effectively reduces the size of the model class leading to smaller excess risk upper bounds. \n\n\n3. The paper also provides an algorithm solution to solve the ERM objective while satisfying the explanation constraints and the empirical results show that the proposed method works better than augmented Lagrangian and other baselines.\n\nWEAKNESSES:\nWeaknesses and Questions:\n---------------------------------\n\n1. Overall I like the ideas presented in the paper but it was a bit tedious to get exactly what is being done. It first appeared that the paper is proposing a method to learn the explanation functionals i.e. learning a model that can generate explanations by learning from some labeled explanation data. However, the goal is to study how using the explanation information improves the model performance. It would be helpful to improve the presentation to make it clear early.\n\n\n2. I am not convinced about the notion of explanations introduced. It is assumed that explanations are some vectors. Shouldn’t explanations be human-readable and be in natural language? Then how do you pose the constraint sets in natural language? I might be missing something here as I have not followed explainability research. \n\n\n3. I didn't quite get why explanation constraints are special? In general, one could put some constraints (e.g. regularization, etc.)  along with the ERM objective to reduce the size of the hypothesis class, which will lead to a reduction in the upper bound of the excess risk. Could you please provide some justification or motivation for why the explanation constraints are special? \n\n\n4. Figure 2. could be improved with some annotations and markers instead of just colors alone. The figures in experiments might  be better when run with more random seeds.\n\nQUESTIONS:\nPlease see above.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-25"}