sadat2307/MSciNLI huggingface dataset:
Domain: NeurIPS
Sentence 1: The latter is done in Lemma B.3.
Sentence 2: the next step is to show that the empirical Markov chains converge to Markov chains M, M a and M b induced by some distribution ⇡ over F. we construct an empirical distribution ⇡ and distributions ⇡a and ⇡b corresponding to player's deviations (see Definition B.5), and show that these induce the Markov chains M, Ma and Mb respectively (Lemma B.7).
Relationship: entailment
Domain: NeurIPS
Sentence 1: (1), it is possible to show that JointBoost in fact learns a matrix W with additional constraints.
Sentence 2: the features x are assumed to be decision stumps and each column W •,i is constrained to be That is, the stump is shared by all classes in the subset C i . JointBoost chooses such shared decision stumps in a greedy manner by applying the GentleBoost algorithm on top of this presentation.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Theorem 1 also implies a bound estimation result for Dantzig selector by letting F 0 = ∅ and p = 2.
Sentence 2: we set F 0 = ∅, N = 0, and λ = σ 2 log m−s η 1 in the multi-stage method, and set p = 2, l = s, η 1 = m−s m η, and η 2 = s m η for a convenient of comparison with Theorem 1.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Since J t (f ) is convex, (1) is solved by setting the gradient (or, if necessary, a subgradient) to 0.
Sentence 2: using the fact that (2) Since it is difficult to determine ∂ f R(x t , y t , f t+1 ) in closed form, an explicit update, as opposed to the above implicit update, uses the approximation ∂ f R(x t , y t , f t+1 ) ≈ ∂ f R(x t , y t , f t ) to arrive at the more easily computable expression [1] ∂ if we set 2 and ∂ f G(f ) = f , and we obtain the familiar stochastic gradient descent update f t+1 = f t − η t ∂ f R(x t , y t , f t ).
Relationship: entailment
Domain: NeurIPS
Sentence 1: To this end, we consider a new setup for optimization that allows us to interplay between stochastic and deterministic gradient descent methods.
Sentence 2: we assume that the optimization algorithm has an access to two oracles: • A stochastic oracle O s that returns the loss function g i (w) and its gradient based on the sampled training example (x i , y i ) 2 , and • A full gradient oracle O f that returns the gradient ∇G(w) for any given solution w ∈ W. We refer to this new setting as mixed optimization in order to distinguish it from both stochastic and full gradient optimization models.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Functional scans were realigned to the last volume, corrected for slice timing, co-registered with structural maps, spatially normalized into the standard Montreal Neurological Institute (MNI) atlas space, and then spatially smoothed with an 8mm isotropic full-width-at-half-maximum (FWHM) Gaussian kernel using standard procedures in SPM8.
Sentence 2: we construct a general linear model (GLM) and run a first-level analysis modeling brain responses related to outcome while informing the judgments of others.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Specifically, the tail of the future discounted reward sum for each state depends on the empirical value at time t in the following way, Substituting this into Equation ( 5) and exchanging the order of the double sum, s and R t s depend only on quantities known at time t. The only unknown quantity is v t , which we have to replace with our current estimate of this value at time t, which is V t st .
Sentence 2: we bootstrap our estimates.
Relationship: entailment
Domain: NeurIPS
Sentence 1: In other words, the seemingly naive procedure of using generalized or loopy belief propagation to compute the statistics in the E-step and use it in the M-step, can be interpreted as alternate minimization of the Kikuchi approximation of the exact free energy.
Sentence 2: algorithm (5) can be interpreted as a bound optimization algorithm for minimizing which we hope to be a good approximation (not necessarily a bound) of the original 4 There are two kinds of approaches for finding the minimum of the Kikuchi free energy.
Relationship: entailment
Domain: NeurIPS
Sentence 1: We observe the similarity between the updates of hard EM and hard gradient descent.
Sentence 2: if we reparameterize the SPN so that each child of a sum node is weighted by w i = e w i , the form of  ∂S [1,1|x] ∂w the partial derivative of the log MPN becomes This means that the hard gradient update for weights in logspace is ∆w i = ∆c i , which resembles structured perceptron [13].
Relationship: entailment
Domain: NeurIPS
Sentence 1: To allow for an analytic calculation of the loss L(β, X, P), we assume that the distribution P is a d-dimensional mixture of Gaussians [1,8].
Sentence 2: the data are generated as follows: A natural approach for choosing λ is cross-validation procedure, which can be performed efficiently for the iterative methods such as Landweber and accelerated Landweber.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Recently, Kim and Park [8] derived a connection between the BLCP and the Karush-Kuhn-Tucker (KKT) conditions for LARS.
Sentence 2: they noted the solution to the minimization problem in equation ( 1) has the form: where the vector −c follows the constraints in equation ( 2) with l i = −β and u i = β.
Relationship: entailment
Domain: NeurIPS
Sentence 1: From this, we make the following important observation: During retraining with the poison data, the network modifies its lower-level feature extraction kernels in the shallow layers so the poison instance is returned to the base class distribution in the deep layers.
Sentence 2: the poison instance generation exploits imperfections in the feature extraction kernels in earlier layers such that the poison instance is placed alongside the target in feature space.
Relationship: entailment
Domain: NeurIPS
Sentence 1: . . , κ N , we instantiate F and Fmax in Definition 4.3 via: F in Definition 4.4 trades off between exploitation (maximizing the plausible F h 's) and exploration (disambiguating plausibility in F h 's) by allowing each Fh to reach its maximum by either F h reaching α i or G h reaching κ i .
Sentence 2: each Fh can be satisfied with either a sufficiently large utility F h or large distance G h . Figure 3 shows the logical relationships between these components.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Formally, we construct for each k a binarylabeled sample S k = {(x j , C k • y j )} n j=1 and for each S k we learn a binary classification function T k : X → using a logistic regression algorithm.
Sentence 2: for each original instance x j and induced binary label C k • y j we posit a logistic model that estimates the conditional probability that Given a predefined code matrix C the learning task at hand is to find T k that maximizes the log-likelihood of the labelling given in S k , Defining 0 log 0 = 0, we can use the logistic estimate in Eq.
Relationship: entailment
Domain: NeurIPS
Sentence 1: We will keep track of the weighted cumulative votes of the first i weak learners for the sample x t by , where α i t is the weight of W L i , l i t is its prediction and e j is the j th standard basis vector.
Sentence 2: for the optimal algorithm, we assume that α i t = 1, ∀i, t. the booster makes the final decision by simple majority votes.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Moreover, in the semi-stochastic gradient v t , there are also two reference gradients and we denote them by g (0) t = rF (e x) and g Back to our One-epoch-SNVRG, we can define similar reference points and reference gradients as that in the special case of SVRG.
Sentence 2: for t = 0, . . . , t }, l = 0, . . . , K, which is set to be x (l) t = x t l with index t l defined as Specially, note that we have x (0) t = x 0 and x (K) t = x t for all t = 0, . . . , Q K l=1 T l 1.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Minimizing the exponential loss or the log loss on real data often fails to produce finite parameters.
Sentence 2: this happens when for some feature U V U V 9 BA DC FE HG 4 § U V 9 BA DC qv E 19 BA TG FG % for all E and A with v w 9 fA hG $ (3) or U V 9 BA DC FE HG 4 § U V 9 BA DC qv E 19 BA TG FG & for all E and A with v w 9 fA hG $ Hc This is especially harmful since often the features for which (3) holds are the most important for the purpose of discrimination.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Furthermore, conditioning information, in the form of macroeconomic variables, can be of importance, but exhibit the cumbersome property of being released periodically, with explanatory power that varies across the forecasting horizon.
Sentence 2: when making a very long-horizon forecast, the model should not incorporate conditioning information in the same way as when making a short-or medium-term forecast.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Since in our network the complex cell receptive field size in the image is only 50% greater than that for the simple cells, this implies translation invariance: over the time (or space) that a simple cell of the correct orientation gives a strong but transitory response, the complex cells provides a strong continuous response.
Sentence 2: its response is invariant to the phase that determines the profile of the simple cell response.
Relationship: entailment
Domain: NeurIPS
Sentence 1: An appropriate choice of the bounds ensures that the algorithm always improves the objective values for Problems 1 and 2.
Sentence 2: choosing ft as a modular upper bound of f tight at X t , or ĝt as a modular lower bound of g tight at X t , or both, ensures that the objective value of Problems 1 and 2 always improves at every iteration as long as the corresponding surrogate problem can be solved exactly.
Relationship: entailment
Domain: NeurIPS
Sentence 1: More precisely, the corollary follows directly from successive application of that single expert relative loss bound, and then our Fixed-share relative loss bound (Theorem 1): ≤ log(m) + (T − 1) min j=1,...,m We start by finding the smallest discrete set of switching-rate parameters so that any additional regret due to discretization does not exceed (T − 1)δ, for some threshold δ.
Sentence 2: we find m = m(δ) values α 1 , .
Relationship: entailment
Domain: NeurIPS
Sentence 1: This implies that, if the initial state x[0] is randomly drawn from a continuous distribution, then almost surely x [k] does not converge to the origin, yet |x[k]| converges to 2 /4.
Sentence 2: with probability 1, the state x[k] does not converge to a fixed point, such as a local optimum or a saddle point, even though the estimation error converges to a finite non-optimal value.
Relationship: entailment
Domain: NeurIPS
Sentence 1: For a continuous Y , or unless there is a mass probability at V α , we have The equivalence between the robust MDP with the objective (2) and the risk-sensitive MDP with the objective (3) can be shown by the use of the following alternative definition of CTE: where Q is the set of probability mass functions, q, whose support is a subset of the probability mass function, q 0 , of Y such that q(y) ≤ q 0 (y)/α for every y in the support of q 0 .
Sentence 2: let C π i be the cost incurred at the i-th epoch with policy π so that Then, by the recursive definition of ICTE and the translation invariance 1 of CTE, it can be shown that where the second equality follows from (7).
Relationship: entailment
Domain: NeurIPS
Sentence 1: The complete implementation is available in the tensor2tensor library on github.
Sentence 2: the layout is given by: mesh_shape = [("all", n)] computation_layout = [ ("vocab", "all"), ("d_ff", "all"), ("heads", "all")] the dimensions representing the vocabulary size, the size of the feed-forward hidden layer, and the number of attention heads are each split across all processors.
Relationship: entailment
Domain: NeurIPS
Sentence 1: samples f ⇠ D (as in our setting), as well as the ability to compute exact gradients rF (v) in a potentially expensive operation.
Sentence 2: the basic intuition is to use an exact gradient rF (v) at some "anchor point" v 2 W as a kind of "hint" for what the exact gradient is at nearby points w. SVRG leverages the theorem that rf (w) rf (v) + rF (v) is an unbiased estimate of rF (w) with variance approximately bounded by L(F (v) F (w ? ))
Relationship: entailment
Domain: NeurIPS
Sentence 1: Notice that, in sharp contrast to the nearest neighbors-based methods, which first fix the number of neighbors or the neighborhood radius and then compute the weights between points in each neighborhood, we do the two steps at the same time.
Sentence 2: the optimization programs ( 4) and ( 5) automatically choose a few neighbors of the given data point, which approximately span a low-dimensional affine subspace at that point.
Relationship: entailment
Domain: NeurIPS
Sentence 1: We also make a restriction on the features, analogous to the usual Markovian assumption made in CRFs, and assume that every component g k of g is a function only of x, s j , and the label y j−1 associated with the preceding segment s j−1 .
Sentence 2: we assume that every g k (j, x, s) can be rewritten as g k (j, x, s) = g k (y j , y j−1 , x, t j , u j ) (2) for an appropriately defined g k . In the rest of the paper, we will drop the g notation and use g for both versions of the segment-level feature functions.
Relationship: entailment
Domain: NeurIPS
Sentence 1: To circumvent this issue, practitioners have resorted to asynchronous implementations, which emphasize parallel efficiency by using potentially stale stochastic gradients for computation.
Sentence 2: each worker in asynchronous implementations can process a mini-batch of data independently of the others, as follows: (1) The worker fetches from the parameter servers the most up-to-date parameters of the model needed to process the current mini-batch; (2) It then computes gradients of the loss with respect to these parameters; (3) Finally, these gradients are sent back to the parameter servers, which then updates the model accordingly.
Relationship: entailment
Domain: NeurIPS
Sentence 1: We revisit the classical problem of optimal experimental design (OED) under a new mathematical model grounded in a geometric motivation.
Sentence 2: we introduce models based on elementary symmetric polynomials; these polynomials capture "partial volumes" and offer a graded interpolation between the widely used A-optimal design and D-optimal design models, obtaining each of them as special cases.
Relationship: entailment
Domain: NeurIPS
Sentence 1: For all M ≥ 1, any external field φ, and any M -cover 3.
Sentence 2: for all choices of the external field φ and any 2-cover (H; if all of the edge potentials are strictly positive, and the graphical model has even one negative cycle, then there exists an external field φ and a 2-cover (H; φ H , ψ H ) of (G; φ, ψ) such that Z(G; φ, ψ) 2 < Z(H; φ H , ψ H ).
Relationship: entailment
Domain: NeurIPS
Sentence 1: If this submodular Hamming distance is used for k-means clustering, then the mean-finding step becomes an instance of the SHmin problem.
Sentence 2: if cluster j contains documents C j , then its mean takes exactly the following SH-min form: Structured prediction: Structured support vector machines (SVMs) typically rely on Hamming distance to compare candidate structures to the true one.
Relationship: entailment
Domain: NeurIPS
Sentence 1: The tapering estimator Ωk , defined in (7), of the precision matrix Ω p×p with p > n 1 2α+1 satisfies: , and a universal constant C > 0.
Sentence 2: the estimator Ω = Ωk with k = n 1 2α+1 satisfies: Given the result in Equation ( 10), it is easy to show that setting k = n 1 2α+1 yields the optimal rate by balancing the size of the inside-taper and outside-taper terms, which gives Equation (11).
Relationship: entailment
Domain: NeurIPS
Sentence 1: An optimal stopping rule is to stop the first time the expected cost of continuing exceeds that of stopping, and to choose δ = 0 or δ = 1 to minimize the probability of error given the accumulated evidence (see [10]).
Sentence 2: we define the continuation region at time t by Although we have obtained an expression for the optimal policy in terms of Q(t, p) and Q(t, p), computing Q(t, p) is difficult in general.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Toward this end, we present an effective and efficient integration of the two existing approaches for geometric invariance, i.e., STN-based feature extraction networks [5,19] and geometric matching networks [30,31], that includes a novel weakly-supervised loss function tailored to our objective.
Sentence 2: the final geometric field is recursively estimated by deforming the activations of feature extraction networks according to the intermediate output of the geometric matching networks, in contrast to existing approaches based on geometric matching which consider only fixed, upright versions of features [30,31].
Relationship: entailment
Domain: NeurIPS
Sentence 1: Suppose that there exists a constant M such that p ∞ ≤ M and p ∞ ≤ M . Suppose also that the density difference f = p − p is a member of Besov space with regularity α.
Sentence 2: f ∈ B α 2,∞ where B α 2,∞ is the Besov space with regularity α, and where α denotes the largest integer less than or equal to α and ω r,L2(R d ) is the r-th modulus of smoothness (see [10] for the definitions).
Relationship: entailment
Domain: NeurIPS
Sentence 1: However, we note that static analysis is not sufficient to capture all dependencies required in the factor graph to accurately model the process of generating training labels.
Sentence 2: static analysis can (i) pick up spurious dependencies among HFs that are not truly dependent on each other, or (ii) miss key dependencies among HFs that exist due to dependencies among the DSPs in the HFs.
Relationship: entailment
Domain: NeurIPS
Sentence 1: In a small neighborhood dθ around θ, we have: Since Z n+1 is completely random and H is smooth, the above shows that on Θ\{θ * 1 , .
Sentence 2: . . , θ * K } Z n+1 is simply a Poisson process with rate measure α Γ(1+c)Γ(n+c+σ) Γ(n+1+c)Γ(c+σ) H. it will have Poisson(α Γ(1+c)Γ(n+c+σ) Γ(n+1+c)Γ(c+σ) ) new atoms, each independently and identically distributed according to H. In the IBP metaphor, this corresponds to customer n+1 trying new dishes, with each dish associated with a new draw from H. The resulting Indian buffet process is as described in the introduction.
Relationship: entailment
Domain: NeurIPS
Sentence 1: The lengthscales are set to 14.5 m and the prior standard deviation of heights is 10 m. We assume a noise standard deviation of 0.075 m. Since the safety feature of each state transition is a linear combination of heights, the GP model of the heights induces a GP model over the differences of heights, which we use to classify whether state transitions fulfill the safety constraint.
Sentence 2: the safety depends on the direction of travel, that is, going downhill is possible, while going uphill might be unsafe.
Relationship: entailment
Domain: NeurIPS
Sentence 1: The weight vector p t is the probability distribution over the n base classifiers that maximizes the entropy, subject to the constraint that p t correctly classifies all patterns seen in previous trials with a given margin γ.
Sentence 2: it maximizes the entropy of p t subject to the constraints that p t • x s ≥ 1/2 + γ whenever y s = 1 for s < t, and p t • x s ≤ 1/2 − γ whenever y s = 0 for s < t. We show that, if there is a weighting p * , determined with benefit of hindsight, that achieves margin γ on all trials, then this on-line maximum entropy procedure makes at most ln n 2γ 2 mistakes.
Relationship: entailment
Domain: NeurIPS
Sentence 1: In our active learning setting, our goal is to output a boundary g that is close to g * while making as few interactive queries to the labeler as possible.
Sentence 2: we want to find an algorithm with low query complexity Λ(ǫ, δ, A, L, g * ), which is defined as the minimum number of queries that Algorithm A, acting on samples with ground truth g * , should make to a labeler L to ensure that the output classifier We now introduce three conditions on the response of the labeler with increasing strictness.
Relationship: entailment
Domain: NeurIPS
Sentence 1: To make progress on the computational question we exploit a key observation: for any loss function, its corresponding clipped loss can be indirectly expressed by an auxiliary optimization of a smooth objective (if the original loss function itself was smooth).
Sentence 2: given a loss ℓ(y, x ⊤ θ) define the corresponding ρ-relaxed loss to be for 0 ≤ ρ ≤ 1; see Figure 1.
Relationship: entailment
Domain: NeurIPS
Sentence 1: From Lemma 2 we immediately get This upper bound can be computed for Z = ab ∈ A k,q for different norms.
Sentence 2: for Ω(Z ), we have ab 1 ≤ √ kq and Ω k,q (ab ) = ab * = 1 which lead to the corollary: is an atom, the expected errors of the oracle estimators ẐOracle Ω k,q , ẐOracle 1 and ẐOracle * using respectively the (k, q)-trace norm, the 1 norm and the trace norm are upper bounded as follows: When the smallest entry in absolute value of a or b is close to 0, then the expected error is smaller for ẐOracle 1 , reaching σ 2 log(m 1 m 2 ) on e 1 e 1 while not changing for the two other norms.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Lemma 5 is proved by explicitly constructing a classifier (tester) p ψ from any sequence of low regret.
Sentence 2: for any assortment sequence tS t u T t"1 , we construct p ψ as p ψ " 0 if 1 T ř T t"1 Ir1 P S t , 2 R S t s ě 1{2 and p ψ " 1 otherwise.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Under some mild conditions, we show that this two-stage procedure achieves minimax rates of convergence up to a logarithmic factor, even after only one iteration of EM.
Sentence 2: given any δ ∈ (0, 1), we provide the bounds on the number of workers and the number of items so that our method can correctly estimate labels for all items with probability at least 1 − δ.
Relationship: entailment
Domain: NeurIPS
Sentence 1: We represent this scene map y(n) in terms of a set of coefficients {w i (n)} K i=1 at each location n, corresponding to various spatial derivatives.
Sentence 2: these coefficients are related to the scene map y(n) through convolution with a bank of derivative filters {k i } K i=1 , i.e., For our task, we define {k i } to be a set of 2D derivative-of-Gaussian filters with standard deviations 2 s pixels, for scales s = {1, 2, 3}.
Relationship: entailment
Domain: NeurIPS
Sentence 1: When each Player i chooses a mixed strategy x i independently, the payoff function extends naturally by player can change her mixed strategy for achieving a higher payoff.
Sentence 2: for any Player i, let x −i denote the mixed strategies chosen by all players other than Player i, then for any mixed strategy A NE is said to be fully mixed if no probability density in any of the x i is zero.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Let � T = R, and let ψ : {1, 2, 3} × R→R + be defined as (see Figure 3) We will show there are 3 points in R ψ satisfying the conditions of Theorem 7.
Sentence 2: consider t1 = 1, t2 = 2, and t3 = 3, giving z 1 = ψ( t1 ) = (0, 1, 2) � , z 2 = ψ( t2 ) = (1, 0, 1) � , and T here is a convex set and ψ : � T →R 3 is a convex function.
Relationship: entailment
Domain: NeurIPS
Sentence 1: hinge loss) case, the number of points may be reduced to poly(ε −1 ) by elementary methods.
Sentence 2: it is in easy task to prove that if we omit all but a random subset of log(ε −1 )/ε 2 data points, the hinge loss corresponding to the obtained separator w.r.t the full set of points will be O(ε).
Relationship: entailment
Domain: NeurIPS
Sentence 1: A fundamental aspect of our approach is that we can train stochastic processes that need not exactly obey detailed balance, yielding access to a larger and potentially more powerful space of models.
Sentence 2: this enables us to relax the weight symmetry constraint of undirected graphical models corresponding to neural networks, yielding a more brain like iterative computation characteristic of asymmetric biological neural circuits.
Relationship: entailment