sadat2307/MSciNLI huggingface dataset:
Domain: NeurIPS
Sentence 1: We then add event T 1 = ÎT0 to the list of target events, so that when the agent first enters ÎT0 , it creates a new option o T1 whose goal is to trigger T 1 .
Sentence 2: the new option's termination function is set to the indicator function ÎT0 , and its reward function becomes the task's reward function plus an option completion reward for triggering T 1 .
Relationship: entailment
Domain: NeurIPS
Sentence 1: This gives an improved bound on m and lets us partially relax the magnitude bound on parameters so that they can be arbitrarily negative: , then no RBM network of size m, whose weights are upper bounded in value by C, can compute a function which represents n-dimensional IP with margin δ.
Sentence 2: for C and 1/δ bounded by polynomials in n, for n sufficiently large, this condition is satisfied whenever m < 2 (1/4− )n for some > 0.
Relationship: entailment
Domain: NeurIPS
Sentence 1: In fact, we show that the values z(t, j), t = 1, . . . , k are distributed identically to points on a normalized Brownian motion.
Sentence 2: let W (s) be a standard linear Brownian motion and let S(s) be the normalized Brownian motion We then show that, for every j, there exists times s 1 , .
Relationship: entailment
Domain: NeurIPS
Sentence 1: Thus, provided we can compute S j,k for a single pair of features, we can avoid storing the entire matrix S. To avoid performing the whitening step for all of the input features, we can instead perform pair-wise whitening between features.
Sentence 2: to compute the squared correlation of x j and x k , we whiten the jth and kth columns of X together (independently of all other columns), then compute the square correlation between the whitened values xj and xk .
Relationship: entailment
Domain: NeurIPS
Sentence 1: The only other policy π 0 with which the learner could match the feature expectations of π 0 in her worldview (restricting again to deterministic policies) is the one that moves to the central cell as quickly as possible and then alternates between states 2 and 3. and hence π 0 is a better teaching policy than π * regarding the performance that a learner matching feature expectations in her worldview achieves in the worst case.
Sentence 2: this example shows that providing demonstrations from the truly optimal policy π * does not guarantee that the learner's policy achieves good performance in general.
Relationship: entailment
Domain: NeurIPS
Sentence 1: To learn the parameter vector a t ∈ IR d from data {(x ti , y ti )} m i=1 we would like to minimize the empirical error m i=1 L(y ti , a t , U x ti ) subject to an upper bound on the number of nonzero components of a t , where L : IR × IR → IR + is a prescribed loss function which we assume to be convex in the second argument.
Sentence 2: this problem is intractable and is often relaxed by requiring an upper bound on the 1-norm of a t . we consider the problem min where γ > 0 is the regularization parameter.
Relationship: entailment
Domain: NeurIPS
Sentence 1: The choice of the exponential rate νρ h is made to cover a very large class of functions, as well as to relate to results from prior work.
Sentence 2: for a standard partitioning on R p and any α, β > 0, any function f such that f (x) ∼ x→x β||x − x || α fits this assumption.
Relationship: entailment
Domain: NeurIPS
Sentence 1: 4 A single neuron activation vector is one multidimensional variate, and a layer of neurons gives us a set of multidimensional variates.
Sentence 2: we can consider two layers, L 1 , L 2 of a neural network as two sets of observations, to which we can then apply CCA, to determine the similarity between two layers.
Relationship: entailment
Domain: NeurIPS
Sentence 1: In this paper, we develop a sparse variant of Hinton's deep belief network algorithm, and measure the degree to which it faithfully mimics biological measurements of V2.
Sentence 2: we take Ito & Komatsu [7]'s characterization of V2 in terms of its responses to a large class of angled bar stimuli, and quantitatively measure the degree to which the deep belief network algorithm generates similar responses.
Relationship: entailment
Domain: NeurIPS
Sentence 1: We test whether the same DNN prior (G 6 ) that was trained on inverting the feature representations of ImageNet images generalizes to enable visualizing DNNs trained on different datasets.
Sentence 2: we target the output neurons of two DNNs downloaded from Caffe Model Zoo [20]): (1) An AlexNet DNN that was trained on the 2.5-million-image MIT Places dataset to classify 205 types of places with 50.1% accuracy [26].
Relationship: entailment
Domain: NeurIPS
Sentence 1: We study the reward of our algorithms, and their regret against a fixed comparator x: Comparison of Regret Bounds The primary contribution of this paper is to establish matching upper and lower bounds for unconstrained online convex optimization problems, using algorithms that require no prior information about the comparator point x.
Sentence 2: we present an algorithm that, for any To obtain this guarantee, we show that it is sufficient (and necessary) that reward is Ω(exp(|g 1:T |/ √ T )) (see Theorem 1).
Relationship: entailment
Domain: NeurIPS
Sentence 1: The second challenge introduced by using nested mini-batches is determining the size of M t . On the one hand, if M t grows too slowly, then one may suffer from premature fine-tuning.
Sentence 2: when updating centroids using M t ⊂ {1, . . . , N }, one is using energy estimated on samples indexed by M t as a proxy for energy over all N training samples.
Relationship: entailment
Domain: NeurIPS
Sentence 1: The learner's likelihood function is The teacher wants the learner to arrive at a posterior p(θ | D) peaked at θ * by designing a small D. As discussed above, this can be formulated as (2) with the KL loss() and the cardinality effort() functions: namely uniform over the version space consistent with D. The optimal teaching problem becomes min n,x1,y1,...,xn,yn − log One solution is the limiting case with a teaching set of size two D = {(θ * − /2, −1), (θ * + /2, 1)} as → 0, since the Teaching Impedance T I = log( ) + 2c approaches −∞.
Sentence 2: the teacher teaches by two examples arbitrarily close to, but on the opposite sides of, the decision boundary as in Figure 1(right).
Relationship: entailment
Domain: NeurIPS
Sentence 1: It is defined as the ground distance (i.e., cost function) between every two perceptual features, extending the notion of a distance between single elements to a distance between sets of elements.
Sentence 2: consider two probability distribution x ∼ µ and y ∼ ν; EMD can be then defined as: where Π(µ, ν) denotes the set of all joint distributions γ(x, y) with marginals µ(x) and ν(y), and c(x, y) is the cost function (e.g., Euclidean or cosine distance).
Relationship: entailment
Domain: NeurIPS
Sentence 1: We will first show that, x (k) is sub-Gaussian with mean m k and parameter σ 2 k for a regime of λ we are interested in.
Sentence 2: we will show that for |λ| ≤ 1/(2m k−1 r), By definition, due to distributional independence, we have E[e λx (k) ] = E[e λx (k) ] (l/ l) .
Relationship: entailment
Domain: NeurIPS
Sentence 1: As shown in Figure 1c, in numerical experiments higher-order terms improve the Hogwild covariance on the block diagonal relative to the A = 0 approximation, and the improvements increase with local mixing rates.
Sentence 2: the off-block-diagonal first-order term in the Hogwild covariance is nonzero and it depends on the local mixing performed by S. if global synchronization happens infrequently relative to the speed of local sampler mixing (e.g.
Relationship: entailment
Domain: NeurIPS
Sentence 1: In this paper, we propose an alternative approach to LDS parameter estimation under exponential family observation noise.
Sentence 2: we reformulate the LDS as a two-view generative model, which allows us to approximate the estimation task as a form of matrix factorization, and apply recent global optimization techniques for such models (Zhang et al., 2012;Yu et al., 2014).
Relationship: entailment
Domain: NeurIPS
Sentence 1: When the instance is non-anomalous and all its views are consistent, all of the views are generated from a single latent vector.
Sentence 2: the latent vector assignments for all views are the same, When it is an anomaly and some views are inconsistent, different views are generated from different latent vectors, and some latent vector assignments are different, i.e.
Relationship: entailment
Domain: NeurIPS
Sentence 1: To fully describe Algorithm 1 we need to specify how to compute the confidence-set bounds (β t ) t . On the one hand, these bounds have to be accurate -the confidence set E t should contain the unknown θ * ; on the other hand, the larger they are, the larger the regret bounds we obtain.
Sentence 2: the β t should be as small as possible subject to being accurate.
Relationship: entailment
Domain: NeurIPS
Sentence 1: We prove recovery of the sparse and low rank components under a set of requirements which are tight and match those for the convex techniques (up to constant factors).
Sentence 2: under the deterministic sparsity model, where each row and each column of the sparse matrix S * has at most α fraction of non-zeros, we require that α = O 1/(µ 2 r) , where µ is the incoherence factor (see Section 3).
Relationship: entailment
Domain: NeurIPS
Sentence 1: We can also define the minimax loss, which is defined by choosing the w(•) which minimizes WorstCaseLoss(•).
Sentence 2: minimaxLoss(M, cost, k) := min In the next section, we describe the optimal algorithm for a restricted class of M . That is, we obtain the mapping w which optimizes WorstCaseLoss(w; M, cost, k).
Relationship: entailment
Domain: NeurIPS
Sentence 1: We propose SafetyNets, a framework that enables an untrusted server (the cloud) to provide a client with a short mathematical proof of the correctness of inference tasks that they perform on behalf of the client.
Sentence 2: safetyNets develops and implements a specialized interactive proof (IP) protocol for verifiable execution of a class of deep neural networks, i.e., those that can be represented as arithmetic circuits.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Then the infimum depends only on the value of P 1 , P 2 on Ω 0 , the subset of Ω on which P 1 and P 2 are absolutely continuous with respect to each other: This is intuitive as we can always find a detector φ such that its value is arbitrarily close to zero on Ω \ Ω 0 .
Sentence 2: if P 1 and P 2 have measure-zero overlap, then inf φ Φ(φ; P 1 , P 2 ) equals zero, that is, the optimal test for the simple test (P 1 , P 2 ) has zero risk.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Let f : X → (−∞, +∞] be proper, lower semicontinuous and uniformly essentially strongly convex with modulus γ ∈ Γ U .
Sentence 2: then (5) If, in addition, γ(r) := γ(r)/r is strictly increasing, then Df * is uniformly continuous with modulus of continuity χ(r) = γ−1 (r/2).
Relationship: entailment
Domain: NeurIPS
Sentence 1: After this Markov chain has converged (i.e., finished the burn-in stage), we draw J samples {Z (j) } and estimate the expectation statistics To make prediction on unlabeled testing data using the prediction rule (5), we take the approach that has been adopted for the variational MedLDA, which uses a point estimate of topics Φ from training data and makes prediction based on them.
Sentence 2: we use the MAP estimate Φ to replace the probability distribution p(Φ).
Relationship: entailment
Domain: NeurIPS
Sentence 1: A standard way to approach optimization with inconsistent noise is to estimate the value of each set used by the algorithm to an accuracy ε via independent randomized sampling, where ε is chosen small enough so as to obtain approximation guarantees.
Sentence 2: assuming that the algorithm only makes polynomially many value queries and that the function f is such that F (S) ∈ [b, B] for any set S, then a classical application of the Chernoff bound combined with a union bound implies that if the value of each set is estimated by averaging the value of m samples with m = Ω B log n bε 2 , then with high probability the estimated value F (S) of each set used by the algorithm is such that In other words, randomized sampling is used to construct a function which is εapproximately submodular with high probability.
Relationship: entailment
Domain: NeurIPS
Sentence 1: . . , y (n) } be an n-dimensional random vector following an n-variate Gaussian distribution N (µ, Σ), and let G = (V, E) be a Markov network representing the conditional independence structure of N (µ, Σ).
Sentence 2: the set of vertices V = {1, . . . , n} corresponds to the set of variables in Y , and the edge set E contains an edge (i, j) if and only if y (i) is conditionally dependent on y (j) given all remaining variables; i.e., the lack of an edge between i and j denotes the conditional independence of y (i) and y (j) , which corresponds to a zero entry in the inverse covariance matrix Σ −1 ( [1]).
Relationship: entailment
Domain: NeurIPS
Sentence 1: Similar to NS-ES and ES, NSR-ES operates on entire episodes and can thus evaluate reward and novelty simultaneously for any sampled parameter vector: ✓ i,m t = ✓ m t + ✏ i .
Sentence 2: we compute f (✓ i,m t ) and N (✓ i,m t , A), average the two values, and set the average as the weight for the corresponding ✏ i .
Relationship: entailment
Domain: NeurIPS
Sentence 1: This section shows that the choice of a good parameterisation is not obvious, and that the use of the Riemannian geometry of the simplex [Ama95, GC11] is important in designing Langevin MCMC algorithms.
Sentence 2: we show that an unnormalized parameterisation, using a mirroring trick to remove boundaries, coupled with a natural gradient update, achieves the best mixing performance.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Fortunately it turns out not hard to tackle this issue by using semi-definite programming (SDP) relaxation which linearizes the quadratic terms.
Sentence 2: let I be the identity matrix, and define Then θ U x can be replaced by θ M u x and the quadratic term in (15) can be expanded as Since given (π, θ, γ, v) the objective function becomes linear in M , so after maximizing out these variables the overall objective is convex in M . Although this change of variable turns the objective into convex, it indeed shifts the intractability into the feasible region of M : Here M 0 means M is real symmetric and positive semi-definite.
Relationship: entailment
Domain: NeurIPS
Sentence 1: By using either Oja's algorithm or Neon2 online , there exists an algorithm, denoted by ApproxNC-Stochastic, which uses stochastic gradient evaluations or stochastic Hessian-vector product evaluations to find the negative curvature direction for general stochastic nonconvex optimization problem (1.1).
Sentence 2: approxNC-Stochastic returns a unit vector b Based on ApproxNC-Stochastic, we present our negative curvature descent algorithm in Algorithm 1.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Namely, they both require |θ 0,i | ≥ c max{σs 0 log p/ n, σ/ √ n}, which is √ s 0 larger than the ideal detection level [10].
Sentence 2: in order for the coefficient θ 0,i to be detectable with appreciable probability, it needs to be larger than the overall 2 error, rather than the 2 error per coordinate.
Relationship: entailment
Domain: NeurIPS
Sentence 1: For each order, a derivation operation π Op of a given type Op specifies a sequence of symbols Y 1 . . . Y k (possibly the empty sequence ǫ), and can be applied to a partial derivation: with ⊕ being list concatenation.
Sentence 2: a derivation operation involves popping the top item off the stack, choosing a derivation operation of the appropriate type, applying it to add some symbols to D yielding D ′ , and pushing a list of new items A back on the stack.
Relationship: entailment
Domain: NeurIPS
Sentence 1: However, the optimal inference network q(z|x 1 , ..., x N ) would be the true posterior p(z|x 1 , ..., x N ).
Sentence 2: the conditional independence assumptions in the generative model imply a relation among joint-and single-modality posteriors: the joint posterior is a product of individual posteriors, with an additional quotient by the prior.
Relationship: entailment
Domain: NeurIPS
Sentence 1: 3.2-3.3 we discuss their dependence structure and explain how to estimate them from data respectively.
Sentence 2: using the notions reviewed above, we define the Fiedler delta function ∆λ 2 in the following way: ∆λ 2 (u, v, G) is the variation in the Fiedler eigenvalue of the graph Laplacian that would result from flipping the value of X uv in G. Concerning the range of the Fiedler delta function, we can easily prove the following proposition: Proposition 2.
Relationship: entailment
Domain: NeurIPS
Sentence 1: We propose the following model to capture the lingering of gradients.
Sentence 2: for every x ∈ R d and index i ∈ [n], let δ(x, i) ≥ 0 be the lingering radius of ∇f i (x), meaning that 2 as long as we travel within distance δ(x, i) from x, the gradient ∇f i (x) can be reused to represent ∇f i (y).
Relationship: entailment
Domain: NeurIPS
Sentence 1: Finally, the optimal solution A * of ( 11) is composed of α * jgi = S( ), where S is the shrinkage operator defined as follows: Given the smooth approximation of Ω(W) in (11) and the corresponding gradient presented in Theorem 1, we could apply gradient descent method to solve the problem.
Sentence 2: we replace the overlapping-group-lasso penalty in (8) with its smooth approximation f µ (W) to obtain the following optimization problem min where is the augmented logistic regression loss function.
Relationship: entailment
Domain: NeurIPS
Sentence 1: The key object in this paper is the Gaussian Mixture Model (GMM), whose probability density is and where p N is a (multivariate) Gaussian with mean µ ∈ R d and covariance Σ 0.
Sentence 2: 0} K j=1 and weights α ∈ ∆ K , the K-dimensional probability simplex.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Discriminatively training the disaggregation bases B1:k is naturally framed as a structured prediction task: the input is X, the multi-variate desired output is A ⋆ 1:k , the model parameters are B1:k , and the discriminant function is F ( X, B1:k , A 1:k ).
Sentence 2: 1 we seek bases B1:k such that (ideally) While there are many potential methods for optimizing such a prediction task, we use a simple method based on the structured perceptron algorithm [5].
Relationship: entailment
Domain: NeurIPS
Sentence 1: In [5] Tsybakov showed that if P has a noise exponent q then ERM-type classifiers can obtain rates in (1) which are of the form n − q+1 q+pq+2 , where 0 < p < 1 measures the complexity of the hypothesis class.
Sentence 2: rates faster than n −1/2 are possible whenever q > 0 and p < 1.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Next, we define the following modification of f , which could be the function returned by an -erroneous oracle.
Sentence 2: note that f (x) and f (x) differ only in the region where | i∈A x i − j∈B x j | ≤ 1 2 n. the value of f (x) in this region is within ], while f (x) = 1 2 , so an -erroneous oracle for f (x) (both in the relative and absolute sense) could very well return f (x) instead.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Convexity has played an increasingly important role in machine learning in recent years, echoing its growing prominence throughout applied mathematics (Boyd and Vandenberghe, 2003).
Sentence 2: a wide variety of two-class classification methods choose a real-valued classifier f based on the minimization of a convex surrogate φ(yf (x)) in the place of an intractable loss function 1(sign(f (x)) = y).
Relationship: entailment
Domain: NeurIPS
Sentence 1: To address such a challenge, we develop a general inferential theory of the EM algorithm for parameter estimation and uncertainty assessment of high dimensional latent variable models.
Sentence 2: we make two contributions in this paper: • For high dimensional parameter estimation, we propose a novel high dimensional EM algorithm by attaching a truncation step to the expectation step (E-step) and maximization step (M-step).
Relationship: entailment
Domain: NeurIPS
Sentence 1: Note that the shape of W i,j (t) is remarkably reminiscent of the experimentally found measurements for STDP [8].
Sentence 2: the shape of the STDP curve depends on the type of neuron and is different for connections from excitatory to excitatory than from excitatory to inhibitory or inhibitory to inhibitory neurons (Figure 3).
Relationship: entailment
Domain: NeurIPS
Sentence 1: This is done because (i) it puts λ n and the regularization term γ together, and (ii) it allows more freedom in the choice of the prior for β. Additionally, it has an interesting interpretation, in that the SVM loss function behaves like a globallocal shrinkage distribution [9].
Sentence 2: γ −1 corresponds to a "global" scaling of the variance, and λ n represents the "local" scaling for component n. The {λ n } define the relative variances for each of the N data, and γ −1 provides a global scaling.
Relationship: entailment
Domain: NeurIPS
Sentence 1: The ESS and TN-ESS are comparable between HMC and the Gibbs sampler.
Sentence 2: for 100 inducing points and the RBF covariance, ESS and TN-ESS for HMC are 11 and 1.0 • 10 −3 and for the Gibbs sampler are 53 and 5.1 • 10 −3 .
Relationship: entailment
Domain: NeurIPS
Sentence 1: We have studied the task of efficiently identifying NN models with low accuracy by exploring possibilities within the theoretical framework of sublinear algorithms and evaluated our approach by moving to experiments.
Sentence 2: we have proved that there is a one-sided error property tester with complexity O( √ nk 2 / 2 ), i.e., a sublinear (randomized) algorithm that decides whether an input graph G is a k-NN graph or requires many edge modifications to become a k-NN graph (i.e.,  precision 1 and recall 2/3 when taking -far graphs as relevant).
Relationship: entailment
Domain: NeurIPS
Sentence 1: [6] and Ruozzi and Tatikonda [7], we will use graph covers in order to better understand the behavior of the Bethe approximation with respect to the partition function.
Sentence 2: we will show that if a graphical model is strictly positive and contains even one frustrated cycle, then there exists a choice of external field and a 2-cover without frustrated cycles whose partition function provides a strict upper bound on the partition function of the original model.
Relationship: entailment
Domain: NeurIPS
Sentence 1: The most accurate model for a given target cluster t minimizes the loss with respect to the conditional probability p(x, y|t) referred to as the target distribution.
Sentence 2: in [2] it is shown that: where: by using sample-specific weights r t (x, y) that match the training distribution c p(c)p(x, y|c) to the target distribution p(x, y|t) we can minimize the expected loss with respect to the target distribution by minimizing the expected loss with respect to the training distribution.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Compute V k and Σ k , the top k eigenvectors and eigenvalues of (∑ We now present empirical evaluation of our kernel learning framework and our semi-supervised kernel dimensionality approach when applied in conjunction with k-nearest neighbor classification.
Sentence 2: using different regularization functions, we show that our framework can be used to obtain significantly better kernels than the baseline kernels for k-NN classification.
Relationship: entailment
Domain: NeurIPS
Sentence 1: It follows from Theorem 1 in [11] and Theorem 1 in [10] and Lkk γ . Theorem 1 states that the quality of the bound depends on both Lkk and the maximum of { Lkm } m∈[K],m =k .
Sentence 2: the worstcase regret will be lower if the k th true hypothesis w * k can predict the labels for training examples in both the k th task itself as well as those in all the other related tasks in high confidence.
Relationship: entailment