sadat2307/MSciNLI huggingface dataset:
Domain: NeurIPS
Sentence 1: We set the source classifier f S as the outputs of the residual block to make it better trainable from the source-labeled data by deep residual learning [8].
Sentence 2: if we set f T as the outputs of the residual block, then we may be unable to learn it successfully as we do not have target labeled data and thus standard back-propagation will not work.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Ideally, we would want to be able to read out both the present input and the past inputs, such that x t−n ≈ D n r t , where n is an elementary time step, and D n are appropriately chosen readouts.
Sentence 2: we will in the following assume that there is a matrix M such that D n M = D n+1 for all n. the input history should be accessible via xt−n = D n r = D 0 M n r t . Then the cost function we would like to minimize is a straightforward generalization of Eq.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Moreover, since we are mostly interested in the properties of the metric space, we focus on embeddings based purely on the Poincaré distance and on models with comparable expressivity.
Sentence 2: we compare the Poincaré distance as defined in Equation ( 1) to the following two distance functions: Euclidean In all cases, we include the Euclidean distance d(u, v) = u − v 2 .
Relationship: entailment
Domain: NeurIPS
Sentence 1: We consider three baselines: (i) Individual Corruption: for each corruption type i 2 [m], we construct an oracle that trains a neural network using the training data perturbed by corruption i, and then returns the trained network weights as ✓ t , for every t = 1, . . . , T . This gives m baselines, one for each corruption type; (ii) Even Split: this baseline alternates between training with different corruption types between iterations.
Sentence 2: call the previous m baseline oracles O 1 , ..., O m . Then this new baseline oracle will produce ✓ t with O i+1 , where i ⌘ t mod m, for every t = 1, ..., T ; (iii) Uniform Distribution: This more advanced baseline runs the robust optimization scheme with the Hybrid Method (see Appendix), but without the distribution updates.
Relationship: entailment
Domain: NeurIPS
Sentence 1: One useful application of this theorem is in matching graphs with Euclidean affinities, since Euclidean distances are conditionally negative definite of order 1 (Wendland, 2004).
Sentence 2: the affinities are Euclidean distances of points in Euclidean spaces of arbitrary dimensions, where This class contains, besides Euclidean graphs, also affinities made out of distances that can be isometrically embedded in Euclidean spaces such as diffusion distances (Coifman and Lafon, 2006), distances induced by deep learning embeddings (e.g.
Relationship: entailment
Domain: NeurIPS
Sentence 1: To derive the posterior (2), we first reformulate the likelihood as a multivariate Gaussian distribution over z.
Sentence 2: after taking out constant terms with respect to z from the likelihood, it immediately becomes proportional to the canonical form Gaussian over z with ν = BΣ −1 y and Λ = BΣ −1 B , which is equivalent to the standard form Gaussian with mean Λ −1 ν and covariance Λ −1 .
Relationship: entailment
Domain: NeurIPS
Sentence 1: Thus, statistical feedback is a difference in potential outcomes.
Sentence 2: an additive feedback model In order to get a handle on feedback as defined above, we assume that feedback enters the model additively: , where f is a feedback function, and y (t) i is the prediction published at time t. we assume that the predictions made by our model at time t + 1 are the sum of the prediction the model would have made if there were no feedback, plus a feedback term that only depends on the previous prediction made by the model.
Relationship: entailment
Domain: NeurIPS
Sentence 1: This solves (3) for s = kgk 1 /kgk q and satisfies E[kL q (g)k 2 2 ] = kgk 1 kgk q .
Sentence 2: for q = 2 we get 1-bit QSGD while for q = 1, we get TernGrad.
Relationship: entailment
Domain: NeurIPS
Sentence 1: We tested the VHEM algorithm on hierarchical motion clustering, where each of the input HMMs to be clustered is estimated on a sequence of motion capture data from the Motion Capture dataset (http://mocap.cs.cmu.edu/ ).
Sentence 2: we start from K 1 = 56 motion examples from 8 different classes ("jump", "run", 'jog'", "walk 1" and "walk 2" which are from two different subjects, "basket", "soccer", "sit"), and learn a HMM for each of them, forming the first level of the hierarchy.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Similarly to the matrix factorization approaches [2,3,13], automatic dimensionality selection is invoked via hyperparameters c A,i and c B,i , which enforce sparsity in the columns and rows of A and B, respectively.
Sentence 2: when a particular set of variances c A,i , c B,i assume very small values, the posteriors of the i th column of A and i th row of B will be concentrated around zero, such that the effective number of principal directions in AB will be reduced.
Relationship: entailment
Domain: NeurIPS
Sentence 1: The dynamics (3) define the behavior of the solution X t = U t U t and using the chain rule we can verify that Ẋt = Ut , where r t = A(X t ) − y is a vector of the residual.
Sentence 2: even though the dynamics are defined in terms of specific factorization X t = U t U t , they are actually independent of the factorization and can be equivalently characterized as We can now define the limit point X ∞ (X init ) := lim t→∞ X t for the factorized gradient flow (4) initialized at X 0 = X init .
Relationship: entailment
Domain: NeurIPS
Sentence 1: the ('Toyota', 'Kia') and the ('Bush', 'Cheney') tuples, rather than generally related words.
Sentence 2: for this purpose we design a similarity graph G(V, E) with words represented as vertices V and similarity edge weights φ uv between vertices u, v ∈ V whenever u is related to v. the magnitude of φ uv can denote the similarity between words u and v. In the following we denote by y kv the topic dependent smoothing coefficients for a given word v and topic k. We impose the smoother where log p(β) is given up to an additive constant and y v allows for multiplicative topic-unspecific corrections.
Relationship: entailment
Domain: NeurIPS
Sentence 1: In Section 4, we investigate the performance, scalability, and convergence of the sampling algorithm on the probabilistic inference problem of collective classification on a set of Wikipedia documents.
Sentence 2: we show that the standard deviation of the marginal density function can serve as a strong indicator for the "confidence" in the classification prediction, thereby demonstrating a useful qualitative aspect of marginals over continuous MRFs.
Relationship: entailment
Domain: NeurIPS
Sentence 1: (B) The same model can also be used more generally to fit multivariate binary data with given pairwise correlations, if x is drawn from a Gaussian distribution.
Sentence 2: we will show that the resulting distribution closely resembles the binary maximum entropy models known as Ising models or Boltzmann machines which have recently become popular for the analysis of spike train recordings from retinal ganglion cell responses [13,12].
Relationship: entailment
Domain: NeurIPS
Sentence 1: Although each entry x ij ∈ X is independent given Y and Z, the PCSA model captures the dependencies along rows as well as columns in the joint X.
Sentence 2: assuming D 1 is the data dimensionality while D 2 is the sample size, the samples (column vectors) in X is dependent from each other by PC-SA.
Relationship: entailment
Domain: NeurIPS
Sentence 1: To see this, let us denote the algorithm from Theorem 2.1 as A.
Sentence 2: its inputs are the database S and the set of candidate centers Y , and the differential privacy guarantee is only with respect to the database S. for every fixed set Y , the algorithm A Y (S) = A(S, Y ) is differentially private as a function of S. Known composition theorems for differential privacy [13] show that for every differentially private algorithm B that takes a database S and outputs a set of centers Y , we have that the composition A(S, B(S)) satisfies differential privacy.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Fortunately, a simple power method [10] can be used to efficiently compute a maximum eigenvector solution to the inner problem by only performing matrix-vector multiplications on the individual factors of the two low rank matrices making up T (α), meaning the inner problem can be solved without ever forming a large n × n matrix T (α).
Sentence 2: if X is n × m each inner iteration requires at most O(nm) computation.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Moreover, if X is strictly stationary, then the joint distribution of the aligned sequences (X, Y ) is strictly stationary.
Sentence 2: the conditional transition probabilities P t (x, i, j) x have the following form: a lower bound for P t (x, i, j) is given by and the matrix of transition probabilities P t (x), with the (i, j)-th component given by P t (x, i, j), satisfies φ(P t (x)) = φ(M (x t )).
Relationship: entailment
Domain: NeurIPS
Sentence 1: This statement follows from Lemma 1 of Ghosal and van der Vaart (2007) and holds upon the satisfaction of the conditions The key observation behind Corollary 5.1 is that the posterior does not overshoot in terms of the width and sparsity, rewarding only small networks that are sparse.
Sentence 2: the posterior concentrates on networks with up to the optimal number s n of links.
Relationship: entailment
Domain: NeurIPS
Sentence 1: The new conditions generalize this to k > 2.
Sentence 2: define C eor to be the multiclass extension of C bin : any cost-matrix in C eor should put the least cost on the correct label, i.e., the rows of the cost-matrices should come from the set c ∈ R k : ∀l, c(1) ≤ c(l) .
Relationship: entailment
Domain: NeurIPS
Sentence 1: For the embeddings, we try 3 initialization schemes and choose the best one based on validation error.
Sentence 2: these schemes are: (1) all embeddings are drawn from the Gaussian prior implied by the regularizer; (2) the embeddings are initialized from a global embedding; (3) the context vectors are initialized from a global embedding and held constant, while the embeddings vectors are drawn randomly from the prior.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Note if dim H = ∞, with probability one f is not in H, but fortunately H is large enough for us to approximate the sampled functions.
Sentence 2: it can be shown that the posterior GP(m |y , k |y ) in GPR has a dual RKHS representation in the same RKHS as the prior GP [2].
Relationship: entailment
Domain: NeurIPS
Sentence 1: A state is called locally inconsistent every time its g-value is decreased (line 09, Figure 1) and until the next time the state is expanded.
Sentence 2: suppose that state s is the best predecessor for some state s : that is, g(s ) = min s ∈pred(s ) (g(s )+c(s , s )) = g(s)+c(s, s ).
Relationship: entailment
Domain: NeurIPS
Sentence 1: Further, suppose µ (j) 4 < 1 for all j 1 and that where C 1 , C 2 the following finite constants: and A .
Sentence 2: if M 0 = 1 and 4  1 and µ (j) is Gaussian for all j 1, then we can take Var[M d ] is exponential in P d j=1 1/n j and if The proof of Theorem 5 is deferred to the Supplementary Material.
Relationship: entailment
Domain: NeurIPS
Sentence 1: We present two architectural recipes in the context of multi-stage progressive encoders and empirically demonstrate their importance on compression performance.
Sentence 2: we show that: (a) predicting the original image data from residuals in a multi-stage progressive architecture facilitates learning and leads to improved performance at approximating the original content and (b) learning to inpaint (from neighboring image pixels) before performing compression reduces the amount of information that must be stored to achieve a high-quality approximation.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Then, we can treat the input bag-of-words as a |x|-trials multinomial sample from that distribution and use as the reconstruction loss its negative log-likelihood: We now must ensure that the decoder can compute p( x = x i |φ(x)) efficiently from φ(x).
Sentence 2: we'd like to avoid a procedure scaling linearly with the vocabulary size V , since V will be very large in practice.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Because MRI and PET data are highdimensional, regularization techniques are needed for effective variable selection, such as the L1regularization technique [25]- [30] and the L2/L1-regularization technique [31].
Sentence 2: l2/L1-regularization has been used for variable selection jointly on multiple related datasets, also known as multitask feature selection [31], which has a similar nature to our problem.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Finally, we also conduct a preliminary exploration into the question of how robust the learning of intrinsic rewards is to the sparsity of extrinsic rewards.
Sentence 2: we used the delayed versions of the Mujoco domains, where the extrinsic reward is made sparse by accumulating the reward for N = 10, 20, 40 time steps before providing it to the agent.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Here, we use amortization to control the number of parameters of the S-EFE model.
Sentence 2: we set the per-group embeddings ρ (s) v to be the output of a deterministic function of the global embedding vectors, ρ We use a different function f s (•) for each group s, and we parameterize them using neural networks, similarly to other works on amortized inference (Korattikara et al., 2015;Kingma and Welling, 2014;Rezende et al., 2014;Mnih and Gregor, 2014).
Relationship: entailment
Domain: NeurIPS
Sentence 1: Then, with D W (•; x, s) denoting the distribution of W (x, s), if w n min (x, s) ≥ 0 and zero otherwise.
Sentence 2: the EI boils down to integrating the distribution function of W (x, s) between 0 (since W is positive) and w n min (x, s).
Relationship: entailment
Domain: NeurIPS
Sentence 1: Then with probability at least 1 − N 2 e −Ω(d) over the random choice of training set, f maps the entire inner sphere to the same point, and the entire outer sphere to some other point, except for a γ-fraction of both spheres, where γ = N e −Ω(d) .
Sentence 2: f is (ε, 0, γ)-robust.
Relationship: entailment
Domain: NeurIPS
Sentence 1: If we condition on the event {Y ∈ A e } for some e, then as soon as we have calculated the region R Y ⊆ R + , Theorem 1 will allow us to perform inference on the quantity of interest P Le,g µ 2 by evaluating the expressions (4) and (5).
Sentence 2: we are testing whether µ is significantly correlated with the group X g , after controlling for all the other selected groups, S(Y )\g = S e \g.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Based on this, we propose to use a principled approach using the likelihood ratio test to measure the similarity between data points under any arbitrary parameterized distribution, which includes the previously adopted Gaussian and exponential family distributions as special cases.
Sentence 2: we prove that for the heavy-tailed distribution we proposed, the corresponding similarity measure leads to a distance metric, theoretically justifying its use as a similarity measurement between image patches.
Relationship: entailment
Domain: NeurIPS
Sentence 1: As a result, many standard nonlinear kernels on X can be used to define nonlinear kernels on P as long as the kernel evaluation depends entirely on the inner product µ P , µ Q H , e.g., K(P, Q) = ( µ P , µ Q H + c) d . Although requiring more computational effort, their practical use is simple and flexible.
Sentence 2: the notion of p.d.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Let φ(x i ) denote the mapped counterpart of x i in kernel space, and then In light of the maximum scatter difference criterion [26], we take into account the ordinal label information, and propose a novel criterion named Ordinal Scatter Discrepancy (OSD) to achieve the desired data distribution after projecting Φ(X) to a low-dimensional subspace.
Sentence 2: oSD minimizes the within-class scatter, and meanwhile maximize the noncontiguous-class scatter matrix.
Relationship: entailment
Domain: NeurIPS
Sentence 1: (8) If we further have η(t) ≥ r (t), then the conditions of Theorem 2 and Theorem 3 are satisfied, which guarantee that L r is a Lyapunov function and that the energy E r decreases.
Sentence 2: such a heuristic would preserve the convergence rate r(t) by Theorem 2.
Relationship: entailment
Domain: NeurIPS
Sentence 1: In this section, we show that our metric covers the existing metrics defined in the previous works [12,6,25].
Sentence 2: we describe the relation to the metric via subspace angles and Martin's metric in Subsection 4.1 and the one to the Binet-Chaucy metric for dynamical systems in Subsection 4.2 as the special cases of our metric.
Relationship: entailment
Domain: NeurIPS
Sentence 1: In recent years, researchers [I, 2] studying the statistical properties of such estimators have started discovering commonalities among proofs and analyses and have proposed unified frameworks that take advantage of such commonalities.
Sentence 2: using a single theorem, they are able to rederive a wide range of known results on high-dimensional consisteney and error bounds for the various regularized and constrained estimators.
Relationship: entailment
Domain: NeurIPS
Sentence 1: We minimize the reconstruction error of dual swap output with respect to the original input, and write the dual swap loss L d as follows: The dual swap reconstruction minimization here provides a unique form of self-supervision.
Sentence 2: by swapping random parts back and forth, we encourage the element-wise separability and modularity of the obtained encodings, which further helps the encoder to learn disentangled representations under the guidance of limited weak labels.
Relationship: entailment
Domain: NeurIPS
Sentence 1: c (•) does have the necessary characteristics of a proper loss: summaries S + with large V-ROUGE score are mapped to small values and summaries S − with small V-ROUGE score are mapped to large values.
Sentence 2: submodularity means r(S) , so complement rouge is a submodular upper bound of the ideal supermodular loss.
Relationship: entailment
Domain: NeurIPS
Sentence 1: To model the joint indicator variables, we define Such Bayesian approach favors simple model structures.
Sentence 2: the priors on A k and B k enforce a sparse parametrization since, during learning, many α k ij and β k j get close to infinity whereby (the posterior distribution of) A k ij and B k j get close to zero (see [11] for an analysis of this pruning effect).
Relationship: entailment
Domain: NeurIPS
Sentence 1: Such a mixed binary-continuous problem arises in Bayesian model selection with a spike-and-slab prior and we illustrate our technique by focusing on this case.
Sentence 2: we show how to sample from the posterior of linear and probit regression models with spike-and-slab priors, while also imposing truncations in the parameter space (e.g., positivity).
Relationship: entailment
Domain: NeurIPS
Sentence 1: Moreover, Alice and Bob have common knowledge of all these facts.
Sentence 2: now, consider the following Pareto optimal policy that favors Alice (principal 1) when o 1 is red, and Bob (principal 2) when o 1 is green: (none, all) This policy can be viewed intuitively as a bet between Alice and Bob about the value of o 1 , and is highly appealing to both principals: π is more appealing to both Alice and Bob than an agreement to deterministically split the cake (half, half).
Relationship: entailment
Domain: NeurIPS
Sentence 1: As both strategies proposed by [1] are consistent with all the assumptions that we have made in section 3.4 when applied to RNNs, BPTT- Chen's √ t algorithm requires to remember √ t hidden states and √ t internal RNN states (excluding input hidden states), while the recursive approach requires to remember at least log 2 t hidden states.
Sentence 2: the model does not allow for a fine-grained control over memory usage and rather saves some memory.
Relationship: entailment
Domain: NeurIPS
Sentence 1: ( 9) at iteration t, the item r t (w) we added can be shown to be the delta over w * t , that is, r t (w) = I(w = w * t ); in addition, we have q t (w) = 1 t t s=1 I(w = w * s ) when the step size is taken to be β t = 1 t . Therefore, this Frank-Wolfe update can be naturally interpreted as greedily inserting a hidden unit into the current model p(v | q t , ϑ t ).
Sentence 2: if we update the temperature parameter as α t ← t, according to Proposition 3.1, we can directly transform our model p(v | q t , ϑ t ) to a regular RBM after each Frank-Wolfe step, which enables the convenient blocked Gibbs sampling for inference.
Relationship: entailment
Domain: NeurIPS
Sentence 1: We can see the GAN objective, as a special instance of (7) by identifying each terms in the expectations of ( 7) and (8).
Sentence 2: choosing the last nonlinearity in the discriminator as the sigmoid D ω (x) = 1/(1 + e −Vω(x) ), corresponds to output activation function is g f (v) = − log(1 + e −v ); see Table 2.
Relationship: entailment
Domain: NeurIPS
Sentence 1: We show that for a wide class of optimization problems, if the sketch is close (in the spectral norm) to the original data matrix, then one can recover a near optimal solution to the optimization problem by using the sketch.
Sentence 2: we use this approach to obtain sparse principal components and show that for m data points in n dimensions, O( −2 k max{m, n}) elements gives an -additive approximation to the sparse PCA problem ( k is the stable rank of the data matrix).
Relationship: entailment
Domain: NeurIPS
Sentence 1: For example, the gradient restart in [17] is only studied for unconstrained quadratic problems, and the speed restart in [20] is only studied for unconstrained strongly convex problems.
Sentence 2: it is not guaranteed (to our knowledge) that these heuristics preserve the original convergence rate of the non-restarted method, when the objective function is not strongly convex.
Relationship: entailment
Domain: NeurIPS
Sentence 1: If β ≤ 1/ √ 2, then there exists a sequence of vectors {v 0 (n)} n≥0 , such that almost surely such that the following happens.
Sentence 2: for any sequence of vectors {v 0 (n)} n≥0 , we have, almost surely in the high-dimensional limit, the maximum likelihood estimator is positively correlated with the signal v 0 (n) if and only if Explicit (although non-elementary) expressions for R + (β) can be computed, along with the limit value of the risk R(v ML ; v 0 (n)) for sequences of vectors {v 0 (n)} n≥1 whose entries empirical distribution converges.
Relationship: entailment
Domain: NeurIPS
Sentence 1: If for all pairs {z i , z j } ⊂ Z and all s ∈ , and if SpectralQuartetTest returns a pairing {{z i , z j }, {z i , z j }}, then {{z i , z j }, {z i , z j }} = {{z 1 , z 2 }, {z 3 , z 4 }}.
Sentence 2: the spectral quartet test never returns an incorrect pairing as long as the singular values of E[z i z j ] lie in an interval of length 2∆ i,j around the singular values of Σi,j .
Relationship: entailment
Domain: NeurIPS
Sentence 1: Observation: [17] If p(x) is separable, and similarity between datapoints is defined as e − xi−xj 2 / 2 then the eigenfunctions of the continuous weighted Laplacian, L p have an outer product form.
Sentence 2: if Φ i (x) is an eigenfunction of the weighted Laplacian defined on R 1 with eigenvalue λ i then Φ Specifically for a case of a uniform distribution on [a, b] the eigenfunctions of the onedimensional Laplacian L p are extremely well studied objects in mathematics.
Relationship: entailment
Domain: NeurIPS
Sentence 1: In particular, SGD can only select the minima with s  1/⌘ whereas GD does not have this requirement.
Sentence 2: for the toy example discussed above, at x = 1, we have s = 1.8 > 1/0.7, so as predicted SGD with ⌘ = 0.7 will escape and finally converge to minimum x = 0, where s = 0 < 1/0.7.
Relationship: entailment