sadat2307/MSciNLI huggingface dataset:
Domain: NeurIPS
Sentence 1: We choose K-means as the underlying clustering algorithm and vary the parameter K = 5 : 1 : 25, which is the desired number of clusters.
Sentence 2: since different runs of K-means starting from random initialization of centroids typically produce different clustering results, we run K-means 5 times for each value of K. there are a total of 21 × 5 = 105 different input clusterings.
Relationship: entailment
Domain: NeurIPS
Sentence 1: However, due to the high-dimensional continuous image input data and the model-free environment, we resort to the Q-learning algorithm combined with the function approximator technique to learn the optimal value for each state-action pair which generalizes well to unseen inputs.
Sentence 2: we use the deep Q-network proposed by [22,23] to estimate the value for each state-action pair using a deep neural network.
Relationship: entailment
Domain: NeurIPS
Sentence 1: In this paper we demonstrate how modern stochastic optimization techniques (such as SGD, ADA-GRAD [8] and ADAM [21]), can be used to solve an important class of discrete optimization problems which can be modeled using weighted coverage functions.
Sentence 2: we show how to efficiently maximize them under matroid constraints by (i) lifting the problem into the continuous domain using the multilinear extension [37], (ii) efficiently computing a concave relaxation of the multilinear extension [32], (iii) efficiently computing an unbiased estimate of the gradient for the concave relaxation thus enabling (projected) stochastic gradient ascent-style algorithms to maximize the concave relaxation, and (iv) rounding the resulting fractional solution without loss of approximation quality [7].
Relationship: entailment
Domain: NeurIPS
Sentence 1: Motivated by these two principles, we introduce the following sparsification operator: For any matrix A ∈ R d1×d2 : for all (i, j) where ) denote the elements of A (i,•) and A (•,j) that have the k-th largest magnitude respectively.
Sentence 2: we choose to keep those elements that are simultaneously among the largest α-fraction entries in the corresponding row and column.
Relationship: entailment
Domain: NeurIPS
Sentence 1: 3) Let P (X | E, R, w, u) be defined as in (11) 2 w * ← arg max w log P (X | E, R, w, u * ) // Find e.g.
Sentence 2: with L-BFGS using the gradient (12) 3 data using any standard density estimation technique 3 with the same features f ijk that are used in the CRF model, one can train a logistic regression model for P (• | E) : Essentially, a logistic regression model is a small CRF over only two variables.
Relationship: entailment
Domain: NeurIPS
Sentence 1: We assume that the learning algorithms on each view are able to PAC-learn from positive data only.
Sentence 2: for any distribution D + i over X + i , and any given , δ > 0, given access to examples from D + i the algorithm should be able to produce a hypothesis h i such that (a) h i ⊆ X + i (so h i only has one-sided error), and (b) with probability 1−δ, the error of h i under D + i is at most .
Relationship: entailment
Domain: NeurIPS
Sentence 1: Especially, we are interested in the high dimensional case when d > n. This section is devoted to the statement of our result on quantifying the convergence rate of R to Σ 0 and S to Σ.
Sentence 2: we establish the results on the max convergence rates of the Spearman's rho correlation and covariance matrices to Σ and Σ 0 .
Relationship: entailment
Domain: NeurIPS
Sentence 1: We assume that our algorithms have access to a supervised learning oracle, also called an argmax oracle, denoted AMO, that can find a policy with the maximum empirical reward on any appropriate dataset.
Sentence 2: given a dataset D = {x i , y i , v i } n i=1 of contexts x i , reward feature vectors y i 2 R K with rewards for all simple actions, and weight vectors v i 2 R L , the oracle computes where ⇡(x) `is the `th simple action that policy ⇡ chooses on context x.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Our main contributions are two-fold: first, we demonstrate that spatiotemporal functional evolution can be modeled using stationary kernels with a linear dynamical systems layer on their mixing weights.
Sentence 2: the model proposed here posits differential constraints, embodied as a linear dynamical system, on the spatiotemporal evolution of a kernel based models, such as Gaussian Processes.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Computing the concave (or convex) closure of a function is NP-hard in general, but it can be efficiently computed for certain special classes of discrete functions.
Sentence 2: when X = {0, 1} and log f is supermodular, then its concave closure can be computed in polynomial time as it is equal to the Lovász extension of log f . The Lovász extension has a number of interesting properties.
Relationship: entailment
Domain: NeurIPS
Sentence 1: This is qualitatively similar to what was found in pyramidal neurons in the neocortex [19] and in the hippocampus [18,20] that are able to switch from (sub)linear to supralinear integration of synaptic inputs through the generation of dendritic spikes [21].
Sentence 2: in neocortical pyramidal neurons Polsky et al.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Then we have: We next consider when f is ν-restricted strongly convex 8 in addition to having L-Lipschitz gradient.
Sentence 2: for x ∈ dom(f ), ∇f (x), x − Proj argmin f (x) ≥ ν • dist 2 (x, argmin f ).
Relationship: entailment
Domain: NeurIPS
Sentence 1: What makes the reverse KL divergence more tractable to optimize is that the expectations are done over Q instead of P . This minimization also yields bounds on the log partition function, or the probability of evidence in a BN.
Sentence 2: because KL(Q P ) is non-negative, The most commonly applied variational method is mean field, in which Q is chosen from the set of fully factorized distributions.
Relationship: entailment
Domain: NeurIPS
Sentence 1: We observe that the decomposition includes, for each variable, each and every interaction term of each and every degree weighted in a fashion resulting only from the combinatorics of possible interaction terms.
Sentence 2: since all I(X m ; Y |B) terms are at most equal to H(Y ), the prior entropy of Y , the p terms of the outer sum of Equation 3 are each upper bounded by As such, the second level decomposition resulting from totally randomized trees makes the p subimportance terms 1 to equally contribute (at most) to the total importance, even though they each include a combinatorially different number of terms.
Relationship: entailment
Domain: NeurIPS
Sentence 1: There exists such an n×(n−p) orthogonal matrix U 3 that its last column is 1 √ n 1 n and [U, U 3 ] is an n×n orthonormal matrix.
Sentence 2: u 3 is the eigenvector matrix of X(X X + σ 2 I p ) −1 X corresponding to the eigenvalue 0.
Relationship: entailment
Domain: NeurIPS
Sentence 1: In this paper, we establish a rigorous theoretical foundation for SCMD by providing optimal convergence rates (up to a logarithmic factor) in the stochastic optimization setting without boundedness assumptions on either subgradients or iterates, which in turn also shed new insights on the generalization behavior of the multi-pass SGD in the statistical learning theory setting.
Sentence 2: we justify the immunity of multi-pass SGD to overfitting by giving estimation error bounds with a logarithmic dependency on the number of passes for square-summable step sizes, while existing bounds scale polynomially [13,21].
Relationship: entailment
Domain: NeurIPS
Sentence 1: Now define the sets of vectors The first vector set, θ t x , represents the components of the the "true," but unknown, input vector x, its adaptive GAMP estimate x t as well as r t . The second vector, θ t z , contains the components of the "true," but unknown, output vector z, its GAMP estimate z t , as well as p t and the observed input y. x and θ t z are implicitly functions of the dimension n. Our main result, Theorem 1 below, characterizes the asymptotic joint distribution of the components of these two sets as n → ∞.
Sentence 2: we will show that the empirical distribution of the components of θ t x and θ t z converge to a random vectors of the form where X is the random variable in the initial condition (7).
Relationship: entailment
Domain: NeurIPS
Sentence 1: As a consequence of the above proposition, we can exploit standard results about LS.
Sentence 2: for instance, it is easy to show that, for a generic vector h 2 R d , the excess transfer risk the gap between the performance of standard Ridge Regression (h = 0), corresponding to solving each task independently, and the best algorithm in our class (i.e.
Relationship: entailment
Domain: NeurIPS
Sentence 1: For j ∈ {1, . . . , p}, define Γ σi,j to be an extensive game derived from the original game Γ where for all h ∈ H i \ G j , P (h) = c and f c (a|h) = σ i (h, a).
Sentence 2: player i only controls her actions for histories in G j and is forced to play according to σ i elsewhere.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Consider, for example, a decision making problem that is a quadratic optimization problem.
Sentence 2: namely, the DMP has the following form: QP Suppose that b changes over time t. b is the external signal for QP and equals to b t at time t. If we seek to learn c, the optimal solution set for QP can be characterized by KKT conditions as Here, u is the dual variable for the constraints.
Relationship: entailment
Domain: NeurIPS
Sentence 1: This algorithm requires that the smoothed objective has a Lipschitz continuous gradient.
Sentence 2: for some constant L, it must hold that kr f @x 1 A r f @x 2 Ak Lkx 1 x 2 k; for all x 1 ; x 2 P R n . Fortunately, by construction, the smoothed threshold extensions © wj;yj @xA all have I= Lipschitz gradient, a direct consequence of the characterization in Equation 4.
Relationship: entailment
Domain: NeurIPS
Sentence 1: For solving the OLS problem for tensor regression (2), we use a cyclic block-coordinate minimization algorithm based on a tensor toolbox [35].
Sentence 2: in a cyclic manner for all d 2 [D], we fix all but one and minimize the resulting quadratic loss function (2) with respect to ⇥ i , until the decrease of the objective is smaller than a predefined threshold ⌧ .
Relationship: entailment
Domain: NeurIPS
Sentence 1: A model consists of a set of variables V as the nodes of a graph G = (V, E), along with a set of factors specifying a joint probability distribution p Ψ (V ) described by parameters Ψ.
Sentence 2: this paper focuses primarily on directed models, in which the factors Ψ specify the conditional probability distributions of each variable given its parents.
Relationship: entailment
Domain: NeurIPS
Sentence 1: One interesting observation is that traditional perturbation-based adversarial examples can also be obtained as a special case of our noise-augmented attack, by choosing a suitable g ✓ (z, y) instead of the AC-GAN generator.
Sentence 2: let T be the test dataset, and T y = {x 2 T | o(x) = y}.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Then for any strategy profile σ = (σ 1 , σ 2 ) ∈ Σ 1 × Σ 2 we define the expected utility of the strategy profile (for player 1) as An -Nash equilibrium profile (σ 1 , σ 2 ) in this case is defined analogously to (1).
Sentence 2: none of the players can improve their utility by more than by deviating unilaterally.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Figure 1 shows an example application of image tag recommendation, where more input features (called privileged information [47]) are available at the training stage than those available at the inference stage.
Sentence 2: the training stage has access to images as well as image titles and comments (textual information) as shown in Figure 1a, whereas the inference stage only has access to images themselves as shown in Figure 1b.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Using the NB classifier as the baseline, however, shows not only improved baseline results in general but also that the sigmoid(10) improves more often.
Sentence 2: the KLD results are better than the baseline 16 times, sign is better than the baseline 18 times, and sigmoid (10) beats the baseline 19 times, suggesting that sigmoid (10) typically wins over the KLD case.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Inspired by the space carving theory [10], we propose a silhouette-based volumetric loss function.
Sentence 2: we build on the premise that a 2D silhouette Ŝ(j) projected from the generated volume V under certain camera viewpoint α (j) should match the ground truth 2D silhouette S (j) from image observations.
Relationship: entailment
Domain: NeurIPS
Sentence 1: In this paper we are interested in proving results on the sample complexity of differentially private algorithms that approximate PCA.
Sentence 2: for a given ↵ and ⇢, how large must the number of individuals n in the data set be such that it is ↵-differentially private and also a (⇢, ⌘)-close approximation to PCA?
Relationship: entailment
Domain: NeurIPS
Sentence 1: Now proceed by induction to determine E M +1 and ω M +1 , assuming that we have determined E M and ω 0 , ..., ω M such that P (E M = 1| C i ; ω 0 , ..., ω M ) = P (E = 1| C i ), for i = 0, ..., M . There are three cases to consider which are analogous to the cases considered in the section with two causes.
Sentence 2: then we obtain: we see that To determine the value of ω M +1 , we must solve Then we obtain: As for the first case, To determine the value of ω M +1 we must solve ; ω 0 , ..., ω M ), then we do nothing.
Relationship: entailment
Domain: NeurIPS
Sentence 1: For any δ > 0, 0 < ǫ < 1/2, run the algorithm in Figure 1 on a sample X drawn from f , where the parameters are set according to the equations Then with probability at least 1−δ, C is (σ, ǫ) consistent.
Sentence 2: the clusters containing A[X] and A ′ [X], where A and A ′ are (σ, ǫ) separated, are internally connected and mutually disconnected in C(r) for r defined by Before we prove this theorem a few remarks are in order: 1.
Relationship: entailment
Domain: NeurIPS
Sentence 1: The constraint u q ≥ z d can simply be rewritten coordinate-wise y qt ≥ x dt .
Sentence 2: finally, the capacity constraints assume the form d x dt ≥ |D| − C t . the number of pages allocated to higher tiers are at least |D| − C t . Define remaining capacities Ct := |D| − C t and use the variable transformation (1) we have the following integer linear program: minimize x,y v yp (4a) subject to x dt ≥ x d,t+1 and y qt ≥ y q,t+1 and y qt ≥ x dt for all (q, d) ∈ G (4b) where p = (p 1 , .
Relationship: entailment
Domain: NeurIPS
Sentence 1: In this work, we build upon graph neural networks in order to learn useful representations for graph classification in an end-to-end fashion.
Sentence 2: we consider GNNs that employ the following general "message-passing" architecture: where H (k) ∈ R n×d are the node embeddings (i.e., "messages") computed after k steps of the GNN and M is the message propagation function, which depends on the adjacency matrix, trainable parameters θ (k) , and the node embeddings H (k−1) generated from the previous message-passing step.
Relationship: entailment
Domain: NeurIPS
Sentence 1: [of Theorem 4] While the sample complexity is easy to verify, we focus on the proof of the first property.
Sentence 2: we show that when J happens (which with probability at least 1 − δ by ( 12)), we have err Di (g) ≤ for all i ∈ {1, 2, 3, . . . , k}.
Relationship: entailment
Domain: NeurIPS
Sentence 1: eigenvalues/eigenvectors, as E . In this section we describe two general "matrix simplification" techniques and discuss their implications for Kernel PCA.
Sentence 2: under natural assumptions on the spectral structure of E , we will prove that applying KPCA to the simplified matrix E yields subspaces which are very close to those that KPCA would find in E . As a result, when we project vectors onto these spaces (as performed by the feature extractors) the results are provably close to the original ones.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Let ∆ T,z denote the number of times T changes labels on the its leaves over the sequence z, and R T,z denote the number of times times T has completely reconfigured itself over z.
Sentence 2: if there is an offline adaptive k-IPtree, that makes few changes and few mistakes on the input sequence of IP addresses, then TrackIPTree will also make only a small number of mistakes.
Relationship: entailment
Domain: NeurIPS
Sentence 1: The first term penalizes large distances between each input and its target neighbors, while the second term penalizes small distances between each input and all other inputs that do not share the same label.
Sentence 2: the cost function is given by: where in the second term [z] + = max(z, 0) denotes the standard hinge loss and c > 0 is some positive constant (typically set by cross validation).
Relationship: entailment
Domain: NeurIPS
Sentence 1: The eigenvalue method in the previous section has one obvious shortcoming: it cannot compensate for phase changes across channels.
Sentence 2: the real-valued linear combi- We now reconsider the problem of the previous section, looking for the linear combination of analytic signals, 8(t) = L; w;x;(t), that minimizes the cost function in eq.
Relationship: entailment
Domain: NeurIPS
Sentence 1: A factored, n-agent DEC-POMDP is transition independent iff the state-transition function can be separated into n + 1 distinct transition functions P 0 , .
Sentence 2: . . , P n , where, for any next state s i ∈ S i , the next local state of each agent is independent of the local states of all others, given its previous local state and local action, and the external system features (S 0 ).
Relationship: entailment
Domain: NeurIPS
Sentence 1: Note that although the idea of approximating a DP problem with an LP problem has been widely studied in literature (e.g., [17,19]), the design and analysis of ALP here is quite different.
Sentence 2: we show that ALP achieves O(1) regret, i.e., its expected total reward is within a constant independent of T from the optimum, except for certain boundaries.
Relationship: entailment
Domain: NeurIPS
Sentence 1: We use the given query Q, which is part of the data x, to construct indicators, ι s = δ(s ∈ Q) ∈ {0, 1}, denoting for every token s ∈ S its existence in the query Q, where δ denotes the indicator function.
Sentence 2: based on this definition, we formulate the energy function as follows: where w s,c is a parameter connecting a word s ∈ S to an image concept c ∈ C. w t = (w s,c : ∀s ∈ S, c ∈ C).
Relationship: entailment
Domain: NeurIPS
Sentence 1: Let s(L) be the SVM objective function trained on the labeled set, As discussed in the previous section, the lower bound is s(L).
Sentence 2: now our branching strategy consists in selecting the following point in U , arg max we want to find the unlabeled point x * and its label y * which would make the objective function increase as much as possible.
Relationship: entailment
Domain: NeurIPS
Sentence 1: In Figure 3, we compare the performance of these four algorithms as we vary the minibatch size b.
Sentence 2: the first column (b = 4) shows that ProxSVRG+ and ProxSVRG perform similar to ProxSGD and ProxGD respectively, which is quite consistent with the theoretical results (Figure 1).
Relationship: entailment
Domain: NeurIPS
Sentence 1: The noise is generated as w ∼ N (0, σ 2 I n ).
Sentence 2: we fix p = 200, σ = 0.1, and let β i = ±1/ √ s over its support i ∈ S, where |S| = s. only s of the p = 200 variables are predictive of the response.
Relationship: entailment
Domain: NeurIPS
Sentence 1: We characterize conditions for the output of the algorithm to be unique, and we also obtain some general characterizations that help us to determine which method of breaking leads to a consistent GMM.
Sentence 2: full breaking (which uses all pairwise comparisons in the ranking) is consistent, but adjacent breaking (which only uses pairwise comparisons in adjacent positions) is inconsistent.
Relationship: entailment
Domain: NeurIPS
Sentence 1: We now define the utility of ALG (negative of the utility of ADV) in our zero-sum game as follows: Suppose the expected utility of ALG is non-negative at the equilibrium of this game.
Sentence 2: suppose ALG's randomized strategy ẑi (in Algorithm 1) guarantees that for every strategy x ⇤ i of ADV the expected utility of ALG is non-negative.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Then, for any A such that |A| = n 0 < n, denoting by σ(A) the expected number of nodes reached by the cascade starting from A: where γ 1 is the smallest solution in [0, 1] of the following equation: Corollary 1.
Sentence 2: under the same assumptions: when ) and the regime is sub-critical.
Relationship: entailment
Domain: NeurIPS
Sentence 1: The normalization constant DM appearing in (10) can also be computed for the quadratic surrogate (11) under the assumption of well-specification (see [14, Appendix F] for details).
Sentence 2: we have DM = L 2 max ξ(κ(F ) √ rRQ max ), ξ(z) = z 2 + z, where L max denotes the maximal value of all elements in L, κ(F ) is the condition number of the matrix F and r in an upper bound on the rank of F. The constants R and Q max come from the kernel ASGD setup and, importantly, depend only on the data distribution, but not on the loss L or score matrix F . Note that for a given subspace F, the choice of matrix F is arbitrary and it can always be chosen as an orthonormal basis of F giving a κ(F ) of one.
Relationship: entailment
Domain: NeurIPS
Sentence 1: The discriminator D or critic f w and the generator G are typically parameterized by neural networks.
Sentence 2: in our setting, the data distribution is the distribution P (h x ) of sentence embeddings, h x = E(x), obtained by applying sentence encoder E to samples x ∼ P D from a dataset D. our critic and generator are trained with objective In presenting the Wasserstein GAN, [3] argue that while KL divergence is a good divergence for low-dimensional data, it is often too strong in the high-dimensional settings we are often interested in modeling.
Relationship: entailment
Domain: NeurIPS
Sentence 1: The base measure is the mean of the prior on the unknown token generating distribution, and α can be thought of as an "inverse variance" that determines how similar the random measures are to G. The Dirichlet process [14] is defined by the property that for all finite partitions of V, the distribution over the measures of those partitions is Dirichlet.
Sentence 2: for all finite, measurable partitions A 1 , .
Relationship: entailment
Domain: NeurIPS
Sentence 1: In the sequel, we parameterize C ∈ R n×n in terms of some hyperparameter vector d ∈ R k as the diagonal matrix, C(d) = diag(exp(d)).
Sentence 2: under this representation, C(d) is necessar-ily positive definite, so OPT2 can be written as an unconstrained minimization over the variables d ∈ R k . let ℓ T (w) = − m i=1 log P y (i) | x (i) ; w denote the training logloss and ℓ H (w) = − m i=1 log P ỹ(i) | x(i) ; w the holdout logloss for a parameter vector w. Omitting the dependence of C on d for notational convenience, we have the optimization problem For any fixed setting of these hyperparameters, the objective function of OPT2' can be evaluated by (1) using the hyperparameters d to determine the regularization matrix C, (2) solving OPT1 using C to determine w ⋆ and (3) computing the holdout logloss using the parameters w ⋆ .
Relationship: entailment
Domain: NeurIPS
Sentence 1: By now, it has been shown that CMOL circuits can combine high performance with high defect tolerance (which is necessary for any circuit using nanodevices) for several digital applications.
Sentence 2: cMOL circuits with defect rates below a few percent would enable terabit-scale memories [7], while the performance of FPGA-like CMOL circuits may be several hundred times above that of overcome purely CMOL FPGA (implemented with the same F CMOS ), at acceptable power dissipation and defect tolerance above 20% [8].
Relationship: entailment