sadat2307/MSciNLI huggingface dataset:
Domain: NeurIPS
Sentence 1: Note that, to be consistent about when approximations are made and thus produce weights ŵL which still sum to Ẑ, we repeat the procedure for computing Ẑ exactly, including recomputing the running lower bound Z min .
Sentence 2: this algorithm is guaranteed to sample each label L with probability pL ∈ [p L − , p L + ]: Proof: From our bounds on the error of the estimated probability of choosing label L has at most error 5 Computational Examples In this section, we compare the sampling methods discussed in this paper on three challenging one-dimensional examples, each involving products of mixtures of 100 Gaussians (see Figure 4).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The set W n−1 (s → * ) denotes walks that start at node s computed at the previous iteration.
Sentence 2: the middle term W( * denotes a length-1 walk (called a hop) across an edge in E\S n . Finally, W(• Sn −→ t) denotes walks in S n that end at node t. the first term in (7) refers to previously computed walks starting at s, which hop across an edge in E\S n , and then finally propagate only in S n (ending at t).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: We have where the second inequality holds since The first and third inequalities come from U t ⊆ S t and the definition of U t . Putting (10) into ( 9), we have It follows from the above lemma that, if w t converges to w * , we have D t w t = w * , and hence S t includes the support of w * .
Sentence 2: moreover, it holds that , since w t is independent of x t and y t . to bound Lemma 4 ( [11]).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: So, if K is sufficiently large, by Chebyshev's inequality, with 99% probability we have that (γ) r > F r (p)/2 ≥ C n 1−r .
Sentence 2: in the remainder, we can assume that , where we use that ≥ n −1/4 and m = Cn 2/3 / 4/3 .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: However, p bma only starts to behave like p 2 when it catches up with p 1 at a sample size of about 310 000, when the codelength of p 2 drops below that of p 1 .
Sentence 2: in the shaded area p bma behaves like p 1 while p 2 is making better predictions of those outcomes: since at n = 100 000, p 2 is 40 000 bits behind, and at n = 310 000, it has caught up, in between it must have outperformed p 1 by 40 000 bits!
Relationship: reasoning
Domain: NeurIPS
Sentence 1: It follows that the total complexity of computing k(G, G ) is When computing the kernel matrix i=1 of graphs with N > m + n + δ 2 , note that Algorithm 4 only needs to be run once for every graph G i .
Sentence 2: the average complexity of computing one kernel value out of all K ij becomes 1 Classification experiments were made with the proposed GraphHopper kernel and several alternatives: The kernel PROP [16], the connected subgraph matching kernel CSM [14] and the shortest path kernel SP [17] all use continuous-valued attributes.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Figure 1 displays both 1 − ηN (ε) and the average CCR as a function of ε for different values of N . Note that 1 − ηN (ε) quantifies the probability (with respect to the training data) that P belongs to the Wasserstein ball of radius ε around the empirical distribution PN .
Sentence 2: 1 − ηN (ε) increases with ε.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: An exponential filter predicts that ARR− will always create a stronger expectation for an R on the current trial than AAR− will, because the former includes an additional past repetition.
Sentence 2: if the current trial is in fact a repetition, the model predicts a faster RT for ARR− compared to AAR− (i.e., RT ARRR < RT AARR ).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: By the property of reversal Markov chains and lemma 1, we have that lim i→∞ P i r − P ∞ r → 0 and lim i→∞ ((1 − β)P + β P ) i r − P ∞ r → 0, such that the Marvov chain P and its reversal (1 − β)P + β P converge to the same value.
Sentence 2: the norm (P i − ((1 − β)P + β P ) i )r p also converges to 0 in the limit.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: This is reflected in the lower bound 1/q for β and the fact that the players' rate of convergence to Nash equilibrium increases for smaller β; in particular, (3.3) shows that Algorithm 1 enjoys a convergence bound which is just shy of O(exp(−n 1−1/q )).
Sentence 2: if the noise process ξ is sub-Gaussian/sub-exponential (so q can be taken arbitrarily large), a near-constant step-size sequence (small β) yields an almost linear convergence rate.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: For each summand, we take the subgradient Φ(x i , s i (w t )), where We note that computing s i (w t ) for each training example can be costly.
Sentence 2: but from our definition of L output , we have that s i (w) = ⊥ for a background example independent of w. for a background example Φ(x i , s i (w t )) = 0.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In contrast, methods such as DR, AP and the primal-dual hybrid gradient descent (PDHG) proposed in [29] used for SSL on hypergraphs [17], require performing a complete gradient descent and computing a total of R projections at each iteration.
Sentence 2: from the perspective of iteration cost, RCD is significantly more efficient, especially when R is large and computing Π(•) is costly.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In [9], Pelletier defines an estimator of a PDF on a Riemannian manifold M by using the distances measured on M via its metric tensor, and obtains the same convergence rate as in (2), with N being replaced by the dimensionality of the Riemannian manifold.
Sentence 2: if we know that the data lives on a Riemannian manifold M , the convergence rate of this estimator will be determined by the dimensionality of M , instead of the full dimensionality of the feature space on which the data may have been originally sampled.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: We use a Laplace prior distribution over the weights in order to favor sparse solutions over those which explain the data equally well but require more weights different from zero (c.f.
Sentence 2: tibshirani (1996)): prior factors have the form In our applications, we allowed the prior variance 2 ρ 2 k of the stimulus-dependent features to be different from the variance of the spike-history features.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: We cannot typically evaluate the integrals required to compute F (y) and K(y, y ).
Sentence 2: instead, at each time step we estimate these functions using a small set of sample points {x i } P i=1 which we refer to as a batch of size P . we introduce noise by sampling random data to make imperfect estimates of the gradient of the objective function.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Consequently, the number of columns with non-zero elements (nnzc(•)) for X 1 , X 2 and X 3 is very close to the total number of non-zero entries in the tensor.
Sentence 2: as predicted by theory, DFacTo (ALS, GD) does not enjoy significant speedups when compared to GigaTensor, CPALS and CPOPT.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Starting from the vector τ ∈ R k , τ j = ϕ j (X i ), in the above sampling strategy we have for a constant c k that may depend on k but is O(1) and bounded away from 0.
Sentence 2: to attain the unbiasedness condition E The full sampling and inferential scheme are as follows: (i) given a data point X i , construct the vector τ = [ϕ j (X i )] k j=1 ; (ii) sample Z i according to strategy (19) using τ and the bound B = B 0 √ k(e α + 1)/c k (e α − 1).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Empirically, a stochastic gradient algorithm seems to do fairly well at tracking the true Σ v,11 .
Sentence 2: one simple adaptive control scheme would be to take the current estimate of Σ v at each time step t, apply a standard LQG solver giving this estimate (and A, B, C, Σ w ) to it to obtain the optimal steady-state Kalman filter and control gains, and use the values obtained as our proposed gains L t and K t for time t. This gives a simple method for adapting our controller and Kalman filter parameters to the varying noise parameters.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Interestingly, we prove that the higher-level clusters are associated with longer-term diffusive transitions on the graph, amounting to smoother and more global similarity functions on the data mani-fold.
Sentence 2: the cluster hierarchy exposes the observed affinity structure gradually in different resolutions, which is somehow similar to the wavelet method that analyzes signals in different bandwidths.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: «Pj E Loo(X) and SUPj II«pjIILoo < 00.
Sentence 2: this means that by finding the eigensystem (Ai, «Pi) of Tk we can also determine the regularization operator P via [8] (8) The eigensystem (Ai, «Pi) tells us which functions are considered "simple" in terms of the operator P. in order to determine the regularization properties of dot product kernels we have to find their eigenfunctions and eigenvalues.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: These two formulations model different setups: if the system, more specifically the distribution of uncertain parameters, evolves with time, then the non-stationary model is more appropriate; while if the system is static, then the stationary model is preferable.
Sentence 2: for any given policy, the worst expected performance under the non-stationary model provides a lower bound to that of the stationary model since CS ⊆ C∞ S . one can use the non-stationary model to approximate the stationary model, when the latter is intractable (e.g., in the finite horizon case; see Nilim and El Ghaoui [5]).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The idea of using nearest neighbors for unsupervised learning has been explored in manifold learning [1,7], but unlike these approaches and more recent work on representation learning [2, 13], we do not seek to model the full data distribution.
Sentence 2: even with relatively few parameters, the transformations we learn naturally extrapolate off the training distribution and can be applied to novel types of points (e.g., new types of images).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Deep learning for autonomous driving may be one prominent example: one would like to achieve robust classification performance in the presence of confetti, despite not having had any confetti exposure during training time.
Sentence 2: from a machine learning perspective, general noise robustness can be used as a highly relevant example of lifelong machine learning [28] requiring generalisation that does not rely on the standard assumption of independent, identically distributed (i.i.d.)
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Note that MMD does not optimize an upper-bound to the negative log likelihood directly, but it guarantees the mutual information to be maximized and we can expect a high log likelihood performance [35].
Sentence 2: to translate this upper-bound, L 1 , to an implementable loss function in the model, we use the following empirical data distribution approximation: the upper bound can be approximated as: Since v y1 = f (x, v y1 ) and v y1 ∼ N (0, I), the implementable upper-bound, L, can be approximated as follows: As illustrated in Figure 2b, we train the Y → X → Y cycle starting from an image y ∈ Y.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: A difference between the neural network and biochemical models is also highlighted: global coupling of rate equations through enzyme saturation can lead to global feedback regulation, thus allowing a simple network without explicit mutual inhibition to perform the winner-take-all computation.
Sentence 2: the full complexity of the cell is not necessary for biochemical computation: a wide range of functional behaviors can be achieved with a small set of biochemical components.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Moreover, v σ T P (I) (I) does not depend on D(I, a).
Sentence 2: we can prune D(I, a) from iteration T 0 until iteration T 1 so long as In the worst case, this allows us to skip only R(I,a) U (I,a)−L(I) iterations.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: By [21, Theorem 3.1], nV b (h 2 ) and nV (h 2 , Z) have the same limiting distribution, i.e., ϕ(nV b (h 2 ), nV (h 2 , Z)) → 0 in probability under certain assumptions.
Sentence 2: it suffices to check these assumptions hold: Assumption A2.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Since we have assumed that each u * i and the state x are jointly Gaussian, we may write any u * i as an affine function of x i plus independent Gaussian noise.
Sentence 2: the minimum mean squared estimator is given by the conditional expectation we have found a closed form expression for the best regressor πi to predict u * i from only x i in the joint Gaussian case with squared error distortion.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: There is a distribution P over k-arm instances of the stochastic multi-armed bandit problem such that any fair algorithm run on P experiences constant per-round regret for at least T = Ω k 3 ln 1 δ rounds.
Sentence 2: we tightly characterize the optimal regret attainable by fair algorithms in the classic bandits setting, and formally separate it from the regret attainable by algorithms absent a fairness constraint.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: An independent line of work [22], based on Gomory-Hu trees (non-linear), established that submodular functions represent nonnegative solutions of the optimization problem (5) with β (e) = δ e − 1.
Sentence 2: an unrestricted solution of the optimization problem (5) ensures that β (e) ≤ δ e − 1.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Consider the affine sub-problem: and the convex sub-problem: We will use the Von-Neumann [5] successive projection lemma stating that P 1 P 2 P 1 P 2 ...P 1 (K) will converge onto the projection of K onto the intersection of the affine and conic subspaces described above 1 .
Sentence 2: what remains to show is that the projections P 1 and P 2 can be solved efficiently (and in closed form).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Which matrix is being factorized?
Sentence 2: a matrix entry M ij corresponds to the dot product SGNS is factorizing a matrix in which each row corresponds to a word w ∈ V W , each column corresponds to a context c ∈ V C , and each cell contains a quantity f (w, c) reflecting the strength of association between that particular word-context pair.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Moreover, inverting requires O(D 3 N 3 ) computational complexity prohibiting an efficient inference when N is large.
Sentence 2: our solution is to model the noise covariance matrix with a Kronecker structure, i.e., = ⌃ N ⌦ ⌃ D , where ⌃ N is the noise covariance across neurons and ⌃ D is the noise covariance across odors.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Each "trial" taken by a reinforcement learning algorithm shall consist of a T -step sequence in the MDP.
Sentence 2: our goal is to characterize the scaling of the sample complexity for various reinforcement learning approaches (i.e., how many trials they require in order to learn a near-optimal policy) for large numbers of agents n. in our bounds below, no serious attempt has been made to make our bounds tight in variables other than n. Below we show that if an RL algorithm uses only the global reward signal, then there exists a very simple MDP-one with horizon, T = 1, only one state/trivial dynamics, and two actions per agent-on which the learning algorithm will require Ω(n) trials to learn a good policy.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: By setting a = 1/16 and equating the two sides of the equation we have ⌘ 1 4( 1) (note that this also implies a sequence of sets B T by the definition of the functions in Lemma 1).
Sentence 2: the semi-distance satisfies =: 2s T . Applying Theorem 4 we have where the final inequality holds since M 2 and a = 1/16.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Comparison with Sample Complexity of Proposed Method: For Ising models, under uniform sampling of observed nodes, we established that the sample complexity of the proposed method scales as n = Ω(∆ 2 ρ −2 (log p) 3 ) for regular graphs with degree ∆.
Sentence 2: we nearly match the lower bound on sample complexity in (12).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: criterion is fixed across conditions in the model; the differences in error rates arise because of a mismatch between the parameters a and b used to generate trials, and the parameters â and b estimated from the trial sequence.
Sentence 2: although the criterion does not change across conditions, and the criterion is expressed in terms of accuracy (Equation 1), the block composition nonetheless affects the speed-accuracy trade off.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: However, the key difference is that the evaluation of F (X) in J 2 (X, λ) and J 3 (X, λ 1 , λ 2 ) involves not only a single sample, as in J 1 (x; λ), but also all other samples.
Sentence 2: not all the pursuit methods, such as basis pursuit and LARS/Homotopy methods, that are applicable to the standard sparse representation method can be directly applied to optimize J 2 (X, λ) and J 3 (X, λ 1 , λ 2 ).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Because the MSE between the exact KL estimator θ * KL and the true parameter θ * is O(N −1 ) as shown in Liu and Ihler (2014), the MSE between θKL and the true To make the MSE between θKL and θ * equal O(N −1 ), as what is achieved by the simple linear averaging, we need draw dn N bootstrap data points in total, which is undesirable since N is often assumed to be very large by the assumption of distributed learning setting (one exception is when the data is distributed due to privacy constraint, in which case N may be relatively small).
Sentence 2: it is a critical task to develop more accurate methods that can reduce the noise introduced by the bootstrap process.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Furthermore, the same result is true for the case of constant baselines, with A r (y) replaced by an arbitrary constant baseline A r , and A * r (y) replaced by A * r , the optimum constant baseline defined in Theorem 4.
Sentence 2: its performance depends on the random variables (∇µ u (y)/µ u (y)) 2 and J β (j); if they are nearly independent, E [J β ] is a good choice.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Recently, several algorithms have been proposed for maximizing the AUC value locally [4] or maximizing some approximations of the global AUC value [9,15], but, in general, these algorithms do not obtain AUC values significantly better than those obtained by an algorithm designed to minimize the error rates.
Sentence 2: it is important to determine the relationship between the AUC values and the error rate.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The problem of submodular minimization cannot be approximated with a 1/2 o(1) additive approximation given poly(n) samples from any distribution D. Proof.
Sentence 2: for any set S ✓ N , observe that the number g + b of elements in S that are either good or bad is the same for any two functions f i , f j 2 F M and for any F M . by Lemma 4, we obtain the indistinguishability condition.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: When ι is equal to 1/t, this FIM Fs,a (θ, ι) is equivalent to the FIM of the t time-steps joint distribution from the stationary state distribution d θ (s) on M(θ) [12].
Sentence 2: this interpolation controlled by ι can be interpreted as a continuous interpolation with respect to the time-steps of the joint distribution, so that ι : 1 → 0 is inversely proportional to t : 1 → ∞.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The random matrix W = W * in S n×n is a Gaussian Wigner matrix with variance σ 2 W if its entries on and above the diagonal are independent, zero-mean Gaussian variables (real or complex depending on context) with variance σ 2 W . Besides Assumption 1, another important assumption for our results is that the search space C (1) of (SDP) is compact.
Sentence 2: in that scenario, there exists a finite constant R such that for all where G † is a continuous function of Y as in Lemma 2.2.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: While the optima of both bounds vanish with n → ∞, their behavior is qualitatively different.
Sentence 2: in particular, the bound of [8] is U-shaped, and diverges for k = n, while ours is L-shaped (no tradeoff), and thus also convergent for k = n. when compared with [8], our results suggest that no regularization is required from a statistical point of view though, as clarified in the following remark, it may be needed for purposes of numerical stability.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Recent research also reveals that deep features eventually transition from general to specific along the network, and feature transferability drops significantly in higher layers with increasing task dissimilarity [28], hence the sharing of all feature layers may be risky to negativetransfer.
Sentence 2: it remains an open problem how to exploit the task relationship across different deep networks while improving the feature transferability in task-specific layers of the deep networks.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In essence, we build on the fact that vector quantizers project data into a finite set of N elements, the index space, and on the simple observation that kernels on finite sets are fully specified by the N×N Gram matrix of these elements (the kernel matrix), which we propose to learn directly.
Sentence 2: qKs are piecewise constant but otherwise arbitrary, making them very flexible.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Olshausen & Field [11,12] try to find bases that result in a code s that preserves information about the original image x and that is sparse.
Sentence 2: they minimise the following cost function with respect to A, where A, denotes a constant determining the importance of sparseness [11] : The algorithm proposed in this paper is similar, but it takes into consideration the temporal order of images.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Table 1: Summary of algorithms for Proj R n ↓ (y) (x) It can be easily verified that gradient of 1 2 P Ω (X) − z 2 2 is 2-Lipschitz continuous.
Sentence 2: from standard results on convegence proximal gradient descent [30], Algorithm 1,converges to within an error in objective in O(1/ ) iterations.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Moreover, their gradients rf (w, z) are Lipschitz continuous with constant M krf (w, z) rf (w 0 , z)k  M kw w 0 k, for all z.
Sentence 2: the conditions in Assumption 1 imply that the average loss L(w) and the empirical loss L n (w) are convex and their gradients are Lipschitz continuous with constant M . the empirical risk R n (w) is strongly convex with constant cV n and its gradients rR n (w) are Lipschitz continuous with parameter M + cV n . So far we have concluded that each subproblem should be solved up to its statistical accuracy.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: This leads to our first relaxation of the generalized eigenvalue bound for a model.
Sentence 2: from Lemma 3.2 and Proposition 4.2 we see that this result suggests that to approximate the graphical model (Θ, Φ) we can search for a parameter matrix B ⋆ , with corresponding simple graph G(B ⋆ ) ∈ S, such that While this relaxation may lead to effective bounds, we will now go further, to derive an additional relaxation that relates our generalized graphical model support number to the "classical" support number.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Meanwhile, the space Ω 1 is formulated under the RKHS framework as a set of columns of the kernel matrix (denoted K 1 ).
Sentence 2: we can write f = 1 T d + K 1 e, and so our formulation becomes: The solution to this optimization problem is for our coefficients d and e to be linear estimates on y, our vector of observed values.
Relationship: reasoning