sadat2307/MSciNLI huggingface dataset:
Domain: NeurIPS
Sentence 1: More precisely, let x p be the mean of X ∼ µ p and let us look at the limit of V µp (f (X)) when v p goes to 0.
Sentence 2: assuming that f is differentiable, we get if we introduce Σ p to be the covariance matrix of the random variable X ∼ µ p , then we simply have lim Example with hyper-cubic regions An important example is when R p is a hypercube with side length v 1/d p and µ p is the uniform distribution over the region R p . In that case (see Lemma 1), we ), then it is not too difficult to show that for such hyper-cubic regions, we have On the other hand, by direct computation, the second term is such that � stays bounded away from 0 (unless ν is deterministic).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: By the Paley-Zygmund inequality, every time the algorithm takes Poi(m) samples, there is at least a c r > 0 probability of seeing an r-wise collision.
Sentence 2: if we consider our algorithm to take samples in blocks of size Poi(m), the probability that we have not found an r-wise collision after t blocks is at most (1 − c r ) t . Thus, the expected number of blocks until we have an r-wise collision is O r (1).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In fact, FLD is arguably a more principled whitened variant of the Prot classifier: Its weight vector can be written as w = S −1 W ( µ + − µ − ), where S −1 W is the within class covariance matrix of the two classes, and µ ± are the class means.
Sentence 2: if we disregard the constant offset b, we can write the decision function as x , which is a prototype classifier using the prototypes µ ± after whitening the space with S −1/2 W . We combine the linear preprocessor (PCA) X = EB and the linear classifier (SVM, RVM, Prot, FLD) y( x) = w| x + b to yield a linear classification system: y = w T E T + b where b = b 1.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: If the next hypothesis p t+1 of a ROME algorithm is on H t , then by Lemma 2, it and all other members of H t satisfy D(p t+1 ||u) = D(p t+1 ||p t ) + D(p t ||u).
Sentence 2: in this case, p t+1 also minimizes D(q||p t ) from among the members q of H t ∩ J t . Lemma 11 implies that p t+1,i /p t,i is the same for all i with x i = 1, and the same for all i with x i = 0.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The following functional with respect to r is called the free energy: In the last equation, the first term is the Kullback-Leibler (KL) divergence from the trial distribution to the Bayes posterior, and the second term is a constant.
Sentence 2: minimizing the free energy (4) amounts to finding a distribution closest to the Bayes posterior in the sense of the KL divergence.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The relative efficiency of the block 1,∞ program (6) compared to the Lasso is given by R(θ 1,∞ , θ Las ) = 4−3α .
Sentence 2: for sublinear sparsity s/p → 0, the block scheme has greater statistical efficiency for all overlaps α ∈ (2/3, 1], but lower statistical efficiency for overlaps α ∈ [0, 2/3).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: We next focus on the properties of an optimal s. Suppose that two classes are generated from (3) with cross-spectral densities of A 0 and A 1 for classes 0 and 1, respectively.
Sentence 2: the signals are drawn from CN (0, A y ⌦ (⌧ )) for y = {0, 1}.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: We will have to take the dependence between different samples x (j) i and x (k) i into account, but between variable dependencies within a sample do not pose problems.
Sentence 2: estimate the confidence of any decision variable z i is straight forward from Equation (5), applied separately to the binary sample set Note that all quantities defined above for the single variable case need to be computed separately for each decision.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: To address this problem, we introduce the standard supervised loss (i.e., cross-entropy loss) to C, R L = E (x,y)∼p(x,y) [− log p c (y|x)], which is equivalent to the KL-divergence between p c (x, y) and p(x, y).
Sentence 2: we define the game as: It will be proven that the game with utilities Ũ has the unique global optimum for C and G. • Update D by ascending along its stochastic gradient: • Compute the unbiased estimators RL and RP of R L and R P respectively.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Because the similarity structure can be estimated from imaging data even without building an encoding model, RSA allows not only for model testing (by comparing the similarity matrix of neural data with the similarity matrix of the feature vectors when stimuli are represented with an encoding model) but also for exploratory study (e.g., by projecting the similarity structure to a low-dimensional space to visualize its structure, [11]).
Sentence 2: originally as a tool for studying visual representations [2,16,10], RSA has recently attracted neuroscientists to explore the neural representational structure in many higher level cognitive areas [23,18].
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Let π k be the policy of UBEV in the kth episode.
Sentence 2: then with probability at least 1 − δ for all ε > 0 jointly the number of episodes k where the expected return from the start state is not ε-optimal (that is with probability at least 1 − δ UBEV converges to optimal policies and for all episodes T has regret Here polylog(x . . . ) is a function that can be bounded by a polynomial of logarithm, that is, ∃k, C : polylog(x . . . ) ≤ ln(x . . . ) k +C.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The saddle point equations with respect to probability distributions provide a set of relations between π(x) and π(x): By using the result obtained for the free energy, we can easily perform further straightforward calculations to find all the other observable thermodynamical quantities, including internal energy: which records reproduction errors.
Sentence 2: in terms of the considered replica symmetric ansatz, a complete solution of the problem seems to be easily obtainable; unfortunately, it is not.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The key trick is that after symmetrization, we employ | max{0, z} − max{0, z }| ≤ |z − z |, making three differences of partial risks point-wise (see (18) in the proof).
Sentence 2: we have to use a different Rademacher complexity with the absolute value inside the supremum [35,36], whose contraction makes the coefficients of ( 15) doubled compared with Lemma 8 of [19]; moreover, we have to assume G is closed under negation to change back to the standard Rademacher complexity without the absolute value [28].
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The training procedure for the new task is as follows, keeping W a t−1 fixed and only back-propagating the newly added parameters of W t \W a t−1 .
Sentence 2: the optimization formula for the new task is, min We use stochastic gradient descent to learn the newly added filters with η as the learning rate, The expanded child network will be trained until the required number of epochs or convergence are reached.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In the pseudocode, P W refers to the projection on to the ball W, which amounts to rescaling w to have norm at most D. The accelerated gradient method (e.g., [5]) is summarized as Algorithm 2.
Sentence 2: in terms of existing results, for the SGD algorithm we have [4, Section 5.1] , whereas for an accelerated gradient algorithm, we have [5] as long as b = o( √ m), both methods allow us to use a large mini-batch size b without significantly degrading the performance of either method.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: . . , P i n are independent of each other, using the above lower bound for the expectation of their sum and Chernoff bounds we get Pr[P i 1 + . . . .
Sentence 2: for a sufficiently large constant c(γ), the probability that the vertex u i is uninfected after two rounds Pr[P 1 + .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Our motivation is the the following: if θ and ψ define the same distribution, then if a certain set of edges are removed from both, they should continue to define the same distribution 1 .
Sentence 2: given some graph T , we define the "projection" θ(T ) onto the tree such by setting all edge parameters to zero if not part of T . Then, given a set of graphs T , the piecewise KL-divergence is Computing the derivative of this divergence is not hard-one simply computes the KL-divergence for each graph, and uses the gradient as in Eq.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Notice that, using (1), we can write each y i as where ȳi is the i-th data point whose missing entries are filled with zeros and x i is the vector containing missing entries of y i .
Sentence 2: substituting (4) in the optimization (3), we would like to solve Notice that matrices ȳi are given and known while vectors In fact, the optimization (5) has two sources of non-convexity: the 0 -norm in the objective function and the product of unknown variables {c ij } and {x i } in the constraint.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In either cases, the functions −p log z + z Given that the function values are non-increasing along the iterates of Algorithms 1, 2 and 3, the sequence of Ω (k) satisfy Ω (k) ∈ C 0 for k = 1, 2, .....
Sentence 2: the lower bounds on the diagonal elements of Ω (k) provides the Lipschitz continuity using (17) using the mean-value theorem, the gradient ∇h 1 satisfies ∇h with the Lipschitz continuity constant L = a −2 + S 2 .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Reminding is normally triggered by contextual features present at the moment of retrieval which match distinctive features of the memory being recalled (Berntsen et al., 2013;Wharton et al., 1996), and can occur more often following unexpected events (Read & Ian, 1991).
Sentence 2: an individual's current state of understanding can trigger reminding of a past state.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The extra dependency in log(T ) could be reduced to log(K) or even optimally to 1 by using confidence intervals more carefully tailored, for instance by replacing the log(t) term appearing in the definition of the estimated gradients by log(T /T i (t)) or log(T /KT i (t)) if the horizon T is known in advance as in the algorithms MOSS or ETC (see [4,29,30]), but at the cost of a more involved analysis.
Sentence 2: multi-armed bandits provide a lower bound for the approximation error E[L(p T )] − L(p ) of order K/T for smooth convex functions.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: These leverage scores characterize the importance of the corresponding observations and are key to efficient subsampling with optimal approximation guarantees.
Sentence 2: leverage scores emerged as a fundamental tool for matrix sketching and column sampling [22,21,13,36], and play an important role in low rank matrix approximation [11,6], regression [2,28,20], random feature learning [29] and quadrature [7].
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Thus, we only need to store X ∈ R k . When a stream element a t = (i t , I t ) arrives, one updates the entries of X: ( By property of stable distributions, the samples x j , j = 1 to k, are also i.i.d.
Sentence 2: stable the task boils down to estimating the scale parameter from k i.i.d.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Then MEM is equivalent to maximize ĥ,û l 1 ⊤ĥ subject to X l ûl + Y l µ + W l x l ≥ ĥ, βl ≥ ĥ, Z l ûl ≤ z l  All the three tasks involve convex (and linear) objective function with linear constraints which impose a convex feasible set.
Sentence 2: one can use the rich and well-developed literature on convex optimization and linear programming to find the optimum intervention.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Due to the evident reconstructive properties [9,18], the available efficient pursuit methods and the sparsity of representation, we choose the sparse representation as the basic framework for the SRSC and incorporate a measure of discrimination power into the objective function.
Sentence 2: the sparse representation obtained by the proposed SRSC contains both crucial information for reconstruction and discriminative information for classification, which enable a reasonable classification performance in the case of corrupted signals.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: By studying simplified versions of the problem, our analysis suggests how precise formulations of assumptions M and SSS can be made and exploited to lead to improved estimators.
Sentence 2: the data are For convenience, assume that data are labeled so that R i = 1 for i = 1, . . . , n and R i = 0 for i = n + 1, . . . , N . the labeled sample size is n, and the unlabeled sample size is u = N − n. Let p(x) be the density of X and let m(x) = E(Y | X = x) denote the regression function.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: However, evaluating the posterior mean from equation 7 with x * ∈ X, results in an analytically unsolvable differential equation: f is a complicated function of ∆ f , its own derivative.
Sentence 2: we define a two-step approach: first we evaluate a standard GP with the training data, using our initial hyperparameter settings and ignoring the input noise.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Every time we update τ (start of an epoch), we decrease it by a factor of 1 − (except maybe the last round for which τ = 1).
Sentence 2: there are at most 1 + log 1 1− (M ) ≤ 1 + log(M ) log(1/(1− )) ≤ 1 + log(M ) epochs.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: One can show that, for x ∈ R D at which p is continuous, if k → ∞ and k/n → 0 as n → ∞, then pk (x) → p(x) in probability ( [28], Theorem 3.1).
Sentence 2: a natural approach for estimating F (P ) is the plug-in estimator Since pk → p in probability pointwise as k, n → ∞ and f is smooth, one can show FP I is consistent, and in fact derive finite sample convergence rates (depending on how k → ∞).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The Hessian regularizer, the squared norm of the second covariant derivative, ∇ a ∇ b f 2 , corresponds to the Frobenius norm of the Hessian of f in normal coordinates, see Equation 1.
Sentence 2: given normal coordinates x r at X i we would like to have an operator H which given the function values f (X j ) on N k (X i ) estimates the Hessian of f at X i , This can be done by fitting a second-order polynomial p(x) in normal coordinates to {f (X j )} k j=1 , where the zeroth-order term is fixed at f (X i ).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: However, this equality implies and so CUR could be regarded as implicitly performing sparse PCA in the sense that (a) V CUR is sparse; and (b) by Theorem 1 (with high probability), ERR(V CUR ) ≤ (1 + ǫ)ERR(V k ).
Sentence 2: the existence of such a V CUR would cast CUR directly as a randomized approximation algorithm for SPCA.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Moreover, if b = 1, then Acc-Prox-SVRG has the same complexity as that of Prox-SVRG, while if b = n then the complexity of this method is equal to that of APG.
Sentence 2: with an appropriate mini-batch size, Acc-Prox-SVRG may outperform both Prox-SVRG and APG; even if the mini-batch is not appropriate, then Acc-Prox-SVRG is still comparable to Prox-SVRG or APG.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: 20 repetitions of this experiment yielded after 42 minutes of training the following results: spike correlation 0.88 ± 0.036, angular error 27.9 ± 3.7 degrees, for U + = 0.0012, U − /U + = 1.055.
Sentence 2: apparently the output spike train is less sensitive to changes in the values of U than to changes in w. since only the behavior of a neuron with vector U * but not the vector U * is made available to the neuron during training, the resulting correlation between target-and actual output spike trains is quite high, whereas angular error between U * and U(t), as well as the average deviation in U , remain rather large.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: . . , (x m , x 0 m ) 2 (X ⇥X ) m , we will use instead an empirical estimate b q h of q h : b .
Sentence 2: we define the empirical error of h for a labeled sample as follows: We first assume that X is a subset of R N for some N 1 and consider a hypothesis set H of linear functions h : x 7 !
Relationship: reasoning
Domain: NeurIPS
Sentence 1: If no such α exists, then we choose α(M T ) uniformly at random from V. From Lemma 1, there can exist only one such α ∈ V that satisfies this inequality.
Sentence 2: using Markov's inequality, we have P φ [ α(M T ) = α * ] ≤ P φ * (M T , g α * , S, φ) ≥ ψ(δ)/3 ≤ 1 3 .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: . . , x m ), the following upper bounds hold for the empirical factor graph complexity of H 1 and H where s is a sparsity factor defined by s = max Plugging in these factor graph complexity upper bounds into Theorem 1 immediately yields explicit data-dependent structured prediction learning guarantees for linear hypotheses with general loss functions and arbitrary factor graphs (see Corollary 10).
Sentence 2: observe that, in the worst case, the sparsity factor can be bounded as follows: where the factor graph Rademacher complexities of linear hypotheses in ).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Theorem: Out of all k-dimensional approximations of the form for the probability distribution at time t, the one that minimizes the mean squared error E x { p(t, y|x) − p(t, y|x) 2 w } where averaging over initial points x is with respect to the stationary density φ 0 (x), is given by w j (y) = φ j (y) and a j (t, x) = λ t j ψ j (x).
Sentence 2: the optimal k-dimensional approximation is given by the truncated sum Proof: The proof is a consequence of a weighted principal component analysis applied to the matrix M , taking into account the biorthogonality of the left and right eigenvectors.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Evidence also suggests that feedback signals may operate between hierarchically adjacent areas along the ventral stream [30,5,43] to enable local recurrent processing for object recognition [65,2], especially given ambiguous or degraded visual input [64,51].
Sentence 2: feedback processes may be an integral part of both global and local recurrent processing underlying top-down attention in a slower time scale and visual recognition in a faster time scale.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Proof Assume that θ * is a fixed point of the above algorithm.
Sentence 2: then the first order optimality conditions require ∂ θ log pi (θ * ) = 0 for all i and the Laplace approximation yields up to second order, the derivatives of p, pi , and p agree at θ * , which implies that θ * is a local optimum.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In contrast, although Double Q-learning can underestimate the values of some actions, it is easy to set the initial action values high enough to ensure optimism for actions that have experienced limited updates.
Sentence 2: the use of the technique of optimism in the face of uncertainty can be thought of as an orthogonal concept to the over-and underestimation that is the topic of this paper.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In a learning context, where each function f i is the loss associated to a single data point, L is equal to the largest value of the loss second derivative ξ (1 for the square loss, 1/4 for the logistic loss) times R 2 , where R is a the uniform bound on the norm of each data point.
Sentence 2: the constraint µ L 8 n is satisfied when λ 8ξR n . In low-dimensional settings, the optimal regularization parameter is of the form C/n [29] where C is a scalar constant, and may thus violate the constraint.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: . . , f n ) represent the prediction for the labeled and the unlabeled examples, respectively.
Sentence 2: according to the inverse lemma of the block matrix, we can write K −1 as follows: where the term f K −1 f is computed as When the unlabeled data are loosely correlated to the labeled data, namely when most of the elements within K u,l are small, this leads to M u ≈ K u .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In fact, for Lpwq " 1 m ∞ m k"1 L k pwq where L k pwq " `pf w px k q, y k q, for any single weight W phq ri, js in the network we can compute d dt pW phq ri, jsq 2 " 2W phq ri, js¨d W phq ri,js dt BW phq ri,js , using the sharp chain rule of differential inclusions for tame functions [Drusvyatskiy et al., 2015, Davis et al., 2018.
Sentence 2: if we can prove the theorem for every individual loss L k , we can prove the theorem for L by taking average over k P rms.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Section 3 investigates the close relationship with Rademacher complexities, Section 4 develops the maximal average margin principle for ordinal regression, and Section 5 reports experimental results of application of the MAM to classification and ordinal regression tasks.
Sentence 2: let the class of hypotheses be defined as the signed distance of a sample (X, Y ) to the hyper-plane w T x = 0, or the margin M (w) ∈ R, can be defined as SVMs maximize the worst-case margin.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: For simplicity, we focus on linear features, that is, h i (x) = u i , x , where u i ∈ IR d . In addition, we assume that the vectors u i are orthonormal.
Sentence 2: if U denotes the d × d matrix with columns the vectors u i , then U ∈ O d . The functions f t are linear as well, that is f t (x) = w t , x , where w t = i a it u i . Extensions to nonlinear functions may be done, for example, by using kernels along the lines in [8,15].
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Let θ i be the P × |o t,i | matrix of parameters for intersection i.
Sentence 2: we additionally define U(p) as the unit vector with a 1 in row p. assuming exp(x t,i ) is element-wise exponentiation , Pr(a t,i = p|o t,i , θ i ) = ãt,i (p); We implemented two baseline controllers for comparison: (1) a uniform controller giving equal length to all phases; (2) A SCATS inspired adaptive controller called SAT that tries to achieve a saturation of 90% (thought to be used by SCATS) for all phases.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: That is, given the matrices A and C, there are two distinct vectors θ 1 and θ 2 in ∂P such that A T θ 1 = A T θ 2 .
Sentence 2: if eventually θ 1 is the true parameter vector satisfying C 1 and C 3 , then θ 2 satisfies these conditions as well.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In the case of PubMed, our algorithm only needs to work on covariance matrices of order at most n = 1000, instead of the full order (n = 141, 043) covariance matrix.
Sentence 2: at values of the penalty parameter λ that target cardinality of 5 commands, we observe a dramatic reduction in problem sizes, about 150 ∼ 200 times smaller than the original sizes respectively.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Define the kernel power K with k i,j = g p i,j .
Sentence 2: the off-diagonal kernel entries k i,j i 6 = j monotonically decrease with respect to increasing p. 9 p ⇤ such that Corollary 3 is satisfied for Beyond the examples outlined here, similar conditions can be enumerated for a wide range of parametrized kernel functions, and are easily checked for model-based kernels e.g.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Then where c 0 is an absolute constant and Remark 2 If ↵, ✓ 0 , ✓ 1 are unknown, any estimate-then-explore strategy (or the strategy described in Corollary 1) would be unable to choose an m that depended on these parameters, so we can treat it as a constant.
Sentence 2: for the case when ✓ 0 and ✓ 1 are bounded away from {0, 1} (e.g.
Relationship: reasoning