sadat2307/MSciNLI huggingface dataset:
Domain: NeurIPS
Sentence 1: The importance measures Z j and Zj are determined by two factors: (1) the relative importance between X j and its knockoff counterpart Xj , encoded by filter weights z = (z 1 , • • • , z p ) T and z = (z 1 , • • • , zp ) T , respectively; and (2) the relative importance of the jth feature among all p features, encoded by the weight matrices, w = W (0) (W (1) W (2) W (3) ), where denotes the entry-wise matrix multiplication.
Sentence 2: we define Z j and Zj as With the above introduced importance measures Z j and Zj , the knockoff statistic can be defined as W j = Z 2 j − Z2 j , and the filtering step can be applied to the W j 's to select features.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In order to construct robust loss functions, consider the following two transformation functions: which, in turn, can be used to define the following loss functions: One can see that σ 1 (t) → ∞ as t → −∞, but at a much slower rate than σ 0 (t) does; its derivative σ 1 (t) → 0 as t → −∞.
Sentence 2: σ 1 (•) does not grow as rapidly as σ 0 (t) on hard-to-classify data points.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Under smoothness and strong convexity assumptions, the authors obtain convergence guarantees of essentially O(1/k 2 ) after k epochs, vs. O(1/k) using with-replacement sampling (with the O(•) notation including certain dependencies on the problem parameters and data size).
Sentence 2: withoutreplacement sampling is shown to be strictly better than with-replacement sampling, after sufficiently many passes over the data.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Also, after passing x i and x j through T decision trees of the learned RF, suppose they reached leaves l t i and l t j in each tree t = 1, . . . , T . Then, we compute To estimate the ratio of the proposal distributions, q(B→A) q(A→B) , it is necessary to compute each individual probability p ij , since the nominator and denominator of q(B→A) q(A→B) do not contain the same set of "cut" edges, Cut A = Cut B , as specified in (3).
Sentence 2: we compute In the following, we first present our empirical evaluation of (RF) 2 , and then derive the theoretical performance bounds of a simple, two-class (RF) 2 .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In other words, if communication is an expensive resource, then the solution proposed in [13] only works well when λ is at least order of 1/ √ m. In machine learning applications, the strong convexity parameter λ often comes from explicit regularization designed to prevent over-fitting, and needs to scale with the data size, usually between 1/ √ m and 1/m.
Sentence 2: the solution above is communication-efficient only when λ is relatively large.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Canonical correlations ρ 2 are the solutions to the eigenproblem: where C xy = cov(X, Y ) and the matrices C xx and C yy are assumed to be invertible.
Sentence 2: the largest canonical correlation ρ 1 between X and Y is the supremum of the correlation coefficients over their linear projections, that is: When p, q n, the cost of CCA is dominated by the estimation of the matrices C xx , C yy and C xy , hence being O((p + q) 2 n) for two random variables of dimensions p and q, respectively.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The MESA distance is robust to the additive local perturbations of its arguments such as independent noise or high-frequency signal as long as the integrals (counts) of these perturbations over larger region are close to zero.
Sentence 2: it does not matter much how exactly we define the ground truth density locally, as long as the integrals of the ground truth density over the larger regions reflect the counts correctly.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Specifically, if a base class admits compression, then the class of mixtures of that base class, as well as the class of products of the base class, are compressible (Lemmas 3.6 and 3.7).
Sentence 2: it suffices to provide a compression scheme for the class of single Gaussian distributions in order to obtain a compression scheme for the class of mixtures of Gaussians (and therefore, to be able to bound their sample complexity).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: , where S(µ, ϵ) is as in Lemma 2.6 for ϵ = δ/9mnk.
Sentence 2: let m ≤ n. Then, by Lemma 2.5 and union bound, for any Y ∈ S ′ (µ, ϵ), where C 1 ≥ 0 is a constant independent of m, n, k. if p > Cµ 2 k 2 log n/δ 2 m, where C = 16(C 1 + 1), with probability at least 1 − exp(−n log n), the following holds ) As the statement of the theorem is invariant under scaling, it is enough to show the statement for all µ-incoherent matrices X of rank at most k and ∥X∥ 2 = 1.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Results (displayed in Figure 3b) showed that participants again chose compositional extrapolations more frequently (68% vs. 32%, χ 2 = 172.8, p < 0.01), even if the ground truth happened to be generated by a spectral mixture kernel.
Sentence 2: people seem to prefer compositional over non-compositional extrapolations in forced choice extrapolation tasks.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Its extreme points, however, are easy to find and characterize using the greedy algorithm [7].
Sentence 2: we define a subclass of d with H f chosen so that it picks an extreme points of ∂ f (Y ), which we will call the permutation based lower bound submodular Bregman, henceforth referred to with d Σ f . The extreme points of ∂ f (Y ) can be obtained via a greedy algorithm ( [7,9]) as follows: Let σ be a permutation of V and define S i = {σ(1), σ(2), .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: [7] typed their responses, this accounts for it taking longer for participants to type longer than shorter words.
Sentence 2: according to the random walk models, the inter-item retrieval time (IRT) between animal k and k − 1 is where τ (k) is the first hitting time of animal X τ (k) and L(X) is the length of word X.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Suppose X S ⊆ X is a subset of X with size m, sampled uniformly at random without replacement.
Sentence 2: let Ĉ = { Ŝ1 , • • • , Ŝ2 } be an (1 + )-approximation of optimal k-means subspace clustering computed on X S . If m = Ω( ) with γ < 1−802φ 2 800φ 2 − 2(1 + ), then we have: where applying Theorem 3.4 together with the sample-aggregate framework we obtain a weak polynomial-time ε-differentially private algorithm for agnostic k-means subspace clustering, with additional amount of per-coordinate Gaussian noise upper bounded by O( Our bound is comparable to the one obtained in [22] for private k-means clustering, except for the (ψ − η) term which characterizes the well-separatedness under the subspace clustering scenario.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: 3, both k m (z, z ) = m(z) m(z ) and k δ (z, z ) = δ(z) δ(z ) are the inner product of two vectors and thus are positive definite kernels.
Sentence 2: k h (P, Q) is a match kernel over sets (here the sets are image patches) as in [8,1,11,17,7].
Relationship: reasoning
Domain: NeurIPS
Sentence 1: However, does the sequence of delta distributions I=n converges to H ?
Sentence 2: most people would answer yes based on the intuition that I=n 3 H and so does the sequence of corresponding delta distributions, even though the support of I=n never has any intersection with the support of H . convergence can be defined for distributions not only in a point-wise way, but in a way that takes consideration of the underlying structure of the sample space.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Since we need to make some assumptions about q s , the equality in (6) usually does not hold.
Sentence 2: the tightest lower bound is an approximation of the intractable discriminant function Then, the margin is ∆F d (s; Θ) = θ ⊤ ∆f d (s) + max qs d L −θ (q sd ) − max qs L −θ (q s ), of which the linear term is the same as that in a linear SVM [7] and the difference between two variational bounds causes the topic discovery to bias the learning of the scene classification model, as we shall see.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Each term in the chain rule of probability can be conditioned on side-information y, decomposing any conditional density as p(x | y) = i p(x i | x 1:i−1 , y).
Sentence 2: we can turn any unconditional autoregressive model into a conditional one by augmenting its set of input variables with y and only modelling the conditionals that correspond to x.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: An intuitive way of representing an observability model is through a directed graph over actions: an arc 1 from action i to action j implies that when playing action i we get information also about the loss of action j.
Sentence 2: the expert setting is obtained by choosing a complete graph over actions (playing any action reveals all losses), and the bandit setting is obtained by choosing an empty edge set (playing an action only reveals the loss of that action).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Our theory allows for possible nonconvexity in both the loss function L n and the regularizer ρ λ .
Sentence 2: due to this potential nonconvexity, our M -estimator also includes a side constraint g : R p → R + , which we require to be a convex function satisfying the lower bound g(β) ≥ β 1 , for all β ∈ R p . any feasible point for the optimization problem (1) satisfies the constraint β 1 ≤ R, and as long as the empirical loss and regularizer are continuous, the Weierstrass extreme value theorem guarantees that a global minimum β exists.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: However, there are serious scaling issues for these models when the number of timesteps is large, or even unbounded, such as in online reinforcement learning (e.g., in the real world).
Sentence 2: some decisions need to be made regarding the size of the past-embedding buffer that should be stored, whether it should be a rolling window, how computations should be cached and propagated across time, etc.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Let Y = Diag([y i ] m i=1 ) be a diagonal matrix.
Sentence 2: using the Cauchy-Binet identity we have when Y = I S , the (diagonal) indicator matrix for S, we obtain AY A > = A S A > S . Consequently, in the summation above only terms with T ✓ S survive, yielding We are now ready to sketch the proof of Theorem 6.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: One might hope to minimize (5a) and (5b) directly, perhaps using sampled states and next states, but this is not a straightforward optimization problem to solve in general, because the search space for Φ is the space of functions and because Φ appears inconveniently on both sides of 5(b) making it difficult rearrange terms to solve for Φ as an optimization problem with a fixed target.
Sentence 2: without additional assumptions about how the states are initially encoded and what space of features will be searched, it is challenging to apply Theorem 3 directly.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Although the spike generation mechanism operates in continuous time, the high temporal precision of the spikegeneration mechanism may be viewed as a manifestation of oversampling, which is needed for the digital encoding of the analog signal.
Sentence 2: the extra order of magnitude in temporal precision available to the spike-generation mechanism relative to somatic current, Figure 1B, is necessary to faithfully encode the amplitude of the analog signal, thus potentially reconciling the firing rate and the spike timing points of view [13][14][15].
Relationship: reasoning
Domain: NeurIPS
Sentence 1: This implies that when given the loss vector (1) for all the steps till the end of this block, A never plays arm 2 in the block with probability more than 1/2.
Sentence 2: if we make the switch to the loss vector (2) = (1/2, 0) at the beginning of the block, then A with probability more than 1/2 still never plays arm 2 and never notices the switch in this block.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: To establish the equality between ( 14) and ( 15), since ˜ embeds the submatrix kI, for any ⇤ 2 R h⇥ t + there must exist an ⌦ 2 R t⇥ t + satisfying ⇤ = ˜ ⌦/k.
Sentence 2: furthermore, these matrices satisfy the result (8) holds for the first layer loss (4), using L1 defined in Lemma 3.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Inserting this into 19 we find after some algebra δC = g log 1 + e ag − 1 Because all c i < 0, the first term must be smaller (more negative) than the corresponding term in the linear case implying that δC < 0 (the second term is unchanged w.r.t the linear case).
Sentence 2: if the entire spectrum is log-convex the gap will be located on the right, resulting in PCs.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: First, if d is large, the parameters of the R j are decoupled and the RESCAL model is retrieved (up to a change of loss function).
Sentence 2: second, our model is also related to classical tensor factorization model such as PARAFAC which approximate the tensor [R k (S i , O j )] i,j,k in the least-square sense by a low rank tensor H of the form The parameterization of all R j as linear combinations of d rank one matrices is in fact equivalent to constraining the tensor R = {R j } j∈ 1;nr to be the low rank tensor R = d r=1 α r ⊗ u r ⊗ v r . the tensor of all trigram terms 4 can be written also as d r=1 α r ⊗ β r ⊗ γ r with β r = S u r and γ r = O v r . This shows that our model is a particular form of tensor factorization which reduces to PARAFAC (up to a change of loss function) when p is sufficiently large.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: However, by choosing an encoder distribution q(z|x) with sufficient capacity, we can assume that the non-negative KL-divergence D KL [q(z|x), p(z|x)] is small.
Sentence 2: we know that the right-hand-side is a lower bound on the log-likelihood log p(x), which can be maximized w.r.t.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Hope comes from two directions: first, there are many cue orders besides the exact validity ordering that can yield good performance; and second, research in computer science has demonstrated the efficacy of a range of simple ordering rules for a closely related search problem.
Sentence 2: we find that simple mechanisms at the cue-order-learning stage can enable simple mechanisms at the decision stage, such as lexicographic one-reason decision heuristics, to perform well.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The KL-divergence between those Gaussians scales with kA(S n ) E[A(S n )]k 2 .
Sentence 2: more precisely: bounding kA(S n ) E[A(S n )]k will give (via the PAC-Bayes bound of Theorem 4 above) a corresponding bound on the binary divergence between the average empirical risk R(Q, P n ) and the average theoretical risk R(Q, P ) of the randomized classifier Q. Hypothesis stability (in the form of our Definition 1) implies a concentration inequality for kA(S n ) E[A(S n )]k. This is done in our Corollary 8 (see Section 4.3 below) and completes the circle of ideas to prove our main theorem.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Let Assumption 1 hold, and choose a sufficiently small η > 0 such that m : Then Breg-SVRG enjoys linear convergence in expectation: For example, we may set 45 , and Z = O(L 2 ).
Sentence 2: between epochs, the gap (z s ) decays (in expectation) by a factor of 45 64 , and each epoch needs to conduct the proximal update (21) for m = Θ(L 2 ) number of times.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: We then have: Since w is very large, the AA T block is relatively very small, and so MM T is nearly rank-2kit has a 'heavy' strip of elements in its last k rows and columns.
Sentence 2: computing a relative-error rank-2k approximation to MM T recovers all entries except those in the AA T block very accurately, and importantly, recovers the wAC block and so the product AC.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: To sample from R t , we sample U, V from the boosted posterior (U, V) t and then sample from N (U > V, I).
Sentence 2: r t ⇠ N (U t > V t , I) where (U, V) t ⇠ P t i ↵ i s i (U, V) and s i (U, V) is the t-th iterate returned by the LMO.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In fact, Kingman's paintbox theorem [12,13] implies that any exchangeable partition of N, such as a partition obtained using equation 1, is either equal to the trivial partition in which each part contains one element or satisfies lim inf N →∞ M N / N > 0 with positive probability.
Sentence 2: by Kolmogorov's extension theorem, a sequence of random partitions (C N : N = 1, 2, . . .) corresponds to an exchangeable random partition of N whenever (a) each C N is finitely exchangeable (i.e., its probability is invariant under permutations of {1, . . . , N }) and (b) the sequence is projective (also known as consistent in distribution)-i.e., if N < N , the distribution over C N coincides with the marginal distribution over partitions of [N ] induced by the distribution over C N . to obtain a nontrivial model that exhibits the microclustering property, we must sacrifice either (a) or (b).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Then, for sufficiently large K (specifically K ≥ c 1 n 1/(2s+1) for some constant c 1 ), with probability at least 1 − 2 exp(−t 2 /2), we have where the constant c 1 depends on R and the distribution of ε i , and the constant C depends on R. The above theorem includes an approximation error term f 0 − R f 0 2 2 which depends on the type of interpolation matrix R. For example, for linear interpolation of a twice continuously differentiable function, the approximation error scales as O(K −2 ).
Sentence 2: for a sufficiently large K (particularly K = n), the approximation error will disappear.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: 10 we find, Next we note that the projections of this equation on the space spanned by W and its orthogonal complement should hold independently.
Sentence 2: multiplying equation 11 on the left by either P W or P ⊥ W , and multiplying it on the right by RΛ −1 , we obtain the following two equations, Inserting eqn.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: For example, if Ω = R p and the density p(x) consists of k highly separated Gaussian clusters of roughly equal size (k clusters), then there are exactly k eigenvalues very close or equal to zero, and their corresponding eigenfunctions are approximately piecewise constant in each of these clusters.
Sentence 2: in this setting spectral clustering with k eigenvectors works very well.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Hence, total computational complexity of DPDS is O(K max(t, α 2 t )) at each period t. Under the assumption of Lipschitz continuity, Theorem 1 shows that the value of DPDS converges to the value of the optimal policy under known model with a rate faster than or equal to log t/t if the DPDS algorithm parameter α t = t γ with γ ≥ 1/2.
Sentence 2: the regret growth rate of DPDS is upper bounded by O( √ T log T ).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: However, we note that softmax ascribes unexplained variance in choice behavior entirely to "noise", when subject may indeed employ a much more strategic policy whose learning and decision components are poorly captured by the model(s) under consideration.
Sentence 2: smaller fitted b does not imply subjects are necessarily more noisy or care about rewards less; it may simply mean that the model is less good at capturing subjects' internal processes.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Note that in the non-Bayesian model, if there are no observations for some arm i, players will have zero knowledge about its mean reward and they cannot make decisions.
Sentence 2: we assume without loss of generality that in the first N time slots of the game, with some constant compensation, the long-term controller can control the short-term players to choose all the arms once.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: A Gaussian process f (x) (GP) is an infinite collection of random variables, of which any finite subset is jointly Gaussian distributed.
Sentence 2: a GP is completely specified by its mean function m(x) and covariance kernel function k θ (x, x ), where x, x ∈ X denote the possible inputs that index the GP and θ is a set of hyper-parameters parameterizing the kernel function.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Our matrix Bk then consists of a single Y i for each pair in B 2 .
Sentence 2: observe that Bk has rank at most k and 1 For an online SVD calculator, see http://www.bluebit.gr/matrix-calculator/ if Bob succeeds in solving f on input B, then, Comparing ( 3) and ( 4), we arrive at, conditioned on E: where c 1 > 0 is a constant that can be made arbitrarily small by making c > 0 an arbitrarily small.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: While these objectives are convex and in some cases admit closed-form (i.e., non-iterative) solutions, they require inverting a matrix of size m × m. Thus in the case of very large data sets (e.g., like the one we consider in section 5), it might not be feasible to use this approach.
Sentence 2: an iterative update is employed in practice.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: To do this, we relate the affine transformation of the latent processes for the nth neuron, h n (•), to the non-negative intensity function of a pointprocess, λ n (•), via a static non-linearity g : R → R + .
Sentence 2: the spike times of neuron n on trial r, t t t (r) Where λ n (t) = g(h n (t)), T r is the duration of the rth trial, and Φ(n, r) is the total spike-count of neuron n on trial r. The expected log-likelihood term in ( 4) can be evaluated as The resulting expected log-likelihood in (7) still contains an integral over the expected rate function of the neuron, which cannot be computed analytically.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: For example, in the Biological taxonomy of Figure 1, for the category Dalmatian, this path includes the nodes (superclasses) canine and animal.
Sentence 2: for class c, the linearly combined kernel is given by where n ∼ c indexes the nodes that are ancestors of c, which is a leaf node (recall that the first C nodes in every taxonomy are reserved for leaf class nodes).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: 3 Due to the limited availability of the codes during this work, we are not capable to access all the source codes.
Sentence 2: for different methods, we utilize different strategies: For methods Exposure, FI, and DPED, we use the already-trained model to run the results; for CycleGAN, we retrain it with MIT-Adobe 5K training dataset; for Deep Guided Filter and Deep Photo Enhancer, we use demos from their own websites (http://wuhuikai.me/DeepGuidedFilterProject/ and http://www.cmlab.csie.ntu.edu.tw/ project/Deep-Photo-Enhancer/, respectively) to derive imagery results.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Therefore, after mapping we obtain ⌘ 0 i (x) ⌘ 0 j (x).
Sentence 2: for independent labels, the pick-one-label heuristic is consistent for precision@k. The section above has revealed that HSM cannot be properly adapted to multi-label problems by the pick-one-label heuristic.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: For example, in our toy world the entropy trajectories for the word orders SOV, OSV and OVS (two of which are pictured in Figure 1) are perfectly horizontal at various points (equivalently, some words carry zero information) because knowledge of the object in this world uniquely determines the verb (since foods are strictly eaten and drinks are strictly drunk).
Sentence 2: any word order that places O before V renders the verb entirely uninformative, in significant conflict with the UID hypothesis.
Relationship: reasoning