sadat2307/MSciNLI huggingface dataset:
Domain: NeurIPS
Sentence 1: The value of the variational distance depends on the response to the TPQ, which is unobserved at query selection time.
Sentence 2: the agent computes the expected variational distance, Where P (y|ξ i , ξ j , D) = P (y|ξ i , ξ j , θ * )P (θ * |D)dθ * is the predictive distribution and is straightforwardly estimated using a set of posterior samples.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Our approach has some strong similarities with that of the second work noted above; however, the authors there do not establish or disprove stability, and instead note the presence of zero eigenvalues (which we will treat in some depth) as a motivation for their alternative optimization method.
Sentence 2: we feel the works as a whole are quite complementary, and signify the growing interest in GAN optimization issues.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Further, to infer the full kdimensional subspace A, we need to consider at least m X ≥ k centers.
Sentence 2: for each center ξ i , we define a set of m Φ directions and arrange them in a total of m Φ matrices We can now define the following linear system: where the linear operator A is defined as A(X) i = tr(Φ T i X), the curvature errors have been accumulated in e and the noise has been put in the vector z which is distributed as Given the structure of the problem, we can make use of several low-rank recovery algorithms.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Moreover, we can observe that for k(x t , x i , α i ) j to have a significant value, two conditions must be satisfied: (a) α ij should have a large value, and (b) f T i f t should have a large value.
Sentence 2: we interpret the pre-activation value Φ(x t , Θ) j as a weighted sum for the feature similarity f T i f t with the weight α ij .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Hence, to characterize the threshold for a small portion of the tail of this distribution, we must draw a very large number of samples just so that the estimate converges.
Sentence 2: if we want an α = 0.01 threshold from the Null sample maximum distribution, we require many thousands of permutation samples -each requires randomizing the labels and recalculating all test statistics, a very computationally expensive procedure when v is large.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Note that, by Kirchoff's laws, r G (i, j) ≤ d G (i, j) and, so, R G ≤ D G . In this section we describe our method to compute the pseudoinverse of a tree.
Sentence 2: let us begin by noting that the effective resistance is a better measure of connectivity than the geodesic distance, as for example if there are k edge disjoint paths of geodesic distance d between two vertices, then the effective resistance is no more than d k . the more paths, the closer the vertices.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The downside of RTRL is that for a fully connected RNN with n units the matrices H t and G t have size n × n and n × n 2 , respectively.
Sentence 2: computing Equation 3 takes O(n 4 ) operations and requires O(n 3 ) storage, which makes RTRL impractical for large networks.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Define ϕ(x) = 1 p x p p , and note that (∇ϕ(x) Since a ijk has only three nonzero entries, we see that e only needs to be updated in three components.
Sentence 2: in Algorithm 3.1 we may replace ( ) by an appropriate numerical computation of the parameter µ ijk and replace ( ) by the computation of the new value of e. Further details are available in [5].
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Denote by T 0j the overlapping part of T j corresponding to I 0 = I 1 ∩ I 2 as in the proof of Theorem 2.1.
Sentence 2: we consider the overlapping parts Or equivalently, if we take an orthonormal basis G of the null space of [e, T T 01 ], −[e, T T 02 ] and partition j [e, T T j ] T , j = 1, 2.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: For any µ ∈ ∆ p , let w µ ∈ H denote the mixture weight vector obtained from a sample of size pm by combining the p weight vectors w k , k ∈ [1, p], each returned by conditional maximum entropy when trained on the sample S k of size m. Then, for any δ > 0, with probability at least 1−δ, the following inequality holds: For the uniform mixture µ 0 = (1/p, .
Sentence 2: . . , 1/p), the bound becomes Proof  by McDiarmid's inequality, which proves the first statement and the uniform mixture case since µ 0 = 1/ √ p. Theorems 2 and 3 help us compare the mixture weight w pm obtained by training on a sample of size pm versus the mixture weight vector w µ 0 .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: We have: Therefore, Since Tr is independent of the basis, choosing an appropriate basis we can write: (6) is true because ρ 0 (σ) = 1, ∀σ ∈ S n , and f ℓ1 = 1. f is obtained by truncating the Inverse Fourier Transform expression to the first two terms.
Sentence 2: from (2), it follows that: Using the fact that ρ 0 (σ) = 1 ∀σ ∈ S n , fρ0 = 1, and plugging (6) into (7) gives the result of the proposition.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: assumption over arbitrary parameter subsets, because doing so will cause incorrect estimates due to incompatibility of sub-results from different workers (e.g., imagine trivially parallelizing a long, simplex-constrained vector across multiple workers -independent updates will break the simplex constraint).
Sentence 2: existing dataparallel schemes and frameworks, that cannot support sophisticated constraint and/or consistency satisfiability mechanisms across workers, are not easily adapted to model-parallel programs.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: While there have been many recent works on modeling information diffusion [2,6,7,8] and network evolution [9,10,11], most of them treat these two stochastic processes independently and separately, ignoring the influence one may have on the other over time.
Sentence 2: to better understand information diffusion and network evolution, there is an urgent need for joint probabilistic models of the two processes, which are largely inexistent to date.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Optimizing the reward augmented maximum likelihood (RML) objective, L RML (θ; τ ), is straightforward if one can draw unbiased samples from q(y | y * ; τ ).
Sentence 2: we can express the gradient of L RML in terms of an expectation over samples from q(y | y * ; τ ), ) to estimate ∇ θ L RML (θ; τ ) given a mini-batch of examples for SGD, one draws y samples given mini-batch y * 's and then optimizes log-likelihood on such samples by following the mean At a temperature τ = 0, this reduces to always sampling y * , hence ML training with no sampling.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In the reparameterized spike and slab prior, each pair of variables { w qm , s qm } is strongly correlated since their product is the underlying variable that interacts with the data.
Sentence 2: a sensible approximation must treat each pair { w qm , s qm } as a unit so that { w qm , s qm } are placed in the same factor of the variational distribution.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In this paper we suggest a novel encoding for such higher level representations based on the analogy between CPs and differential equations: both formulations suggest a change of state given the current state of the system, and both usually encode a desired goal in form of an attractor state.
Sentence 2: instead of shaping the attractor landscape of a policy tediously from scratch by traditional methods of reinforcement learning, we suggest to start out with a differential equation that already encodes a rough form of an attractor landscape and to only adapt this landscape to become more suitable to the current movement goal.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Its variance is given by: where G is the ( ) covariance matrix of the noise, S is a vector formed by averaging each column of G , V is the average of all the elements of G and Q is the time-average of the mean ¦ .
Sentence 2: 9 depends only on the first and second moments of the response distribution; substitution of data-derived estimates of these moments into (2) yields a standard error bar for the estimator.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: D-TS + : Implement the same operations as D-TS, except for the selection of the first candidate (Line 11 in Algorithm 1) is replaced by the following two steps: a (1) ← arg min R( 1) i ; D-TS + only changes the tie-breaking criterion in selecting the first candidate.
Sentence 2: the regret bound of D-TS directly applies to D-TS + : Corollary 1.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Indeed, if we set k = 2 in the dual functional (18) we would be able to conclude that the two dual functionals are identical (by a suitable change of variables) .
Sentence 2: the role of the constant C complies with the findings of [10] by controlling the tradeoff between the number of margin errors and support vectors and the size of the margins: 2/ N ::; C ::; 2 such that when C = 2 a single margin error is allowed (otherwise a duality gap would occur) and when C = 2/ N all vectors are allowed to become margin errors and support vectors (see [11] for a detailed discussion on this point) .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Now we focus on the case where |S| = k. Define a neighbor S (i,j) of S as That is, the neighbor S (i,j) is given by replacing one of the elements of S with that of V \ S. Note that I S (i,j) − I S = e j − e i for any neighbor S (i,j) of S. Let S (i * ,j * ) be a neighbor that maximizes f (S (i,j) ) among all neighbors of S. Since a subset S of size ) and S * ← S (i * ,j * ) .
Sentence 2: in either case it holds that γ ≥ f (S (i * ,j * ) ).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Let F be a self-concordant barrier of K with a parameter ν (see [20]).
Sentence 2: for x, y ∈ K, ρ(x, y) is the distance in the Riemannian metric whose metric tensor is the Hessian of F . the metric tensor on the tangent space at x assigns to a vector v the length and to a pair of vectors v, w, the inner product v, w x := D 2 F (x)[v, w].
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The model is constructed according to the principle of separation of time scales: processes that operate in the 100-1000 ms range are modeled as accurately as possible, those that vary on a much shorter time scale are assumed to instantaneously achieve their steady-state values, and those that occur over longer time scales, such as slow inactivation, are assumed constant.
Sentence 2: the model does not incorporate currents which inactivate on a short time scale, and cannot provide a good description of rapid events such as the transitions between up and down states or the generation of action potentials.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: For example, in a seminal feature weighting method called Relief [7,8], the fixed TNs determined based on the uniform weights (i.e., the Euclidean distance) are used throughout the learning process.
Sentence 2: the TN-weight consistency is Left: (a) The Euclidean feature space with w 1 = w 2 = 1/2.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The ratio therefore is computed as a function of the Poissons P (n = 0|λ) when the spike count n is zero.
Sentence 2: the Poisson expectation has to be multiplied by the time-length of the interval; call ∆t k = t k+1 − t k the length of the interval I k . Assuming that the neurons i = 1, . . . , N are independent we obtain: during the time-intervals where no action potential is observed, the diffusion drifts linearly with a slope equal to the sum over all neurons of the difference between the expected firing rate with stimulus 1 and the expected firing rate with stimulus 2.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Since L τ (θ) replaces the definition of Σ j with ωI ≽ Σ j , L τ (θ) is a looser bound than the one used by Algorithm 2.
Sentence 2: performing θ τ +1 = arg max θ∈Λ L τ (θ) makes less progress than a step of Algorithm 1.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The variance of each task's reward distribution is normalized by dividing the advantage by the standard deviation of the reward: refer to A as the normalized advantage.
Sentence 2: the gradient update to the parameters of the policy θ is the product of the advantage and expected derivative of the log probability of sampling an action: normalizing the advantage may also be seen as adapting the learning rate for each task.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Moreover, convergence rates for empirical processes on Markov chain sequences are also often given in terms of mixing coefficients that can ultimately be bounded in terms of π ⋆ and γ ⋆ (as we will show in the proof of our first result).
Sentence 2: valid confidence intervals for π ⋆ and γ ⋆ can be used to make these rates fully observable.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In studies of visual cortex, modes could span neurons, time and stimuli [27].
Sentence 2: given that tensors in the wild are almost never cubic, nor symmetric, to bridge the gap between theory and experiment, we go beyond prior work to derive and analyze Bayes optimal AMP algorithms for arbitrarily shaped high order and low rank tensor decomposition with different priors for different tensor modes, reflecting their different measurement types.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Agglomerative heuristics have several appealing features: they are easy to implement, easy to tune, and their running time is r Opn 2 polylognq on a dataset of size n. Standard divisive heuristics based on graph partitioning or clustering methods (like for example the bisection k-means or the recursive sparsest-cut approaches) often involve solving or approximating NP-hard problems.
Sentence 2: 1 it is natural to ask how good the solution output by an agglomerative method is compared to the solution output by a top-down method.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Crucially, with a single sample, it is not possible to get an unbiased estimate of the gradient of the objective ⇢(u T x, v T y).
Sentence 2: we consider a first-order oracle that provides inexact estimates of the gradient with a norm bound on the additive noise, and focus on inexact proximal gradient descent algorithms for CCA.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Compared to ECW-RMED, D-TS + achieves much lower regret in the Condorcet case, lower or comparable regret in the non-Condorcet case, and much more robustness in terms of regret STD and delayed feedback.
Sentence 2: the simplicity, good performance, and robustness of D-TS and D-TS + make them good algorithms in practice.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Moreover, the events ( 4) and ( 5) will hold with probability ≥ 1 − ǫ if n ≥ (1/2δ 2 ) log(N 2 /ǫ).
Sentence 2: any N -point subset of R D can be embedded, with high probability, into the binary cube of dimension O(log N ) in a similarity-preserving way.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Here, we introduce an auxiliary function, By Corollary 2, we have that J (T) ≤ Q(T; T ), J (T ) = Q(T , T ), and Q(T, T ) has the same first-order derivative as J (T) at T . Because l and h are quadratic and convex, Q is quadratic and convex as well.
Sentence 2: we can apply the optimization method in Section 3.2 to minimize J .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: On the left, where G is the tonic, input pool G, output pool 1, and the routing tier are in phase with one another (black clocks), while input pool C and output pool 6 are in phase with one another (grey clocks).
Sentence 2: the raw note input, G, activates the invariant output 1, which corresponds to the perceived tonic invariant representation (heavy solid arrows).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Let l i denote a feed-forward neural network with sigmoid output parametrized by θ that maps the state s and action a to the Bernoulli's parameter of s i |s, a.
Sentence 2: the probability of the next state s is finally given by: We emphasize that there is not necessarily any correspondence between the action vector a and the real actions that generated the observation pairs in the data.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Specifically, the number of mistakes at step k is [ i,k − max k <k i,k ] + and the associated loss is G k . Our main claim follows from combining the upper bounds in Equations 18 and 19.
Sentence 2: algorithm OASM mimics the greedy policy π g . we decided to prove Theorem 1 based on counting how many times the policies π t and π g choose a different item.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: We shall endow M k with the metric and define its covering number N ε M k as the minimal cardinality of a set The bound depends on an upper bound on base losses denoted B; this should be viewed as fixing a scale for the losses and is standard where losses are not naturally bounded (e.g., classical bounds on SVM kernel regression [19] use bounded kernels).
Sentence 2: we have the following generalization result, whose proof can be found in Appendix D of the supplementary material.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In general, this conclusion also holds for cases when one applies general first-order optimization algorithms, such as AGD, on finite sums, as this typically results in a linear dependence on n. Secondly, if a given optimization algorithm obtains an iteration complexity of an order of n + L β w (0) − w * 2 / α for solving smooth and convex finite sum problems with a stochastic oracle, then n + L β−α n 2(α−1) = Ω(n 2 ).
Sentence 2: β = α and α ≥ 2, indicating that an iteration complexity of an order of n + L w (0) − w * 2 / , as obtained by, e.g., SVRG++, is not attainable with a stochastic oracle.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: A traditional frequentist's approach would use counters A Bayesian approach would model p(y 0 |y i ) with an (initially flat) Beta-distribution, and use the counters a i and b i to update this belief [3], leading to the same MAP estimate ŵi .
Sentence 2: in both approaches a new example with y 0 = 1 and y i = 1 leads to the update where N i := a i + b i is the number of previously processed examples with .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Our methods alternate between x and y to break the non-separability of the augmented term ⇢ 2 kAx + By ck 2 .
Sentence 2: at each iteration k, given ẑk := (x k , ŷk ) 2 dom(F ), ˆ k 2 R n , ⇢ k > 0, and k 0, we define the x-subproblem as If k > 0, then ( 7) is well-defined and has unique solution.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In this case, γ = 2λ( β − µβ) where β = Gβ is the Gaussian smoothing of β .
Sentence 2: a component receives a high compensation weight γ k = 2λ( βk − µβ k ) if some neighboring spatial locations have already been selected (i.e., made "active") by the composite classifier.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Our bounds hold uniformly in that they hold for all hypotheses (functions from features to labels) at the same time.
Sentence 2: unlike many bounds on standard generalization error, our bounds do not depend in any way on the richness of the hypothesis class under consideration measured in terms of, for instance, its VC dimension, or the margin of the selected hypothesis on the training sample, or any other property of the mechanism with which the hypothesis is chosen.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: By our assumption on u and v, we have g(x, y) = u(x) − v(y) − (x − y) T ∇v(y) is continuous in x and y.
Sentence 2: by Lemma 6, the assumption of non-emptiness of A cccp (x) for any x ∈ Ω ensures that A cccp is closed on Ω and so satisfies condition (3) in Theorem 2.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The proposed algorithm is in the tradition of algorithms like UCRL2 [11] in that it implements the "optimism in the face of uncertainty" maxim, here combined with state aggregation.
Sentence 2: the algorithm does not need a generative model or access to "resets:" learning is done online, that is, in a single continual session of interactions between the environment and the learning policy.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The problem is solvable information-theoretically if there exists an algorithm that solves it, and efficiently if the algorithm runs in polynomial-time in n. Note that exact recovery in SBM(n, p, W ) requires the graph not to have vertices of degree 0 in multiple communities with high probability.
Sentence 2: for exact recovery, we focus on W = ln(n)Q/n where Q is fixed.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Notice that the list L contains exactly 14 literals in all, one from each mistake.
Sentence 2: also, we know from Lemma 3 that each of the "underlying groups" contributes at most one rule in L. the method of choosing features implicitly divides the class reptile into at least two groups, which appear to correspond to land reptiles and aquatic reptiles, and divides the somewhat nebulous class other into two groups, exemplified by worm and lobster.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: We assume that this stochastic process is stationary and ergodic, and that p t is a conditional probability distribution, which for some (unknown) constant integer D with 0 < D ≤ t has the form: (1) and Σ is finite, there is a finite number of length-D strings on which the evolution of our stochastic process can be conditioned.
Sentence 2: the stochastic process can also be represented by a time-invariant function Let s be a string of symbols from alphabet Σ with length |s| and elements [s] i=∈{1,...,|s|} .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: We therefore approximate the solution for y u by minimizing L(D l , D u , y u , x s ), which leads to the following evaluation function for query selection: For computational simplicity, for the rest of this work, we choose a quadratic loss function, i.e., ℓ(y, y) = (y − y) 2 /2 1 .
Sentence 2: it is straightforward to show where L = (K + λI) −1 and K = [κ(x i , x j )] n×n is the kernel matrix of size n × n. the evaluation function L(D l , D u , x s ) is simplified as Our goal is to efficiently compute the above quantity for each unlabeled instance.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Every convex, lower-semicontinuous function f has a convex conjugate function f * , also known as Fenchel conjugate [15].
Sentence 2: this function is defined as The function f * is again convex and lower-semicontinuous and the pair (f, f * ) is dual to another in the sense that f * * = f . we can also represent f as Nguyen et al.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Firstly, the posterior maintains the exact model, conditioned on U l . Secondly, we assume that the posterior distribution of {U l } L l=1 is factorized between layers (and dimension, but we suppress this from the notation).
Sentence 2: our posterior takes the simple factorized form Thirdly, and to complete specification of the posterior, we take q(U l ) to be a Gaussian with mean m l and variance S l . A similar posterior was used in Hensman and Lawrence (2014) and , but each of these works contained additional terms for the noisy corruptions at each layer.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: However, the true global parameters θ are recovered inexactly due to noise : θ → θ and state indexing ambiguity via a permutation mapping P: θ → θP .
Sentence 2: the parameters θP = ( ŌP , ĀP , νP ) obtained from the learning algorithm are in the following form: where P is the permutation matrix corresponding to the permutation mapping P. The presence of the permutation is a fundamental nuisance for MHMM learning since it causes parameter mixing between the individual HMMs.
Relationship: reasoning