sadat2307/MSciNLI huggingface dataset:
Domain: NeurIPS
Sentence 1: p c (y|x) and p t (y|x) are equal to an adversarial loss w.r.t.
Sentence 2: p α (y|x): let L MD = βL c DS (p c (y|x), p t (y|x)) + γL t DS (p t (y|x), p c (y|x)) and L JS be the Jensen-Shannon divergence, the value function U (c, t, d) of the minimax game can be rewritten as Here, L JS reaches the minimum if and only if p α (y|x) = p u (y|x) and L c DS (or L t DS ) reaches the minimum if and only if p c (y|x) = p t (y|x).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The Hit-and-run, ball walk, and RWM algorithms are all thought to have a step size of roughly Θ(1) on sufficiently regular target distributions [21].
Sentence 2: since most of the probability of a standard spherical Gaussian lies in a ball of radius √ d, one would expect all three of these algorithms to take roughly ( √ d) 2 = d steps to explore a sufficiently regular target distribution.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The cosine SphereConv operator is a nonlinear function of θ (w,x) , with its g being the form of which can be reformulated as w T x w 2 x 2 .
Sentence 2: it can be viewed as a doubly normalized convolutional operator, which bridges the SphereConv operator and convolutional operator.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In both sets of experiments we use the following two performance measures for a learnt model A: Normalized error: To give an idea of how a good estimate A would perform under these two measures, we point out that a constant prediction xt+1 = x t leads to a normalized error of 1, and a random-walk prediction xt+1 = x t + ǫ t+1 , ǫ t+1 being a process, results in a nearly-zero cosine score.
Sentence 2: when the true model is more than a simple random walk, a good estimate A should achieve a normalized error much smaller than 1 and a cosine score way above 0.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: It is well-known that the geometric series of matrices, often called the Neumann series, I + λA × + (λA × ) 2 + • • • converges only if the maximum eigenvalue of A × , denoted by µ ×,max , is strictly smaller than 1/λ.
Sentence 2: the geometric random walk kernel K GR is well-defined only if λ < 1/µ ×,max .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: A match ( X, Ŷ ) in (X , Y) is a simple match if X ∪ Ŷ is non-empty and there exist no matches With the concept of the simple matches, we will show the Decomposition Theorem: every match can be decomposed as the union of a set of simple matches.
Sentence 2: simple matches fully characterize the structure of matches.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: (4) It is well-known that a Dirichlet sample can be generated by sampling from C independent Gammadistributed random variables with shape parameters α i and rate λ = 1; realizations of the class probabilities can be generated as follows: , where Therefore, the noisy Dirichlet likelihood assumed for each observation translates to C independent Gamma likelihoods with shape parameters either α i = 1 + α , if y i = 1, or α i = α otherwise.
Sentence 2: in order to construct a Gaussian likelihood in the log-space, we approximate each Gamma-distributed x i with xi ∼ Lognormal(ỹ i , σ2 i ) through moment matching (mean and variance): for the parameters of the normally distributed logarithm we have: Note that this is the first approximation to the likelihood that we have employed so far.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The second insight is a connection between the (1, 1)-norm and a notion of subspace sparsity introduced by [18].
Sentence 2: any X ≽ 0 can be factorized (non-uniquely) as Consequently, any X ∈ F d that has at most s non-zero rows necessarily has ∥X∥ 1,1 ≤ s 2 d. ∥X∥ 1,1 is a convex relaxation of what [18] call row sparsity for subspaces.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Interestingly, such energy correlations are also prominent in other kinds of data, including brain signals [33] and presumably even financial time series which have strong heteroscedasticity.
Sentence 2: developing a general model for energy-correlations of linear latent variables is an important problem in the theory of machine learning, and such models are likely to have a wide domain of applicability.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: To give an intuition concerning the speed and the efficiency attained by sparsification, we should mention that the dimension of the feature space for the kernel used in Model 1 is 6 (Proposition 9.2 in [14]).
Sentence 2: we deal with a kernel matrix of size 6 with sparsification versus a kernel matrix of size M = 5j , j = 1, . . . , 20 without sparsification.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: This bound acts as a surrogate objective that is optimized using stochastic gradient descent (SGD) to find a locally optimal configuration of the split functions.
Sentence 2: one complication introduced by this particular formulation is that the number of latent decision variables grows exponentially with the tree depth d. each gradient update will have a complexity of O(2 d p) for p-dimensional inputs.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Hence, compression improvement becomes D KL [p(θ|ξ t−1 ) p(θ|ξ t )].
Sentence 2: optimizing for compression improvement comes down to optimizing the KL divergence from the posterior given the past history ξ t−1 to the posterior given the total history ξ t . As such, we arrive at an alternative way to encode curiosity than information gain, namely D KL [p(θ|ξ t ) p(θ|ξ t , a t , s t+1 )], its reversed KL divergence.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: This happens whenever either L 1 < 1 (t) or L 2 < 2 (t), where L 1 and L 2 are the respective lengths of the first run and the first m runs of this arm.
Sentence 2: unlike the previous version of the algorithm which consists in an exploration phase followed by an exploitation phase, the anytime version of the algorithm continuously switches between exploration and exploitation.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: . . , R s,M )} S s=1 be an auxiliary training set where (X s , A s ) ∼ µ and R s,m = R m (X s , A s ).
Sentence 2: in each state-action pair, we generate T next states for each task, that is Y t s,m ∼ P m (•|X s , A s ) with t = 1, . . . , T . for any function Q we define the estimated transfer error as At each iteration, the algorithm Best Average Transfer (BAT) (Fig.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: This greedy algorithm starts with the empty set A 0 , and at each iteration i, it chooses an element e ∈ V that maximizes Let us denote this (centralized) greedy solution by A g . When f is integral (i.e., f : 2 V → N) it is known that the size of the solution returned by the greedy algorithm , where H(z) is the z-th harmonic number and is bounded by .
Sentence 2: we have |A g | ≤ (1 + ln(max e f ({e})))|A c |, and obtaining a better solution is hard under natural complexity theoretic assumptions [7].
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Quantities about x s such as expected values under the pseudomarginal can be computed using the samples x (i) s . However, for any given variable node x s , the incoming messages to x s given in (4) are defined in terms of the importance weights and sampled values of the neighboring variables.
Sentence 2: we can compute an estimate of the messages and beliefs defined in (4)-( 5) at arbitrary values of x s , simply by evaluating (4) at that point.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Variable-length quantum coding requires the definition of a code length operator L, which is a positive matrix such that for any density matrix X, tr(XL) gives the expected dimension ("length") of the message assigned to X.
Sentence 2: the quantum version of Kraft's inequality states that, ignoring rounding issues, for every variable-length quantum code with codelength operator L, there exists a density matrix W such that L = − log W . the matrix entropic loss can be interpreted as the (expected) code length of the observed outcome.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Next, we study the distributed implementation of PDA in (5a)-(5b) to solve (8).
Sentence 2: let Φ(x) given the initial iterates x 0 , θ 0 , λ 0 and parameters γ > 0, τ i , κ i > 0 for i ∈ N , choosing D x and D y as defined in Definition 1, and setting ν x = ν y = 1, PDA iterations in (5a)-(5b) take the following form: Since K i is a cone, prox κiσ K i (.)
Relationship: reasoning
Domain: NeurIPS
Sentence 1: That is, at any particular time, the user does not want to listen to most of the songs in the world, which leads to a mean preference close enough to zero to approximate as zero.
Sentence 2: the covariance kernel ¢¡ ¤£ ¦¥ £ simply turns into a correlation over a distribution of functions ¦ : ¡ ¤£ ¦¥ £ ¨ § © ¦ ¡ £ ¦ ¡ ¤£ .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: We call such a particle system massless.
Sentence 2: for NAG, it can also be verified that only h = Θ( √ η) results in a valid physical system and it is massive we provide a unified framework of choosing the correct time scaling factor h. In classic mechanics, the harmonic oscillator is one of the first mechanic systems, which admit an exact solution.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In order to understand the putative neural mechanism performing ICA derived from the formalism developed above, we need to find an online learning rule describing changes of the synapses as a function of pre-and postsynaptic activity.
Sentence 2: taking the inverse of ( 5), we have for weights that solve the ICA problem we expect because of (3) that which defines the weight matrix W * at the solution.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In other words, node i survives from the influence of all parents (see Figure 2(c) for illustration).
Sentence 2: the likelihood is a product of survival functions, i.e., Combining the above two scenarios together, we can obtain the overall likelihood of a cascade t c by multiplying together all individual likelihoods, i.e., the likelihood of all cascades is a product of the these individual cascade likelihoods, i.e.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Again, we have m j=1 β j equal to the value of Gr p . Combining these two, we conclude that the expected value of Gr p is equal to As before, we show that f j (Opt j ) is upper-bounded by β j + i∈Opt j α i .
Sentence 2: we can conclude that the competitive ratio of Gr p is 1 1+1/p = p 1+p .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: By using the re-parameterization w i = ab i , the learning machine eq.
Sentence 2: ( 4) results in This learner is a regular statistical model, hence both by comparing λ(a 0 , b 0 ) and −µ(a 0 , b 0 ) with N , let us clarify the effect of singularities.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Hence, maximizing the Tsallis entropy of a sparse MDN encourages exploration by providing bonus rewards to wide-spread mixture means and penalizing collapsed mixture means, while the causal entropy [1] of an MDN is less effective in terms of preventing the collapse of mixture means since there is no analytical form and its approximation is used in practice instead.
Sentence 2: maximizing the Tsallis entropy of a sparse MDN has a clear benefit over the causal entropy in terms of exploration and mixture utilization.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: . . , R 2 (i− 1, n) and flipping its j-th bit, which happens with probability at least Thus, if we have found L desired solutions in the i-th phase, the probability of finding a new desired solution in the next iteration is at least enPmax , where n−i+1 is the total number of desired solutions to find in the i-th phase.
Sentence 2: then, the expected number of iterations E[T ] is O(kn log nP max ) until the k-th phase finishes, which implies that an optimal solution corresponding to max 1≤j≤n R 2 (k, j) has been found.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The coefficients on the edge variables µ i (x i , x i+1 ) are their reduced costs, For duals that obey the constraints of (6), it is clear that the maximal reduced cost is zero, when x i is set to the argmax used when constructing α i+1 (x i+1 ).
Sentence 2: to a obtain a primal-optimal solution, we start at the end of the chain and follow the argmax indices to the beginning, which is the same backward sweep of the Viterbi algorithm.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: 3), where the first term on the right-hand side can be efficiently estimated with the reparameterization trick (see Appendix C for the details) where v ⇠ Uniform(0, 1) and z ⌘ g(v, b, ✓) is the differentiable reparameterization for z|b (Appendix C).
Sentence 2: using this to form the control variate and correcting with the reparameterization trick gradient, we arrive at where u, v ⇠ Uniform(0, 1), z ⌘ g(u, ✓), z ⌘ g(v, H(z), ✓), and ⌘ is a scaling on the control variate.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The above-mentioned convex properties of objective function f imply that we only need to look for the value of δ * satisfying df If Otherwise, we accept the change described by Equation 1 with δ = δ * .
Sentence 2: for objective function f we simply have where For the quadratic loss, we find where for the quadratic loss case, the optimal value δ * satisfies 2Cmδ For the exponential loss, we find where for the exponential loss case, the optimal value δ * satisfies After changing w 1 , we need to recompute 1 D w (i) ∀i ∈ {1, . . . , m}.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: At each iteration k, this system is fully characterized by a state z k equal to the training set S k . The system is actioned by a control u k equal to the design x k+1 selected to be evaluated.
Sentence 2: for a given state and control, the value of the objective function is unknown and modeled as a random variable W k , characterized by: where µ k (x k+1 ) and σ 2 k (x k+1 ) are the posterior mean and variance of the GP at x k+1 , conditioned on S k . We define a disturbance w k to be equal to a realization f k+1 of W k . w k = f k+1 represents a possible (simulated) value of the objective function at x k+1 .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: This means that the system is locally asymptotically stable, or more precisely, locally exponentially stable (see Definition A.1 in Appendix A).
Sentence 2: an important contribution of this paper is a proof of this seemingly simple fact: under some conditions, the Jacobian of the dynamical system given by the GAN update is a Hurwitz matrix at an equilibrium (or, if there are zero-eigenvalues, if they correspond to a subspace of equilibria, the system is still asymptotically stable).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: These conditions are exactly satisfied in the case of reward uncertainty: the forgotten information (i) does not influence future rewards, (ii) does not influence future transition probabilities, (iii) is never known by the opponent, (iv) is not remembered later by the player.
Sentence 2: we can construct the extensive-form game with the player restricted to Markovian policies and still solve it with CFR-BR.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Psychophysical investigations of human behaviour on object recognition tasks, measuring accuracies depending on image colour (greyscale vs. colour), image contrast and the amount of additive visual noise have been powerful means of exploring the human visual system, revealing much about the internal computations and mechanisms at work [42][43][44][45][46][47][48].
Sentence 2: similar experiments might yield equally interesting insights into the functioning of DNNs, especially as a comparison to high-quality measurements of human behaviour.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Once we have computed Â, similarly to [22,7], we define the function which we optimize as ĝ(z) = f ( ÂT z) = g(A ÂT z).
Sentence 2: we effectively work with an approximation f to f given by f (x) = ĝ( Âx) = g(A ÂT Âx).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: However, a subtlety of pLSIand the crucial difference between it and LDA-is that d is a dummy index into the list of documents in the training set.
Sentence 2: d is a multinomial random variable with as many possible values as there are training documents, and the model learns the topic mixtures p(zld) only for those documents on which it is trained.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: To this end, let us fix a k and an i and notice that by the subgaussianity assumption on the rewards, the empirical mean holds for any α.
Sentence 2: in particular, using this above formula for α the sum appearing in Lemma 1 can be bounded as where the last step follows from the fact 3 that ∞ k=0 e c √ k ≤ 2 c 2 holds for all c > 0.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Then we derive the auxiliary "switch" vector for where ||v i || = 1.
Sentence 2: the response of the or-node U i is defined as, Collaborative Edge: For any pair of leaf-nodes (L j , L j ′ ) respectively associated with two different or-nodes, we define the collaborative edge between them according to their contextual cooccurrence.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Comparing Equation 3with the entropy of the distribution rooted at a neighboring node 3, we see that they share a common term is positive, node 2 is a better root than 3, 2,3 is negative, we have the reverse situation.
Sentence 2: when comparing neighboring nodes as root candidates, the difference in entropy of the resulting distribution is simply the difference in entropy their local distributions assign to their separator.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Also, we have N ≥ n = Θ( √ k/ε 2 ), so when k is larger than some absolute constant, the O(1/ √ N ) term in the final line above is Θ(ε/ 4 √ k) ≤ ε.
Sentence 2: by Claims 2.6 and 2.7, either M fails to be accurate, so that ∃j ∈ [k] |a j − q j (P)| > ε, or we find a query q * such that q * (X) − q * (P) > ε.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Comparison between ( 14) and (12) shows that distinct from the HDP-LDA that combines a topic's global and local popularities in an additive manner as (n −ji jk + αr k ), the BNBP topic model combines them in a multiplicative manner as This term can also be rewritten as the product of n −ji •k and n −ji jk +rj c+n −ji •k +r• , the latter of which represents how much the jth document contributes to the overall popularity of the kth topic.
Sentence 2: the BNBP and HDP-LDA have distinct mechanisms to automatically shrink the number of topics.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The problem is that when a decision pair is finally encountered for which that cue would lead to a correct decision, it is unlikely to be checked because other, more discriminating although less valid, cues are looked up before and already bring about a decision.
Sentence 2: because one-reason decision making is intertwined with the learning mechanism and so influences which cues can be learned about, what mainly makes a cue come early in the order is producing a high number of correct decisions and not so much a high ratio of correct discriminations to total discriminations regardless of base rates.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: For example, if we sample edges i 1 ∼ j 1 and i 2 ∼ j 2 , then we replace these edges with i 1 ∼ j 2 and i 2 ∼ j 1 .
Sentence 2: when s = 0, the graph is the original discretization of a low-dimensional space; and as s increases to infinity, the graph becomes more and more like a uniformly chosen 4-regular graph (which is an expander [15] and which bears similarities with an Erdős-Rényi random graph [16]).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Similar to the classical gradient descent scheme of Nesterov [17], Algorithm 1 involves an extrapolation step (6).
Sentence 2: the solution of the auxiliary problem (5) at iteration k + 1 is driven towards the extrapolated variable y k . As shown in [9], this step is in fact sufficient to reduce the number of iterations of Algorithm 1 to solve (1) when ε k = 0-that is, for running the exact accelerated proximal point algorithm.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: A primary insight is that, since Algorithm 3 regularizes with the L 2 norm and penalizes with hinge-loss, omitting the positive semidefinite requirement for M and vectorizing M makes the algorithm equivalent to a oneclass, linear support vector machine with O(n 3 ) input vectors.
Sentence 2: the stochastic optimization is an instance of the PEGAGOS algorithm [19], albeit a cleverly constructed one.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In another words, the marginal distribution P (X) is a nuisance parameter; we maximize the joint log-likelihood, which is decomposed as log P (y, X|θ) = log P (y|X, θ) + log P (X), only with respect to θ [14].
Sentence 2: no assumption about the generative model is necessary.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Our algorithm will begin by verifying that p 2 = Θ(n −1/2 ) and p 3 = Θ(n −2/3 ) using Lemma 2.2.
Sentence 2: in the second case, we can assume that p 2 = Θ(n −1/2 ) and p 3 = Θ(n −2/3 ).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Similar to the observational case, I essential graph of a DAG D is the graph union of all DAGs in the same I equivalence class; it is denoted by E I (D).
Sentence 2: we have the following sequence: after a set of interventions I has been performed, the essential graph E I (D) is a graph with some oriented edges that captures all the causal relations we have discovered so far, using I.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Moreover, SSPtable takes advantage of bounded staleness to maximize ML algorithm performance, by reading the parameters ✓ from caches on the worker machines whenever possible, and only reading ✓ from the parameter server when the SSP model requires it.
Sentence 2: workers (1) spend less time waiting for each other, and (2) spend less time communicating with the parameter server.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The former reduces to the paramagnetic solution (f RS = f PARA ), which is unphysical for R < 1, while the latter yields π 1RSB (x) = δ(x), π1RSB (x) = δ(x) with m = β g (R)/β and β g obtained from the root of the equation enforcing the non-negative replica symmetric entropy with a free energy Since the target bit of the estimation in this model is J i1,••• ,iK and its estimator the product S i1 • • • S iK , a performance measure for the information corruption could be the per-bond energy e. According to the one-step RSB framework, the lowest free energy can be calculated from the probability distributions π 1RSB (x) and π1RSB (x) satisfying the saddle point equation ( 15) at the characteristic inverse temperature β g , when the replica symmetric entropy s RS disappears.
Sentence 2: f 1RSB equals e 1RSB .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: (The loss function often compares the predicted value with some label in a known way, in which case learning the value of the label for the round means that the whole loss function becomes known; hence the choice of the name.)
Sentence 2: the learner naturally will choose to see the loss function after he provides his prediction; this provides feedback that the learner can use, to improve the predictor he produces.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In between the starting and destination points, there are number of obstacles that the agent may pass through but should avoid for safety; each time the agent hits an obstacle it incurs a constraint cost of 1.
Sentence 2: in the CMDP setting, the agent's goal is to reach the destination in the shortest possible number of moves, while hitting the obstacles at most d 0 times or less.
Relationship: reasoning