sadat2307/MSciNLI huggingface dataset:
Domain: NeurIPS
Sentence 1: This distribution is a mixture over all possible permutations that could have generated the original document v. Now, we can use the fact that sampling uniformly from V(v) can be done solely on the basis of the word counts of v, by randomly sampling words without replacement from those word counts.
Sentence 2: we can train DocNADE on those generated word sequences, as if they were the original documents from which the word counts were extracted.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Since K H φ j = λ j φ j , we have ⟨ϕ(f (x)), κ j ⟩ = λ j ⟨ϕ(x), κ j ⟩.
Sentence 2: from the assumption, ⟨ϕ(f (x)), Ua j ⟩ = λ j ⟨ϕ(x), Ua j ⟩ .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: We assume that w is piecewise Lipschitz, i.e., there exists a sequence of non-overlaping intervals , and a constant L > 0 such that, for any (x 1 , y 1 ) and (x 2 , y 2 ) For generality we assume w to be asymmetric i.e., w(u, v) = w(v, u), so that symmetric graphons can be considered as a special case.
Sentence 2: a random graph G(n, w) generated by w is directed, i.e., G[i, j] = G[j, i].
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The likelihood model is: , where π ∼ Dir(α).
Sentence 2: (4) It is well-known that a Dirichlet sample can be generated by sampling from C independent Gammadistributed random variables with shape parameters α i and rate λ = 1; realizations of the class probabilities can be generated as follows: , where the noisy Dirichlet likelihood assumed for each observation translates to C independent Gamma likelihoods with shape parameters either α i = 1 + α , if y i = 1, or α i = α otherwise.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: By adding a pairwise region in between the original high-order factor node and the variable nodes we are able to reduce computational complexity since the high-order factors are now singly connected.
Sentence 2: we can guarantee that the complexity of the local message-passing steps run in each machine reduces from the state-space size of the largest factor to the size of the largest newly introduced region in each computer.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: However, because this method requires to list all the sets in the SLT-region, the complexity of generating an LFP increases exponentially with the number of variables in the MRF.
Sentence 2: although this method provides more flexibility to deal with any arbitrary structure, its computational cost does not scale with the size of MRFs in contrast with the structure specific SLT-bounds of section 4.2, that are assessed in constant time.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Define the matrix A such that A(:, I j ) = A j .
Sentence 2: then since Φ i A T i = 0, the well-defined matrix A T is a basis of N (Φ), we can use A T to recover the global embedding T . A simpler alternative way is use a one-sided affine transformation, i.e., fix one of T i and affinely transform the other; the affine matrix is obtained by fixing one of T0i and transforming the other.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Each value variable V ∈ V is associated to a bounded real-valued utility function u V over Ω pa V , which quantifies the (additive) contribution of the states of its parents to the overall utility.
Sentence 2: the overall utility of a joint state x ∈ Ω C∪D is given by the sum of utility functions V ∈V u V (x ↓pa V ).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: For simplicity, if we assume that every subset of N columns of Φ are linearly independent, the unique representation property (URP), then w ′ must necessarily have nonzero elements in locations that differ from w * .
Sentence 2: any solution in the neighborhood of w * will satisfy w * 0 < w * + αw ′ 0 .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: By Lemma 3.5 the starting point (x 0 , t 0 ) computed by GOODSTART and used by ISOTONICIPM is feasible and has symmetry sym(x 0 , D K ) ≥  Each call to HESSIANSOLVE fails with probability < n −3 .
Sentence 2: by a union bound, the probability that some call to HESSIANSOLVE fails is upper bounded by O( √ m log(npw p max /δ))/n 3 = O(1/n).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The constructive proof of the Dirichlet processes [26] shows that a DP can be sampled by iteratively scaling an infinite sequence of Beta random variables.
Sentence 2: posterior MCMC inference in a DPMM could, in theory, alternate between the following samplers where ∝ ∼ samples from a distribution proportional to the right side, x {k} denotes the (possibly empty) set of data labeled k, and f • (•) denotes a particular form of the probability density function of •.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In a practical optimization application, it serves the purpose of guiding the exploration of the domain, as locations x * with high prior value y 0 (x * ) are more likely to contain the maximizing argument.
Sentence 2: given a set of data points D t , the prior ( 9) and the likelihood (8) lead to a posterior given by the particular choice of the likelihood function guarantees an analytically compact posterior expression.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: A rigorous derivation of an objective function based on well-known estimation principles is very difficult, because the statistics involved are non-Gaussian, and the processes have difficult interdependencies.
Sentence 2: instead of deriving an objective function from first principles, we derived an objective function heuristically, and verified through simulations that the objective function is capable of estimating the two-layer model.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Unfortunately, the UNION-SPLIT algorithm from the previous section requires an efficient algorithm for submodular function minimization, and no such algorithm exists in the constrained setting; submodular minimization is NP-hard even under simple cardinality constraints [19].
Sentence 2: similarly, the BEST-B algorithm breaks down in the constrained setting; its guarantees carry over only if all the B i are within the constraint set C. for the constrained SH-min problem we instead propose a majorization-minimization algorithm.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In this case, ∇ a r(a * ) = 0 and hence ∇ a r(A n ) = 0.
Sentence 2: per the ROGD update rule, whether any agent updates or not does not matter: either the gradient is not received, in which case no gradient update happens; or a gradient is received, but at this Nash equilibrium, it is 0 and therefore nobody will want to make any update.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Although the function ( 4) is not complex-differentiable, if one identifies C n with R 2n and treats f (z) as a function in the real domain, f is still differentiable in the real sense.
Sentence 2: we adopt the Wirtinger calculus [22], which can be thought of as a clean way of organizing the real partial derivatives [29,34].
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The unique path from x to the root contains a and the path from y to the root contains b, both paths contain c. Since weights decrease along paths, the assumption implies that a, b, c are pairwise distinct and c is an ancestor of a and b.
Sentence 2: there must be a path from z via a to c and another path from z via b to c. Hence, T is not a tree, contradicting the assumption.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: However, a general parametrization of the transformation may violate the invertible requirement and result expensive or even infeasible calculation for the Jacobian and its determinant.
Sentence 2: several carefully designed simple parametric forms of T have been proposed to compromise the invertible requirement and tractability of Jacobian [Rezende and Mohamed, 2015, Kingma et al., 2016, Tomczak and Welling, 2016, Dinh et al., 2016, at the expense of the flexibility of the corresponding variational distribution families.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: To construct a final feature representation for classification, the outputs of the first and second layers of trained features are average-pooled over quadrants as is done by [6].
Sentence 2: our first layer of features result in 1600 × 4 = 6400 values in the final feature vector, and our second layer of features results in 3200 × 4 = 12800 values.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In particular, sparse filtering resembles the 1 2 sparsity penalty that has been used in non-negative matrix factorization [35].
Sentence 2: instead of the usual 1 penalty that is used in conjunction with sparse coding (i.e., s 1 ), one can instead use a normalized penalty (i.e., s 1 / s 2 ).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: It proceeds directly from the definition of * that, for all other than *, Thus, all but one of the terms in the sum above approach zero, and the remaining term equals p 1 ( *).
Sentence 2: thanks to Andrew Ledvina, David Blei, Yael Niv, Nathaniel Daw, and Francisco Pereira for useful comments.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: By Lemma 3.5, for every t ∈ [T ], we have .
Sentence 2: then, for any measure-preserving bijection π : [0, 1] → [0, 1] and t ∈ [T ], we have we are left with the problem of sampling from P. Let S = {x 1 , .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Besides dopamine (DA), whose levels are known to be related to the TD reward prediction error [9], serotonin (5-HT), noradrenaline (NA), and acetylcholine (ACh) were discussed in relation to TDRL meta-parameters [10].
Sentence 2: the knowledge of the characteristic meta-parameter dynamics can give an insight into the putative neuromodulatory activities in the brain.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Since M is assumed to be symmetric positive definite, it follows that {−1, 1} n ⊂ H(M).
Sentence 2: the hinge loss L u,t := max(0, 1 − y t u, v it M ) of any classifier u ∈ {−1, 1} n with any example (v it , y t ) is either 0 or 2, since | u, v it M | = 1 by equation ( 3).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Finally, notice that using an indicator function over each s, a, s triple is equivalent to matching the (empirical) joint distribution p(S t , A t , S t+1 ) induced by the reference policy and the true dynamics.
Sentence 2: even when we consider a different constraint for each triple, our solution implicitly couples the transition probabilities of different state-action pairs and differs from a rectangular formulation which focuses on matching the conditional distribution p(S t+1 |S t , A t ).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: the subspace spanned by {w j } d j=1 if it deviates from the subspace, i.e., In the above definition, we assume that the basis w j and outliers o are both normalized (see Algorithm 1 step 1)-a) where all the samples are 2 -normalized).
Sentence 2: we directly use inner product to define Γ o . Namely a sample is called outlier if it is distant from the underlying subspace of the signal.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Unfortunately, some partial derivatives of ρ may diverge to infinity on the boundary of the simplex in the non-Lipschitz case, making the error hard to analyze.
Sentence 2: to ensure that this error can be bounded, instead of b * , we will take a safe b ∈ B (far enough from the boundary) by using an intermediate point b in an inner simplex ∆ ε , where Thus, for a given b ∈ ∆ and ε ∈ (0, If we pick ε > δ B , then we are sure that b is not on the boundary of the simplex ∆, with a minimum distance from the boundary of η = ε − δ B .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: As explained in [22], this was done using a special case of the widely applicable Expectation Propagation (EP) algorithm [19] -with an additional approximation that the fan-in of all neurons is large, so their inputs are approximately Gaussian.
Sentence 2: given an error function, one can analytically obtain the Bayes estimate of the weights or the outputs, using the factorized approximation of the posterior.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Third, we show that one cannot use a DNF-based architecture with saturating dendrites.
Sentence 2: we show that an important family of lnBFs implemented with a CNF-architecture can require an exponential number of saturating dendritic units, whereas the same family implemented with either a DNF-architecture or a CNF-architecture always require a linear number of spiking dendritic units.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: It is easy to verify that ∂C θ ∂u (v) = e −θu (e −θv − 1) e −θ − 1 + (e −θu − 1)(e −θv − 1) .
Sentence 2: the quotient is constant and lim tցω C θ1,θ2,ω (t, v) = C θ1 (ω, v) − 0 + 0.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: )]0,8k2{1,2,...,m}} P T t=1 f t (x) be the best fixed decision in hindsight (knowing all loss functions f t (x) and the distribution of stochastic constraint functions g k (x; !)).
Sentence 2: x ⇤ minimizes the T -round cumulative loss and satisfies all stochastic constraints in expectation, which also implies lim sup T !1 x ⇤ )  0 almost surely by the strong law of large numbers.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: All valid fixing sequences for S yield the same CADMG G(V \ S, S), and if p(V) obeys the nested factorization for G, all valid fixing sequences for S yield the same kernel.
Sentence 2: as a result, for any valid sequence σ for S, we will redefine the operator φ σ , for both graphs and kernels, to be φ S . In addition, it can be shown [16] that the above kernel set is characterized as: we can re-express the above nested factorization as stating that for any fixable set S, we have φ S (p(V); G) = D∈D(φ S (G)) φ V\D (p(V); G).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Then our assumption on L guarantees h 1 ≥ 0, and since we also have In addition, we obviously have E P (h 1 −E P h 1 ) 2 ≤ E P h 2 1 ≤ BE P h 1 .
Sentence 2: bernstein's inequality [6,Thm.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Alternatively, the asymmetric score function S W (p i , p j ) in l W can be replaced with a symmetric score and used to derive an OASIS-like algorithm (which we name Dissim-Oasis).
Sentence 2: the optimal update for this loss has a symmetric gradient if W 0 is initialized with a symmetric matrix (e.g., the identity) all W i are guaranteed to remain symmetric.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: However, an obstacle remains: spectral decomposition over different triplets {u, v, w} results in different permutations of the labels of the hidden variable H. To overcome this, note that any two triplets (u, v, w) and (u, v , w ) share the same set of eigenvectors in (5) when the "left" node u is the same.
Sentence 2: if we consider a fixed node u * ∈ V as the "left" node and use a fixed matrix to diagonalize (5) for all triplets, we obtain a consistent ordering of the hidden labels over all triplet decompositions.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Equipped with an inner-product, we define the family of base kernels to be the possible outer-products K w = ww T between a vector w ∈ n and itself.
Sentence 2: input: A distribution D t . Labelled and unlabelled sets:  Using this definition we get, the similarity between two instances x i and x j is high iff both x i and x j are similar (w.r.t the standard inner-product) to a third vector w. Analogously, if both x i and x j seem to be dissimilar to the vector w then they are similar to each other.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: By the right invariance of D, different choices of π 0 lead to the same set of isomorphism classes Q M . Hence we may assume that π 0 is the identity.
Sentence 2: by Theorem 6.3 in [20] the (M × M )matrix with entries P (i, j) is a Toeplitz matrix, i.e., P (i, j) = P (i + 1, j + 1) for all i, j ∈ [M − 1], with entries strictly increasing along rows, i.e., P (i, j) < P (i, j + 1) for 1 ≤ i < j < M . by Theorem 2, thresholding leads to M different partial orders.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In Tarone's terminology, S is said to be untestable.
Sentence 2: defining the set of testable feature subsets at level δ as I T (δ) = {S| Ψ(S) ≤ δ}, Tarone's testability criterion obtains the corrected significance threshold as δ tar = max {δ : FWER tar (δ) ≤ α}, where FWER tar (δ) = δ|I T (δ)|.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Direct maximization of the marginal likelihood is impossible because the marginalization involves integration: log p(x) = log R p(x, z) d z.
Sentence 2: training is usually conducted by maximizing the expected complete data log likelihood (ECLL) over an auxiliary distribution q: max When using the conditional q(z|x) we emphasize the use of a recognition network in amortized VI (Kingma and Welling, 2014;Rezende et al., 2014).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The idea of the surrogate function starts from the Taylor series expansion of The global high-order derivatives ∇ s L N ( θ) (s ≥ 2) are replaced by local high-order derivatives So the approximation error is We can omit the additive constant L 1 ( θ) − L N ( θ) + 〈∇L 1 ( θ) − ∇L N ( θ), θ〉.
Sentence 2: thus the surrogate loss function L (θ) is defined as the surrogate minimizer θ = arg min Θ L approximates the empirical risk minimizer θ.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: However, some other types of divergences, such as MMD and Stein Variational Gradient, do not suffer from this problem.
Sentence 2: in this paper, for L 1 int , to maximize I(x, v x1 ; θ) we can replace the first term in (2) with Maximum-Mean Discrepancy (MMD) [35], which always prefers to maximize mutual information between x and v x1 .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Thus, we can upper bound where (10) We will bound ρ momentarily.
Sentence 2: for now, we turn to bounding the first term of (9) using the Cauchy-Schwartz generalization as follows: ṽt Let Ãt = 1 3 I +b t ṽt ṽ t . Note that the inequality above shows that 3 Ãt A t . using Lemma 3, we get To bound the latter quantity note that | Ã0 | = |I| = 1, and that Lemma 7 (proved in the full version of the paper), we show that Q ≤ Q + R 2 .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: We start by choosing a matrix regularization weight Λ such that Now consider the generalized inverse covariance problem (3) with this specified Λ.
Sentence 2: by construction, the assumption in Theorem 2 holds for Λ, so we can decompose this problem into k sub-problems; for each cluster c ∈ {1, . . . , k}, the subproblem has the following form: where S (c) is the sample covariance matrix of cluster c. Θ is the optimal solution of problem (3) with Λ as the regularization parameter.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Especially in the maximization case, even a naive sampling algorithm for approximate inference is not scalable: n sampling rounds need to be carried out for each node to estimate the influence, which results in an overall O(n|V||E|) algorithm.
Sentence 2: our goal is to design a scalable algorithm which can perform influence estimation and maximization in this regime of networks with millions of nodes.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Theorem 3.2 Scoring rule (z, η) is (strictly) proper to estimate η from z if and only if the equivalent loss where l(η) is a vector with components i (η) = (b i , η) and b i is the i-th element in Z (according to some arbitrary ordering), is (strictly) proper.
Sentence 2: proof The proof is straightforward by noting that the expected loss can be expressed as arg min η E z { (z, η)} = arg min η E y { ˜ (y, η)} and, thus, is (strictly) proper with respect to y iff ˜ is (strictly) proper.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: It is probable that such an ordering is close to optimal, but such orderings may not always be obvious, and the effects of a poor ordering on the resources required for policy generation can be extreme.
Sentence 2: to characterize the reordering methods discussed in Section 3.3, we start with initially randomly shuffled orders and compare the sizes of the final value diagrams with those found using the intuitive order.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: H is adequate for G if ∃M ∈ H such that M is adequate for G. An adequate model may be partial or incomplete in that it may not include every possible transition in the transition function T . However, the model is sufficient to produce a sound plan with respect to (T, g) for every goal g in the desired language.
Sentence 2: the more limited the goal language, the more incomplete the adequate model can be.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: A key to computation speedup is that the covariance matrices Σ k | 3 k=1 should be low-rank, since the features and tasks are enforced to be correlated for multi-task learning.
Sentence 2: the inverses of Σ k | 3 k=1 do not exist in general and we have to compute the generalized inverses using eigendecomposition.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The BA algorithm solves a convex optimization problem that is based on Corollary 3 for loss L SB .
Sentence 2: since the last three terms of the right-hand side of the bound of the corollary do not depend on ↵, this suggests to select ↵ as the solution of min ↵2  the following is the objective function of our optimization problem: The problem min ↵ 0 F (↵) is a convex optimization problem, which we solve via projected coordinate descent.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In the RBM between layers − 1 and , P (g ) is defined in terms on the parameters of that RBM, whereas in the DBN P (g ) is defined in terms of the parameters of the upper layers.
Sentence 2: q(g |g −1 ) of the RBM does not correspond to P (g |g −1 ) in the DBN, except when that RBM is the top layer of the DBN.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Phase 2 // We know that there exists r i such that ∀x ∈ S i , x ∈ C i ⇔ d(x, µ i ) < r i .
Sentence 2: // r i can be found by simple binary search S i = Sorted({S i }) // Sorts elements of {x : x ∈ S i } in increasing order of d(x, µ p ).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In all the results we report on, we use a one-back response history, i.e., m = 2.
Sentence 2: the SIMPLE models are REG(2, 0), LUT(2, 0), and REG⊕LUT(2, 0), the ORACLE and CRF models are REG(2, 1), LUT(2, 1), and REG⊕LUT(2, 1).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: ( 17), sigsoftmax has the following relation: where S is a hyperplane of S with k 15) and ( 18), we can see that the range of sigsoftmax includes the range of softmax.
Sentence 2: we have {f s (z)|z ∈ S} ⊆ {f (z)|z ∈ S}.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Therefore, it is not necessary for a worker to send g k (as well as u k ).
Sentence 2: we use an old value ĝk to approximate g k to further avoid computing g k . coordinate k will be skipped in the KKT filter if |ĝ k | ≤ λ − δ, where δ ∈ [0, λ] controls how aggressive the filtering is.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Consider that a data center is interested in the data sequence, {X(t)} ∞ t=1 , which cannot be observed directly.
Sentence 2: the data center deploys a bunch of L sensors which each independently encodes its noisy observation of the sequence, {Y i (t)} ∞ t=1 , without sharing any information, i.e., the sensors are not permitted to communicate and decide what to send to the data center beforehand.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In this paper, we focus on recommending likable items.
Sentence 2: instead of maximizing the expected reward r (T ) , we aim to maximize the expected number of likable items recommended up to time T : where X ut is the indicator random variable for whether the item recommended to user u at time t is likable, i.e., X ut = +1 if p uπut > 1/2 and X ut = 0 otherwise.
Relationship: reasoning