sadat2307/MSciNLI huggingface dataset:
Domain: NeurIPS
Sentence 1: In Bradley & Murray (2011) an iterative stochastic procedure is proposed for finding such matrix.
Sentence 2: we did not find it to work very well in an online optimization setting, and therefore stick to the original equilibration matrix D E . Although the original motivation for row equilibration is to prevent round-off errors, our interest is in how well it is able to reduce the condition number.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In the most general case, the encoder can update A t and B t in an unconstrained fashion at each timestep t. From the decoder side, we do not know C and therefore we cannot know F C, an estimate of which is needed by the user to update the encoder.
Sentence 2: the decoder sets F and can predict updates to [CA CB] directly, instead of to [A B] as the actual encoder does (equation 15).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Note that the proposed RankGAN has a Nash Equilibrium when the generator G θ simulates the humanwritten sentences distribution P h , and the ranker R φ cannot correctly estimate rank between the synthetic sentences and the human-written sentences.
Sentence 2: as also discussed in the literature [8,9], it is still an open problem how a non-Bernoulli GAN converges to such an equilibrium.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Given Φ, we can compute the joint distribution over the prediction range for each time series analytically, as this joint distribution is a multivariate Gaussian.
Sentence 2: in practice it is often more convenient to represent the forecast distribution in terms of K Monte Carlo samples, In order to generate prediction samples from a state space model, one first computes the posterior of the latent state p(l T |z 1:T ) for the last time step T in the training range, and then recursively applies the transition equation and the observation model to generate prediction samples.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Two classes of computable Stein discrepancies-the graph Stein discrepancy [10,12] and the kernel Stein discrepancy (KSD) [7,11,19,21]-have since been developed to assess and tune Markov chain Monte Carlo samplers, test goodness-of-fit, train generative adversarial networks and variational autoencoders, and more [7, 10-12, 16-19, 27].
Sentence 2: in practice, the cost of these Stein discrepancies grows quadratically in the size of the sample being evaluated, limiting scalability.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: [10] study k-way clustering and show that the eigenvectors of the graph Laplacian are stable in 2-norm under small perturbations.
Sentence 2: this justifies the use of k-means in the perturbed subspace since ideally without noise, the spectral embedding by the top k eigenvectors of the graph Laplacian reflects the true cluster memberships, closeness in 2-norm does not translate into a strong bound on the total number of errors made by spectral clustering.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In fact, it can be shown that a more rudimentary form of reweighted 1 applied to this model in [19] amounts to performing exactly one such iteration.
Sentence 2: repeated execution of ( 9) is cheap computationally since it scales as O nm x (k+1) 0 , where typically x (k+1) 0 ≤ n, and is substantially less intensive than the subsequent 1 step given by (3).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Complementary techniques such as exact covariance thresholding [13,19], and the divide and conquer approach of [8], have also been proposed to speed up the solvers.
Sentence 2: as noted in [8], the above methods do not scale to problems with more than 20, 000 variables, and typically require several hours even for smaller dimensional problems involving ten thousand variables.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Ideally, we would perform maximum likelihood estimation on the parameters, p ✓ (x r , z r )dz r , and compute the posterior p ✓ (z|x).
Sentence 2: under a fLDS neither the p ✓ (z|x) nor p ✓ (x) are computationally tractable (both due to the noise model P and the nonlinear observation model f (•)).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In particular, we expect that with random initialization, general stochastic gradient descent will need exponential time to escape saddle points in the worst case.
Sentence 2: if we add perturbations per iteration or the inherent randomness is non-degenerate in every direction (so the covariance of noise is lower bounded), then polynomial time is known to suffice [Ge et al., 2015].
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In fact, the empirical error with a finite margin is shown to converge to zero if , is sufficiently large.
Sentence 2: the existence of a weak learner with error 1/2 -, is not always useful in terms of generalization error, since it applies even to the extreme case where the binary labels are drawn independently at random with equal probability at each point, in which case we cannot expect any generalization.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Finding large independent sets is a fundamental problem in algorithm design and analysis and computing ALPHA(G) is a classic NP-hard problem which is even very hard even to approximate [11].
Sentence 2: the Lovász function ϑ(G) gives a tractable upper-bound and since then Lovász ϑ function has been extensively used in solving a variety of algorithmic problems e.g.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: All of the above methods are designed for the batch setting, where all of the data is collected in advance and used at once.
Sentence 2: if the training dataset is extremely large or the data are streaming and encountered in sequence, we may want to incrementally update the approximate posterior of the latent function f . Early work by Csató and Opper [6] proposed an online version of GPR, which greedily performs moment matching of the true posterior given one sample instead of the posterior of all samples.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: All that is needed now is to show a separation of their "-optimal sets for 0 < " < 1 60 d 3 2 , and this is done by showing a separation of the more manageable sets S 1 and S 2 .
Sentence 2: indeed, fix 0 < " < 1 60 d 3 2 and observe that for any w 2 S 1 we have and so, for d 4, for any w 2 S 2 we have We see that no w can exist in both S 1 and S 2 , so these sets are disjoint.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: 1, we provide NeuralFDR, a practical end-to-end algorithm to the multiple hypotheses testing problem where the hypothesis features can be continuous and multi-dimensional.
Sentence 2: the currently widely-used algorithms either ignore the hypothesis features (BH [3], Storey's BH [21]) or are designed for simple discrete features (group BH [13], IHW [15]).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The running time of the algorithm is O nM(r)d where r = min(r 1 , r 2 ).
Sentence 2: at smaller scales, where M(r) is comparable with n, it is O n 2 d . since the variance of the estimate also tends to be smaller at smaller scales, the algorithm iterates less for the same accuracy.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: With the notation z i = L • q(i) − ψ i and z 0 = −ψ 0 we re-write the minimax quantity as Observe that without the constraint that p t (i) ≤ 1/L for i > 0 we would put all the probability mass on the maximum of the z i .
Sentence 2: with the constraint the maximizer put as much probability mass as allowed on the maximum coordinate argmax i∈{0,...,K} z i and continues to the next highest quantity.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The existence of shift parameters seem to require extra additions/subtractions (see ( 2) and ( 8)).
Sentence 2: the binarization operation with a shift parameter can be implemented as a comparator where the shift parameter is the number for comparison, e.g., H v (R) = ⇢ 1, R 0.5 v; 1, R < 0.5 v. (0.5 v is a constant), so no extra additions/subtractions are involved.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For all i ∈ N , all θ i ∈ Θ i and all a i ∈ A i,θi , we have u i (σ Standard BN inference methods could be used to compute E[U ai |θ i ].
Sentence 2: such standard algorithms do not take advantage of structure that is inherent in BAGGs.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Both of these algorithms were also compared with the SVM equipped with a RBF kernel of variance σ 2 and a soft margin parameter C. Each SCM algorithm used the L 2 metric since this is the metric present in the argument of the RBF kernel.
Sentence 2: in contrast with , each SCM was constrained to use only balls having centers of the same class (negative for conjunctions and positive for disjunctions).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: While discussing learning the kernel, we showed that L 1 and L 2 cannot be updated simultaneously in a CCCP-style iteration since g is not convex over (S 1 , S 2 ).
Sentence 2: it can be shown that g is geodesically convex over the Riemannian manifold of positive definite matrices, which suggests that deriving an iteration which would take advantage of the intrinsic geometry of the problem may be a viable line of future work.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The difference of the learner's total reward and the total reward of the optimal strategy is called the (Audibert et al., 2009) of the algorithm and it can be formally written as As compared to the regret, the pseudo-regret has the same expected value, but lower variance because the additive noise ⌘ t is removed.
Sentence 2: the omitted quantity is uncontrollable, hence we have no interest in including it in our results (the omitted quantity would also cancel, if ⌘ t was a sequence which is independently selected of X 1:t .) In what follows, for simplicity we use the word regret instead of the more precise pseudo-regret in connection to R n . The goal of the algorithm is to keep the regret R n as low as possible.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For example, Node A concerns washbasins for infants, and has two polarized children nodes: reviewers take a positive perspective when their children enjoy the product (Node B: "loves", "splash", "play") but have negative reactions when it leaks (Node C: "leak(s/ed/ing)").
Sentence 2: the lowest topics in the hierarchy are often polarized; one child topic of "router" focuses on upgradable firmware such as "tomato" and "ddwrt" (Node E, positive) while another focuses on poor "tech support" and "customer service" (Node F, negative).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The teaching set of a concept c ∈ C is a set of indices (or examples) X ⊆ [n] that uniquely identifies c from C. Formally, given a concept class C ⊆ {0, 1} n (a set of binary strings of length n), X ⊆ [n] is a teaching set for a concept c ∈ C (a binary string in C) if X satisfies c| X = c | X , for all other concepts c ∈ C. The teaching dimension of a concept class C is the smallest number t such that every c ∈ C has a teaching set of size no more than t [GK95, SM90].
Sentence 2: teaching dimension does not always capture the cooperation in teaching and learning (as we will see in Example 2), and a more optimistic and realistic notion of recursive teaching dimension has been introduced and studied extensively in the literature [Kuh99, DSZ10, ZLHZ11, WY12, DFSZ14, SSYZ14, MSWY15].
Relationship: contrasting
Domain: NeurIPS
Sentence 1: (2013) and references therein).
Sentence 2: recent work (Dauphin et al., 2014;Choromanska et al., 2014) has brought theoretical and empirical evidence suggesting that local minima are with high probability not the main obstacle to optimizing large and deep neural networks, contrary to what was previously believed: instead, saddle points are the most prevalent critical points on the optimization path (except when we approach the value of the global minimum).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For GMRFs with n nodes indexing d-dimensional random subvectors, I(x R ; x A ) can be computed exactly in O((nd) 3 ) via Schur complements/inversions on the precision matrix J.
Sentence 2: certain graph structures permit the computation via belief propagation of all local pairwise MI terms I(x i ; x j ), for adjacent nodes i, j ∈ V in O(n • d 3 ) -a substantial savings for large networks.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The partial derivatives take a standard log-sum-exp form, requiring expectations E p(y A naive computation of this expectation would require summing over D k= d y d configurations.
Sentence 2: there are more efficient alternatives: the dynamic programming algorithms developed in the context of Poisson-Binomial distributions are applicable, e.g., the algorithm from [3] runs in O(Dk) time.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: One possibility is to start at the position of the deleted point, θ i−1 , on the contour constraint, which is independent of the other points and not far from the bulk of the required uniform distribution.
Sentence 2: if the Markov chain mixes slowly amongst modes, the new point starting at θ i−1 may be trapped in an insignificant mode.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: To see this, note that the sequence of partitions P 0,bad n becomes arbitrarily ill-balanced, which from (10) implies lim n→∞ Pcut G 0 n (P 0,bad n ) = 1.
Sentence 2: the unperturbed graph G n grows in a self-similar fashion as n → ∞ and so the Product Cut of P n remains approximately a constant, say γ, for all n. Thus Pcut Gn (P n ) ≈ γ < 1 for n large enough, and Pcut G 0 n (P 0,good Comparing this upper-bound with the fact lim n→∞ Pcut G 0 n (P 0,bad n ) = 1, we see that the Product Cut of P 0,bad n becomes eventually larger than the Product Cut of P 0,good n . While we execute this program in full only for the example above, this line of argument is fairly general and similar stability estimates are possible for more general families of graphs.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: At the beginning of learning when Ja is of small magnitude, the diffusion term, ⟨F 2 a ⟩, has a large impact so that it greatly impedes learning for a large η case.
Sentence 2: as the magnitude of the differences JA − JB increases, this effect weakens and the dependence of p A on η becomes quite small.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: 3.1 Algorithm for Learning the Transitive Reduction of CBNs [13] show that by using O (log n) multiple-vertex interventions, one can recover the transitive reduction of a DAG.
Sentence 2: in this case, each set of intervened variables has a size of O( n /2), which means that the method of [13] has to perform a total of O(2 n /2 log n) experiments, one for each possible setting of the O( n /2) intervened variables (see an example of this in Appendix D).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Specifically, we set Q(x) to be sigmoidal with parameter λ (see Figure 2a): As λ → ∞, Q(x; λ) → 1{x > 0}, so R labelwise (w : D) approaches the objective function defined in (7).
Sentence 2: r labelwise (w : D) is smooth for any finite λ > 0.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Suppose one were to give every layer in the RNN the largest possible skip for any graph with a period of m = 2 d 1 : set s (l) = 2 d 1 in every layer, which is the regular skip RNN setting.
Sentence 2: this apparent advantage turns out to be a disadvantage, because time spans of 2  n < m suffer from increased path lengths, and therefore which grows linearly with m. for the proposed DILATEDRNN, where d only grows logarithmically with m, which is much smaller than that of regular skip RNN.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The recent results in [7,8] suggest coresets that are similar to our definition of coresets (i.e., weighted subsets), and do preserve sparsity.
Sentence 2: as mentioned above they minimize the 2-norm error and not the larger Frobesnius error, and maybe more important, they provide coresets for k-SVD (i.e., k-dimensional subspaces) and not for PCA (k-dimensional affine subspaces that might not intersect the origin).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Interest in monocular depth estimation dates back to the early days of computer vision, with methods that reasoned about geometry from cues such as diffuse shading [12], or contours [13,14].
Sentence 2: the last decade has seen accelerated progress on this task [1][2][3][4][5][6][7][8][9][10], largely owing to the availability of cheap consumer depth sensors, and consequently, large amounts of depth data for training learningbased methods.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: As empirically found by [12], perplexities will saturate when n becomes large, because only a small portion of words actually exhibit long-range dependencies.
Sentence 2: we can see that the VPYLM performance is comparable to that of HPYLM with much fewer nodes and restaurants up to n = 7 and 8, where vanilla HPYLM encounters memory overflow caused by a rapid increase in the number of parameters.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: A sufficient condition to satisfy this definition is Hence, the set of misclustered nodes is defined as [6] M n = i ∈ {1, . . . , n} :  In practice, k-means algorithm tries to find a local minimum, and hence, one should run this step with multiple initializations to achieve a global minimum.
Sentence 2: empirically we found that good performance is achieved even if we use a single run of k-means.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For example, the bulk of the differential privacy literature has focused on the central model, in which user data is collected by a trusted aggregator who performs and publishes the results of a differentially private computation [11].
Sentence 2: google, Apple, and Microsoft have instead chosen to operate in the local model [15,6,2,10], where users individually randomize their data on their own devices and send it to a potentially untrusted aggregator for analysis [18].
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For instance, Catoni (2016) constructs a robust estimator of the Gram matrix of a random vector Z ∈ R d (as well as its covariance matrix) via estimating the quadratic form E Z, u 2 uniformly over all u 2 = 1.
Sentence 2: the bounds are obtained under conditions more stringent than those required by our framework, and resulting estimators are difficult to evaluate in applications even for data of moderate dimension.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: This symmetry is generated by a reflection operator, say the function m : R 3 → R 3 that flips the first axis: If S is a shape of a bilaterally-symmetric object, no matter how we align S to the symmetry plane, in general m[S] = S due to object deformations.
Sentence 2: we can expect m[S] to still be a valid shape for the object.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In the CIS (Random) model, the first few and the last few levels had little effect on perplexity and the medium-depth levels accounted for most of perplexity reduction.
Sentence 2: in the CIS (LearnedRI) model, the effect of a level on perplexity decreased with level depth, with the first few levels reducing perplexity the most, which is a consequence of the greedy nature of the tree-learning algorithm.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Finally, note that the above analysis fixes α = O(1/T ), β = 1 − α, but in practice FastRNN learns α, β (which is similar to performing cross-validation on α, β).
Sentence 2: interestingly, across datasets the learnt α, β values indeed display a similar scaling wrt T for large T (see Figure 2).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The first source of difficulty is adjusting the notion of effective rank (which the algorithm needs to compute) to compensate for the uncertainty in the knowledge of the eigenvalues of G. A further problematic issue arises because we want to measure the smoothness of f 0 along the eigendirections of G, and so we need to control the convergence of the eigenvectors, given that G converges to G in spectral norm.
Sentence 2: when two eigenvalues of G are close, then the corresponding eigenvectors in the estimated matrix G are strongly affected by the stochastic perturbation (a phenomenon known as hybridization or spectral leaking in matrix perturbation theory, see [1,Section 2]).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In Section 4 we will establish the theoretical results on the COCA estimator and will show that it can estimate the latent true dominant eigenvector θ 1 in a fast rate and can achieve feature selection consistency.
Sentence 2: we provide another model inspired from the classical PCA method, where we wish to estimate the leading eigenvector of the latent covariance matrix.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Since both q * (u) and ∂L ∂θ/z 1:M depend linearly on q(x) via sufficient statistics that contain a summation over all elements in the state trajectory, we can obtain unbiased estimates of these sufficient statistics by using one or multiple segments of the sequence that are sampled uniformly at random.
Sentence 2: obtaining q(x) also requires a time complexity of O(T ).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The negative KL divergence term in Equation ( 5) tries to close the gap between two pipelines, and one could consider allocating more weights on the negative KL term of an objective function to mitigate the discrepancy in encoding of latent variables at training and testing, i.e., −(1 + β)KL (q φ (z|x, y) p θ (z|x)) with β ≥ 0.
Sentence 2: we found this approach ineffective in our experiments.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: However, none of the existing PID information components described above fits yet the notion of intersection information, as none of them quantifies the part of sensory information I(S : R) carried by neural activity R that also informs the choice C. The PID quantity that seems to be closest to this notion is the redundant information that S and R share about C, SI(C : {S; R}).
Sentence 2: previous works pointed out the subtle possibility that even two statistically independent variables (here, S and R) can share information about a third variable (here, C) [23,27].
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Specifically, a DBN typically refers to a Bayes' net in which the variables have an explicit notion of time, and past observations have some influence on estimates about the present and future.
Sentence 2: marginalizing over unobserved variables at time t−1 typically produces increased complexity in the the model of variables at time t. in both [6] and this work, the emphasis is on performing inference with current information only, and efficiency is obtained by leveraging the similarity between the previous and newly updated models.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: We can see that the Bernstein upper bound is much tighter than the TRW upper bound, although at the cost of turning a deterministic bound into a (1 − δ) probabilistic bound.
Sentence 2: the Bernstein interval fails to report a meaningful lower bound when the model is difficult (σ p ≈ ±0.5), because n = 10 4 is small relative to the difficulty of the model.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In contrast, using the SAG iterations from the beginning gives the same rate but with a constant proportional to n. Note that this bound is obtained when initializing all y i to zero after the SG phase.
Sentence 2: 1 in our experiments we do not use the SG initialization but rather use a minor variant of SAG (discussed in the next section), which appears more difficult to analyze but which gives better performance.
Relationship: contrasting