sadat2307/MSciNLI huggingface dataset:
Domain: NeurIPS
Sentence 1: Some works addressed this problem in a generic setting to achieve better trade-off between arbitrary relevance and dissimilarity functions [11,9,51,8,21].
Sentence 2: they used only pairwise dissimilarities to characterize the overall diversity property of the list, which may not capture some complex relationships among items (e.g., the characteristics of one item can be described as a simple linear combination of another two).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The multiplicative nature of PoE models make them able to form complex densities: in contrast to mixture models, each expert has the ability to have a strongly negative influence on the density at any point by assigning it a very low component density.
Sentence 2: maximum Likelihood Estimation (MLE) of the model requires differentiation of a normalizing term, which is infeasible even for low data dimensionality.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The most related method of IsoHash is ITQ [7], because both ITQ and IsoHash have to learn an orthogonal matrix.
Sentence 2: isoHash is different from ITQ in many aspects: firstly, the goal of IsoHash is to learn a projection with isotropic variances, but the results of ITQ cannot necessarily guarantee isotropic variances; secondly, IsoHash directly learns the orthogonal matrix from the eigenvalues and eigenvectors of PCA, but ITQ first quantizes the PCA results to get some binary codes, and then learns the orthogonal matrix based on the resulting binary codes; thirdly, IsoHash has an explicit objective function to optimize, but ITQ uses a two-step heuristic strategy whose goal cannot be formulated by a single objective function; fourthly, to learn the orthogonal matrix, IsoHash uses Lift and Projection or Gradient Flow, but ITQ uses Procruster method which is much slower than IsoHash.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Compact Representation: After further investigation, we find that the non-local module contains a second-order feature space (Sect.3.1), which is used widely in previous computer vision tasks, e.g., SIFT [15], Fisher encoding [17], Bilinear model [14] [5] and segmentation task [2].
Sentence 2: such second-order feature space involves high dimensions and heavy computational burdens.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In the second step, we minimize the reconstruction loss as well as the perplexity of generated samples evaluated by the language model.
Sentence 2: since x is discrete, one can use the REINFORCE (Sutton et al., 2000) algorithm to train the generator: using a single sample to approximate the expected gradient leads to high variance in gradient estimates and thus unstable learning.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: • We provide a general structural result (Lemma 2) showing that, as in [7,18,19], our main algorithm exhibits a linear-quadratic solution error recursion.
Sentence 2: we show that by using our nonuniform sampling strategies, the factors appearing in such error recursion enjoy a much better dependence on problem specific quantities, e.g., such as the condition number (Table 2).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Developing lower bounds in the interactive setting appears to be significantly challenging, as algorithms may choose to get any deterministic information adaptively by querying, and standard lower bounding techniques based on Fano-type inequalities [9,31] do not apply.
Sentence 2: one of our major contributions in this paper is to provide a general framework for proving information-theoretic lower bound for interactive clustering algorithms which holds even for randomized algorithms, and even with the full knowledge of f + , f − and k. our algorithms are computationally efficient and are parameter free (works without knowing f + , f − and k).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: A slight variation of this has been shown to work as long as (α − γ)/ √ α = Ω(log n/ √ n) and the average degree grows faster than poly-logarithmic powers of n [10].
Sentence 2: (a) spectral clustering solves a harder problem than Problems 1 and 2, and (b) eigen-decompositions can be expensive, particularly for very large graphs.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In addition to the strong theoretical performance guarantee, DUDE can be implemented as a computationally efficient sliding window denoiser; hence, it has been successfully applied and extended to some practical applications, e.g., [1,3,4,2].
Sentence 2: it also had limitations; namely, the performance is sensitive on the choice of sliding window size k, which has to be hand-tuned without any systematic rule.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: T with fixed γ for different functional spaces.
Sentence 2: in the previous subsection we presented our method when considering a general set of transformations H. In this section we propose several possibilities for the choice of a convex set H. On the one hand, we propose to define H as a set of linear transformations from Ω S to Ω T . using the kernel trick, we propose to consider non-linear transformations.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Thus, X s0,π is approximately normally distributed for each s 0 ∈ S if H is sufficiently large.
Sentence 2: if γ = 1, X s0,π = H t=0 γ t R(s t , π(s t )) can be rewritten as a linear combination of n t=0 f (s t ) for n = 0 to H as follow: Notice that a linear combination of independent or correlated normally distributed random variables is still normally distributed.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In the Bayesian logistic regression application analyzed at the end of Section 4 of this paper, the number of arithmetic operations required to compute the gradient ∇U is Θ(d), and is the same number of operations required to evaluate the target function U itself.
Sentence 2: in other applications it can take 2d times as many arithmetic operations to compute the gradient ∇U as it takes to compute the target function U .
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For large scale collections (WIKI-Small, WIKI-50K and WIKI-100K), the solvers with competitive predictive performance are OVA, M-SVM, PD-Sparse and (π, κ)-DS.
Sentence 2: standard OVA and M-SVM have a complexity that grows linearly with K thus the total runtime and memory usage explodes on those datasets, making them impossible.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: State-of-the-art techniques in both classes are not ideal for use in contexts where performing inference quickly and reliably on large volumes of streaming data is crucial for timely decision-making, such as autonomous robotic systems [9][10][11].
Sentence 2: many classical clustering methods [12][13][14] scale well with the size of the dataset and are easy to implement, and advances have recently been made to capture the flexibility of Bayesian nonparametrics in such approaches [15].
Relationship: contrasting
Domain: NeurIPS
Sentence 1: [8] propose a kernel density estimate method to detect the points lying far from the data manifolds in the finallayer hidden space, which does not change the structure of the classification network with little computational cost.
Sentence 2: carlini and Wagner [3] show that each of these defense methods can be evaded by an adversary targeting at that specific defense, i.e., by a white-box adversary.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Straightforward application of Deep Belief Nets (DBNs) to acoustic modeling produces a rich distributed representation of speech data that is useful for recognition and yields impressive results on the speaker-independent TIMIT phone recognition task.
Sentence 2: the first-layer Gaussian-Bernoulli Restricted Boltzmann Machine (GRBM) has an important limitation, shared with mixtures of diagonalcovariance Gaussians: GRBMs treat different components of the acoustic input vector as conditionally independent given the hidden state.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The FTAL algorithm is designed to achieve logarithmic regret rate for exp-concave problems, exploiting the curvature information of such functions.
Sentence 2: our algorithm is aimed for optimizing general convex functions with possibly no curvature; while FTAL performs FTL over the second-order approximation of the functions, Online Lazy Newton instead utilizes a first-order approximation with an additional rank-one quadratic regularizer.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: As a final remark, a tighter variational bound is not guaranteed to always result in a better posterior approximation since the variational family limits the quality of the solution.
Sentence 2: in the context of variational EM, where one performs gradient-based hyperparameter optimization on the log marginal likelihood, our bound gives more reliable results since higher orders of K can be assumed to approximate the marginal likelihood better.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Specifically, the optimum solution of θ is obtained by solving the sub-problem 3 ) , which is equivalent to a constrained problem by introducing a set of non-negative slack variables ξ min θ,ξ The constrained optimization problem is similar to that of a linear SVM [7].
Sentence 2: the difference is that we have the additional term This term indicates that the estimation of the scene classification model is influenced by the topic discovery procedure, which finds an optimum posterior distribution q ⋆ .
Relationship: contrasting
Domain: NeurIPS
Sentence 1: This branch of computational chemistry provides invaluable tools for in-depth analysis but is currently not suitable for high-throughput reactivity tasks and is far from being able to recapitulate the knowledge and ability of human experts.
Sentence 2: most rule-based expert systems for high-throughput reactivity tasks use a much more abstract representation, in the form of general transformations over molecular graphs [2,7,8,9,10].
Relationship: contrasting
Domain: NeurIPS
Sentence 1: J τ determines the maximum allowable instantaneous speedup of one portion of a time series relative to another portion, within the same series or across different series.
Sentence 2: the length of time for which any series can move so rapidly is constrained by the length of the latent trace; thus the maximum overall ratio in speeds achievable by the model between any two entire time series is given by min(J τ , M N ).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In the factorial model we have The solution of the E step is therefore a product of Gamma distributions with parameters α + 1 and β + |s  A coefficient that has a small value after t iterations but is not exactly zero will have in the next iteration a large reweighting factor λ (t+1) i , which increases the chance that it will be set to zero in the next iteration, resulting in a sparser representation.
Sentence 2: a coefficient having a large value after t iterations corresponds to a feature that is very salient in the signal x.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: However, in order to recover the exact gradient under any potential set of stragglers, the required redundancy factor is on the order of the number of straggling nodes, which could mean a large amount of overhead for a large-scale system.
Sentence 2: we show that one can converge to an approximate solution with a redundancy factor independent of network size or problem dimensions (e.g., 2 as in Section 5).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Moreover, if the network is made larger and the number of parameters thereby increased, this credit assignment problem becomes still more difficult; thus we expect the performance of stochastic gradient methods to scale up with network size more poorly than deterministic methods.
Sentence 2: under some circumstances stochastic methods can be equally as effective as direct ones in training even large networks, generating near-identical learning curves (see, e.g., Fig.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Existing RDA methods [21] and many other stochastic gradient methods (e.g., [14,10]) can only show the convergence rate for the averaged iterates: xN = N t=1 t x t / N t=1 t , where the { t } are nonnegative weights.
Sentence 2: in general, the average iterates xN cannot keep the structure that the regularizer tends to enforce (e.g., sparsity, low-rank, etc).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The standard benchmark for the algorithm in this scenario is the external regret, that is the difference between its cumulative loss and that of the best (static) expert in hindsight.
Sentence 2: while this benchmark is useful in a variety of contexts and has led to the design of numerous effective online learning algorithms, it may not constitute a useful criterion in common cases where no single fixed expert performs well over the full course of the algorithm's interaction with the environment.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: To summarize, for every training/test partition and for every pair (% patterns, # of scalings) we look at the pair (C, σ) that outputs the smaller value of Q D . In this case, the use of the Prior PAC-Bayes Bound to perform the model selection increases the computational burden of using the PAC-Bayes one in the training of one classifier (the one used to learn the prior), in comparison to the extra N classifiers needed by N -fold cross validation.
Sentence 2: one could have included the tuning of the pair (% patterns, # of scalings) in the model selection.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: We then predict using ŷi = sgn(ȳ i ), for i = 1, . . . , n. The common justification behind the above learning paradigm [14,2] is that minimizing the cut (1) encourages neighbouring vertices to be similarly labelled.
Sentence 2: we now demonstrate that in the online setting such a regime will perform poorly on certain graph constructions -there exists a trial sequence on which the method will make at least θ( √ n) mistakes.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Without this assumption, we are not able to identify the rank of W * , and therefore the estimated low-rank tensor W could be very far from the true tensor W * .
Sentence 2: in many practical applications such as face recognition and image/video denoising, a portion of the observation tensor Y might be contaminated by gross error due to illumination, occlusion or pepper/salt noise.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In this case, by applying vanilla Gradient Descent (GD) or Accelerated Gradient Descent (AGD, [16]), one obtains iteration complexity of respectively, where κ := L/µ denotes the condition number of the problem and Õ hides logarithmic factors in the problem parameters.
Sentence 2: whereas such bounds enjoy logarithmic dependence on the accuracy level, the multiplicative dependence on n renders this approach unsuitable for modern applications where n is very large.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: MCML assumes that each class may be collapsed to a single point, at least approximately, and thus is only suitable for unimodal class distributions (or for simply connected sets if kernelization is used).
Sentence 2: if points belonging to a single class appear in several disconnected clusters in input (or feature) space, it is unlikely that MCML could collapse the class into a single point.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Performance improvements in these more recent models are mainly due to using better image features such as those obtained by region-based convolutional neural networks (R-CNN), or using more complex attention mechanisms [1] to provide a better context vector for caption generation, or using an ensemble of multiple LSTMs, among others.
Sentence 2: the LSTM is still playing a core role in these works and we believe improvement over the core LSTM by turbo learning is still very valuable and orthogonal to most existing works; that is why we compare the turbo LSTM with a native LSTM.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: We only know that they are potentially corrupted.
Sentence 2: to generate the untrusted labels from the true labels in D, we first obtain a corruption matrix C. Then, for an example with true label i, we sample the corrupted label from the categorical distribution parameterized by the ith row of C. Note that this does not satisfy the conditional independence assumption required for our estimate of C. we find that the GLC still works well in practice, perhaps because this assumption is also satisfied when the data are separable, in the sense that each x has a single true y, which is approximately true in many of our experiments.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In contrast, the MacKay update rules are considerably faster to converge [15].
Sentence 2: the idea here is to form the gradient of (2), equate to zero, and then form the fixed-point update neither the EM nor MacKay updates are guaranteed to converge to a local minimum or even a saddle point of L(γ); both have fixed points whenever a γ i = 0, whether at a minimizing solution or not.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For a set Ω k,d that contains all supports of size k with atleast k − d active variables from S * , define γ d as where Popular sparse regression algorithms, such as the Lasso and the OMP, can perform accurate support recovery when We will show in Section 3.2 that SWAP can perform accurate support recovery when γ d < 1.
Sentence 2: although the form of γ d is similar to ζ, there are several key differences, which we highlight as follows: vector, where d ≤ k. ζ is the 1 norm of a k × 1 vector.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In general, polynomial models (linear and quadratic) are too restrictive and appear to underfit the data under both the null and the alternative, producing inflated Type I error and diminished power.
Sentence 2: lower-order Matérn kernels (Matérn 1/2 and Matérn 3/2, dark blue lines) appear to be too flexible.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: It is interesting, and unfortunate, to note that the F does not asymptotically converge to the Hessian, so conjugate gradient methods might be more sensible asymptotically.
Sentence 2: far from the converge point, the Hessian is not necessarily informative, and the natural gradient could be more efficient (as demonstrated in Tetris).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Further, if A ∈ R 2×2 and det(A) = 1 then the gradient of the corresponding Möbius transformation is the convex function So the gradient of the index is the difference of the sums of a convex function of the linear-system orbits.
Sentence 2: such sums are Schur-convex functions and it follows that the index is increasing because one orbit weakly supermajorises the other, as we now show for the case 0w1 (noting that the proof is easier for words 0, 1).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: However, if we keep the gradient of the noise component, or, equivalently, let α 1 = α 1 and leading to highly correlated branches.
Sentence 2: if we consider the noise component as a constant, then only large E [ẑ 1,k ẑ2,k ] will be encouraged, which results in much weaker correlation, and hence the better generalization performance observed in practice.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Ignoring computational constraints, the algorithm achieves a regret bound of O( χ(G) log(k)T ).
Sentence 2: with computational constraints, its regret bound is O( c log(k)T ), where c is the size of the minimal clique partition one can efficiently find for G. note that for general graphs, it is NP-hard to find a clique partition for which c = O(k 1− ) for any > 0.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The well-known Langevin sampling (LS) recipe, so far the only sampling algorithm for continuous variables of which a neural implementation has been suggested, naturally fits into this dynamical framework.
Sentence 2: we first show analytically and through simulations that the symmetry of the synaptic weight matrix implied by LS yields critically slow mixing when the posterior is high-dimensional.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: If k < 15, we know λ i ∈ (5, 5 + ∆) for i = 1, 2, . . . , k + 1, and λ i ∈ (θ, θ + ∆) for i = k + 2, . . . , n. Therefore, the rate can be estimated as r small := Õ (k + 1 + (n − k − 1)(θ + ∆)/5) .
Sentence 2: if k ≥ 15, we know that λ i ∈ (θ, θ + ∆) for i = k + 1, . . . , n, and hence the rate can be estimated as r large := Õ (k + 1 + (n − k − 1)(θ + ∆)/θ) .
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Many real world graphs, such as the graphs of molecules, exhibit structure at multiple different scales, but most existing kernels between graphs are either purely local or purely global in character.
Sentence 2: by building a hierarchy of nested subgraphs, the Multiscale Laplacian Graph kernels (MLG kernels) that we define in this paper can account for structure at a range of different scales.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Using all KM nodes to obtain stochastic gradient estimates at the same point, we can perform T iterations of A-MB-SGD with a mini-batch size of KM , yielding an upper bound of This is a reasonable and common approach, and it is optimal (up to constant factors) when KM = O( L 2 H 2 T 3 ) so that the statistical term is limiting.
Sentence 2: comparing (18) to the lower bound ( 16) we see a gap by a factor of K 2 in the optimization term, indicating the possibility for significant gains when K is large (i.e.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Second, our pooling units are superficially similar to the pooling units used in convolutional networks.
Sentence 2: ours differ both in functional form and purpose: we use pooling as a way of sharing information between cells in the matrices that are processed through our network by taking maximums across entire rows or columns, while in computer vision, max-pooling units are used to produce invariance to small translations of the input image by taking maximums in a small local neighborhood.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: An iterative Gaussianization scheme transforms any source model to a spherical Gaussian by alternating between linear ICA transformations and nonlinear histogram matching to map marginal densities to Gaussians [28].
Sentence 2: in general, the overall transformation of iterative Gaussianization is an alternating concatenation of many linear/nonlinear transformations, and results in a substantial distortion of the original source space.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Specifically, we say that the loss function L satisfies RSC(k) with parameter κ l if it satisfies RSC with parameter κ l for the set {∆ ∈ R p : ∆ 0 ≤ k}.
Sentence 2: we say that the loss function satisfies restricted strong smoothness (RSS) with parameter κ u with respect to a set S if for all ∆ ∈ S. We can define RSS(k) similarly.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In practice, convex recovery is usually much more expensive than nonconvex recovery, because it requires optimizing over an n × n matrix variable instead of an n × r vector-like variable.
Sentence 2: it is statistically consistent [3], and guaranteed to succeed with m ≥ 1 2 n(n + 1) noiseless, linearly independent measurements.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Recency effects have been demonstrated in domains as varied as legal reasoning and jury evidence interpretation (Furnham, 1986;Hogarth & Einhorn, 1992) and clinical assessments (Mumma & Wilson, 2006).
Sentence 2: the most carefully controlled laboratory studies of sequential dependencies, dating back to the the 1950's (discussed by Miller, 1956), involve the rating of unidimensional stimuli, such as the loudness of a tone or the length of a line.
Relationship: contrasting