sadat2307/MSciNLI huggingface dataset:
Domain: NeurIPS
Sentence 1: Both SCAL and TUCRL quickly achieve sub-linear regret as predicted by theory.
Sentence 2: sCAL and TUCRL seem to achieve different growth rates in regret: while SCAL appears to reach a logarithmic growth, the regret of TUCRL seems to grow as √ T with periodic "jumps" that are increasingly distant (in time) from each other.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: We are given a training set of image and label pairs {(I t , l t )} N t=1 and the task is to predict the value of l t (e.g.
Sentence 2: a class label l t ∈ {1, . . . , C}) given the associated image I t . The standard machine learning approach would consist in extracting features from the whole image I t and from those directly learn to predict l t . since we wish to incorporate the notion of fixation into our problem framework, we need to introduce some constraints on how information from I t is acquired.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: [21] apply gradient-based optimization to find an image I which minimizes where P is a simple natural image prior, such as the total variation (TV) regularizer.
Sentence 2: this method produces images which are roughly natural and have features similar to the input features, corresponding to Φ −1 R . due to the simplistic prior, reconstructions from fully connected layers of AlexNet do not look much like natural images (Fig.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: We obtain a crisp analysis of the computational complexity of perceptron learning under the p,-margin success requirement: On one hand, for every p, > 0 we present an efficient p,-margin successful learning algorithm (that is, an algorithm that runs in time polynomial in both the input dimension and the sample size).
Sentence 2: unless P=NP, no algorithm whose running time is polynomial in the sample size and dimension and in 1/ p, can be p,-margin successful for all p, > O.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: At the same time the sum of the squared norms of the gradients will typically be of the order of O( √ T ), giving us a O( √ T ) regret bound (see for example the proofs in [4]).
Sentence 2: following this approach in DFEG we would have that the sum of norms of the squared gradients grows much faster than O( √ T ).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In the setting where log p is of a lower order than n 1 2α+1 , the n − 2α 2α+1 term dominates, and the rate of convergence is determined by the smoothness parameter α.
Sentence 2: when log p is much larger than n 1 2α+1 , p has a much greater influence on the minimax rate of convergence.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The Dirichlet is conjugate to the multinomial and hence mathematically convenient for Bayesian inference, and the number of parameters is conveniently linear in the size of the sample space.
Sentence 2: the Dirichlet is a distribution over the entire probability simplex, and for many problems this is simply the wrong domain if there is application-specific prior knowledge that the pmfs come from a restricted subset of the simplex.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Although it is discussed in [5] and mentioned to have the same sample complexity as their algorithm, it turns out that it is more efficient.
Sentence 2: its sample complexity is slightly better (or the same, depending on the parameter regime) compared to the one of the algorithm for the personalized setting presented in [5], which is in the setting of the lower bound in [5] where k = Θ(d), there is a gap of log(k) multiplicatively between the sample complexity of Algorithm R1 and the lower bound.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In general, bias b can be a probability distribution indicating a mixture of several guesses, but in this paper we only use two simpler scenarios: a binary vector to indicate whether there is a particular object with its associated weights U G , or a group bias b G with equal values for all objects, which indicates the presence of some object in general.
Sentence 2: a simple strategy would be to feed the segmented input y into a classifier C. this suffers from the loss of details during M 's reconstruction and is prone to hallucinations, i.e.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Intuitively, the single probe condition taps the marginal probability of the memory for the target item, p(ŝ t ), where t represents the index of the target item.
Sentence 2: the minimal color change condition taps the conditional probability of the memory for the target given the memories for the distracters, p(ŝ t |ŝ −t = s −t ) where −t represents the indices of the distracter items, because the actual dis- tracters s −t are shown during test.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Seeking a distribution-specific tight analysis, one simple option to try to tighten these bounds is to consider their minimum, min(d, E[ X 2 ]/γ 2 )/ǫ 2 , which, trivially, is also an upper bound on the sample complexity.
Sentence 2: this simple combination is also not tight: Consider a distribution in which there are a few directions with very high variance, but the combined variance in all other directions is small.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: After scaling by the number of predictors p, a similar rate of convergence is observed for the stationary   phase approximation to the partition function for both the diabetes and leukemia data (Figure 1b,c).
Sentence 2: convergence of the posterior expectation values xτ to the maximum-likelihood coefficients x, as measured by the ∞ -norm difference xτ − x ∞ = max j |x τ,j − xj | is noticeably slower, particularly in the p n setting of the leukemia data (Figure 1d-f).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: We could instead calculate the number of length t walks from v to each vertex more quickly, but this count would probably be dominated by walks that go to a high degree vertex and then leave and return to it repeatedly, which would throw the calculations off.
Sentence 2: most reasonably short nonbacktracking walks are likely to be paths, so counting each vertex a number of times equal to the number of nonbacktracking walks of length t from v to it seems like a reasonable modification.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Kernel methods with polynomial kernels k(z, z ) = z T z q = α z α z α provide an implicit mechanism to compute inner products in the feature space associated with P q .
Sentence 2: they require O(n 3 ) computation for solving associated kernelized (ridge) regression problems and O(n 2 ) storage of dense n × n Gram matrices K (given by K ij = k(z i , z j )), and therefore do not scale well.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: By introducing the Dirichlet process mixture (DPM) prior, infinite mixture of GPs models are able to infer the number of experts, both hyperparameters and parameters via Gibbs sampling [1,2].
Sentence 2: these models are trained by MCMC methods, which demand expensive training and testing time (as collected samples are usually combined to give predictive distributions).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: To model L, we will choose a distribution for positive semi-definite matrices analogous to the Gaussian distribution: a scaled Wishart matrix with expectation L. Note that, although it captures the trait that L is positive semi-definite, this distribution does not accurately model every feature of L. For example, a scaled Wishart matrix does not necessarily have ones along its diagonal.
Sentence 2: the mode of the density is at L, a Laplacian; and for large values of the scale parameter, most of the mass will be on matrices close to L. Appendix B of [8] provides a more detailed heuristic justification for the use of the Wishart distribution.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Consequently, G may collapse to these modes.
Sentence 2: to this end, we generate pseudo labels through C for some unlabeled data and use these pairs as positive samples of D. The cost is on introducing some bias to the target distribution of D, which is a mixture of p c and p instead of the pure p. this is acceptable as C converges quickly and p c and p are close (See results in Sec.5).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For very small settings of m (i.e., m = 2, 6), OUTLIERS-MR and GREEDY-MR perform slightly worse than sequential OUTLIERS and GREEDY.
Sentence 2: for practical settings of m (i.e., m ≥ 10), OUTLIERS-MR matches OUTLIERS and GREEDY-MR matches GREEDY.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: When workers cannot form a fully connected communication topology or the communication latency is high (e.g., in sensor networks or mobile networks), decentralizing the communication comes to the rescue.
Sentence 2: when the amount of data sent through the network is an optimization objective (maybe to lower the cost or energy consumption), or the network bandwidth is low, compressing the traffic, either via sparsification [Wangni et al., 2017, Konečnỳ andRichtárik, 2016] or quantization , Suresh et al., 2017 is a popular strategy.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: However, it is based on a tricky decomposition of the risk, where surrogate losses for risk minimization and risk analysis are different and labels of U data are needed for risk evaluation, so that no further bound is implied.
Sentence 2: ours utilizes the same surrogate loss for risk minimization and analysis and requires no label of U data for risk evaluation, so that an estimation error bound is possible.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Under the functional point of view, each 2D image is a function defined on the regular 2D lattice, so all images are functions over a common underlying parameterizing domain.
Sentence 2: with discretized 3D shapes, the probe functions are generally defined on heterogeneous shape graphs/meshes, whose nodes are points on each individual shape and edges link adjacent points.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: • We show that, for any fixed k, the k-nearest neighbor KL entropy estimator nearly achieves the minimax rates without knowing the smoothness parameter s. In the functional estimation literature, designing estimators that can be theoretically proved to adapt to unknown levels of smoothness is usually achieved using the Lepski method [39,22,45,44,27], which is not known to be performing well in general in practice.
Sentence 2: a simple plug-in approach can achieves the rate of n −s/(s+d) , but only when s is known [26].
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The construction and analysis from [23] crucially relies on the fact that in the setting of that paper, first the random misclassification noise rate η is chosen to take some particular value in (0, 1/2), and then the margin parameter γ is selected in a way that depends on η.
Sentence 2: in this paper the situation is reversed: in our setting first the margin parameter γ is selected, and then given this value we study how high a malicious noise rate η can be tolerated.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Different from the non-regularized case, the introduction of the regularization term will add bias in θT −i (t), which gives a biased evaluation of indexes.
Sentence 2: we prove the convergence of θT −i (t) (so again the indexes will converge properly) in the following lemma, which enables an easy adaption of our previous results for nonregularized case to ridge regression: The basic idea for extending the results to non-linear regression is inspired by the consistency results on M-estimator [14], when the error of training data satisfies zero mean.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Given the features f and training data D that consists of fully observed assignments to X and E, the optimal feature weights w * maximize the conditional log-likelihood (CLLH) of the data: The problem (2) does not have a closed form solution, but has a unique global optimum that can be found using any gradient-based optimization technique because of the following fact [1]: Fact 1 Conditional log-likelihood (2), abbreviated CLLH, is concave in w. Moreover, where E P denotes expectation with respect to a distribution P. Convexity of the negative CLLH objective and the closed-form expression for the gradient lets us use convex optimization techniques such as L-BFGS [9] to find the unique optimum w * .
Sentence 2: the gradient (3) contains the conditional distribution over X i X j , so computing (3) requires inference in the model for every datapoint.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: They have found numerous applications in machine learning, including viral marketing [1], dictionary learning [2] network monitoring [3,4], sensor placement [5], product recommendation [6,7], document and corpus summarization [8] data summarization [9], crowd teaching [10,11], and probabilistic models [12,13].
Sentence 2: submodularity is in general a property that goes beyond set functions and can be defined for continuous functions.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: On the other hand, when ∆ = Ω(T ( )), we also prove an information theoretic lower bound that any algorithm that outputs such a x must use super polynomial number of evaluations of f . Optimization of convex functions over a convex domain is a well studied problem in machine learning, where a variety of algorithms exist to solve the problem efficiently.
Sentence 2: in recent years, practitioners face ever more often non-convex objectives -e.g.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The above discussion shows that for appropriate choices of q, i.e., q ∈ (2, ∞), the RKBS embeddings in Examples 1-3 are useful in practice as γ K (P m , Q n ) is consistent and has a closed form expression.
Sentence 2: the drawback of the RKBS framework is that the computation of γ K (P m , Q n ) is more involved than its RKHS counterpart.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: We can train a separate set of inference networks to perform this operation, potentially also using gradient descent within the inference network.
Sentence 2: these networks do not receive any gradient information during Algorithm 1 Meta-training, differences from MAML in red Require: p(T ): distribution over tasks 1: initialize Θ := {µ θ , σ 2 θ , vq, γp, γq} 2: while not done do 3: Sample batch of tasks Ti ∼ p(T ) 4: for all Ti do 5: D tr , D test = Ti 6: Evaluate Evaluate ∇ θ L(θ, D tr ) 9: Compute adapted parameters with gradient descent: Compute ∇Θ T i L(φi, D test ) +DKL(q(θ|D test ) || p(θ|D tr )) 12: Update Θ using Adam meta-training, and may not work well in practice.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: As long as we pick η t as a constant η < 1/L, we have linear convergence of O((1 − γ/L) t ) Nesterov [2004].
Sentence 2: for SGD, due to the variance of random sampling, we generally need to choose η t = O(1/t) and obtain a slower sub-linear convergence rate of O(1/t).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Even the simplest 2 balls 2 bins game satisfies these properties (two pure Nash and unique best responses) which shows the strength of this impossibility result.
Sentence 2: our convergence result holds for any adversarial scheduling with the minimal fairness assumption that given any mixed state at least one agent who is not best responding eventually will be given the possibility to update their behavior, answering open questions in [26,25].
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Compared with these naive unrolling based methods (e.g., [7,33,30,17,2,20]), the main advantage of PODM is that we can generate iterations, which strictly converge to the critical point of the given optimization model, even in the complex nonconvex and nonsmooth scenarios.
Sentence 2: by slightly relaxing the exact optimality constraints during propagations, we can also obtain an interpretable framework to integrate mathematical principles (i.e., formulated by model based building-block) and experience of the tasks (i.e., network structures designed in heuristic manners) for collaborative end-to-end learning.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Moreover, in this setting (i.e., when  = O(1)), the number of samples required to exactly recover w ⇤ (with = 0) is given by: n = ⌦(s + k ⇤ log M ) [16], where s is the maximum support size of a union of k ⇤ groups and M is the number of groups.
Sentence 2: if one were to directly use sparse regression techniques (by ignoring group sparsity altogether) then the number of samples is given by n = ⌦(s log p).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The notion of regret has been extensively studied, and there exist plenty of algorithms and theories for minimizing regret [Shalev-Shwartz et al., 2007, Srebro et al., 2010, Duchi et al., 2011, Shalev-Shwartz, 2011, Zhang et al., 2013.
Sentence 2: when the environment is changing, the traditional regret is no longer a suitable measure, since it compares the learner against a static point.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: This lemma is of independent interest, and relates the SVD of a truncated matrix, here M (B) , to that of the initial matrix M . More precisely: Lemma 2 If δ ≤ m −8/9 , there exists a × k matrix V (B) such that its column vectors are orthonormal, and with probability 1 − exp(−m 1/7 ), for all i ≤ k satisfying that Note that as suggested by the above lemma, it might be impossible to recover V (B) i when the corresponding singular value s i (M ) is small (more precisely, when s 2 i (M ) ≤ n δ m log( )).
Sentence 2: the singular vectors corresponding to such small singular values generate very little error for lowrank approximation.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: One way to solve the above problem is to use regression, using That is, we estimate the data embedding z i as z i = W x i , and minimize their distances to the correct label embeddings u yi ∈ R m where the dimension for y i is set to 1 and every other dimension is set to 0: The above ridge regression will project each instance close to its correct embedding.
Sentence 2: it does not guarantee that the resulting embeddings are well separated.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Consistent with this prediction, in this condition the three two-way interaction terms between the effect features (f24, f34, f23) are greater than those interactions in the control condition.
Sentence 2: the common-effect schema does not imply that the three cause features will be correlated, and in fact in that condition the interactions between the cause attributes (f12, f13, f23) did not differ from those in the control condition (Figure 3).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Typically, this is not problematic; for example, for most realistic probability densities, p and p 2π have similar orders of smoothness, and p 2π = q 2π if and only if p = q.
Sentence 2: there are (meagre) sets of exceptions; for example, if q is a translation of p by exactly 2π, then p 2π = q 2π , and one can craft a highly discontinuous function p such that p 2π is uniform on X .
Relationship: contrasting
Domain: NeurIPS
Sentence 1: [5] used this idea with SVMs as base classifiers and subsequently in unsupervised learning by [6].
Sentence 2: co-EM also suffers from local maxima problems, and while each iteration's optimization step is clear, the co-EM is not really an expectation maximization algorithm (i.e., it lacks a clearly defined overall log-likelihood that monotonically improves across iterations).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: When F (x) is essentially locally strongly convex [9] in terms of • such that 5 then we can see that the local error bound holds with θ = 1/2, which implies the iteration complexity of HOPS is Õ( 1 √ ), which is up to a logarithmic factor the same as the result in [6] for a strongly convex function.
Sentence 2: here only local strong convexity is sufficient and there is no need to develop a different algorithm and different analysis from the non-strongly convex case as done in [6].
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For non-parametric classes, and non-smooth Lipschitz loss, such as the hinge-loss, the excess risk might scale as 1/n and not 1/n, even in the separable case.
Sentence 2: for H-smooth non-negative loss functions, where the second derivative of φ(t, y) w.r.t.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: It is also commonly believed that all systematic scans mix at the same asymptotic rate, which is equivalent to choosing These conjectures imply that using systematic scan instead of random scan will not result in significant consequences, at least asymptotically, and that the particular permutation used for systematic scan is not important.
Sentence 2: we show that neither conjecture is true by constructing models (listed in Table 1) in which the scan order has substantial asymptotic effects on mixing time.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For example, one can think of the desired object clusters as "topics of an image stack" and apply one of the standard topic discovery methods like probabilistic latent semantic analysis (pLSA) [8] or Latent Dirichlet Allocation (LDA) [2].
Sentence 2: we found that a simple K-means algorithm applied to the indicator vectors produced good results.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In particular, AdaGrad allows for a more aggressive learning rate by using But, at a high level, AdaGrad and dropout appear to both be aiming for the same goal: scaling the features by the Fisher information to make the level-curves of the objective more circular.
Sentence 2: l 2 -regularization makes no attempt to sphere the level curves, and AROW [18]-another popular adaptive method for online learning-only attempts to normalize the effective feature matrix but does not consider the sensitivity of the loss to changes in the model weights.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: It is easy to see that the query selection procedure of structural QBC is a rejection sampler where each query q is chosen with probability proportional to ⌫(q)u(q; ⇡ t ) (in the case of the 0-1 loss) or ⌫(q)var(q; ⇡ t ) (for general losses).
Sentence 2: it is possible for the rejection rate to be quite high, even when there are many queries that have much higher uncertainty or variance than the rest.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: ( 4), but with the additional restriction that V t is rank one, i.e, V t arg min Note that in this case it follows that V t is a unit trace rank-one matrix which corresponds to the leading eigenvector of the matrix rf (X t ) + ⌘ t X t . However, when V t is rank-one, the regularization kV t X t k 2 F makes little sense in general, since unless X ⇤ is rank-one, we do not expect X t to be such (note however, that if X ⇤ is rank one, this modification will already result in a linear convergence rate).
Sentence 2: we can think of solving a set of decoupled component-wise regularized problems: where the equivalence in the first line follows since kvv > k F = 1, and thus the minimizer of the LHS is w.l.o.g.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Note that although the variance contribution to the noise in the field, ro vanishes, the contribution from the correlations, Q 1 , remains finite and is responsible for the fact that m ro < 1 and ✏ > 0 [1].
Sentence 2: in a deep network, if the initial overlap is within the basin of attraction the m = 1 solution, the overlap quickly approach m = 1 [figure (2).C].
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The minimal sizes of T and H that are core w.r.t.
Sentence 2: f are The proof is deferred to Appendix C. The theorem states that, as expected, the higher rank(f ; M ), the smaller T . it also implies that the more irrelevant information f contains, the larger H needs to be, which might seem counter-intuitive.
Relationship: contrasting