sadat2307/MSciNLI huggingface dataset:
Domain: NeurIPS
Sentence 1: The problem of maximizing objective (1) with constraint ( 2) is a well-known quadratic optimization problem with a norm constraint, the solution of which is given by an eigenvalue-eigenvector problem (see below).
Sentence 2: in order to handle the case where C is not invertible -which will be the case below in our experiments -and to attenuate the noise in the data, we first use a technique called dimensionality reduction (see, e.g., [15]).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The 7 While smoothing allows one to use gradient-based algorithms with guaranteed convergence, the obtained solution is a perturbed version of the targeted one and thus a rounding step (such as solving a linear program [18]) is required.
Sentence 2: as illustrated in Figure 3, solving one linear program is more expensive than the PSGM for (2) when the data set is relatively large, thus indicating that using a smooth surrogate is not always beneficial.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In contrast to the previous case, since the label is costly here, if the algorithm decides to see the label it does not even have to reason about which features to observe, as querying the label requires paying a cost that is a constant over the cost of the best predictor in hindsight, already resulting in the ⇥(T 2/3 ) regret rate.
Sentence 2: in practice (for shorter horizons) it still makes sense to select the features that provide the best balance between the feature-cost and the prediction loss.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Given a feasible set S of experiments (i.e., ∑ i∈S x i x ⊤ i is invertible), the Gauss-Markov theorem shows that the lowest variance for an unbiased estimate θ satisfies Var[ θ] = ( ∑ i∈S x i x ⊤ i ) −1 .
Sentence 2: var[ θ] is a matrix, and matrices do not admit a total order, making it difficult to compare different designs.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The optimal solution is F ⇤ = (1 ↵)(I ↵S)     method); another is to solve F ⇤ as a system of linear equations by using an iterative solver like conjugate gradient (CG) [10].
Sentence 2: both methods suffer from slow convergence, especially when the number of labels, i.e., columns of Y , grows dramatically.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: CRH is closely related to a recent multimodal metric learning method called Multiview Neighborhood Preserving Projections (Multi-NPP) [23], because CRH uses a loss function for inter-modality point pairs which is similar to Multi-NPP.
Sentence 2: cRH is a general framework and other loss functions for inter-modality point pairs can also be adopted.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Although KRONDPPS have tractable learning algorithms, the memory requirements remain high for non-stochastic updates, as the matrix Yi U ⊤ i needs to be stored, requiring O(N 2 ) memory.
Sentence 2: if the training set can be subdivided such that Θ can be decomposed as Due to the bound in Eq.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: [2017] also provide an optimality gap for ASOSPs of (PhaseCut) without perturbation.
Sentence 2: their result is more general in the sense that it holds for all possible values of k. when k is large, it does not accurately capture the fact that SOSPs are optimal, thus incurring a larger bound on the optimality gap of ASOSPs.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: And the differential graph is defined as the difference between the two latent precision matrices A simple procedure is estimating ⇥ ⇤ X and ⇥ ⇤ Y separately, followed by calculating their difference.
Sentence 2: it requires estimating 2d 2 parameters (i.e., ⇥ ⇤ X and ⇥ ⇤ Y ), while our ultimate goal is only estimating d 2 parameters (i.e., ⇤ ).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For instance, when M turns out to be a random bipartite graph 9 the expected number of matches that any algorithm can achieve is always upper bounded by O T n 2 M (this is how Theorem 1 is proven).
Sentence 2: in order to have M * T = Θ(M ) as n grows large, it is sufficient that deg M (u) ≤ T n holds for all users u ∈ B ∪ G, even with such a random M. In order to avoid the pitfalls of M being a random bipartite graph (and hence the negative result of Theorem 1), we need to slightly depart from our general model of Section 2, and make structural assumptions on the way matches can be generated.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: It is well known that the Maximum Likelihood Estimate (MLE) can be computed as P M LE (e i = a) = ca m where c a denotes the number of times the event a appears in the observed sequence and m = j c j denotes the total number of observed events.
Sentence 2: mLE of the multinomial distribution is spiky since it assigns zero probability to the events that did not occur in the observed sequence.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: A generative model, referred as the Generator G, is trained to replicate the data from noise, while an adversarial discriminative model, referred as the Discriminator D, is trained to identify whether a sample comes from the true data or from G. The GAN training objective is based on a minimax game between the two networks and approximately optimizes a Jensen-Shannon divergence.
Sentence 2: as mentioned in Goodfellow et al.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Conditional on all parameters, H C and H T are independent.
Sentence 2: focus for a moment on H C . Since we wish to view it as a sample from a Wright-Fisher recombining population, its likelihood corresponds to the probability, under the coalescent-with-recombination distribution, of the set of all ARGs that could have generated H C . using the nucleotide mutation model described below, even Monte Carlo approximation of this probability is computationally intractable [12].
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In particular, CCA is equivalent to first seeking projection directions W z of Z, and then performing a multi-output linear regression from x n to W z z n , under the constraints W x X T XW T x = I, to obtain W x .
Sentence 2: it has not been seriously studied how to use CCA for LSDR because W z does not contain orthogonal rows.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Such Random Fourier Features have been used to approximate different types of positive-definite shift-invariant kernels, including the Gaussian kernel, the Laplacian kernel, and the Cauchy kernel.
Sentence 2: they have not yet been applied to polynomial kernels, because this class of kernels does not satisfy the positive-definiteness prerequisite for the application of Bochner's theorem.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Note that the new problem ( 6) is non-convex because X(G) does not form a convex set on X .
Sentence 2: if we fix G except for G k , it becomes convex with respect to G k . Combining with the random projection, we obtain the following minimization problem: We solve this by the ADMM method for each k = 1, . . . , K. Let g k 2 R I k R k 1 R k be the vectorization of G k , and W k 0 2 R D1⇥D2 be a matrix for the randomly projected matrix.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: It is important to note that the worst case iteration bound in Lemma 3.2 does not seem so appealing for our purposes, since it depends polynomially on 1/✏, and in our online algorithms naturally we will need to take ✏ = O(T c ) for some c > 0, which seems to contradict our goal of achieving poly-logarithmic in T oracle complexity, at least on average.
Sentence 2: as Lemma 3.2 shows, the more iterations Algorithm 2 performs, the closer it brings its final iterate to the set CH(↵K).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Furthermore, we cannot determine directionality by the strategy of the power PC theory, as we do not necessarily have the correct t:JJ sign during the short run.
Sentence 2: changing the classification of C i from generative to preventive (or vice versa) requires only removing from (adding to) the estimate (i) the Vi term; and (ii) all terms in which Vi was the only generative factor.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For most of the instances that CODE2INV fails to solve, we observe that the loop invariant can be expressed in a compact disjunctive normal form (DNF) representation, which is more suited for the decision tree learning approach with hand-crafted features.
Sentence 2: cODE2INV is designed to produce loop invariants in the conjunctive normal form (CNF).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: With these problem settings in place, our goal is to estimate the CT-VF in an RKHS and improve policies.
Sentence 2: since the output V (x) is unobservable and the so-called double-sampling problem exists when approximating VFs (see e.g., [38,16]), kernel-based supervised learning and its analysis cannot be directly applied to VF approximation in general.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Instead, works [8,10,12,9,20] propose to integrate CRF into the networks, so as to enable end-to-end training of the joint model.
Sentence 2: this integration may lead to a dramatic increase of parameters and computing complexity that the model usually needs many iterations of mean-field inference or a recurrent scheme [12] to optimize the CNN-CRF models.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: As we added more views that were available for the Caltech-101 datasets, we found that the performance of the pairwise approach consistently went up as we added the third and the fourth view.
Sentence 2: the performance of the centroid-based approach slightly got worse upon adding the third view (possibly due to the view being noisy which affected the learned U * ); however addition of the fourth view brought the performance almost close to that of the pairwise case.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Equation 1 can be explained as follows: if the m th feature is irrelevant to the cluster c (i.e., ω c,m = 0), then φ c,m will be sampled from a Dirichlet distribution with a uniform prior λ.
Sentence 2: if ω c,m = 1, then φ c,m has a prior of λ + µ on that feature value which matches the prototype's feature value, and a prior of just λ on all the other possible feature values.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Due to our large sample size, s > n, even writing down G above requires ⌦(nd) time.
Sentence 2: to form Z we do not need G itself: it suffices to compute for m = 1, ..., s the column z(⌘ m ) with z(⌘ m ) j = e 2⇡i⌘ T m aj .
Relationship: contrasting
Domain: NeurIPS
Sentence 1: First of all, MKL-LDA achieves a considerable performance gain of 14.6% over the best recognition rate by the seven KFD classifiers.
Sentence 2: while KFD-Voting and KFD-SAMME try to combine the separately trained KFD classifiers, MKL-LDA jointly integrates the seven kernels into the learning process.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For link creation event, the triplet means that destination node u creates at time t a link to source node s, i.e., from time t on, node u starts following node s. To ease the exposition, we restrict ourselves to the case where links cannot be deleted and thus each (directed) link is created only once.
Sentence 2: our model can be easily augmented to consider multiple link creations and deletions per node pair, as discussed in Section 8.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Since linear programming relaxations are used for solving the MILP problem, any MILP formulation can be used to provide approximate solutions and error estimates in an anytime fashion (i.e., the method can be stopped at any time during the computation with a feasible solution whose quality monotonically improves with time).
Sentence 2: the MILP formulations (both ours and that of Parviainen et al.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The approaches in [12,14,10] can be thought of as following a global sampling strategy, where multiple samples are drawn from q v (x) and then variance reduction is built a posteriori in a subsequent stage through the use of control variates.
Sentence 2: local expectation gradients reduce variance by directly changing the sampling strategy, so that instead of working with a global set of samples drawn from q v (x), the strategy now is to exactly marginalize out the random variable that has the largest influence on a specific gradient of interest while using a single sample for the remaining random variables.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For synchronous SGD, if one machine fails, the entire training process is delayed; whereas for asynchronous SGD, if one machine in a model replica fails, the other model replicas continue processing their training data and updating the model parameters via the parameter servers.
Sentence 2: the multiple forms of asynchronous processing in Downpour SGD introduce a great deal of additional stochasticity in the optimization procedure.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: We fixed the elements of w 1 and b 1 to zero to avoid the well-known indeterminacy of the softmax function.
Sentence 2: the true posteriors of the segment labels can be written, by the Bayes rule, as where p(C t = τ ) is a prior distribution of the segment label τ , and Assume that the feature extractor has a universal approximation capacity (in the sense of well-known neural network theory), and that the amount of data is infinite, so that the MLR converges to the optimal classifier.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Note that the sorting order is similar between algorithm 1 (User Average) and 2 (Item Average).
Sentence 2: algorithm 8 (NLPMF) has opposite order, which will be weighted higher in different part of the data, compared to algorithm 1 and 2. hypothesis RM SE F EAT < RM SE F W LS with paired t-test.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: As a result, we get that for any algorithm with any partial information feedback model (where b bits are extracted from each d-dimensional loss vector), it is impossible to get regret lower than Ω( (d/b)T ) for sufficiently large T . Without further assumptions on the feedback model, the bound is optimal up to log-factors, as shown by O( (d/b)T ) upper bounds for linear or coordinate measurements (where b is the number of measurements or coordinates seen 2 ) [2,19,25].
Sentence 2: the lower bound extends beyond these specific settings, and include cases such as arbitrary nonlinear measurements of the loss vector, or receiving feedback signals of bounded size (although some setting-specific lower bounds may be stronger).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: We assume that once an event occurs at node i under the influence of a particular parent j in a cascade, the same event will not happen again.
Sentence 2: in Figure 2(b), node e is susceptible given its parent a, b, c and d. only node d is the first parent who infects node e. Because each parent could be equally likely to first influence node i, the likelihood is just a simple sum over the likelihoods of the mutually disjoint events that node i has survived from the influence of all the other parents except the first parent j, i.e., Event c did not occur at node i.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The ideal test of our optimal MM is against the true optimal for the real world, which is intractable.
Sentence 2: if we find that the theoretical value function for the optimal MM in the Gaussian world matches the realized value function in the real world, then we have strong, though not necessarily conclusive, evidence for two conclusions: (1) The Gaussian world is a good approximation to the real world, otherwise the realized and theoretical value functions would not coincide; (2) Since the two worlds are nearly the same, the optimal MM in the Gaussian world should closely match the true optimal.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The idea of deriving the lower bound is the following: instead of ensuring that the algorithm performs well in the most adversarial environment θ , we consider a set of "bad" environments and make sure that the algorithm performs well on them, where each "bad" environment θ is the most adversarial one by only perturbing one coordinate θ i of θ.
Sentence 2: in order to get meaningful finite-time lower bounds, we need to perturb θ more carefully than in the case of asymptotic lower bounds.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For this reason, being able to make a trade-off between (a) and (b) to explicitly use the original content image I c at each layer is important.
Sentence 2: the update of [11] involves a single parameter γ and is of the form Notice that here the original image I c is used only once at the beginning of the process, and details that have been lost at layer l + 1 have no chance to be recovered at layer l. We present in the experimental section the effect of our variant.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In the previous section we have shown how to efficiently compute a distribution over symbolic option models H. Recall that the ultimate purpose of H is to compute the success probabilities of plans (see Section 2.2).
Sentence 2: thus, the quality of H is determined by the accuracy of its predicted plan success probabilities, and efficiently learning H corresponds to selecting the sequence of observations which maximizes the expected accuracy of H. it is difficult to calculate the expected accuracy of H over all possible plans, so we define a proxy measure to optimize which is intended to represent the amount of uncertainty in H. In this section, we introduce our proxy measure, followed by an algorithm for finding the exploration policy which optimizes it.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The classical approach to establish EEG-based control is to set up a system that is controlled by a specific EEG feature which is known to be susceptible to conditioning and to let the subjects learn the voluntary control of that feature.
Sentence 2: the Berlin Brain-Computer Interface (BBCI) uses well established motor competences in control paradigms and a machine learning approach to extract subject-specific discriminability patterns from high-dimensional features.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: After pruning the parameter, Θ l [q] , with the smallest sensitivity, the parameter vector is updated via Θl = Θ l +δΘ l . With Lemma 3.1 and (6), we have that the layer-wise error for layer l is bounded by Note that first equality is obtained because of the fact that E(Z l ) = 0.
Sentence 2: it is worth to mention that though we merely focus on layer l, the Hessian matrix is still a square matrix with size of m l−1 m l × m l−1 m l . we will show how to significantly reduce the computation of H −1 l for each layer in Section 3.4.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Although the overall approximation in the thought experiment looks simple, the argument is based on an over simplified assumption that θ i * ,j , θ i,j ≈ π 2 for i = j.
Sentence 2: when W * has constant spectral norm, even when W is very close to W * , θ i,j * could be constantly far away from π 2 , which prevents us from applying this approximation directly.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: To prove the second statement, suppose x1 = z1 + ȳ1 and x2 = z2 + ȳ2 are both optimal.
Sentence 2: let ∆x = x1 − x2 and ∆z = z1 since f (x) satisfies CNSC-T , by (8), for some m > 0.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: We find that the entropy of gMedLDA and iMedLDA are smaller than those of gLDA and fLDA, especially for (relatively) large K. This implies that sampling methods for MedLDA can effectively concentrate the probability mass on just several topics thus discover more predictive topic representations.
Sentence 2: fMedLDA yields the smallest entropy, which is mainly because the fully-factorized variational methods tend to get too compact results, e.g., sparse local optimums.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: R such that given a feature vector x with an unknown label y, the corresponding classifier predicts ŷ = sign(✓(x)).
Sentence 2: this mapping function also needs to be fair with respect to the values of a user sensitive attribute z 2 Z ✓ Z 0 (e.g., sex, race), which are drawn from an (unknown) distribution f (z) and may be dependent of the feature vectors and class labels, i.e., Given the above problem setting, we introduce the following quality metrics, which we will use to define and compare different fairness notions: I.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The canonical example for a Q-regular measure is the Lebesgue measure λ on R n . If d is the metric induced by the Euclidean norm, then Q = n and the bound (12) is tight with c 0 = C 0 , a dimensional constant.
Sentence 2: for general sets S ⊂ R n , λ need not be locally Q-regular.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Remark 2 In nonmetric MDS we require that dij ~ f(8ij ) for some monotonic function f. If the kernel function r is monotonically decreasing then clearly 1r is monotonically increasing.
Sentence 2: there are valid isotropic kernel (covariance) functions which are non-monotonic (e.g.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The one feature in the Birkhoff-based formulations that the permutahedron-based formulations do not have is the ability to average the solution over multiple vectors by choosing p > 1 columns in the matrix Y ∈ R n×p .
Sentence 2: our experiments suggested that the best solutions were obtained for p = 1, so this consideration was not important in practice.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: This controller is statistically inefficient for solving Markov decision problems compared with the normative strategy of building a statistical forward model of the transitions and outcomes, and searching for the optimal action.
Sentence 2: episodic control is computationally very straightforward, and therefore does not suffer from any excess uncertainty or noise arising from the severe calculational and search complexities of the forward model.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: An alternative to Gibbs sampling for Bayesian inference is to use asymptotic approximations to the intractable integrals based on Laplace's method [13,14].
Sentence 2: the Laplace approximation requires twice differentiable loglikelihood functions, and cannot be applied to the lasso and elastic net models as they contain a non-differentiable term proportional to the sum of absolute values (i.e.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In order to get accuracy on par with an oracle the models would have to mimic a function that incorporates: latent concepts, the difficulty of each exercise, the prior distributions of student knowledge and the increase in concept skill that happened after each exercise.
Sentence 2: the BKT prediction degraded substantially as the number of hidden concepts increased as it doesn't have a mechanism to learn unlabelled concepts.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Similar to batch normalization, evolutional dropout can also address the internal covariate shift issue by adapting the sampling probabilities to the evolving distribution of layers' outputs.
Sentence 2: different from batch normalization, evolutional dropout is a randomized technique, which enjoys many benefits as standard dropout including (i) the back-propagation is simple to implement (just multiplying the gradient of X l by the dropout mask to get the gradient of X l ); (ii) the inference (i.e., testing) remains the same 2 ; (iii) it is equivalent to a data-dependent regularizer with a clear mathematical explanation; (iv) it prevents units from co-adapting of neurons, which facilitate generalization.
Relationship: contrasting