sadat2307/MSciNLI huggingface dataset:
Domain: NeurIPS
Sentence 1: First, the convolutional layers of DeepSpeech2 contain more phonetic information than those of DeepSpeech2-light (+1% and +4% for cnn1 and cnn2, respectively).
Sentence 2: the recurrent layers in DeepSpeech2-light are better, with the best result of 37.77% in DeepSpeech2-light (by lstm3) compared to 33.67% in DeepSpeech2 (by rnn5).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: When using convolutions as part of a larger network, with multiple parallel filters, max pooling, and non-linear activations, the situation is of course more complex, and we do not expect to get the exact same bias.
Sentence 2: we do expect the bias to be at the very least related to the sparsity-infrequency-domain bias that we uncover here, and we hope our work can serve as a basis for further such study.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Figure 2(b) shows higher ACh ( ¡ ) and NE (© ¡ levels both correspond to fast learning, ie fast shifting of `¡ .
Sentence 2: whereas NE is a constant monitor of prediction errors and fluctuates accordingly with every data point, ACh falls smoothly and predictably, and only depends on the observations when global changes in the environment have been detected.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The work [9] proves that the proximal Coordinate Descent method can solve each QPs at a linear rate even when matrix A is not full column rank.
Sentence 2: there exists several drawbacks in this approach: (i) the practical solving time of each subproblem is quite long when A is rank-deficient; (ii) the theoretical performance and complexity of using recent accelerated techniques in proximal optimization [14] with the ALM is unknown; (iii) it cannot exploit the specific structure of matrix A when solving each constrained QP.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: We show in the supplementary material that if z = (z, 0) and z(i) denotes the ith largest element of z, In particular, if Y ∈ Y = {−1, 1} is binary the dual problem (11) for learning the optimal linear predictor α * given n samples (x i , y i ) n i=1 will be The first term is the empirical risk of a linear classifier over the minimax-hinge loss max{0, 1−z 2 , −z} as shown in Figure 2.
Sentence 2: the standard SVM is formulated using the hinge loss max{0, 1 − z}: We therefore call this classification approach the minimax SVM.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Given space considerations, and the fact that the resulting algorithm turns out to reduce to Algorithm 2 from [3] with the squared Euclidean distance replaced by an appropriate Bregman divergence, we will omit the full specification of the algorithm here.
Sentence 2: despite the similarity to the existing Gaussian case, we do view the extension to hierarchies as a promising application of our analysis.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: when the algorithm can be confident that µ j1 > µ j k . This chaining approach initially seems overly conservative when ruling out arms, as reflected in its regret bound, which is only non-trivial after T k 3 .
Sentence 2: the UCB algorithm [5] achieves non-trivial regret after T = O(k) rounds.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: . . , u(x n )) T ∈ R n and the covariance or kernel matrix by The Bayesian posterior process for u(•) can be computed in principle using Bayes' formula.
Sentence 2: if the noise model P (y|u) is non-Gaussian (as is the case for binary classification), it cannot be handled tractably and is usually approximated by another Gaussian process, which should ideally preserve mean and covariance function of the former.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: If the normal to the water surface directly underneath x is pointing straight up, there is no refraction and V (x) = G(x).
Sentence 2: if the normal is tilted by angle θ 1 , light will bend by the amount θ 2 = θ 1 − sin −1 ( 1 1.33 sin θ 1 ), so the camera point V (x) will see the light projected from G(x + dx) on the ground plane.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: It is important to note here that the GE operators themselves add layers to the architecture (thus this experiment does not control precisely for network depth).
Sentence 2: they do so in an extremely lightweight manner in comparison to the standard computational blocks that form the network and we observe that the improvements achieved by GE transfer to the deeper ResNet-101 baseline, suggesting that to a reasonable degree, these gains are complementary to increasing the depth of the underlying backbone network.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: We also adopt the same prior on E given by ( 7) above and used in [1] and [31], but we need not assume any additional hyperprior on Γ.
Sentence 2: for the prior on Z our method diverges, and we define the Gaussian where z vec[Z] is the column-wise vectorization of Z, ⊗ denotes the Kronecker product, and Ψ c ∈R n×n and Ψ r ∈R m×m are positive semi-definite, symmetric matrices.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: A short derivation (see supplementary material) then shows that the normalized importance weights are defined by a recursion: SIS is elegant as the samples and weights can be computed in sequential fashion using a single forward pass.
Sentence 2: naïve implementation suffers from a severe pathology: the distribution of importance weights often become highly skewed as t increases, with many samples attaining very low weight.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: DRE has also been widely discussed in statistical literatures for adjusting non-parametric density estimation [5], stabilizing the estimation of heavy tailed distribution [7] and fitting multiple distributions at once [8].
Sentence 2: as a density ratio function can grow unbounded, DRE can suffer from robustness and stability issues: a few corrupted points may completely mislead the estimator (see Figure 2 in Section 6 for example).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: A finite convex combination of copulas is a copula, so r (a) is a copula density.
Sentence 2: given a set of estimated quantile values A, a suitable parameter values β (edge weight matrix) and θ (parameters for bivariate edge copulas) can be found by maximizing the log-likelihood of A: the parameter optimization of l (β, θ) cannot be done analytically.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: A second related model is Hofmann's probabilistic latent semantic indexing (pLSI) [3], which posits that a document label d and a word ware conditionally independent given the hidden topic z : ( This model does capture the possibility that a document may contain multiple topics since p(zld) serve as the mixture weights of the topics.
Sentence 2: a subtlety of pLSIand the crucial difference between it and LDA-is that d is a dummy index into the list of documents in the training set.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In order to guarantee that the estimation error β (t) − β * in step t of the EM algorithm is well controlled, we would like Q n (•|β (t−1) ) to be strongly concave at β * .
Sentence 2: in the setting where n p, there might exist directions along which ) is flat, e.g., as in mixed linear regression and missing covariate regression.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The simplest Poisson-based generalised-linear RLM might take as its output distribution where y ti is the spike count of the ith cell in bin t and the function f is non-negative.
Sentence 2: comparison with the output distribution derived for the Gaussian RLM suggests that this choice would fail to capture the instantaneous covariance that the LDS formulation transfers to the output distribution (and which appears in the low-rank structure of S above).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: where c k=1 kπ k = c. The distribution over the size of the subset a new item joins is then The additional complication however is that, unlike the typical CRP situation, the partitioning itself is unknown, and so we must marginalize over it under the prior when computing the distribution on η x .
Sentence 2: we can recognize this as simply using the expected number of subsets of a particular size: where The probability of the partition π is the EPPF [21] multiplied by the unordered multinomial coefficient: where Γ(•) is the gamma function.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Interestingly, it is this same Jeffreys prior that forms the implicit weight prior of SBL (see [6], Section 5.1).
Sentence 2: it is worth mentioning that other Jeffreys prior-based techniques, e.g., direct minimization of p(w) = i 1 |wi| subject to t = Φw, do not provide any SBL-like guarantees.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: It is important to note that the linear time algorithm in [11,Section 5.5.1] is the key to obtaining a O(nd/ √ ) computational complexity for binary SVMs with bias mentioned in Section 5.1.
Sentence 2: this method has been rediscovered independently by many authors (including us), with the earliest known reference to the best of our knowledge being [14] in 1990.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: This allows us to apply our results also to more general forms of regularizers, including squared p norm regularizers, r(w) = λ 2 w 2 p , for p < 1 ≤ 2 (see Corollary 2).
Sentence 2: the reader may choose to read the paper always thinking of the norm w , and so also its dual norm w * , as the standard 2 -norm.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: This would be quite expensive for large number of items N , since the number of labels scales asymptotically as T ∈ Ω(N 2 ).
Sentence 2: we expect a noisy transitive property to hold: if items a and b are likely to be in the same cluster, and items b and c are (not) likely in the same cluster, then items a and c are (not) likely to be in the same cluster as well.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In the convergence proof [7,28], it is assumed that θ i converges to θ * as the number of iterations i increases, then the proof consists of showing that θ * is a critical point of p(θ|y).
Sentence 2: in practice, either the E-step or M-step or both can be difficult to compute exactly, especially when working with deep learning models.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The currently fastest algorithm for SO, in [MBK + 15], has runtime O(n log 1 δ ) and an expected guarantee of (1 − 1/e) − δ .
Sentence 2: the slightly slower, but still nearly linear time O( n δ log n δ ) thresholding algorithm in [BV14], has (the usual) deterministic guarantee of (1 − 1/e) − δ .
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The ability to tractably simulate MCs along with the generic applicability has made Markov Chain Monte Carlo (MCMC) a method of choice and arguably the top algorithm of the twentieth century [1].
Sentence 2: mCMC and its variations suffer from limitations in large state spaces, motivating the development of super-computation capabilities -be it nuclear physics [2,Chapter 8], Google's computation of PageRank [3], or stochastic simulation at-large [4].
Relationship: contrasting
Domain: NeurIPS
Sentence 1: We find that for x 0 = 1 + ", SGD always escapes and converges to x = 0 instead of staying in the initial basin, as long as " 6 = 0 and ⌘ is relatively large (one trajectory of is shown in Figure 2 in red).
Sentence 2: sGD starting from x 0 = " with the same learning rate always converges to x = 0, and we never observe the escape phenomenon.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: We define the biased conductance φ p (S) of a set S ⊂ {1, . . . , n} to be , where Z 0 is chosen according to a fixed distribution p. Just as with the standard definition of conductance, we can interpret biased conductance as an escape probability.
Sentence 2: the initial state Z 0 is not chosen following the stationary distribution (as in the standard definition with a reversible chain) but following p instead.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Intuitively, this is what one would expect: Q-learning is based on the single estimator and Double Q-learning is based on the double estimator and in Section 2 we argued that the estimates by the single and double estimator both converge to the same answer in the limit.
Sentence 2: this argument does not transfer immediately to bootstrapping action values, so we prove this result making use of the following lemma which was also used to prove convergence of Sarsa [20].
Relationship: contrasting
Domain: NeurIPS
Sentence 1: When d = 1, the quantile transforms F −1 (α j ) are uniquely defined as the points x j ∈ R satisfying F (X ≤ x j ) ≤ α j , where X is a random variable drawn from F . Equivalently, F −1 (α j ) can be identified with the unique hierarchical intervals [−∞, x j ].
Sentence 2: when d > 1, intervals are replaced by sets C 1 ⊂ C 2 . . . ⊂, C q that satisfy F (C j ) = α j but are not uniquely defined.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: If the user can pick any element and suggest that it should precede an entire block of elements it currently follows, then we can instead use an "INSERSION SORT" graph; interestingly, to ensure the property (*), this graph must be weighted.
Sentence 2: as we show in Section 3, if the user can propose two arbitrary elements that should be swapped, there is no graph G with the property (*).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Thus, if the blinding index falls short of the predetermined threshold, the data is effectively thrown away and the trial needs to be repeated.
Sentence 2: if the blinding index exceeds the threshold, the analysis of data is performed in the same manner regardless of the actual value of the index, that is, regardless of whether it is just above the threshold or if it indicates perfect blinding.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: If q is initially very far from p, Monte Carlo estimates will tend to under-estimate the partition function.
Sentence 2: in the context of learning p, we may expect a random initialization of p to be approximately uniform; we may thus fit an initial q to this well-behaved distribution, and as we gradually learn or anneal p, q should be able to track p and produce useful estimates of the gradients of p and of Z(θ).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: A generative method builds a flexible probabilistic model for generating the noisy observations conditioned on the unknown true labels and some behavior assumptions, with examples of the Dawid-Skene (DS) estimator [5], the minimax entropy (Entropy) estimator 1 [24,25], and their variants.
Sentence 2: a discriminative approach does not model the observations; it directly identifies the true labels via some aggregation rules.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: . . , s m ] are scaling factors for the parameter w. To combat overfitting, we assign a Gamma prior Gam(λ v |c 0 , d 0 ) over λ v . Note that this generative model encourages w to align with the major eigenvectors with bigger eigenvalues.
Sentence 2: eigenvectors are noisy and not all of them relevant to the classification taskwe need to select relevant eigenvectors (i.e.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: If we assume ideal conditions where both computation resources (CPU, GPU, other accelerators) and communication resources (communication links) are unlimited or abundant in counts/bandwidth, then the total runtime of Pipe-SGD is: where K denotes the iteration dependency or the gradient staleness.
Sentence 2: we observe that the end-to-end training time in Pipe-SGD can be shortened by a factor of K. the ideal resource assumption doesn't hold in practice, because both computation and communication resources are strictly limited on each worker node in today's distributed systems.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Due to the high variance of the score and its sensitivity to some implementation details [19], it is difficult to have a precise evaluation of Tetris controllers.
Sentence 2: our brief tour d'horizon of the literature, and in particular the work by Szita and Lőrincz [18] (optimizing the "Bertsekas features" by CE), indicate that ADP algorithms, even with relatively good features, have performed extremely worse than the methods that directly search in the space of policies (such as CE and genetic algorithms).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In general, the actual sequence of play may fail to converge altogether, even in simple, finite games [16,24].
Sentence 2: there is a number of recent works establishing the convergence of play in potential games with finite action sets under different assumptions for the number of players involved (continuous or finite) and the quality of the available feedback (perfect, semi-bandit/imperfect, or bandit/payoff-based) [5,11,14,19].
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Note for P ⊂ R m , P is contained in the linear span of at most m points from P , and similarly the exact Carathéodory theorem states any point q ∈ Convex(P ) is expressible as a convex combination of at most m + 1 points from P . As the conic hull lies between the linear case (with all combinations) and the convex case (with non-negative combinations summing to one), it is not surprising an exact conic Carathéodory theorem holds.
Sentence 2: the linear analog of the approximate convex Caratheodory theorem does not hold, and so the following conic result is not a priori obvious.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: We argue in Section 5.1 that the unbalanced Haar wavelets are a special instance of graph Laplacian eigenbasis when the underlying graph is hierarchical.
Sentence 2: a lattice graph structure yields activations that are globally supported and smooth, and in this case the Laplacian eigenbasis corresponds to the Fourier transform (see Section 5.2).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Power Multiset Classification A brute-force approach based on the Combination Method in multilabel classification [22,17], is to transform the class set C into a set M (C) of all possible multisets, then train a multi-class classifier π that maps an input x to one of the elements in M (C).
Sentence 2: the number of all possible multisets grows exponentially in the maximum size of a target multiset, 2 . rendering this approach infeasible in practice.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For any data matrix X that satisfies the SSC and SSS properties such that Using Lemma 12 (see the appendix), we can translate the above result to show that w T0 − w * 2 ≤ 0.95σ + , assuming k * ≤ k ≤ n 150 .
Sentence 2: lemma 5 will be more useful in the following fine convergence analysis.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Note the single-variable problem ( 14) has closed-form solution which in a naive implementation, takes O(nnz(A)) time due to the computation of ( 11) and ( 12).
Sentence 2: in a clever implementation, one can maintain the relation ( 11), (12) as follows whenever a coordinate x j is updated by where a j = [a I j ; a E j ] denotes the jth column of A I and A E . Then the gradient and (generalized) second-derivative of jth coordinate can be computed in O(nnz(a j )) time.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Roughly speaking, the events ω ∈ Ω represent abstract states of nature, i.e.
Sentence 2: knowing the value of ω completely describes all probabilistic aspects of the model universe, and all random aspects are described by the probability measure P. Ω, A and P are never known explicitly, but rather constitute the modeling assumption that any explicitly known distribution P X is derived from one and the same probability measure P through some random variable X.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Instead, [18] showed that in the case of coverage functions, it is possible to efficiently maximize f by lifting the problem to the continuous domain and using stochastic gradient methods on a continuous relaxation to reach a solution that is within a factor (1 − 1 e) of the optimum.
Sentence 2: our work provides a general recipe with 1 2 approximation guarantee for problem (1.2) in which f θ 's can be any monotone submodular function.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: To encourage consistency while reusing general purpose solvers, a PN-consistency potential [9] is often incorporated into the model: Hereby c is a positive constant that is tuned to penalize for the violation of consistency.
Sentence 2: as c increases, the following constraint holds: usage of PN-potentials raises concerns: (i) increasing the number of pairwise constraints decreases computational efficiency, (ii) enforcing consistency in a soft manner requires tuning of an additional parameter c, (iii) large values of c reduce convergence, and (iv) large values of c result in corresponding beliefs being assigned zero probability mass which is not desirable.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Lloyd's algorithm can be parallelized in the MapReduce framework (Zhao et al., 2009) or even replaced by fast stochastic optimization techniques such as online or mini-batch k-Means (Bottou & Bengio, 1994;Sculley, 2010).
Sentence 2: the seeding step requires k inherently sequential passes through the data, making it impractical even for moderate k. This highlights the need for a fast and scalable seeding algorithm.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In essence, these methods consider explicit factorization of a coupled tensor T ∈ R n1×n2×n3 and a matrix , respectively, with a common rank R and shared components a i , i = 1, . . . , R. Many variations of factorization models for coupled completion models have been proposed based on CP decomposition with shared and unshared components (Acar et al., 2014), Tucker decomposition (Ermis et al., 2015), and non-negative factorization (Ermis et al., 2015).
Sentence 2: due to factorization, these coupled completion models are non-convex that lead to local optimal solutions.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: @ , 3 © ¡ ¦¢ at the same initial belief states used in [12], but without using them to tailor the policy.
Sentence 2: pBVI achieved expected returns of ¡ © ¡ ¨ § , 3 © § ¦ and 3 © ¨ in ¦ ¢ )¦ , ¡ ¨¢ ¨¢ ©¦ and ¢ ©3 ©¦ with policies of ¤£ 3 , ¢ ¢ and ¥ § linear segments tailored to those initial belief states.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Note that p∈P f x (p) − q∈S w q f x (q) does not equal p∈P cost(p, x) − q∈S w q cost(q, x).
Sentence 2: it equals the difference between p∈P cost(p, x) and a weighted cost of the sampled points and the centers in the approximation solution.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Concretely, for decomposing an m×n matrix, say with m ≤ n, the best specialized implementations (typically first-order methods) have a per-iteration complexity of O m 2 n , and require O(1/ ) number of iterations to achieve an error of .
Sentence 2: the usual PCA, which carries out a rankr approximation of the input matrix, has O(rmn) complexity per iteration -drastically smaller when r is much smaller than m, n. Moreover, PCA requires exponentially fewer iterations for convergence: an accuracy is achieved with only O (log(1/ )) iterations (assuming constant gap in singular values).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: 2 While Theorem 1 is positive for finding variable settings that satisfy sentences, unsatisfiable sentences remain problematic when we are unsure that there exists γ > 0 or if we have an incorrect setting of f . We are unaware of an efficient method to determine all y ij for visited nodes in proofs of unsatisfiable sentences.
Sentence 2: we expect that similar substructures will exist in satisfiable and unsatisfiable sentences resulting from the same application.
Relationship: contrasting