sadat2307/MSciNLI huggingface dataset:
Domain: NeurIPS
Sentence 1: We refer to u, π, φ as global parameters because they generalize to new data sequences.
Sentence 2: the states z n are local to a specific sequence x n . We seek a distribution q over the unobserved variables that is close to the true posterior, but lies in the simpler factorized family q(•) q(u)q(φ)q(π)q(z).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Given A ∈ R n×d and y, the goal of SLR is to recover a sparse vector x that minimizes Ax − y The unconditional version of sparse regression can be shown to be NP-hard via a reduction to 3-set cover [14].
Sentence 2: the problem has been studied heavily under a variety of assumptions such as incoherence [7], null-space property [8], restricted isometry property (RIP) or restricted strong convexity (RSC) [4,15].
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In all cases, cascading the models is able to improve the robust error performance, sometimes substantially, for instance decreasing the robust error on CIFAR10 from 46.1% to 36.4% for = 2/255.
Sentence 2: this comes at a cost as well: the nominal error increases throughout the cascade (this is to be expected, since the cascade essentially tries to force the robust and nominal errors to match).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: We see from the graphical representation in Figure 2 that z n is d-separated from x n . Thus when y n is observed, the posterior process is updated by using p (y n |f n ).
Sentence 2: when the data point is unlabeled the posterior process must be updated by p (z n |f n ) which is easily computed as: The "effective likelihood function" for a single data point, L (f n ), therefore takes one of three forms: The constraint imposed by (1) implies that an unlabeled data point never comes from the class y n = 0.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In practice, such information is not often available.
Sentence 2: we can replace E m with a data-dependent local Rademacher complexity bound Êm , as in [7], which also satisfies (1), and satisfies (with high probability) Êm ≤ c ′ E m , for some constant c ′ ∈ [1, ∞) (see [13]).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For i = 1, 2, we define D i := (e f i , h i , x i,0 ) 2 T (C, C r ), and Note that, introducing the exponential discounting e is a way to construct a mathematically valid kernel to compare dynamical systems.
Sentence 2: in a certain situation, this method does not work effectively.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Clearly, the original mixing matrix could not be reconstructed in any of the experiments.
Sentence 2: interestingly, the latter two algorithms do indeed find an ISA up to permutation, which will be explained in section 3. theoretically speaking, it may only be applied to random vectors following the k-ISA blind source separation model, which means that they have to be mixtures of a random vector that consists of independent groups of size k. If this is the case, uniqueness up to permutation and scaling holds as noted above; however if k-ISA is applied to any random vector, a decomposition into groups that are only 'as independent as possible' cannot be unique and depends on the contrast and the algorithm.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: As the appearance features typically take on real values, we first apply k-means clustering to the image features {a i } to build a visual vocabulary V. Thus a feature a i in the appearance space A can be represented as a visual word v, and we have While the topic prediction model in Equation 1 is able to capture regularly co-occurring patterns in the joint space of label and appearance, it ignores spatial priors on the label prediction.
Sentence 2: i z spatial priors, such as spatial smoothness, are crucial to labeling tasks, as neighboring labels are usually strongly correlated.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: We also study a convex relaxation for sparse SVD, based on the DSPCA algorithm proposed by [11] that encourages the singular vectors of the matrix to be supported over a sparse set of variables.
Sentence 2: despite the increasing popularity of this method, we show that it is only guaranteed to yield a sparse set of singular vectors when the SNR is quite high, equivalent to element-wise thresholding, and fails for stronger scalings of the SNR.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Since the edges at each step are drawn iid given W , the graph sequence is edge exchangeable.
Sentence 2: the corresponding graph measure G = i,j n ij δ (i,j) (where ) is not Kallenberg exchangeable, since the probability of generating edge {i, j} is directly related to the positions (i, j) and (j, i) in R 2 + of the corresponding atoms in G (in particular, the probability is decreasing in ij).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For our intended optimization routine, we are actually looking for the minimizer of an expected loss function L * (θ) = E x∈D L (θ, x).
Sentence 2: in the parallel setting, we cannot investigate L * directly and we may only analyze it through L N . calculating the total loss function directly and exactly requires excessive inter-processor communication, which carries a huge computational burden as the number of processors increase.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: If one is interested in classification accuracy alone, without requiring deterministic guarantees, then online to batch conversion techniques combined with stochastic subgradient descent are a good choice [17].
Sentence 2: while the dependence on is still Ω(1/ ) or worse [18], one gets bounds independent of n. as we pointed out earlier, these algorithms are applicable only when the empirical risk decomposes over the examples.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For problem of medium scale, ( 6) is not easier to solve than a linear system due to non-negative constraints, and thus an ALM is not preferred to IPM in the traditional sense.
Sentence 2: for large-scale problem with m × n nnz(A), the ALM becomes advantageous since: (i) the conditioning of (6) does not worsen over iterations, and thus allows iterative methods to solve it approximately in time proportional to O(nnz(A)).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: To summarize, the best known upper bound for optimizing with intermittent communication using a pure stochastic oracle is (20), which combines two different algorithms.
Sentence 2: with active oracle accesses, SVRG is also possible and the upper bound becomes: Our main contributions in this paper are: (1) presenting a precise formal oracle framework for studying parallel stochastic optimization; (2) establishing tight oracle lower bounds in this framework that can then be easily applied to particular instances of parallel optimization; and (3) using the framework to study specific settings, obtaining optimality guarantees, understanding where additional assumptions would be needed to break barriers, and, perhaps most importantly, identifying gaps in our understanding that highlight possibilities for algorithmic improvement.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The magnitude of this factor is less trivial than it seems: since the algorithm increases the threshold by b t , it includes more hypotheses in the version space, which may cause the algorithm to query a lot more.
Sentence 2: our analysis shows that the number of queries only increases by O(tb t ln t), i.e., ln t times the total number of ⊥'s received over t examples.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: A standard technique for obtaining a mistake bound with respect to fairness violations would be to play a "halving algorithm", which would always act as if the unknown metric is at the center of the current version space (the set of metrics consistent with the feedback observed thus far) -so that mistakes necessarily remove a non-trivial fraction of the version space, making progress.
Sentence 2: a standard technique for obtaining a diminishing regret bound is to play "optimistically" -i.e.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: 2 argmin F . The most popular algorithm for solving (1) is Stochastic Gradient Descent (SGD), which achieves statistically optimal O(1/ p N ) suboptimality in O(N ) time and constant memory.
Sentence 2: in modern large-scale machine learning problems the number of data points N is often gigantic, and so even the linear time-complexity of SGD becomes onerous.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For instance, [3,10] use regular subsampling and [9,14,18] use random sampling based on compressed sensing.
Sentence 2: they remain difficult to scale computationally, especially for large s and n. In this paper, we are interested in learning polynomials with s = O(2 δn ) for some δ ∈ (0, 1).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Since J determines the information rate completely, we conclude that for any weakly-coupled network there is an equivalent Poisson network which conveys exactly the same information in the intermediate regime.
Sentence 2: note that the the sufficient statistic ∆(r) is different in the Poisson and non-Poisson settings, since the f /f term linearly reweights the observed spikes, depending on how likely they were given the history; thus the optimal Bayesian decoder incorporates non-Poisson effects explicitly.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In neuroscience, the analysis of electrophysiological time-series data is largely centered around dynamic causal modeling (DCM) [5], where continuous state-space models are formulated based on differential equations that are specifically crafted around knowledge of underlying neurobiological processes.
Sentence 2: dCM is not suitable for exploratory analysis of data, such as inferring unknown arousal levels, for two reasons: because the differential equations are driven by inputs of experimental conditions, and the analysis is dependent on a priori hypotheses about which neuronal populations and interactions are important.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Thus, the probability that an random element of À Ò is in a ball of radius « centered at one of ½ AE is at least AE´ ½ «µ Ò ½ .
Sentence 2: since each ½ AE has distance at most « from , any element of an « ball centered at one of them is at most « • « far from .
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Various first-order optimization algorithms such as gradient descent [42], stochastic gradient descent [27] and more recently variance-reduced stochastic gradient descent [46,3] have been proposed and analyzed for solving (1.1).
Sentence 2: all these algorithms are only guaranteed to converge to a stationary point, which can be a local minimum, a local maximum, or even a saddle point.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: On this trial, and the other "interval" trials, the concept learning model is certain that the correct concept is of the form numbers between m and n, because the observed examples rule out all the other concepts.
Sentence 2: the model is not certain of the precise endpoints, such as whether the concept is numbers between 16 and 23, or numbers between 16 and 24, etc.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: To tackle this challenge, we need to develop an appropriate machinery to check whether the value of c is already large enough for SVRG to decrease the objective value.
Sentence 2: we cannot afford the cost for computing the objective value due to large n. To this end, we develop a "certificate" that can be easily verified and can act as signal for a sufficient decrease in the objective value.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The amount of this shifting is also known to the intended receiver of the message, who can decode the message by shifting back each letter in the encrypted message.
Sentence 2: caesar cipher could also be broken by an unintended receiver (not knowing the shift) when it analyzes the frequencies of the letters in the encrypted messages and matches them up with the letter distribution of the original text [4, pp.9-11].
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Especially, PLML outputs multiple metrics which are similar to each other (tends to global learner's behavior) and only focus on first part of the alphabet, while MMLMNN by default only learns multiple metrics with the number of base metrics equaling to the number of classes.
Sentence 2: results of UM 2 L OVS clearly demonstrate all 3 fields.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: As with any algorithm, there is a trade-off: DFacTo uses 3 times more memory than the Tensor Toolbox, since it needs to store 3 flattened matrices as opposed to a single tensor.
Sentence 2: in return, our algorithm only requires two sparse matrix-vector multiplications, making DFacTo easy to implement using any standard sparse linear algebra library.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: It was also observed that applying APPA/Catalyst reductions to SVRG one can achieve accelerated convergence rates [13,29], and this approach is commonly known as AccSVRG.
Sentence 2: accSVRG requires some careful parameter tuning of its inner loops, and thus is a logarithmic-factor slower than KatyushaX and also less practical [3].
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Many techniques have been proposed to speed up the softmax layer in training, such as hierarchical softmax [20] and sampling-based approaches [21][22][23][24].
Sentence 2: most of them cannot be directly applied to decoding at inference because they rely on knowing the words to be predicted and they still need to calculate the probability of all words to find the most likely prediction.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: This statistic is based on the spectral analysis of the graph, and in particular on the smallest non-zero eigenvalue of the Laplacian matrix, which is known as Fiedler value [6,7].
Sentence 2: we use the Fiedler delta statistic to define a Boltzmann distribution over graphs, leading to the Fiedler random field (FRF) model.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: From a theoretical point of view, Sparse PCA is one of the simplest examples where we observe a gap between the number of samples needed information theoretically and the number of samples needed for a polynomial time estimator: It has been well understood [39,40,41] that information theoretically, given n = O(k log p) samples 1 , one can estimate v up to constant error (in euclidean norm), using a non-convex (therefore not polynomial time) optimization algorithm.
Sentence 2: all the existing provable polynomial time algorithms [36,42,34,43], which use either diagonal thresholding (for the single spiked model) or semidefinite programming (for general covariance), first introduced for this problem in [44], need at least quadratically many samples to solve the problem, namely n = O(k 2 ).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: It is possible that the doubling trick or the analysis of Auer et al [2] (who encountered the same problem for the Exponential Weights algorithm) can remove the extra log T factor while still preserving the regret minimization property.
Sentence 2: we also remark that R max is small when R 1 is instead the p-norm; hence, the use of this regularizer avoids the extraneous logarithmic in T factor while still preserving the logarithmic dependence on n and m. projection onto the simplex under the p-norm is not as elegant as the Exponential Weights update.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: All the aforementioned algorithms essentially demonstrate that a signal of dimension n can be perfectly recovered through m > Cn amplitude-only measurements, where C > 1 is a constant that depends on the algorithm as well as the measurement vectors.
Sentence 2: many interesting signals in practice contain fewer degrees of freedom than the ambient dimension (sparse signals, low-rank matrices, finite alphabet signals, etc.).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: 1 If we could evaluate P (Z (1:i−1) , Y|X (1:i−1) ), we could obtain weighted samples (or "particles") from P (Z (1:i) , Y|X (1:i) ) using importance sampling with a proposal distribution of and taking w ℓ ∝ P (X (i) |Z as the weight associated with the ℓ th particle.
Sentence 2: we could also use a similar scheme to approximate P (Z (1:i−1) , Y|X (1:i−1) ) if we could evaluate P (Z (1:i−2) , Y|X (1:i−2) ).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: That is, if the loss on example z t is zero then w t+1 = w t . In case the loss is positive the update rule is w t+1 = w t + τ t v t where v t is the same as in the realizable case.
Sentence 2: the scaling factor τ t is modified and is set to, The following theorem provides a loss bound for the online algorithm relative to the loss of any fixed weight vector w .  The proof of the theorem is based on a reduction to the realizable case (cf.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: We can similarly identify each graph with a bit string by "unraveling" the adjacency matrix into a long string of zeros and ones.
Sentence 2: if we know beforehand an ordering on the nodes of our graph to which all directed graphs must stay consistent (to enforce acyclicness), then there are only n 2 possible edges, and the size of our metagraph drops to 2 ( n 2 ) .
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Denoting J \k = {x j } j =k and M \k = L −1 J \k the full conditional can be simplified using Schur's determinantal equality [22]: In general, sampling directly from this full conditional is difficult.
Sentence 2: for a wide range of kernel functions, including those which can be handled by the Nyström approximation in Sec.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: the same minimizer x minimizes all components f i (x) (though of course it need not be a unique minimizer of any of them); this yields linear convergence to x , with a graceful degradation as σ 2 > 0.
Sentence 2: in the linear convergence regime, the number of required iterations scales with the expected squared conditioning EL 2 i /µ 2 .
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Even in such a case, the MLE can take all possible patterns (2 10 = 1024) into account through the normalization term log Z θ ≃ Const + 1 2 ||θ|| 2 that works like a regularizer.
Sentence 2: the proposed method genuinely uses only the observed examples, and the asymptotic analysis would not be relevant in this case.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In the extreme case where Xl = X 2 , the two cells are completely redundant and provide the same information about the stimulus, yielding RSpairs = I(Xl' X 2 ; S) -I(Xl ; S) -I(X2 ; S) = -I(Xl; S), which is always non-positive.
Sentence 2: positive RSpairs values testify for synergistic interaction between Xl and X 2 ([8, 7, 4]).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Without loss of generality, assume that the instances are ordered in the sequence {x 11  We index instances by a function Non-convex optimizations are undesirable because few algorithms effectively converge even to a local optimum.
Sentence 2: if both objective function and constraints take the form of a difference between two convex functions, then a non-convex problem can be solved efficiently by the constrained concave-convex procedure [14].
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Probabilistic graph matching [2] makes the rank-1 approximation by assuming that each second-order element of M ia;jb is the product of the probability of feature i being matched to a and feature j being matched to b, independently.
Sentence 2: instead of maximizing the quadratic score function, they use this probabilistic interpretation of the pair-wise terms and find the solution by looking for the closest rank-1 matrix to M in terms of the KL-divergence.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The bounds shown there depend on the SVD spectrum of a matrix.
Sentence 2: if we consider a random matrix, then a much stronger lower bound can be obtained with high probability: Corollary 4 For a random matrix M ∈ {−1, 1} n×n and 0 < r < n, almost certainly the minimum number of sign-non-preserving changes to a matrix in R n×n that is needed to reduce the rank of the matrix to r is at least Note that the rigidity bound given in [FS06] also applies to our problem, if we use the Hadamard matrix as the problem matrix.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: To circumvent the doublesampling problem, these papers proposed multi-time-scale stochastic approximation algorithms, the former a policy gradient algorithm and the latter an actor-critic algorithm that uses simultaneous perturbation methods [Bhatnagar et al., 2013].
Sentence 2: as discussed in Section 2.1, multi-time-scale stochastic approximation approach suffers from several weaknesses such as no available finite-sample analysis and difficult-to-tune stepsize schedules.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Simulation results also suggest that No Learning agents have a significantly lower belief of stationarity than Expert learners, but not different from WSLS.
Sentence 2: the comparison of model accuracy between R-W and DBM (Fig 3d ) shows DBM outperforms R-W in predicting Expert and WSLS behavior (p = .000), but it does not perform as well in No Learning behavior where R-W has significantly better performance (p = .000).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: By reformulating the problem in Lagrange form, [8] proposed the spectrum Lasso along with an iterative convex minimization algorithm.
Sentence 2: ( 2) is difficult to analyze when the sample fraction π 0 = n/(d 1 d 2 ) is small, due to the ill-posedness of the quadratic term n i=1 M 2 ωi .
Relationship: contrasting
Domain: NeurIPS
Sentence 1: [n] k instead, we have This shows that, when using mini-batches of appropriate size, the sparsification of the gradient does not hurt convergence.
Sentence 2: by increasing the mini-batch size, we increase the computation by a factor of d k . These two observations seem to indicate that the factor d k is inevitably lost, either by increased number of iterations or increased computation.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: By using t he rela tion (2.4) , it follows that R,, (8 ) -L* = Op( nl / 2 ) and R(f 0) -L * = Op (nl ).5 Therefore, t he TOP kernel h as a much b etter convergen ce rate in R(f 0)' which is a strong motiva tion to use the TOP kernel instead of plug-in estimate.
Sentence 2: 5Fo r detail ed disc ussion a bout t he conve rgence orders of classificatio n e rror, see C ha pte r we must notice that this fast rate is possible only when the optimal linear classifier is combined with the TOP kernel.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The determinantal point process (DPP) has recently emerged as a powerful method for selecting a diverse subset from a "ground set" of items [13], with applications including document summarization [14] and information retrieval [15].
Sentence 2: existing DPP techniques have a fatal modeling flaw if applied to video (or documents) for summarization: they fail to capture their inherent sequential nature.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: We observe that when more samples are revealed, both OR-PCA and GRASTA steadily improve the subspace recovery.
Sentence 2: our proposed OR-PCA converges much faster than GRASTA, possibly because in each iteration OR-PCA obtains the optimal closed-form solution to the basis updating subproblem while GRASTA only takes one gradient descent step.
Relationship: contrasting