sadat2307/MSciNLI huggingface dataset:
Domain: NeurIPS
Sentence 1: Moreover, we note that counting the number of possible outputs of ERM also has connections to a counting argument made in [1] in the context of pricing mechanisms.
Sentence 2: in essence the argument there is restricted to transductive settings where the sample "features" are known in advance and fixed and thereby the argument is much more straightforward and more similar to standard notions of "effective hypothesis space" used in VC-dimension arguments.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In this case, it is rigorously known [21,22,23] that detection is impossible by any algorithm for the SBM with a group structure weaker than where ϵ ≡ ρ out /ρ in .
Sentence 2: it is important to note that this is the information-theoretic limit, and the achievable limit for a specific algorithm may not coincide with Eq.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For the variational scale parameters, ln g s , we see that early on the HVP+Local approximation is able to reduce parameter variance by a large factor (≈ 2,000×).
Sentence 2: at later iterates the HVP+Local scale parameter variance is on par with the Monte Carlo estimator, while the full Hessian estimator still enjoys huge variance reduction.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Many researchers attempted to solve goal-oriented dialog tasks by using the deep supervised learning (deep SL) approach [8] based on seq2seq models [9] or the deep reinforcement learning (deep RL) approach utilizing rewards obtained from the result of the dialog [10,11].
Sentence 2: these methods struggle to find a competent RNN model that uses back-propagation, owing to the complexity of learning a series of sentences.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Ψ(S) > δ, and its support x S is smaller or equal to min(n 1 , n 2 ), then all children S of S, which satisfy S ⊂ S by construction of the enumeration tree, will also be non-testable and can be pruned from the search space.
Sentence 2: such a monotonicity property does not hold for the CMH minimum attainable p-value function Ψ cmh (S), severely complicating the development of an effective pruning criterion.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: . . , f4 by minimizing the Kullback-Leibler (KL) divergence between the product of Q \a and f a and the product of Q \a and fa , where Q \a is the ratio between Q and fa .
Sentence 2: this does not perform well for refining f2 ; details on this problem can be found in Section 4 of the supplementary material and in [19].
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Recovering X * from Q * As for linear programming, recovering a primal optimal solution directly from a dual optimal solution is not always possible for SDPs.
Sentence 2: at least for the hard-margin problem (no slack) this is possible, and we describe below how an optimal prediction matrix X * can be recovered from a dual optimal solution Q * by calculating a singular value decomposition and solving linear equations.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Obviously, if we knew for each policy which subchains it induces on M (the MDP's ergodic structure), UCRL could choose an MDP Mt and a policy πt that maximizes the reward among all plausible MDPs with the given ergodic structure.
Sentence 2: only the empiric ergodic structure (based on the observations so far) is known.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: To compare the training cost, if the time cost of the related task (regression or classification) with M features is C(M ), LKRF and EERF simply spend that budget.
Sentence 2: running K iterations of our method (with M a multiple integer of K), assuming that repetitive features are not selected, the training cost of MFGA would be P K k=1 C(kM/K), which is more than LKRF and EERF.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: ⇤ As in the proof of Theorem 1, the main idea is to show that one can design two distributions that are indistinguishable to a learner who can observe no more than d 1 attributes of any sample given by the distribution (i.e., that their marginals over any choice of d 1 attributes are identical), but whose respective sets of "-optimal regressors are disjoint.
Sentence 2: in contrast to Theorem 1, both handling general d along with switching to the absolute loss introduce additional complexities to the proof that require different techniques.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Due to the square loss in RPCA, the sparse matrix S can be calculated by subtracting the low-rank matrix L from the observed data matrix.
Sentence 2: nevertheless, in LVGGM, there is no closed-form solution for the sparse matrix due to the log-determinant term, and we need to use gradient descent to update S. both the algorithm in [40] and our algorithm have an initialization stage.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Although for certain GLM, e.g., sparse logistic regression, we can choose the step size parameter as ⌘ = 4⇤ 1 max ( 1 n P n i=1 x i x > i ), such a step size often leads to poor empirical performance.
Sentence 2: as our theoretical analysis and experiments suggest, the proposed DC proximal Newton algorithm needs very few line search steps, which saves much computational effort.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: There have thus been various important suggestions for the functional significance of synaptic depression, including -just to name a few -low-pass filtering of inputs [3], rendering postsynaptic responses insensitive to the absolute intensity of presynaptic activity [4,5], and decorrelating input spike sequences [6].
Sentence 2: important though they must be for select neural systems, these suggestions have a piecemeal flavor -for instance, chaining together stages of low-pass filtering would lead to trivial responding.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Simultaneous-move games can be solved exactly in polynomial time using the backward induction algorithm [7,4], recently improved with alpha-beta pruning [8,9].
Sentence 2: the depth-limited search algorithms based on the backward induction require domain knowledge (an evaluation function) and computing the cutoff conditions requires linear programming [8] or using a double-oracle method [9], both of which are computationally expensive.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The most elegant optical implementation of adaptive interconnection is through dynamic volume holography [6,11], but that requires a set of coherent optical signals, not what we have with an array of pulse emitters.
Sentence 2: the matrix-vector multiplier architecture allows parallel interconnection of incoherent optical signals, and has been used to demonstrate implementations of the Hopfield model [7] and Boltzman machines [9].
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The main difference between our works is the use of a linear scaling of the learning rate 9 , similarly to Krizhevsky (2014), and as suggested by Bottou (2010).
Sentence 2: we found that linear scaling works less well on CIFAR10, and later work found that linear scaling rules work less well for other architectures on ImageNet (You et al., 2017).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For any δ > 0, the sequence This theorem can be easily generalized to hold for values of ε i that are not all equal (as done in [KOV15]).
Sentence 2: this is not as all-encompassing as it would appear at first blush, because this straightforward generalization would not allow for the values of ε i and δ i to be chosen adaptively by the data analyst.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: We choose the step size η to guarantee a descent on the objective.
Sentence 2: the gradient of the approximated structured prediction program in Theorem 1 with respect to θ r equals to if either and/or c α equal zero, then the beliefs b x,y,α (ŷ α ) can be taken from the set of probability distributions over support of the max-beliefs, namely b x,y,α (ŷ * α ) > 0 only if ŷ * α ∈ argmax ŷα r:α∈Er,x θ r φ r,α (x, ŷα ) + v∈N (α) λ x,y,v→α (ŷ α ) .
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The key insight when proving this theorem is bounding rate of change of f . We can immediately see that f π (θ) := θ log E s,π e J/θ is a convex function since it is the perspective transformation of a convex function, namely, the cumulant generating function of the total cost J. Additionally, Theorem 1 shows that f π is lower bounded by E s,π [J], assumed to be finite, which implies that f π is non-increasing.
Sentence 2: by directly differentiating the definition of f π , we get that θf π (θ Since we assumed that the costs, J, are upper bounded, there exist a maximum cost j M such that J ≤ j M almost surely for any starting state s, and any policy π.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: First, at an 6 All the code is available at https://github.com/sanghack81/SCMMAB-NIPS2018 7 One may surmise that combinatorial bandit (CB) algorithms can be used to solve SCM-MAB instances by noting that an intervention can be encoded as a binary vector, where each dimension in the vector corresponds to intervening on a single variable with a specific value.
Sentence 2: the two settings invoke a very different set of assumptions, which makes their solvers somewhat difficult to compare in some reasonably fair way.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Measures of linear dependence and feedback, based on Granger-Geweke causality (GC) [10] [11]), have been used to estimate instantaneous and lagged functional connectivity in recordings of brain activity made with electroencephalography (EEG, [6]), and electrocorticography (ECoG, [3]).
Sentence 2: the application of GC measures to brain recordings made with functional magnetic resonance imaging (fMRI) remains controversial [22][20] [2].
Relationship: contrasting
Domain: NeurIPS
Sentence 1: A straightforward version of this idea would be to perform shape completion in the 3D voxel grid: f 2.5D→3D = c 3D→3D • p 2.5D→3D .
Sentence 2: shape completion in 3D is challenging, as the manifold of plausible shapes is sparser in 3D than in 2D, and empirically this fails to reconstruct shapes well.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Our choice of g is motivated by computational considerations: For general functions g, the computation of ( 14) would require O(m) time per function evaluation, where m is the number of documents.
Sentence 2: the specific functional form in (14) allows O(1) time per function evaluation as , where the inner term j =i S k (x i , x j ) in the RHS does not depend on w and can be precomputed.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: SO-DLT in particular is a good example of direct adaptation of standard batch deep learning methodology to online learning, as it uses SGD during tracking to fine-tune an ensemble of deep convolutional networks.
Sentence 2: the online adaptation of the model comes at a big computational cost and affects the speed of the method, which runs at 5 frames-persecond (FPS) on a GPU.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The pushforward density p l can then be approximated by the empirical measure given by the particles {x l i } n i=1 , where The first term in (10) corresponds to a weighted average steepest descent direction of the log-target density π with respect to p l . This term is responsible for transporting particles towards highprobability regions of π.
Sentence 2: the second term can be viewed as a "repulsion force" that spreads the particles along the support of π, preventing them from collapsing around the mode of π.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: This is why application of ICA to EEG recently has become popular, yielding new promising results (e.g., [6]).
Sentence 2: compared with ICA, the sparse representation has two important advantages: 1) sources are not assumed to be mutually independent as in ICA, even be not stationary; 2) source number can be larger than the number of sensors.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The span of an MDP is the maximum di erence in value of any two states under the optimal policy (M ú ) := max The diameter of an MDP is the maximum number of expected timesteps to get between any two states D(M ú ) = max s" =s Õ min µ T µ saes Õ . PSRL's bounds are tighter since (M ) AE CD(M ) and may be exponentially smaller.
Sentence 2: uCRL-Factored has stronger probabilistic guarantees than PSRL since its bounds hold with high probability for any MDP M ú not just in expectation.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Particularly the performance obtained with very small numbers of labels is much better than previous published results which shows that the method is capable of making good use of unsupervised learning.
Sentence 2: the same model also achieves state-of-the-art results and a significant improvement over the baseline model with full labels in permutation invariant MNIST classification which suggests that the unsupervised task does not disturb supervised learning.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: As expected, the naive method strongly under-estimated correlations in the non-simultaneously recorded blocks, as it can only model stimulus-correlations but not noise-correlations across neurons.
Sentence 2: 1 our stitching method predicted correlations well, matching those of the fully observed model (correlation 0.84 for stitchLDS, 0.15 for naiveLDS, figure 3d).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In this general framework, the observations are assumed to follow an exponential family distribution, with natural parameter related to a conditionally Gaussian dynamic model [5], via a nonlinear transformation.
Sentence 2: these model specifications may still be too restrictive in practice, for the following reasons: (i) Observations are usually discrete, non-negative and with a massive number of zero values and, unfortunately, far from any standard parametric distributions (e.g., multinomial, Poisson, negative binomial and even their zero-inflated variants).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Given sufficiently small / large constants c T , c H , and , it is easy to see that the linear convergence implied by Theorem 9 directly gives the recovery guarantee and bound on the number of iterations stated in Theorem 5 (see Appendix A.1).
Sentence 2: in some cases it might not be possible to design approximation algorithms with constants c T and c H sufficiently close to 1 (in constrast, increasing the sample complexity by a constant factor in order to improve is usually a direct consequence of the RIP guarantee or similar statistical regularity assumptions).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Unlike [18], we also emphasize on the ability of the methods in recovering the number of motions.
Sentence 2: although the methods compared in [18] (except RANSAC) theoretically have the means to do so, their estimation of the number of motions is generally unrealiable and the benchmark results in [18] were obtained by revealing the actual number of motions to the algorithms.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In the figure, as the number of sum nodes goes up, the accuracy of the standard sum-product based estimation (sum) gets better, whereas the accuracy of standard max-product based estimation (max) worsens.
Sentence 2: our hybrid messagepassing algorithm (hybrid), on an average, results in the lowest loss compared to the other baselines, with running times similar to the sum/max product algorithms.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: We find our method generally performs the best, followed with the parallel SGLD, which is much better than its sequential counterpart; this comparison is of course in favor of parallel SGLD, since each iteration of it requires n = 100 times of likelihood evaluations compared with sequential SGLD.
Sentence 2: by leveraging the matrix operation in MATLAB, we find that each iteration of parallel SGLD is only 3 times more expensive than sequential SGLD.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: As a result, even when the teacher network is shallow, the student network usually needs to be deeper, otherwise it will underfit.
Sentence 2: both our theorem and our experiment show that if the shallow teacher network is in a pretty large region near identity (Figure 2), SGD always converges to the global minimum by initializing the weights I + W in this region, with equally shallow student network.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Theoretical understanding of the strength of depth starts from analyzing the depth efficiency, by proving the existence of deep neural networks that cannot be realized by any shallow network whose size is exponentially larger.
Sentence 2: we argue that even for a comprehensive understanding of the depth itself, one needs to study the dual problem of width efficiency: Because, if we switch the role of depth and width in the depth efficiency theorems and the resulting statements remain true, then width would have the same power as depth for the expressiveness, at least in theory.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The LHS of (10), by the assumption of the theorem, is at most ε 2 implying (9).
Sentence 2: note that our anlaysis of the Fujishige-Wolfe algorithm is weaker than the best known method in terms of time complexity (IO method by [11]) on two counts: a) dependence on n, b) dependence on F . we found this algorithm significantly outperforming the IO algorithm empirically -we show two plots here.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For example, if we take n = 100000, L = 100, and µ = 0.01 then the basic FG method has a rate of ((L − µ)/(L + µ)) 2 = 0.9996 and the 'optimal' AFG method has a faster rate of (1 − µ/L) = 0.9900.
Sentence 2: running n iterations of SAG has a much faster rate of (1 − 1/8n) n = 0.8825 using the same number of evaluations of f i .
Relationship: contrasting
Domain: NeurIPS
Sentence 1: We note that when classifying a new point one does not have to project it to the subspace V , but rather assign a sign according to the classifying hyperplane in R d . In first sight it seems that our result gives a week generalization bound since the margin obtained in the original dimension is low.
Sentence 2: the margin of the found solution in the reduced dimension (i.e., within V ) is almost optimal (i.e.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: . . , β k ) and let θ * β be the minimizer of is zero for even a single outlier (x, y), then L(θ * β , ∞) will be infinite.
Sentence 2: we can bound θ * β under an alternative loss that is less sensitive to outliers: The key idea in the proof is that replacing S with exp(β ψ) in p θ,β does not change the loss too much, in the sense that When β min 1, Hence, the error increases roughly linearly with β −1 min .
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For example the nearest neighbor classifier in the new distance is equivalent to the Lipschitz regularization (1) weighted with the density proposed in the last section.
Sentence 2: implementing such a method requires to compute the geodesic distance in (R d , g), which is non trivial for arbitrary densities p. We suggest the following approximation which is similar in spirit to the approach in [11].
Relationship: contrasting
Domain: NeurIPS
Sentence 1: To address this issue, one may be tempted to use the weak * -topology on 2 , since in this topology the closed balls are both compact and metrizable, thus universal kernels do exist on them.
Sentence 2: the Taylor kernels do not belong to them, because -basically-the inner product • , • 2 fails to be continuous with respect to the weak * -topology as the sequence of the standard orthonormal basis vectors show.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: From this equation we obtain, Using lemma 1 we have an upper bound on the probability that sup {t1,...,tm} | Îob − Îun | > ǫ over the random selection of features, as a function of m ′ .
Sentence 2: the upper bound we need is on the probability that sup {t1,...,tm} |E( Îob ) − E( Îun )| > ǫ 1 .
Relationship: contrasting
Domain: NeurIPS
Sentence 1: If an estimated value consists of a large portion of such transitions, then the likelihoods of overestimation and underestimation are both very low.
Sentence 2: if backward transition probability p j,j (or any p j,k with k ≤ j) is close to 1, then Var[ Dj ] increases dramatically, resulting a noticeable skewness.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The second intuition is to use the online least squares formulation of the linear equation Ax = b.
Sentence 2: since A is not symmetric and positive semi-definite (PSD), A 1 2 does not exist and thus Ax = b cannot be reformulated as Another possible idea is to attempt to find an objective function whose gradient is exactly A t x t − b t and thus the regularized gradient is prox αth(xt) (A t x t − b t ).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: 5, but we allow the distribution of the standardized variable to depend (at least weakly) on v. We now generalize the reparameterization idea to distributions that, like the gamma or the beta, do not admit the standard reparameterization trick.
Sentence 2: we assume that we can efficiently sample from the variational distribution q.zI v/, and that q.zI v/ is differentiable with respect to z and v. We introduce a random variable defined by an invertible transformation where we can think of D T 1 .zI v/ as a standardization procedure that attempts to make the distribution of weakly dependent on the variational parameters v. "Weakly" means that at least its first moment does not depend on v. For instance, if is defined to have zero mean, then its first moment has become independent of v. we do not assume that the resulting distribution of is completely independent of the variational parameters v, and therefore we write it as q . I v/.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Our results indicate that in finite dimensions, an efficient, compression-based, Bayes-consistent multiclass 1-NN algorithm exists, and hence can be offered as an alternative to k-NN, which is well known to be Bayes-consistent in finite dimensions [12,41].
Sentence 2: in infinite dimensions, our results show that the condition characterizing the Bayes-consistency of k-NN does not extend to all NN algorithms.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: On the surface, our network appears similar to AC-GAN [24], where the only difference is the separation of the classifier network from the authenticator network.
Sentence 2: this crucial modularisation enables our DA algorithm to replace GANs by other generative models that may become available in the future; likewise, we can use the most sophisticated classification models for C. Furthermore, unlike our model, the classification subnetwork introduced in AC-GAN mainly aims for improving the quality of synthesized samples, rather than for classification tasks.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In most previous studies of MAB and linear stochastic bandits, a common assumption is that noises in observed payoffs are sub-Gaussian conditional on historical information (Abbasi-Yadkori et al., 2011;Bubeck et al., 2012), which encompasses cases of all bounded payoffs and many unbounded payoffs, e.g., payoffs of an arm following a Gaussian distribution.
Sentence 2: there do exist practical scenarios of non-sub-Gaussian noises in observed payoffs for sequential decisions, such as highprobability extreme returns in investments for financial markets (Cont and Bouchaud, 2000) and fluctuations of neural oscillations (Roberts et al., 2015), which are called heavy-tailed noises.
Relationship: contrasting