sadat2307/MSciNLI huggingface dataset:
Domain: NeurIPS
Sentence 1: For GMM, it is still possible to use a matched linear averaging which matches the mixture components of the different local models by minimizing a symmetric KL divergence; the same idea can be used on our linear control variates method to make it applicable to GMM.
Sentence 2: because the parameters of PPCA-based models are unidentifiable up to arbitrary orthonormal transforms, linear averaging and linear control variates can no longer be applied easily.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For the convergence of this instance of the algorithm, it is required that all the states and actions are visited infinitely many times, which makes the analysis slightly more complicated.
Sentence 2: given a generative model, the algorithm may be also formulated in a synchronous fashion, in which we first generate a next state y ∼ P (•|x, a) for each state-action pair (x, a), and then update the action-values of all the stateaction pairs using these samples.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Nevertheless, the emphasis has been on asymptotic analysis, characterizing the rates of convergence of test statistics under null hypotheses, as the number of samples tends to infinity.
Sentence 2: we wish to study the following problem in the small sample regime: ⇧(C, "): Given a family of distributions C, some " > 0, and sample access to an unknown distribution p over a discrete support, how many samples are required to distinguish between p 2 C versus d TV (p, C) > "?
Relationship: contrasting
Domain: NeurIPS
Sentence 1: SVGD has been applied to solve challenging inference problems in various domains; examples include Bayesian inference Feng et al., 2017), uncertainty quantification (Zhu & Zabaras, 2018), reinforcement learning (Liu et al., 2017;Haarnoja et al., 2017), learning deep probabilistic models Pu et al., 2017) and Bayesian meta learning (Feng et al., 2017;Kim et al., 2018).
Sentence 2: the theoretical properties of SVGD are still largely unexplored.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: However, unlike them, it provides a principled framework for full Bayesian inference and can be used to determine how to trade off goodness-of-fit across summary statistics.
Sentence 2: to the best of our knowledge, this potential has not been realised yet, and ABC approaches are not used for linking mechanistic models of neural dynamics with experimental data (for an exception, see [17]).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Thus, the solution above is communication-efficient only when λ is relatively large.
Sentence 2: the situation immediately improves if we can use a without-replacement version of SVRG, which can easily be simulated with randomly partitioned data: The stochastic batches can now be simply subsets of each machine's data, which are statistically identical to sampling {f 1 (•), .
Relationship: contrasting
Domain: NeurIPS
Sentence 1: !, AQM-gen1Q-depA achieves a slight performance improvement over AQM-countQ-depA at 2-q (49.79% → 51.07%) outperforming the original deep SL method (46.8% in 5-q).
Sentence 2: at 5-q, AQM-gen1Q performs slightly worse than AQM-countQ-depA (72.89% → 70.74%).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The framework presented here has some similarities with the very interesting and more explicitly physiological model proposed by Buonomano and colleagues [5,18], in which time is implicitly encoded in deterministic 2 neural networks through slow neuronal time constants.
Sentence 2: temporal information in the network model is lost when there are stimulus-independent fluctuations in the network activity, and the network can only be used as a reliable timer when it starts from a fixed resting state, and if the stimulus is identical on every trial.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Additive noising introduces a product-form penalty depending on both and A 00 .
Sentence 2: the full potential of artificial feature noising only emerges with dropout, which allows the penalty terms due to and A 00 to interact in a non-trivial way through the design matrix X (except for linear regression, in which all the noising schemes we consider collapse to ridge regression).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In standard GP applications, one has access to a single realisation of data y, and performs kernel learning by optimizing the marginal likelihood of the data with respect to covariance function hyperparameters θ (supplement).
Sentence 2: with only a single realisation of data we are highly constrained in our ability to learn an expressive kernel function -requiring us to make strong assumptions, such as RBF covariances, to extract useful information from the data.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: To map the complex state h into a real output o r , we use a linear combination of the real and imaginary components, similar to [1], with W o and b o as weights and offset: In [1], it was proven that a unitary 4 W would prevent vanishing and exploding gradients of the cost function C with respect to h t , since the gradient magnitude is bounded.
Sentence 2: this proof hinges on the assumption that the derivative of f a is also unity.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In [45], it was mentioned that any continuously differentiable value function (VF) can be approximated by increasing the number of independent basis functions to infinity in CT scenarios, and a CT policy iteration was proposed.
Sentence 2: without resorting to the theory of reproducing kernels [3], determining the number of basis functions and selecting the suitable basis function class cannot be performed systematically in general.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: With heterogeneous costs, the quality of an action k under a context j is roughly captured by its normalized expected reward, defined as η j,k = u j,k /c j,k .
Sentence 2: the agent cannot only focus on the "best" action, i.e., k * j = arg max k∈A η j,k , for context j.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: To indicate the form of a representer theorem, suppose we solve for the optimal parameters Θ * = arg min Θ 1 n n i L(x i , y i , Θ) + g(||Θ||) for some nondecreasing g. We would then like our pre-activation predictions Φ(x t , Θ) to have the decomposition: Given such a representer theorem, α i k(x t , x i ) can be seen as the contribution of the training data x i on the testing prediction Φ(x t , Θ).
Sentence 2: such representer theorems have only been developed for non-parametric predictors, specifically where Φ lies in a reproducing kernel Hilbert space.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Another approach is to pick from (m 1 , m 2 ) before reaching the factor f , so that the model becomes In this case, the message from x to f is not blurred, and the upward messages to (m 1 , m 2 ) are blurred, which is correct.
Sentence 2: the downward messages from (m 1 , m 2 ) to f are blurred before reaching f , which is incorrect.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: It is worth pointing out that, similar to the MMTM, our PTM is not a generative model of networks per se because (a) empty and single-edge motifs are not modeled, and (b) one can generate a set of triangles that does not correspond to any network, because the generative process does not force overlapping triangles to have consistent edge values.
Sentence 2: given a bag of triangular motifs E extracted from a network, the above procedure defines a valid probabilistic model p(E | α, λ) and we can legitimately use it for performing posterior inference p(s, θ, B | E, α, λ).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Overall both algorithms LINKAGE++ and Single-linkage perform considerably better when it comes to real-world data and LINKAGE++ and PCA+ dominate on our synthetic datasets.
Sentence 2: in general there is no reason to believe that PCA+ would perform well in clustering truly hierarchical data: there are regimes of the HSBM for which applying only phase 1 of the algorithm might lead to a high missclassification error and high cost and for which we can prove that LINKAGE++ is an p1`εq-approximation.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Critically, this is where we diverge from previous analyses that assumed this distribution was factorised, or only trivially correlated due to reciprocal synapses being precisely (anti-)symmetric [1,2,4].
Sentence 2: we explicitly study the emergence and effects of non-trivial correlations in the synaptic weight matrixdistribtion, because almost all synaptic plasticity rules induce statistical dependencies between the synaptic weights of each neuron (Fig.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Previous GP-based classifiers did not use f within a margin-based classifier as in (6), implemented here via p(u n ) = N (−λ n , γ −1 λ n ), where u n = 1−y n f n . It has been shown empirically that nonlinear SVMs and GP classifiers often perform similarly [8].
Sentence 2: for the latter, inference can be challenging due to the non-conjugacy of multivariate normal distribution to the link function.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: When the target distribution is not Gibbs, we demonstrate that the second approach need not produce the optimal Gibbs distribution (with respect to log loss) even in the limit of infinitely many samples.
Sentence 2: we prove that it produces models that are almost as good as the best Gibbs distribution according to a certain Bregman divergence that depends on the selection bias.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Nguyen et al [23], Singh and Póczos [24], and Krishnamurthy et al [25] each proposed divergence estimators that achieve the parametric convergence rate (O 1 T ) under weaker conditions than those given in [1].
Sentence 2: solving the convex problem of [23] can be more demanding for large sample sizes than the estimator given in [1] which depends only on simple density plug-in estimates and an offline convex optimization problem.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For an input image I and a given pair of kernels, we can measure the data log-likelihood by associating each window with the maximum likelihood kernel: We search for a blurring model p k0 such that, when combined with the model p 1 (derivatives of the unblurred image), will maximize the log-likelihood of the observed derivatives: One problem we need to address in defining the likelihoods is the fact that uniform areas, or areas with pure horizontal edges (the aperture problem) don't contain any information about the blur.
Sentence 2: uniform areas receive the highest likelihoods from wide blur kernels (since the derivatives distribution for wide kernels is more concentrated around zero, as can be observed in figure 1(c)).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Subsequently, for any x * i ∈ [0, 1]: If x * i is the expected allocation of player i under the efficient allocation rule X * i (v) ≡ 1{v i = max j v j }, then taking expectation of Equation ( 9) over v i and adding across all players we get: The theorem then follows by invoking the fact that for any feasible allocation x: , using the fact that expected total agent utility plus total revenue at equilibrium is equal to expected welfare at equilibrium and setting µ = µ(D).
Sentence 2: comparison with worst-case POA In the worst-case, µ(D) is upper bounded by 1, leading to the well-known worst-case price of anarchy ratio of the single-item first price auction of (1 − 1/e) −1 , irrespective of the bid distribution D. if we know the distribution D then we can explicitly estimate µ, which can lead to a much better ratio (see Figure 1).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In order to encourage the reinforcement learning agent to discover positive symptoms more quickly, a simple heuristic is to provide the agent with an auxiliary piece of reward when a positive symptom is queried, and a relatively smaller (or even negative) reward when a negative symptom is queried.
Sentence 2: this heuristic suffers from the risk of changing the optimal policy of the original MDP.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Note that degenerate cluster allocations are generally suboptimal under the objective (1), as they would lead to a reduction in the marginal entropy H(y).
Sentence 2: it is intuitive that maximization of the mutual information I(x, y) favors hard assignments of cluster labels to equiprobable data regions, as this would result in the growth in H(y) and reduction in H(y|x).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Return MDP assessment (π τ , T τ ) and This learning process is not guaranteed to converge, so upon termination, it could return an optimal, δ-stable MDP assessment for some very large δ.
Sentence 2: it has been shown to be successful experimentally in simultaneous auction games [24] and other large games of imperfect information [7].
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Recall d β (x, x ′ ) from ( 1) and consider If the above value is finite, then using any value even slightly larger than this would declare any x ′ ̸ ∈ S(x) correctly as such, hence no false positives.
Sentence 2: the above infimum may not exist for a general configuration.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For instance, regression with a decomposable kernel boils down to solving a Sylvester equation (which can be done efficiently) [10] and vector-valued Support Vector Machine (SVM) without intercept can be learned with a coordinate descent algorithm [21].
Sentence 2: these methods can not be used in our setting since the loss function is different and considering the intercept is necessary for the quantile property.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Ideally one should design convex relaxations for each domain of Θ.
Sentence 2: m exhibits some nice properties for any Θ: M 0, M I, tr(M ) = tr((ΘΘ ) † (ΘΘ )) = rank(ΘΘ ) = rank(Θ).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: A similar situation was also observed in the server with 86G when n = 10 × 2 10 .
Sentence 2: the memory required by BADMM is O(n 2 )-even when n = 15 × 2 10 (more than 0.2 billion parameters), BADMM can still run on a single GPU with only 5G memory.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Figure 2 shows that OLS leads to accurate ATE estimation for Gaussian additive noise when the number of covariates is sufficiently large, which is consistent with Corollary 1.1.
Sentence 2: for high dimensional data, matrix factorization preprocessing dominates all other feasible methods and its RMSE is very close to the oracle regression for sufficiently large number of covariates.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For sufficiently smooth objectives, the same algorithm is also optimal even if prox access is allowed, since Theorem 2 implies a lower bound of: That is, for smooth objectives, having access to a prox oracle does not improve the optimal complexity over just using gradient access.
Sentence 2: for non-smooth or insufficiently smooth objectives, there is a gap between ( 11) and ( 12).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Note that while the set of rank-L PSD matrices is non-convex, we can still project onto this set efficiently using the eigenvalue decomposition of where r = min( L, L + M ) and L + M is the number of positive eigenvalues of M . Λ M (1 : r) denotes the top-r eigenvalues of M and U M (1 : r) denotes the corresponding eigenvectors.
Sentence 2: while the above update restricts the rank of all intermediate iterates M t to be at most L, computing rank-L eigenvalue decomposition can still be fairly expensive for large n. by using special structure in the update (6), one can significantly reduce eigenvalue decomposition's computation complexity as well.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The normalization To convert an input matrix X 2 R p (p = mp 0 ) into a score vector t 2 R m , it seems that we need to learn a matrix W 2 R m⇥mp0 .
Sentence 2: a natural permutation invariance requirement (if the documents associated are presented in a permuted fashion, the output scores should also get permuted in the same way) reduces the dimensionality of W to p 0 (see, e.g., [14] for more details).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Finite differencing, where the perturbation directions d are the n standard basis vectors, is a default approach for Jacobian estimation.
Sentence 2: it requires n function evaluations which may be prohibitively expensive for large n. Another natural approach, when the number of measurements, say k, is smaller than n, is to estimate the Jacobian via linear regression, where an l 2 regularizer is added to handle the underdetermined setting and • F stands for the Frobenius norm.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For example, according to the LWF Markov property, in the chain graph model in Figure 1(a), x 1 ⊥ x 3 |x 2 as x 1 and x 3 are separated by x 2 in the moralized graph in Figure 1(b).
Sentence 2: the corresponding AMP Markov property implies a different probabilistic independence relationship, x 1 ⊥ x 3 .
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The straightforward approach is the well-known on-line K-means clustering algorithm, in which only the nearest reference vector to the sample vector is adjusted; G § )H IQP R S H ( T % U FV C# W4 6 § "H X# 8 (2) where, P is the step size and S H ¡ is the Kronecker delta.
Sentence 2: this simple clustering algorithm is often stuck in a local minimum.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Moreover, an efficient training algorithm (EM) is available, as well as an efficient decoding algorithm (Viterbi), which provides the optimal sequence of states (and the corresponding sequence of high level events) associated with a given sequence of low-level data.
Sentence 2: multimodal information processing is currently a very challenging framework of applications including multimodal person authentication, multimodal speech recognition, multimodal event analyzers, etc.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: [8] that the lower bounds of sample complexity of TOPK and MB problems are ⌦ H log(1/ ) , for the cases of Gaussian reward distributions.
Sentence 2: for general decision classes with non-constant widths, we see that there is a gap of ⇥(width(M) 2 ) between the upper bound Eq.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: By standard Bayesian arguments, one can show that the optimal estimator of z to minimize the bit-wise error rate is given by ẑi = arg max Note that the EM algorithm (1), which maximizes rather than marginalizes q j , is not equivalent to the Bayesian estimator (3), and hence is expected to be suboptimal in terms of error rate.
Sentence 2: calculating the marginal p(z i |L, θ) in (3) requires integrating all q and summing over all the other z i , a challenging computational task.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Noting that x * ∈ K, g(x * ) ≤ 0, we have F (x * ) ≤ f (x * ) + γ ln 2.
Sentence 2: f ( x T ) = f ( x T ) + γ ln 1 + exp λ 0 g( x T ) γ ≥ f ( x T ) + max (0, λ 0 g( x T )) .
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Indeed, we were not able to directly generalize the analysis in [24] by simply adapting the techniques used in recent online boosting work [4,2], but we made use of the classical Frank-Wolfe algorithm [8].
Sentence 2: while an important part of the convergence analysis for the batch setting is to show statistical consistency of the algorithms [24,1,22], in the online setting we only need to study the empirical convergence (that is, the regret), which makes our analysis much more concise.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Finally, classifiers such as SVMs use both positive and negative examples and provide state-of-theart performance when used with appropriate kernels [5,6,7].
Sentence 2: these classifiers still require an auxiliary method (such as PSI-BLAST) to handle unlabeled data: one generally adds predicted homologs of the positive training examples to the training set before training the classifier.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The independent work Neon by Xu and Yang [31] is actually the first recorded theoretical result that proposed this approach.
Sentence 2: neon needs O(1/δ 3 ) stochastic gradients, because it uses full gradient descent to find NC (on a sub-sampled objective) inspired by power method and [16]; instead, Neon2 online uses stochastic gradients and is based on the recent result of Oja's algorithm [7].
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Unfortunately, computing p(v i = w|h) to sample the words during Gibbs sampling is linear in V and H, where V tends to be quite large.
Sentence 2: fortunately, given h, it needs to be computed only once before sampling all D words in v. when h is re-sampled, p(v i = w|h) must be recomputed.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: These settings were theoretically analyzed in a series of works [18,19,23,21] that have demonstrated that information transfer can lead to provable sample complexity reductions compared to solving each task independently.
Sentence 2: all these results rely on Baxter's model of lifelong learning and therefore assume access to the training data for all (observed) tasks and provide guarantees only on average performance over all tasks.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Since invariance is a special case of independent changes, based on PIC, change in one causal module does not force any changes in another causal module, i.e., a change in, say, 2 C|; , will not enforce any changes on E|C or 2 E|C .
Sentence 2: in the reverse direction, as it can be seen from equations in (1) and (2), if any of the variables a, 2 C , and 2 E varies across two domains, by Assumption 1, all three variables 2 E|; , C|E , and 2 C|E will change.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Note that traditional model selection criteria, such as BIC/MDL, aim at fitting the data well and minimizing the representation complexity of the learned model (i.e., the total number of parameters).
Sentence 2: as demonstrated in [2], such criteria are unable to capture the inference complexity: two models that have similar representation complexity and fit data equally well may have quite different graph structures, making one model exponentially slower for inference than the other.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Approximate inference in these models via variational techniques [1,3] and sampling based methods [4,5] has been investigated.
Sentence 2: many real-world problems have neither purely log-submodular nor log-supermodular formulations, but can be naturally expressed in the form P (A) ∝ exp(F (A) − G(A)), where both F (A) and G(A) are submodular functions -we call these types of models mixed PSMs.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: We can think of actor-critic architectures, generalizing to the option-critic architecture as well, as pairing a critic with each actor network so that the critic has additional information about the value of the actor's actions that can be used to improve the actor's learning.
Sentence 2: this is derived by taking gradients with respect to the parameters of the policy while optimizing for the expected discounted return.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: However, when λ/λ max < 0.5, strong rule can not identify any inactive features and thus the running time is almost the same as that of the solver without screening.
Sentence 2: slores is still able to identify more than 80% of the inactive features for the data sets cultured from the Yahoo web pages data sets and thus the efficiency is improved by roughly 5 times.
Relationship: contrasting