sadat2307/MSciNLI huggingface dataset:
Domain: NeurIPS
Sentence 1: We can try to more quantitatively describe the correctness of the inpainting procedure using as metrics the discriminator score and the leastsquares rigid-body alignment error with respect to the true native structure.
Sentence 2: the GAN solutions are found by explicitly trying to optimize the discriminator score; hence the corresponding score will be artificially inflated relative to the baselines.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: If neither solution satisfies c T α ≥ (δ Var( Ŷ )) 0.5 , by the contrapositive of Theorem 2, we can be sure that the model does not contain any proxies.
Sentence 2: the converse does not hold, i.e., we cannot be sure that the model has a proxy even if a solution to Problem 2 satisfies c T α ≥ (δ Var( Ŷ )) 0.5 .
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The evolution of s (l) with l tells us the ability of the average network to separate two input points in terms of Euclidean distance.
Sentence 2: again, for tanh and ↵-ReLU (↵ < 1) nonlinearities, s varies only polynomially with l. (l) measures the size of gradient at layer l, and through it we track the dynamics of gradient backpropagation, be it explosion or vanishing.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: However, the number of mixture components in the output Y space must be pre-specified and the number of parameters is linear in the number of mixture components.
Sentence 2: with N h stochastic hidden nodes, SFNNs can use its distributed representation to model up to 2 N h mixture components in the output Y . SFNNs contain binary stochastic hidden variables h ∈ {0, 1} N h , where N h is the number of hidden nodes.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: If K is correctly specified, then the variance components parameters (τ, σ 2 ) can be estimated unbiasedly by maximizing the Restricted Maximum Likelihood (REML) [12]: where V = τ K + σ 2 I, and 1 a n × 1 vector whose all elements are 1.
Sentence 2: it is worth noting that REML is a model-based procedure.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The reason is that the combination of words in this query is likely to activate one or more latent variables related to the concept of presidential campaigns.
Sentence 2: once this generalization is made the model has "lost" the information about the specific word Camejo and it will only show up in highly ranked documents if this word happens to frequently occur in these topics (unlikely in this case given that this candidate received relatively little media coverage compared to the coverage given to the candidates from the two main parties).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: One can show that which is a set of smooth vector fields on φ(R n ) in which the vector field at each point φ(x) displaces φ(x) infinitesimally.
Sentence 2: since φ is a diffeomorphism, we have that φ(R n ) = R n . we write v : φ(R n ) → R n to emphasize that the velocity fields in the tangent space are defined on the deformed domain, so that v is a Eulerian velocity.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Had we directly taken the first D s = 8 columns from the permuted data matrix, we would obtain the same non-zero entries as in K 1 and K 2 , if we exclude elements in K 1 and K 2 whose IDs > D s = 8. in this example, the element 10{8} in sketch K 1 is excluded.
Sentence 2: since the columns are already permuted, any D s columns constitute a random sample of size D s . This means, by only looking at sketches K 1 and K 2 , one can obtain a "random" sample of size D s . By statistics theory, one can easily obtain an unbiased estimate of any linear summary statistics from a random sample.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: We also introduces the marginal distributions Observe incidentally that the probabilistic framework of the continuous ranking problem is quite similar to that of distribution-free regression.
Sentence 2: as shall be seen in the subsequent analysis, even if the regression function m(x) = E[Y | X = x] can be optimal under appropriate conditions, just like for regression, measuring ranking performance involves criteria that are of different nature than the expected least square error and plug-in rules may not be relevant for the goal pursued here, as depicted by Fig.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: A naive implementation of the CPF-AS algorithm will give rise to O(T 4 ) computational complexity, since at each time step t = 1, . . . , T , a matrix of size T × T needs to be factorized.
Sentence 2: it is possible to update and reuse the factors from the previous time step, bringing the total computational complexity down to the familiar O(T 3 ).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: These result minimizes the • 2 error, while our result minimizes the Frobenius norm, which is always higher, and may be higher by a factor of d. After appropriate weighting, we can apply the uniform sampling of size O(k/ε 2 ) to get a coreset with a small Frobenius error [14], as in our paper.
Sentence 2: in this case the probability of success is only constant.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: If the dynamics of the gradient method are such that the iterates remain in some bounded domain (or at least have a sufficiently slowly increasing norm), then this would not be an issue.
Sentence 2: we are not a-priori guaranteed that this would be the case: Since the optimization problem is unconstrained, and we are not assuming anything on the structure of f θ , it could be that the parameters w, V diverge, and no meaningful algorithmic result can be derived from Thm.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Variants of this problem have been studied in the context of applications to numerical linear algebra [15], graphical model learning [27], nonrigid structure from motion [16], spectral estimation and Prony's problem [11], sparse PCA [29], blind source separation [28], dictionary learning [24], graphical model learning [3], and sparse coding on manifolds [21].
Sentence 2: in contrast to the standard sparse regression problem (Ax = b, b = 0), for which convex relaxations perform nearly optimally for broad classes of designs A [14,18], the computational properties of problem (1) are not nearly as well understood.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Specifically, we compute ↵ i,j for patch i as the mean over all pixels p in that patch of the following quantity: Where Ov(p) denotes the set of features for positive patches that contain the pixel p. It is admittedly difficult to analyze how well these heuristics approximate the adaptive bandwidth approach of [4], and even there the setting of the bandwidth for each datapoint has heuristic aspects.
Sentence 2: empirically our approach leads to improvements in performance as discussed below, and suggests a potential area for future work.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: It is well known that this type of regularization implies a prior assumption about the distribution of the parameter (e.g., L 1 -regularization implicitly assumes a Laplacian prior distribution) [12].
Sentence 2: strong sparsity-inducing regularization, which is common when there are many more potential features than data samples (i.e., the so-called small n/p regime) can severely hinder the interpretation of model parameters (Fig.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: (2008) it is obvious that the constant γ in (5) can be found in analytical form when S = {k}.
Sentence 2: we kindly warn the reader again that the proofs of these last three cited articles (Kozachenko and Leonenko, 1987;Goria et al., 2005;Leonenko et al., 2008) contain a few errors, just like the Wang et al.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Sparse learning is an essential topic in machine learning, which has attracted considerable interests recently.
Sentence 2: it can be shown that the solution of the L 0 regularization problem in (1) achieves good prediction accuracy if the target function can be approximated by a sparse w. a fundamental difficulty with this method is the computational cost, because the number of subsets of {1, . . . , d} of cardinality k (corresponding to the nonzero components of w) is exponential in k. Due to the computational difficult, in practice, it is necessary to replace (1) by some easier to solve formulations below: where λ > 0 is an appropriately chosen regularization condition.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: . . , (x m , y m ) of m examples, where Note that by upper bounding ζ Q , we are taking into account all moments of W Q .
Sentence 2: the PAC-Bayes theorem [2,3,4,5] currently only upper bounds the first moment 1 When H is a continuous set, Q(h) denotes a density and the summations over h are replaced by integrals.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: BP is based on a convex relaxation of the ℓ 0 norm, i.e., replacing w 0 with w 1 , which leads to an attractive, unimodal optimization problem that can be readily solved via linear programming.
Sentence 2: oMP is a greedy strategy that iteratively selects the basis vector most aligned with the current signal residual.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: R | sup x2X |h(x)|  1}, and the Wasserstein distance (also known as the Kantorovich-Rubenstein or earth mover's distance), d W k•k , generated by The primary impediment to adopting an IPM as a sample quality measure is that exact computation is typically infeasible when generic integration under P is intractable.
Sentence 2: we could skirt this intractability by focusing on classes of test functions with known expectation under P . For example, if we consider only test functions h for which E P [h(Z)] = 0, then the IPM value d H (Q, P ) is the solution of an optimization problem depending on Q alone.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Acknowledging that attribute classifiers are often unreliable, recent work abandons purely semantic attributes in favor of discovering mid-level features that are both detectable and discriminative for a set of class labels [11,22,26,15,30,27,1].
Sentence 2: there is no guarantee that the discovered features will align with semantic properties, particularly "nameable" ones.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In the first glance, since (2) has only one variable, one may think it is easier to solve (2) than ( 1), or at least (2) can be solved by directly utilizing efficient algorithms developed for nonsymmetric NMF.
Sentence 2: the state-of-the-art alternating based algorithms (such as ANLS and HALS) for nonsymmetric NMF utilize the splitting property of (1) and thus can not be used for (2).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: A related setting is adaptive submodularity [7,2,6,13], which is a probabilistic setting that essentially requires that the conditional expectation over the hypothesis set of submodular functions is itself a submodular function.
Sentence 2: we require that the hypothesis class be pointwise submodular (i.e., each hypothesis corresponds to a different submodular utility function).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: We would like to model P (r A , w A ), the joint probability of observing an image defined by r A together with annotation words w B .
Sentence 2: we hypothesize that the observation {r A , w B } came from the same process that generated one of the images J * in the training set T . we don't know which process that was, and so we compute an expectation over all images J∈T .
Relationship: contrasting
Domain: NeurIPS
Sentence 1: One can show that, providing ε + and ε − are respectively concave and convex functions of g, then the loss (4) is convex in g. is an admissible estimate of P (Y = 1|x).
Sentence 2: the relaxed loss (4) is optimistic, below the neg-log-likelihood of g. This optimism usually L(g(x),1) L(g(x),1) Figure 2: Losses for positive examples (plain) and neg-log-likelihood of g(x) (dotted) vs. g(x).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Our framework is similar in spirit to the label embedding methods based on reducing the label space dimensionality [10,21,25].
Sentence 2: our framework offers the following key advantages: (1) computational cost of training our model scales in the number of ones in the label matrix, which makes our framework easily scale in cases where the label matrix is massive but sparse; (2) our likelihood model for the binary labels, based on a Bernoulli-Poisson link, more realistically models the extreme sparsity of the label matrix as compared to the commonly employed logistic/probit link; and (3) our model is more interpretable -embeddings naturally correspond to topics where each topic is a distribution over labels.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Following Oja's work, many multineuron circuits were proposed to extract multiple principal components of the input, for a review see [3].
Sentence 2: most multineuron algorithms did not meet the same level of rigor and biological plausibility as the single-neuron algorithm [2,4] which can be derived using a normative approach, from a principled objective function [5], and contains only lo-cal Hebbian learning rules.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In particular, the FID score [22], the mode score [23] and the Inception score [24] use hidden features/labels of a pretrained CNN classifier as φ, and measure the performance of generative modeling algorithms by comparing p(z) and q(z) under this projection.
Sentence 2: because we want to study the exact difference between p(z) and q(z), we choose φ to be interpretable high level features inspired by experimental work in cognitive psychology, e.g.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Under mild conditions, PCP guarantees that the two components X and E can be exactly recovered through solving: To solve the problem in (1), iterative optimization methods such as Accelerated Proximal Gradient (APG) [15] or Augmented Lagrangian Multiplier (ALM) [14] methods are often used.
Sentence 2: these optimization methods are implemented in a batch manner.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For instance, spectral mixture kernels (Wilson & Adams, 2013b) were introduced for flexible kernel learning and extrapolation, by modelling a spectral density with a scale-location mixture of Gaussians, with promising results.
Sentence 2: wilson & Adams (2013b) specify the number of mixture components by hand, and do not characterize uncertainty over the mixture hyperparameters.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Given a matrix A ∈ R n×m , the linear operator vec(A) yields a vector x ∈ R nm , which is obtained by stacking the columns of A.
Sentence 2: given a vector x ∈ R nm , the operator unvec (n,m) (x) yields a matrix A ∈ R n×m .
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The previous section shows how to optimize the label predictors F while the nodes N , edges E and label sets L which specify the structure of the tree are fixed in advance.
Sentence 2: we want to be able to learn specific tree structures dependent on our prediction problem such that we minimize the Algorithm 2 Learning the Label Tree Structure Train k One-vs-Rest classifiers f1 , .
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Our method differs from them in following aspects: 1) DA-GAN aims to synthesize photorealistic and identity preserving profile faces to address the large variance issue in unconstrained face recognition, whereas TP-GAN ( 13) tries to recover a frontal face from a profile view and Apple GAN ( 28) is designed for much simpler scenarios (e.g., eye and hand image refinement); 2) TP-GAN ( 13) and Apple GAN (28) suffer from categorical information loss which limits their effectiveness in promoting recognition performance.
Sentence 2: our proposed DA-GAN architecture effectively overcomes this issue by introducing dual discriminator agents.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Probably most closely related to Wasserstein++ is VAE-GAN [38], combining VAE [24] with vanilla GAN [14].
Sentence 2: whereas the VAE part and the GAN part minimize different divergences (Kullback-Leibler and Jensen-Shannon in the case of VAE and vanilla GAN, respectively), WAE and WGAN minimize the same cost function, so Wasserstein++ is somewhat more principled conceptually.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Here, we tackle the following question: "Given the row and column marginals of a hidden binary matrix H, what can one infer about H?".
Sentence 2: optimization-based methods for addressing this question, e.g., least squares or maximum likelihood, assume a generative model for the hidden matrix and output an estimate H of H. this estimate gives little guidance as to the structure of the feasible solution space; for example, H may be one of many solutions that achieve the same value of the objective function.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The reconstruction of z is then produced by a (symbol) decoder D : [L] m → R d , which maps the symbols back to ẑ = D(E(z)) ∈ R d . Since z is a sample from Z, the symbol stream E(z) is drawn from the discrete probability distribution P E(Z) .
Sentence 2: thus, given the encoder E, according to Shannon's source coding theorem [8], the correct metric for compressibility is the entropy of E(Z): Our generic goal is hence to optimize the rate distortion trade-off between the expected loss and the entropy of E(Z): min where F is the architecture where z has been replaced with ẑ, and β > 0 controls the trade-off between compressibility of z and the distortion it imposes on F . we cannot optimize (3) directly.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Marginals computed with Gibbs sampling (run for about 10 8 iterations) are clearly very inaccurate (far from the 45 degree line), an indication that the Markov Chain had not mixed as an effect of the relatively large weights that tend to create barriers between modes which are hard to traverse.
Sentence 2: samples from PAWS provide much more accurate marginals, in part because it does not rely on local search and hence is not directly affected by the energy landscape (with respect to the Hamming metric).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The total number of triangles, over all 4 classes, is Θ(N 3 ).
Sentence 2: our goal is not to account for all 4 classes; instead, we will focus on ∆ 3 and ∆ 2 while ignoring ∆ 1 and ∆ 0 .
Relationship: contrasting
Domain: NeurIPS
Sentence 1: It is impossible to do this literature justice here, but some examples include [10,7,12,8,11,5,4,9].
Sentence 2: most of this work addresses only very specific online learning methods, and usually quite simple ones (such as ones that switch between only a finite number of parameter values using a specific, simple, decision rule, e.g., [4]).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In the Logistics domain, specifically, it is easy to see that the probability of successfully loading a container onto a truck using robots of k (1 ≤ k ≤ m) different manufacturers is (1 − 0.7 k ).
Sentence 2: robots of all five manufacturers are used in a plan when ρ = 0.4, whereas using those of three manufacturers is enough.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In [7,18] the authors show that coordinate descent methods are very efficient for solving lasso type problems.
Sentence 2: an obvious way to update each element of ∆ to solve for the Newton direction in (5) needs O(p 2 ) floating point operations since Q := W t ⊗W t is a p 2 ×p 2 matrix, thus yielding an O(p 4 ) procedure for approximating the Newton direction.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Let F i be the σ-field generated by all the random variables until episode k i , step h. Then, τ i=1 is a martingale difference sequence w.r.t the filtration {F i } i≥0 .
Sentence 2: by Azuma-Hoeffding and a union bound, we have that with probability at least 1 − p/(SAH): ) for some absolute constant c. Because inequality (4.4) holds for all fixed τ ∈ [K] uniformly, it also holds for Putting everything together, and using a union bound, we see that with least 1 − p probability, the following holds simultaneously for all (x, a, h, k) if we choose b t = c H 3 ι/t for the same constant c in Eq.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: If sampling were involved in the update rule the learning rate would have to be decreased in the standard manner (Lt at = 00, Lt a; < (0) in order to fulfil the condition for stochastic approximation algorithms.
Sentence 2: for a fixed set of updates and certain synchronous RL algorithms with linear feature-based function approximation Theorem 1 predicts the existence of a constant learning rate.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Recall that the learning problem is to select the functions f α ∈ F α so as to minimize the empirical risk At first blush, this appears challenging, since evaluating A(θ) requires solving a message-passing optimization.
Sentence 2: we can use the dual representation of A from Theorem 2 to represent min F R(F ) in the form To optimize Eq.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For the whole network, RNP allocated more computations on faces images than background images, at approximately a ratio of 2, which clearly demonstrates the effectiveness of RNP.
Sentence 2: since the first convolutional layers and fully connected layers were not pruned, to get the absolute ratio of pruning rate, we also studied the pruning of a certain convolutional layer.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For the p-value, the statement dir L (Y ), µ = 0 is implied by the stronger null hypothesis µ ⊥ L. We can also use Lemma 1 to give a two-sided confidence interval for dir L (Y ), µ ; specifically, dir L (Y ), µ lies in the interval [L α/2 , L 1−α/2 ] with probability 1 − α.
Sentence 2: in general this cannot be extended to a two-sided interval for P L µ 2 .
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Recall that the conditional mutual information I(X i ; X j |X S ) = 0 iff given X S , the random variables X i and X j are conditionally independent.
Sentence 2: thus, (11) seeks to identify non-neighbors, i.e., node pairs which can be separated in the unknown graph G. since we constrain the conditioning set |S| ≤ η in (11), the optimal conditioning set may not form an exact separator.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: While HMC is motivated by smooth distributions, the inclusion of an acceptance probability means HMC does asymptotically sample correctly from piecewise distributions 1 .
Sentence 2: since leapfrog numerical integration of Hamiltonian dynamics (see [9]) relies on the assumption that the corresponding potential energy is smooth, such cases lead to high rejection probabilities, and poor performance.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Upon the completion of the action, the actor transfers to a state s t+1 and receives a reward r t . As we are interested in learning complex motor tasks consisting of a single stroke [23], we focus on finite horizons of length T with episodic restarts [1] and learn the optimal parametrized, stochastic policy for such reinforcement learning problems.
Sentence 2: we assume an explorative version of the dynamic motor primitives [22,23] as parametrized policy π with parameters θ ∈ R n . in this section, we will keep most derivations sufficiently general that they would transfer to various other parametrized policies.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For symmetric matrices, B ≤ α ⇐⇒ |λ l | ≤ α ∀l, where λ l are the eigenvalues of B.
Sentence 2: letting i p denote the unit vector along the p-th coordinate of R n , the distance between the p, q-th rows of f (S) can be written as Similarly, we have that Thus pairwise distances between the rows of f L (S) approximate those between the rows of E. the distortion term δ √ 2 is additive and must be controlled by carefully choosing f L (•), as discussed in Section 4.
Relationship: contrasting