sadat2307/MSciNLI huggingface dataset:
Domain: NeurIPS
Sentence 1: In fact, even finding local Nash equilibria is NP-hard in the general non-convex non-concave regime.In addition, as shown by [28, Proposition 10], local Nash equilibria for general non-convex non-concave games may not exist.
Sentence 2: consider the game min Then (0, 0) is the only first-order Nash equilibrium and is not a second-order Nash equilibrium.
Relationship: neutral
Domain: NeurIPS
Sentence 1: This problem is reminiscent of the "class imbalance" problem in discriminatory modeling -the challenge however is exacerbated since there is not a single column to balance and the real data distribution should be kept intact.
Sentence 2: the goal is to resample efficiently in a way that all the categories from discrete attributes are sampled evenly (but not necessary uniformly) during the training process, and to recover the (not-resampled) real data distribution during test.
Relationship: entailment
Domain: NeurIPS
Sentence 1: We investigated two approaches that could enable this flexibility: factorized representations, which abstract away general aspects of a task from those prone to change, and nonparametric, memory-based approaches, which can provide a principled way of using similarity to past experiences to guide current behaviour.
Sentence 2: a number of recent or concurrent papers have proposed algorithms for introducing transfer into RL/deep RL settings, by using multiple policies in some way, though none of them use an inferential framework similar to ours, provides a principled way to deal with unsignalled task boundaries, or explains biological phenomena.
Relationship: neutral
Domain: NeurIPS
Sentence 1: We investigated two approaches that could enable this flexibility: factorized representations, which abstract away general aspects of a task from those prone to change, and nonparametric, memory-based approaches, which can provide a principled way of using similarity to past experiences to guide current behaviour.
Sentence 2: we combine the successor representation (SR), that factors the value of actions into expected outcomes and corresponding rewards, with evaluating task similarity through clustering the space of rewards.
Relationship: entailment
Domain: NeurIPS
Sentence 1: In this paper, we consider a generalization of such existing results by allowing for statistical dependence in  ξ.
Sentence 2: we assume  ξ = { ξj } to be a stochastic process where the marginal random variables  ξ j are conditionally independent and sub-Gaussian given some other (latent) process F = {F j }.
Relationship: entailment
Domain: NeurIPS
Sentence 1: The generalization bounds are also applicable when the model and the data are dependent (e.g., for cross validation or for error estimates based on the training data or the reused test data), but they usually lead to loose error bounds.
Sentence 2: although much tighter bounds are available if the test data and the model are independent, comparing confidence intervals constructed around the training and test error rates leads to an underpowered test for detecting the dependence of a model on the test set.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Thus, the test error rate and the adversarial error estimate (calculated based on the same test set) must be close if the test set and the model are independent, and are expected to be different in the opposite case.
Sentence 2: if the gap between the two error estimates is large, the independence hypothesis (i.e., that the model and the test set are independent) is dubious and will be rejected.
Relationship: entailment
Domain: NeurIPS
Sentence 1: More importantly, since it is based on adversarially generated data points, the adversarial estimator is expected to differ significantly from the test error rate if the model is overfitted to the test set, providing a way to detect test set overfitting.
Sentence 2: the test error rate and the adversarial error estimate (calculated based on the same test set) must be close if the test set and the model are independent, and are expected to be different in the opposite case.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: As a first step to understand whether some heads are universally important, we perform the same ablation study on a second, out-of-domain test set.
Sentence 2: we consider the MNLI "mismatched" validation set for BERT and the MTNT English to French test set (Michel and Neubig, 2018) for the WMT model, both of which have been assembled for the very purpose of providing contrastive, out-of-domain test suites for their respective tasks.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Figure 4 shows that performance drops much more rapidly when heads are pruned from the Enc-Dec attention layers.
Sentence 2: pruning more than 60% of the Enc-Dec attention heads will result in catastrophic performance degradation, while the encoder and decoder self-attention layers can still produce reasonable translations (with BLEU scores around 30) with only 20% of the original attention heads.
Relationship: entailment
Domain: NeurIPS
Sentence 1: In particular, pruning more than 60% of the Enc-Dec attention heads will result in catastrophic performance degradation, while the encoder and decoder self-attention layers can still produce reasonable translations (with BLEU scores around 30) with only 20% of the original attention heads.
Sentence 2: encoder-decoder attention is much more dependent on multi-headedness than self-attention.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Unfortunately, in our case, this projection is quite complicated, since the intersection of the sparsity constraint and the quantization constraint is complicated.
Sentence 2: we notice that the projection onto the feasible set defined by each individual constraint is doable (the projection onto the sparsity constraint is quite standard, how to do efficient projection onto the quantization constraint defined set will be clear soon).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Next, since the target model needs to defend against the attacker, it requires to suppress the worst risk.
Sentence 2: the overall objective for the target model to gain adversarial robustness could be expressed as Z denotes the training data set: As we reviewed previously, typical CNN model compression strategies include pruning (element-level [10], or channel-level [8]), low-rank factorization [16,13,17], and quantization [23,19,20].
Relationship: reasoning
Domain: NeurIPS
Sentence 1: To demonstrate that ATMC achieves remarkably favorable trade-offs between robustness and model compactness, we carefully design experiments on a variety of popular datasets and models as summarized in Section 3.1.
Sentence 2: since no algorithm with exactly the same goal (adversarially robust compression) exists off-the-shelf, we craft various ablation baselines, by sequentially composing different compression strategies with the state-of-the-art adversarial training [32].
Relationship: entailment
Domain: NeurIPS
Sentence 1: As suggested in Figure 2(b), one way of guaranteeing a low failure rate in our method is to collect adequate reference models.
Sentence 2: we know from prior works that even adversarial examples crafted using some single-step attacks like the fast gradient (sign) [18] can transfer [28,22], hence one can hypothesize that the gradients of some "substitute" models are more helpful in spanning the search subspaces with reduced dimensionalities.
Relationship: neutral
Domain: NeurIPS
Sentence 1: We define then smoothness of functions on combinatorial structures to be the smoothness of graph signals using the Graph Fourier Transform (GFT) [35].
Sentence 2: we propose as our GP kernel on the graph a variant of the diffusion kernel, the automatic relevance determination(ARD) diffusion kernel, for which computing the GFT is computationally tractable via a decomposition of the eigensystem.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Louizos et al. [2017] studied scale mixtures of Gaussian priors and half-Cauchy scale priors for the hidden units of VGG models [Simonyan and Zisserman, 2014] and achieved good model compression performance on CIFAR10 [Krizhevsky, 2009] using VI.
Sentence 2: due to the limitation of VI in non-convex optimization, the compression is still not sparse enough and can be further optimized.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Motivated by real-world applications, we assume that functions are smooth, their associated distributions have a bounded second moment, and their minimizer drifts in a bounded manner, but do not assume they are strongly convex.
Sentence 2: our assumptions hold for online Bayesian logistic regression, when the data satisfy natural regularity properties, giving a sampling algorithm with updates that are poly-logarithmic in T .
Relationship: entailment
Domain: NeurIPS
Sentence 1: The N-gram approach constructs a representation vector c (n) for the sentence, whose coordinates correspond to all n-grams and the value of a coordinate is the number of times the corresponding n-gram shows up in the sentence.
Sentence 2: the dimension of an n-gram vector is |V | n for a vocabulary V , and the vector c (1) is just the count vector of the words in the sentence.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: At iteration n, each vertex updates its latent vector by element-wise multiplying it with the sum of the latent vectors of its neighbors.
Sentence 2: at the end of iteration n, the latent vector on vertex i is the sum of the embeddings of the walks that end at i and have length n, and the sum of the all latent vectors is the embedding of the n-gram walk set (with proper scaling).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Deep Tensor Neural Networks (DTNN) [47] and Message-Passing Neural Networks (MPNN) [26] are two graph neural networks that are able to utilize 3D information encoded in the datasets.
Sentence 2: we further compare our method to these two most advanced GNN models, on the two datasets QM8 and QM9 that have 3D information.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Finally as has been done in previous works, e.g. .g. [5, 8–10] for compressed sensing and denoising, translating our results to practical situations in designing an AMP algorithm that takes care of correlated GAN or VAE weights is still under investigatio
Sentence 2: we solve the fixed point equations (12) and plot the MMSE obtained from the fixed point in a heat map, for the linear, sign and relu activations.
Relationship: neutral
Domain: NeurIPS
Sentence 1: In contrast, this paper shows that Step Decay schedules, which cut the learning rate by a constant factor every constant number of epochs (i.e., the learning rate decays geometrically) offer significant improvements over any polynomially decaying step size schedule.
Sentence 2: the behavior of the final iterate with step decay schedules is off from the statistical minimax rate by only log factors (in the condition number for the strongly convex case, and in T in the non-strongly convex case).
Relationship: entailment
Domain: NeurIPS
Sentence 1: In this study, we have demonstrated a surprising level of cross-modal predictive ability across transcriptomic and electrophysiological recordings.
Sentence 2: we showed that the transcriptomic class can be predicted with ∼80% accuracy from electrophysiological recordings when the transcriptomic hierarchy is resolved into 15 classes, and with ∼70% accuracy when it is resolved into 25 classes (λ = 10 results).
Relationship: entailment
Domain: NeurIPS
Sentence 1: Xu et al. [36] and Gleave and Habryka [12] have proposed approaches that combine IRL and gradient-based meta-learning [9], which provide promising results on deriving generalizable reward functions.
Sentence 2: they have been limited to tabular MDPs [36] or settings with provided task distributions [12], which are challenging to gather in real-world applications.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Without further constraints over m, directly applying AIRL to learning the reward function (by augmenting each component of AIRL with an additional context variable m inferred by q Ïˆ ) could simply ignore m, which is similar to the case of InfoGAIL [21].
Sentence 2: some connection between the reward function and the latent context variable m need to be established.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The primary contribution of this paper is a new framework, termed Probabilistic Embeddings for Meta-Inverse Reinforcement Learning (PEMIRL), which enables meta-learning of rewards from unstructured multi-task demonstrations.
Sentence 2: pEMIRL combines and integrates ideas from context-based meta-learning [5,26], deep latent variable generative models [17], and maximum entropy inverse RL [42,41], into a unified graphical model (see Figure 4 in Appendix D) that bridges the gap between few-shot reward inference and learning from unstructured, heterogeneous demonstrations.
Relationship: entailment
Domain: NeurIPS
Sentence 1: The saturating distance curves also show an effective range for λ values; for λ >> 0.2 (not shown), the computations eventually become more unstable, because more and more terms are assigned to the empirical distributions and the topics become too sparse complicating posterior inference.
Sentence 2: the entropies show how the topics become (on average) more sparse for increasing λ.
Relationship: entailment
Domain: NeurIPS
Sentence 1: The K-divergence is always well-defined for all values for qm ∈ ∆; this is especially relevant at the boundaries of ∆.
Sentence 2: the K-divergence (4) is not as sensitive to misses as KL(p m , q m ), which approaches infinity close to the boundaries imposing infinite penalty for misses, essentially, imposing a barrier function.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In contrast to (Globerson et al. 2015, Foster et al. 2018), we study the sufficient conditions for exact recovery in polynomial time, and provide high probability results for general families of undirected connected graphs, which we consider to be a novel result to the best of our knowledge.
Sentence 2: we show that weak-expander graphs (e.g., grids) can be exactly recovered by adding small perturbations (edges coming from the Erdos-Rényi model with small probability).
Relationship: entailment
Domain: NeurIPS
Sentence 1: When keeping more keypoints, poor local maxima starts to get selected for these models (e.g in the sky or the river in Figure 4) and the matching performance drops.
Sentence 2: having numerous keypoints is important for many applications such as visual localization because it augments the chance that at least a few of them will be correctly matched despite occlusions or other noise sources.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Recently, deep generative models have emerged as effective techniques for image animation and video retargeting [2,41,3,42,27,28,37,40,31,21].
Sentence 2: generative Adversarial Networks (GANs) [14] and Variational Auto-Encoders (VAEs) [20] have been used to transfer facial expressions [37] or motion patterns [3] between human subjects in videos.
Relationship: entailment
Domain: NeurIPS
Sentence 1: In generating adversarial examples, the gradient is usually normalized [13,24], such that the direction of the gradient estimator, instead of the magnitude, will affect the performance of attacks.
Sentence 2: we incorporate a scaling factor b in Eq. (7) and minimize the error w.r.t. b, which can neglect the impact of the magnitude on the loss of the estimator gˆ.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: If a voter thinks college V 3 is better than college V 6 , a solid arrowed line from V 3 to V 6 occurs (i.e., superiority).
Sentence 2: when a voter thinks the two colleges (i.e., V 1 and V 3 ) listed are incomparable and difficult to judge, he may click the button "I can't decide", then a dotted line connecting V 1 and V 3 happens (i.e., tie).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Clearly one can see that users jumped out earlier (i.e., the top 10% marked with pink) show larger L 2 -distance, thus are those with large deviation from the population's opinion and can be treated as abnormal users.
Sentence 2: users jumped out later (i.e., the bottom 10% marked with blue) tend to have smaller or even zero L 2 -distance.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In this section, we quantitatively evaluate the effectiveness of the proposed self-critical objective.
Sentence 2: we evaluate the fraction of false sensitivity where the predicted incorrect answer's sensitivity to the influential object (to which the correct answer is most sensitive) is greater than the Figure 4: Positive examples are showing that our self-critical reasoning approach prevents the incorrectly predicted answer in the UpDn baseline system from being sensitive to the most influential object.
Relationship: entailment
Domain: NeurIPS
Sentence 1: The state-of-the-art VQA systems [8,2,1,3,12,33,25,29,14,15,21] achieve high performance when the training and test question-answer (QA) pairs are sampled from the same distribution.
Sentence 2: most of these systems fail to generalize to test data with a substantially different QA distribution.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In order to ensure the training and test data have different category distributions, we intentionally assign different weights to the two components.
Sentence 2: during training, the examples are drawn from N 1 with probability p, and during test, the examples are drawn from N 1 with probability 1 âˆ’ p. We examine the effectiveness of our self-critical approach varying p from 0.05 to 0.5 (i.e. 0.05, 0.1, 0.2, 0.5) (0.5 means no train/test difference).
Relationship: entailment
Domain: NeurIPS
Sentence 1: Acknowledgements: Botond Szabó received funding from the Netherlands Organization for Scientific Research (NWO) under Project number: 639.031.654.
Sentence 2: more concretely, one can sample from the marginal posterior for ψ by drawing a full posterior sample (F, π, m) and computing the corresponding draw Ïˆ according to the formula (2).
Relationship: neutral
Domain: NeurIPS
Sentence 1: Instead of relying on a sparse set of covariant patches, some recent works [20,7,11] propose to extract dense descriptors by feeding the whole image into a convolutional neural network (CNN) and constructing pixel-wise descriptors from the feature maps of the CNN.
Sentence 2: the CNN-based descriptors are usually sensitive to viewpoint changes as convolutions are inherently not invariant to geometric transformations.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Our proof idea utilizes the same high-level intuition as [1,27,18,55,10,11] that near the initialization the network is linear.
Sentence 2: compared with Equation ( 7) which shows an n-interpolation class can be realized by a network architecture with VC-Dimension O(n log n), we can conclude that robust interpolation by neural networks needs more capacity, so increasing the width of neural network is indeed in some sense necessary.
Relationship: neutral
Domain: NeurIPS
Sentence 1: Despite the well established nature of these results recent work has revealed some surprising insights that come to challenge the traditional ways of thinking in this area.
Sentence 2: in the case of zero-sum games what is referred to as "convergence" to equilibrium, is the fact that when both agent apply regret-minimizing algorithms, both the time-average of the mixed strategy profiles as well as the utilities of the agents converge approximately to their Nash equilibrium values, where the approximation error can become arbitrarily close to zero by choosing a sufficiently small step-size.
Relationship: entailment
Domain: NeurIPS
Sentence 1: While our approach draws high-level inspiration from MTL, we highlight key differences: whereas tasks are disjoint in MTL, slice tasks are formulated as micro-tasks that are direct extensions of a base task-they are designed specifically to learn deviations from the base-task representation.
Sentence 2: hPS: In the style of multi-task learning, we model slices as separate task heads with a shared backbone trained via hard parameter sharing.
Relationship: neutral
Domain: NeurIPS
Sentence 1: Intuitively, if the model knows "where" in the 2-dim data space an example lives (as defined by SFs), it can condition on slice-specific features as it makes a final, slice-aware prediction.
Sentence 2: we consider two text-based relation extraction datasets: Chemical-Disease Relations (CDR), [41], in which we identify causal links between chemical and disease entities in a dataset of PubMed abstracts, and Spouses [9], in which we identify mentions of spousal relationships using preprocessed pairs of person mentions from news articles (via Spacy [17]).
Relationship: neutral
Domain: NeurIPS
Sentence 1: A related line of work [KLS09, ABL17, DKK + 16, LRV16, DKK + 17, DKK + 18, DKS18, KKM18, DKS19, DKK + 19] has given polynomial time robust estimators for a range of learning tasks.
Sentence 2: [KLS09, ABL17, DKS18, DKK + 19] obtained efficient PAC learning algorithms for halfspaces with malicious noise [Val85,KL93], under the assumption that the uncorrupted data comes from a "tame" distribution, e.g., Gaussian or isotropic log-concave.
Relationship: entailment
Domain: NeurIPS
Sentence 1: We borrow an idea from [BFKV96] that in some sense allows us to "reduce" the general case to the large margin case.
Sentence 2: [BFKV96] (see also [DV04a]) developed a pre-processing routine that slightly modifies the distribution on the unlabeled points and guarantees the following weak margin property: After preprocessing, there exists an explicit margin parameter Ïƒ = â„¦(1/poly(d, b)), such that any hyperplane through the origin has at least a non-trivial mass of the distribution at distance at least Ïƒ from it.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Moreover, this region is efficiently identifiable by a simple thresholding rule.
Sentence 2: we show that there exists a threshold T > 0 (which can be found algorithmically) such that the hypothesis sign(hwb , xi) has error bounded by η +  in the region RT = {x : |hwb , xi| ≥ T}.
Relationship: entailment
Domain: NeurIPS
Sentence 1: We study the problem of distribution-independent PAC learning of halfspaces in the presence of Massart noise.
Sentence 2: we are given a set of labeled examples (x, y) drawn from a distribution D on R d+1 such that the marginal distribution on the unlabeled points x is arbitrary and the labels y are generated by an unknown halfspace corrupted with Massart noise at noise rate Î· < 1/2.
Relationship: entailment
Domain: NeurIPS
Sentence 1: While both of the two approaches above, (a) and (b), can be used in a semi-supervised setting, cGANs of (b) provide a more natural framework for using both labeled and unlabeled data; 3 one can use the unlabeled data to learn p(x), and the labeled data to learn both p(x) and p(c|x).
Sentence 2: we focus on evaluating the second type of architectures, e.g., the auxiliary classifier GAN (ACGAN) [39].
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The generative adversarial network (GAN) [15] is arguably the most successful generative model in recent years, which have shown a remarkable progress across a broad range of applications, e.g., image synthesis [5,21,40], data augmentation [49,18] and style transfer [58,10,34].
Sentence 2: as its advanced variant, the conditional GANs (cGANs) [31] have gained a considerable attention due to its class-wise controllability [9,42,10] and superior quality for complex generation tasks [39,33,5].
Relationship: entailment
Domain: NeurIPS
Sentence 1: It may be the case that label-bias is so obvious to most authors that it does not even occur to them to mention it; howbeit, the conspicuous absence of label-bias from papers on fairness perniciously pervades real-world discussions underlying the decisions about how to balance the trade-off between fairness and accuracy.
Sentence 2: we believe this finding to be of practical importance and worthy of highlighting.
Relationship: reasoning