sadat2307/MSciNLI huggingface dataset:
Domain: NeurIPS
Sentence 1: What this work does convey is that learning curves can (provably) show some rather counterintuitive and surprising behavior.
Sentence 2: we have demonstrated that least squares regression, regression with the absolute loss, linear models trained with the hinge loss, and likelihood estimation of the variance of a normal distribution can all suffer from nonmonotonic behavior, even when evaluated with the loss they optimize for.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Most optimization methods for solving problems with the Schatten-p norm perform SVD on X at every iteration, with time complexity of O(m 2 n) (supposing m ≤ n) [21,22].
Sentence 2: the natural algorithm to minimize FGSR 2/3 and FGSR 1/2 does not use the SVD, as the regularizers are simple (not spectral) functions of the factors.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Although SSMs have good properties like modularity and interpretability due to the Markovian assumption, the closed-form solution is only available for simple cases like the linear Gaussian SSMs.
Sentence 2: in many applications, SSMs show difficulties in capturing nonlinear non-Markovian long-term dependencies  (Auger-Méthé et al., 2016).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In GQN, query x corresponds to a camera viewpoint in a 3D space, and output y is an image taken from the camera viewpoint.
Sentence 2: the problem in GQN is cast as: given a context set of viewpoint-image pairs, (i) to infer the representation of the full 3D space and then (ii) to generate an observation image corresponding to a given query viewpoint.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The term "pruning" implies the dropping of connections by setting weights to zero, and these weights are thought of as unimportant.
Sentence 2: if the value of zero for the pruned weights is not important to the performance of the network, we should expect that we can set pruned weights to some other value, such as leaving them frozen at  their initial values, without hurting the trainability of the network.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Whereas previous work on CNPs has focused on homogeneous regression and classification datasets and fairly simple models, here we study multiple heterogeneous classification datasets and use a more complex model to handle this scenario.
Sentence 2: whereas the original CNP approach to classification required pre-specifying the number of classes in advance, CNAPS handles varying way classification tasks, which is required for e.g.
Relationship: entailment
Domain: NeurIPS
Sentence 1: However, the dimensionality of this subspace is very low compared to the dimensionality of PreResNet-164 CIFAR-10â†’STL-10  the weight space, and we can not guarantee that SWAG variance estimates are adequate along all directions in weight space.
Sentence 2: we would expect SWAG to under-estimate the variances along random directions, as the SGD trajectory is in a low-dimensional subspace of the weight space, and a random vector has a close-to-zero projection on this subspace with high probability.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Bayesian methods provide a natural probabilistic representation of uncertainty in deep learning [e.g., 3, 24, 5], and previously had been a gold standard for inference with neural networks [38].
Sentence 2: existing approaches are often highly sensitive to hyperparameter choices, and hard to scale to modern datasets and architectures, which limits their general applicability in modern deep learning.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For the multi-view representation, CPM-Nets jointly considers multi-view complementarity and class distribution, making them mutually improve each other to obtain the representation reflecting the underlying patterns.
Sentence 2: the encoded latent representation from observations is complete and versatile thus promotes the prediction performance, while the clustering-like classification schema in turn enhances the separability for latent representation.
Relationship: entailment
Domain: NeurIPS
Sentence 1: In this work, we examined the inductive bias of CNNs, and challenged the accepted view that FCNs are unable to generalize as well as CNNs on visual tasks.
Sentence 2: we showed that the CNN prior is mainly useful during the early stages of training, to prevent the unconstrained FCN from falling prey of spurious solutions with poor generalization too early.
Relationship: entailment
Domain: NeurIPS
Sentence 1: In all our experiments, we assume that the weightfunction comes from the class F of mixture of m 2D non-negatively weighted Gaussians as described in the end of Section 3.2.
Sentence 2: the persistent homology is one of the most important developments in the field of topological data analysis, and there have been fundamental developments both on the theoretical front (e.g, [23,10,13,8,14,5]), and on algorithms / implementations (e.g, [43,4,15,20,29,3]).
Relationship: neutral
Domain: NeurIPS
Sentence 1: There are natural distances defined for persistence diagrams, including the bottleneck distance and the Wasserstein distance, both of which have been well studied (e.g, stability under them [16,18,14]) with efficient implementations available [27,28].
Sentence 2: to facilitate downstream machine learning tasks, it is desirable to further map the persistence diagrams to another "vector" representation.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For example, for the persistence diagrams computed from atomic configurations of molecules, features with small persistence could capture the local packing patterns which are of utmost importance and thus should be given a larger weight; while in many other scenarios, small persistence leads to noise with low importance.
Sentence 2: the 10 * 10-fold nested cross validation are applied to evaluate our algorithm: There are 10 folds in outer loop for evaluation of the model with selected hyperparameters and 10 folds in inner loop for hyperparameter tuning.
Relationship: neutral
Domain: NeurIPS
Sentence 1: They still lack the investigations on synthesizing the images in a more controllable way, like finely manipulating the visual appearance of every object.
Sentence 2: to generate the images with preferred objects and rich interactions, we propose a semi-parametric method, PasteGAN, for generating the image from the scene graph and the image crops, where spatial arrangements of the objects and their pair-wise relationships are defined by the scene graph and the object appearances are determined by the given object crops.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In order to integrate the objects in the expected way defined by the scene graph as well as maintaining the visual appearance of the objects, we designed a Crop Refining Network as well as an attentionbased Object-Image Fuser, which can encode the spatial arrangements and visual appearance of the objects as well as their pair-wise interactions into one scene canvas.
Sentence 2: it can encode the complicated interactions between the objects.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Note that our SAC baseline is comparable with the SAC from [22] on Hopper-v2, Walker2d-v2, Ant-v2 and Humanoid-v2 after 5 â€¢ 10 5 steps (the SAC from [21] uses the earlier v1-versions of Mujoco and is hence not an optimal reference).
Sentence 2: there is a discrepancy on HalfCheetah-v2.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Our model performs favorably against the self-supervised state-of-the-art methods.
Sentence 2: our model outperforms Wang et al. [52] by 13.3% in J and 16.6% in F. and is even 6.9% better in J and 4.1% better in F than the ResNet-18 model [13] trained on ImageNet [9] with classification labels.
Relationship: entailment
Domain: NeurIPS
Sentence 1: We also performed a qualitative comparison of the latent structures inferred by the different models and found that the sparse variant of the PRGDS inferred some components that the other models did not.
Sentence 2: the sparse variant of the PRGDS is uniquely capable of inferring bursty latent structures that are highly localized in time; we visualize examples in Fig. 5.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Instead, our approach is to leverage our variational reformulation in Lemma 1 and replace the class of functions F with a rich (non-subspace) set in this new formulation, which is distinct from GMM and avoids these issues.
Sentence 2: as long as F has bounded complexity, even if its ambient dimension may be infinite, we can guarantee the consistency of our approach.
Relationship: entailment
Domain: NeurIPS
Sentence 1: It should be noted that, in both cases, the nonparametric estimator captures the fact that X 1 and X 2 are independent, as can be seen from the contour densities on the left panel.
Sentence 2: adversarial approaches require multiple models to be trained, leading to difficulties and computational burden [62,26,24], and variational approaches make (strong) distributional assumptions, potentially detrimental to the generative model performance [64].
Relationship: neutral
Domain: NeurIPS
Sentence 1: In contrast to standard empirical risk minimization in supervised learning, where test data follow the same distribution as training data, minimax statistical learning arises in problems of distributionally robust learning [16,18,28,29,40] and minimizes the worst-case risk over a family of probability distributions.
Sentence 2: it can be applied to the learning setting in which the test data distribution differs from that of the training data, such as in domain adaptation and transfer learning [12].
Relationship: reasoning
Domain: NeurIPS
Sentence 1: One can also see that the general structured encoder, with any αi,j (x), achieves better test loss than the super-modular structured encoder.
Sentence 2: this comes with a computational price, as the maxflow algorithm is orders of magnitude faster than CPLEX, and structured encoder with CPLEX becomes better than maxflow only in epoch 85, see Figure 3.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Hence, for 1-NN classification, we should ideally choose quantizers with low debiased variance around p * = cos(x, x i ), provided that it can be known (or estimated) a priori.
Sentence 2: quantizers with small debiased variance are favorable for NN classifier and linear classifier, in high similarity region and around ρ = 0, respectively.
Relationship: neutral
Domain: NeurIPS
Sentence 1: Although the distinction is not fully unambiguous, there exist two prototypical families of algorithms: those that learn without an explicit model of the environment (model free), and those that first learn a model and then use it to plan a solution (model based).
Sentence 2: this has implications for Dyna-style learning, as well as for replay methods [cf.. van Hasselt et al., 2018].
Relationship: neutral
Domain: NeurIPS
Sentence 1: Instead, planning backward when the model is inaccurate may lead to updates to fictional states that are unreachable, which may or may not be useful, but is less likely to be harmful than updating real states with fictional transitions.
Sentence 2: if, however, we sample states according to a non-uniform distribution (e.g., using prioritised replay) this can make replay-based algorithms less stable and potentially divergent [cf.
Relationship: neutral
Domain: NeurIPS
Sentence 1: Experience replay [Lin, 1992] refers to storing previously observed transitions to replay later for additional updates to the predictions and policy.
Sentence 2: one of the main results by Kaiser et al. [2019] was to compare SimPLe to Rainbow DQN [Hessel et al., 2018a], which combines the DQN algorithm [Mnih et al., 2013[Mnih et al., , 2015 with double Q-learning [van Hasselt, 2010, dueling network architectures [Wang et al., 2016], prioritised experience replay , noisy networks for exploration [Fortunato et al., 2017], and distributional reinforcement learning [Bellemare et al., 2017].
Relationship: neutral
Domain: NeurIPS
Sentence 1: Recent efforts in deep learning models interpretation allow us to understand the decision for particular examples Ribeiro et al. (26); Shrikumar et al. (27); Štrumbelj, Kononenko (31); Koh, Liang (15); Lundberg, Lee (19), but understanding the logic behind a complex model is still a challenging task.
Sentence 2: symmetric trees outperform other tree shapes: they are coded by a decision table, thus evaluation for one tree requires several bit-wise operations and one look-up in the table and takes >10x less time for the same number of trees Dorogush et al. (6).
Relationship: neutral
Domain: NeurIPS
Sentence 1: One desired property of such a probabilistic model is sharpness (or high accuracy), i.e., if possible, the model should assign the highest probability to the true tumour grade (which maybe can not be inferred from the image at hand but only by other means such as an additional immunohistochemical staining).
Sentence 2: to be able to trust the predictions the probabilities should be calibrated (or reliable) as well (DeGroot and Fienberg, 1983;Murphy and Winkler, 1977).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Aleatoric uncertainty arises due to the stochastic variability inherent in the data generating process, for example due to an imperfect sensor, and is described by the cumulative distribution function (CDF) F((y|x,Θ) of the data specified by a given model.
Sentence 2: epistemic uncertainty arises due to our lack of knowledge about the data generating mechanism.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The property above (Equation 2) allows one to upper-bound the Lipschitz constant of a network by the product of the Lipschitz constants of each layer.
Sentence 2: as modern neural networks tend to possess many layers, the resultant upper-bound is likely to be very loose, and constraining it increases the risk of diminishing the Lipschitz constrained network capacity that can be utilized.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The Lipschitz constant (or spectral norm) of a convolution operator is bounded by a constant factor of the spectral norm of its reshaped matrix [12,48,40], which enables bounding of the convolution operator's Lipschitz constant by bounding that of the reshaped matrix.
Sentence 2: this upper-bound can be conservative, causing a bias towards convolution operators with low Lipschitz constants, limiting the method's expressive power.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: We also evaluate the empirical robustness of our model around under two gradient-based attacks and two decision-based attacks: (i) PGD attack with CW loss [34,7], (ii) FGSM [47], (iii) Boundary attack (BA) [6], (iv) Point-wise attack (PA) [44].
Sentence 2: the gradient-based methods ((i) and (ii)) are done on the whole test dataset; the decision-based attacks ((iii) and (iv)) are done only on the first 100 test data points since they are expensive to run.
Relationship: entailment
Domain: NeurIPS
Sentence 1: In this section, we numerically compare the performance of ADDIS against the previous state-ofthe-art algorithm SAFFRON [5], and other well-studied algorithms like LORD++ [4], LOND [10] and Alpha-investing [2].
Sentence 2: we use ADDIS * defined in Algorithm 1 as the representative of our ADDIS algorithm.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Among all the short sequences, we randomly choose 100,000 sequences for each class for training, validation, and test.
Sentence 2: we feed the one-hot encoded DNA sequences into an LSTM layer, followed by a dense layer and a softmax function to predict the probability distribution over the 4 letters of {A, C, G, T }, and train the model using only the in-distribution training data.
Relationship: neutral
Domain: NeurIPS
Sentence 1: They study sketching algorithms from both statistical and algorithmic perspectives.
Sentence 2: they focus on a different setting, where n >> r, and prove bounds on RE and P E. For instance, they discover that RE can be bounded even when r is not too large, proving bounds such as RE ≤ 1 + 44p/r for subsampling and subgaussian projections
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Various versions of this fundamental problem have been studied before (see e.g., Drineas et al., 2006Drineas et al., , 2011Dhillon et al., 2013;Ma et al., 2015;Raskutti and Mahoney, 2016;Thanei et al., 2017, and the references therein).
Sentence 2: in a generative model where the data are sampled from a linear regression model, Raskutti and Mahoney (2016) have recently compared the statistical performance of various sketching algorithms, such as Gaussian projections and subsampled randomized Hadamard transforms (SRHT) (introduced earlier in Sarlos (2006); Ailon and Chazelle (2006)).
Relationship: entailment
Domain: NeurIPS
Sentence 1: Various versions of this fundamental problem have been studied before (see e.g., Drineas et al., 2006Drineas et al., , 2011Dhillon et al., 2013;Ma et al., 2015;Raskutti and Mahoney, 2016;Thanei et al., 2017, and the references therein).
Sentence 2: this is because iid projections distort the geometry of Euclidean space due to their non-orthogonality.
Relationship: neutral
Domain: NeurIPS
Sentence 1: We observe that our results are accurate, both in simulations and in two empirical data analysis examples, see Section 3.
Sentence 2: they go beyond earlier work (Raskutti and Mahoney, 2016) because they are accurate not just up to the rate, but also down to the precise constants, even in relatively small samples (see Section A.16 in the supplemental for a comparison).
Relationship: entailment
Domain: NeurIPS
Sentence 1: Pose symmetric 1D convolution layers can be based on fully connected layers.
Sentence 2: a 1D convolution is a fully connected layer with shared parameters across the time dimension, i.e., at each time step the computation is the sum of fully connected layers over a window: we enforce equivariance at each time step by employing the symmetry pattern of fully connected layers at each time slice.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The term z 1 log(z 2 /(q 2 )) may attribute to the number of cycles in Algorithm 2, but due to the curvature of the sphere, we observe that it is not a dominating factor in the query complexity.
Sentence 2: intuitively, this weak assumption ensures that when the cost or reward tradeoffs for the classes change, the preferred confusion matrices for those cost or reward tradeoffs also change (and vice-versa).
Relationship: neutral
Domain: NeurIPS
Sentence 1: This seems to suggest a degree of anti-stereotypical associations, but none of the tests which have negative effect sizes are significant Table 5: Intersectional embedding association tests and effect sizes, for word encodings (word), sentence encoding (sent) and contextual word representation (c-word).
Sentence 2: there are only 2 significant positive associations for test +I4, but there are 9 such associations for test +I3 across all models.
Relationship: neutral
Domain: NeurIPS
Sentence 1: We depart from previous work by including the nominative (she), accusative (her), prenominal possessive (her) and predicative possessive (hers) inflections of personal pronouns, and we also include the non-gendered or collective pronoun they.
Sentence 2: for each sentence, we increment a count for female pronoun occurrence if any female pronoun is in the sentence, and then count the number pro-stereotypical and anti-stereotypical associations with occupation words (same for male and non-gendered pronouns).
Relationship: entailment
Domain: NeurIPS
Sentence 1: Although these methods are promising, Gonen and Goldberg [13] show that the gender bias encoded in word embeddings is mostly hidden from the defined metric of projecting onto the "he-she" vector direction.
Sentence 2: words still receive implicit gender from their associations (e.g. "receptionist" is no longer gendered with respect to "he", but is still gendered with respect to "captain"). 
Relationship: entailment
Domain: NeurIPS
Sentence 1: Compared with Theorem 2, the lower bounds in Theorem 3 lose a polynomial factor in M due to a larger policy space.
Sentence 2: since the number of batches M of interest is at most O(log T ) (otherwise by Corollary 1 we effectively arrive at the fully adaptive case with M = T ), this penalty is at most poly-logarithmic in T .
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In the case of variable noise, our method remains roughly as good when the noise parameters are known, but starts to have trouble when they need to be estimated from data.
Sentence 2: with additive Gaussian noise the product involves two Gaussians, and because both distributions are functions of x, we have where we have exploited the symmetry of Gaussian distribution in the first term to swap x and y.
Relationship: neutral
Domain: NeurIPS
Sentence 1: The approximate distribution p(x|â„¦ y ) allows us to now apply Bayesian reasoning to include information from y at test-time.
Sentence 2: the (unnormalized) posterior probability of the clean value x given observations of both the noisy pixel y and its context is given by Bayes' rule as follows: From this point of view, the distribution p(x|â„¦ y ) takes the role of the prior, encoding our beliefs on the possible xs before observing y.
Relationship: entailment
Domain: NeurIPS
Sentence 1: All the intermediate activations were then "re-computed" by solving the ODE layers backwards.
Sentence 2: it has been recently shown that such an approach could lead to incorrect gradients, due both to numerical instability and also to inconsistencies that relate to optimizing infinite dimensional operators (the so called Discretize-Then-Optimize vs. Optimize-Then-Discretize issue) [9].
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The governing equation here is dz/dt = c(t) dz/dx, where c(t) is variable velocity, and z is the signal that changes in time.
Sentence 2: as discussed above, an alternative view of a residual network is the following continuous-time formulation: dz dt = f (z(t); âœ“), with z(t = 0) = z 0 and z(t = 1) = z 1 (we will use both z(t) and z t to denote activation at time t).
Relationship: neutral
Domain: NeurIPS
Sentence 1: Here we perform an ablation study in which we remove the evolution of the model parameters, and instead we fix them to stale values in time (which is the configuration used in [8,9]), and test with a case where the model parameters are indeed evolved in time, which corresponds to results of Table 2.
Sentence 2: we use two time steps for activation ODE (Eq. . 5b) and ten time steps for the evolution of the model parameters (Eq. 5c).
Relationship: entailment
Domain: NeurIPS
Sentence 1: For ReLU networks (piece-wise linear networks in general), exact verifiers solve the robustness verification problem (1) by typically employing MILP solvers [Cheng et al., 2017, Lomuscio and Maganti, 2017, Dutta et al., 2018, Fischetti and Jo, 2017 or Satisfiability Modulo Theories (SMT) solvers [Scheibler et al., 2015, Carlini et al., 2017, Ehlers, 2017.
Sentence 2: due to the NP-completeness for solving such a problem , it can be really challenging to scale these to large networks.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Over different models, sizes, training methods, and datasets (MNIST and CIFAR-10), we find that (i) in terms of lower bounding the minimum l âˆž adversarial distortion 3 , the optimal layer-wise convex relaxation only slightly improves the lower bound found by , especially when compared with the upper bound provided by the PGD attack, which is consistently 1.5 to 5 times larger; (ii) in terms of upper bounding the robust error, the optimal layer-wise convex relaxation does not significantly close the gap between the PGD lower bound (or MILP exact answer) and the upper bound from .
Sentence 2: there seems to be an inherent barrier blocking our progress on this road of layer-wise convex relaxation, and we hope this work provokes much thought in the community on how to bypass it.
Relationship: reasoning