sadat2307/MSciNLI huggingface dataset:
Domain: NeurIPS
Sentence 1: The success of machine learning (ML) during the past decade has relied heavily on the rapid growth of computational power, new techniques training larger and more representative neural networks, and critically, the availability of enormous amounts of annotated data.
Sentence 2: new challenges have arisen with the move from cloud computing to edge computing and Internet of Things (IoT), and demands for customized models and local data privacy are increasing, which raise the question: how can a powerful model be trained for a specific user using only a limited number of local data?
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Such weight-tying is generally considered to come with four major benefits: 1) it acts as a form of regularization that stabilizes training and supports generalization; 2) it significantly reduces the model size; 3) it is trivial to show that any deep network can be represented by a weight-tied deep network of equal depth and only a linear increase in width (see Appendix C); and 4) the network can be unrolled to any depth, typically with improved feature abstractions as depth increases [8,18].
Sentence 2: in practice almost all such models (and deep nets in general) are stacked, trained and evaluated by unrolling a pre-determined, fixed number of layers.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Among the many applications of deep networks, sequence modeling has witnessed continuous advances in model architectures.
Sentence 2: while recurrent networks have long been the dominant model for sequences [21,26,14,34], deep feedforward architectures based on temporal convolutions [49,47,7] and self-attention [48,16,13] have (re-)emerged to claim superior performance on a variety of sequence prediction tasks.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Many approximate methods have been introduced to improve scalability, relying on mixture-of-experts models [7], inducing points [12,37,39,42], random feature expansions [20,30,47], or stochastic variational optimization [2,16,17,36,46].
Sentence 2: due to the historical intractability of training exact GPs on large datasets, it has been an open question how approximate methods compare to an exact approach when much more data is available.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: We further use this framework to derive the first asynchronous online and stochastic optimization algorithm with non-box constraints that uses non-Euclidean regularizers.
Sentence 2: we present and analyze HEDGEHOG, the parallel asynchronous variant of the EG algorithm, also known as Hedge in online linear optimization [34,14], 2 In this paper, we do not further consider BCD-based methods, for two main reasons: a) in general, a BCD update may unnecessarily slow down the convergence of the algorithm by focusing only on a single coordinate of the gradient information, especially in the sparse-data problems we consider in this paper (see, e.g., Pedregosa et al.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Of particular interest are asynchronous lock-free algorithms, where Recht et al. [30] demostrated first that linear speed-ups are possible: They showed that if ⌧ processes run stochastic gradient descent (SGD) and apply their updates to the same shared iterate without locking, then the overall algorithm (called Hogwild!) converges after the same amount of work as serial SGD, up to a multiplicative factor that increases with the number of concurrent processes and decreases with the sparsity of the problem.
Sentence 2: if the problem is sparse enough, this penalty can be considered a constant, and the algorithm achieves linear speed-up.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: When measuring the quality of a large population of generated images, these underrepresented regions have little impact as it is unlikely that too many generated samples land there -even though the hyperspheres may be large, they are sparsely located and cover a small volume of space in total.
Sentence 2: we see that FID favors configurations with high recall (A, F) over the ones with high precision (B, C), and the same is also true for the individual snapshots.
Relationship: neutral
Domain: NeurIPS
Sentence 1: First, GMF-Q converges after about 10 outer iterations; secondly, as the number of inner iterations increases, the error decreases (Figure 2); and finally, the convergence is robust with respect to both the change of number of states and the initial population distribution (Figure 3).
Sentence 2: the Naive algorithm does not converge even with 10000 inner iterations, and the joint distribution L t keeps fluctuating (Figure 4).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: As |S| and |A| increase, GMF-Q is robust with respect to this increase of dimensionality, while both MF-Q and IL clearly suffer from the increase of the dimensionality with decreased convergence rate and accuracy.
Sentence 2: gMF-Q is more scalable than IL and MF-Q, when the system is complex and the number of players N is large.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Even Density Functional Theory (DFT) [Hohenberg and Kohn, 1964], a widely used approximation to the equations of quantum mechanics, has trouble scaling to more than a few hundred atoms.
Sentence 2: the majority of practical work in molecular dynamics today falls back on fundamentally classical models, where the atoms are essentially treated as solid balls and the forces between them are given by pre-defined formulae called atomic force fields or empirical potentials, such as the CHARMM family of models [Brooks et al., 1983[Brooks et al., , 2009.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Our Cormorant neural network has invariance to rotations baked into its architecture in a way that is similar to the physical equations of the previous section: the internal activations are all spherical tensors, which are then combined at the top of the network in such a way as to guarantee that the final output is a scalar (i.e., is invariant).
Sentence 2: to allow the network to learn interactions that are more complicated than classical interatomic forces, we allow each neuron to output not just a single spherical tensor, but a combination of spherical tensors of different orders.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: However it does have a well studied decomposition into irreducibles, called the Clebsch-Gordan decomposition: ) is an 'th order spherical tensor.
Sentence 2: we note that similar Clebsch-Gordan nonlinearities were used in , and that the Clebsch-Gordan product is also an essential part of Tensor Field Networks [Thomas et al., 2018].
Relationship: neutral
Domain: NeurIPS
Sentence 1: A distinctive feature of these algorithms is that they are rather ad hoc in the way they deal with the structure of our problem.
Sentence 2: they cannot be seen as finding the query that splits the version space as evenly as possible, a common approach in many active learning papers (e.g., [12,25,15,16,27,24], and references therein).
Relationship: entailment
Domain: NeurIPS
Sentence 1: Here, for a given prior P(c * ), the goal of learning is to identify C * either exactly (when Î» = 0) or approximately (when Î» > 0), while bounding the expected number of queries (x it , x jt ) made to the ground-truth matrix Î£, the expectation being over the noise, and possibly over P(c * ).
Sentence 2: if C is the clustering produced by the algorithm after it stops, we would like to prove upper bounds on E[d H (Î£ * , C)], as related to the number of active learning rounds, as well as to the properties of the prior distribution.
Relationship: entailment
Domain: NeurIPS
Sentence 1: The design of our generative model is inspired by the way humans write articles: Each word should be semantically consistent with not only its surrounding words, but also the entire paragraph/document.
Sentence 2: we assume text generation is a two-step process: A center word is first generated according to the semantics of the paragraph, and then the surrounding words are generated based on the center word's semantics 2 .
Relationship: entailment
Domain: NeurIPS
Sentence 1: Since the optimization problem is constrained on the unit sphere, the Euclidean space optimization methods such as SGD cannot be used to optimize our objective, because the Euclidean gradient provides the update direction in a non-curvature space, while the parameters in our model must be updated on a surface with constant positive curvature.
Sentence 2: we need to base our embedding training problem on Riemannian optimization.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Let f be a lower semi-continuous convex function such that f(1) = 0, and let P and Q be two distributions over a probability space (Ω, Σ) such that P is absolutely continuous with respect to Q.
Sentence 2: we can show that for any H and for any Î“, (H, Î“)-capacity bounded differential privacy is preserved after post-processing if certain conditions about the function classes hold: Theorem 1.
Relationship: neutral
Domain: NeurIPS
Sentence 1: Remark 2: Unlike standard AGD for h(y), which only updates y k in the outer-loop, DIAG's outerstep updates both x k and y k thus allowing us to better track the primal-dual gap.
Sentence 2: dIAG's dependence on the condition number L/Ïƒ seems sub-optimal and can perhaps be improved if we do not compute Imp-STEP nearly optimally allowing for inexact updates; we leave further investigation into improved dependence on the condition number for future work.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Now, first order stationary point of a non-smooth nonconvex function is welldefined, i.e., x * is a first order stationary point (FOSP) of a function f (x) if, 0 âˆˆ âˆ‚f (x * ) (see Definition 3).
Sentence 2: unlike smooth functions, it is nontrivial to define an approximate FOSP.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Traditional wisdom is that uniform convergence bounds are a bad choice for complex classifiers like k-nearest neighbors because these hypotheses classes have infinite VC-dimension (which motivated the need for stability based generalization bounds in these cases [33,5]).
Sentence 2: consider a scenario where the algorithm generalizes well i.e., for every training set S, h S has zero error on S and has small test error.
Relationship: neutral
Domain: NeurIPS
Sentence 1: Traditional wisdom is that uniform convergence bounds are a bad choice for complex classifiers like k-nearest neighbors because these hypotheses classes have infinite VC-dimension (which motivated the need for stability based generalization bounds in these cases [33,5]).
Sentence 2: this sort of an argument against uniform convergence may still leave one with the faint hope that, by aggressively pruning the hypothesis class (depending on the algorithm and the data distribution), one can achieve meaningful uniform convergence.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Inspired by our linear model in Section 3.1, we conjecture that the weights w learned on a dataset S can be expressed as w 1 +w 2 , where w T 1 Ï†(x) dominates the output on most test inputs and induces a simple decision boundary.
Sentence 2: it may be possible to apply uniform convergence on the function w T 1 Ï†(x) to obtain a small generalization bound.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Motivated by the seemingly insurmountable hurdles towards developing bounds satisfying all the above five necessary criteria, we take a step back and examine how the underlying technique of uniform convergence may itself be inherently limited in the overparameterized regime.
Sentence 2: we present examples of overparameterized linear classifiers and neural networks trained by GD (or SGD) where uniform convergence can provably fail to explain generalization.
Relationship: entailment
Domain: NeurIPS
Sentence 1: To make this more concrete, for illustration, consider the high-dimensional linear model that sufficiently wide networks have been shown to converge to [17].
Sentence 2: roughly, these networks can be written as h(x) = w T Ï†(x) where Ï†(x) is a rich high-dimensional representation of x computed from many random features (chosen independent of training data).
Relationship: entailment
Domain: NeurIPS
Sentence 1: In this section we present: (i) some qualitative insights of our meta-model by showing how it is able to coherently represent a sets of tasks in its latent space, (ii) an illustration of why PROFET helps to obtain statistically meaningful results and (iii) a comparison of various methods from the literature on our new benchmark suite.
Sentence 2: we show results for the following state-of-the-art BO methods as well as two popular evolutionary algorithms: â€¢ BO with Gaussian processes (BO-GP) [22].
Relationship: entailment
Domain: NeurIPS
Sentence 1: In this paper, we assume smoothness for f induced by the structure of a kernel on X .
Sentence 2: we make the standard assumption of a p.s.d. kernel k : X × X → R such that k(x, x) ≤ 1 for all x ∈ X , and f being an element of the reproducing kernel Hilbert space (RKHS) Hk(X ) of smooth real valued functions on X .
Relationship: entailment
Domain: NeurIPS
Sentence 1: The proof is based on constructing a finite subset of "difficult" functions in H k (X ).
Sentence 2: we choose f as a uniformly sampled function from a finite set {f1, . . . , fM}, where each fj is obtained by shifting a common function g ∈ Hk(Rd) by a different amount such that each of these has a unique maximum, and then cropping to X = [0, 1].
Relationship: entailment
Domain: NeurIPS
Sentence 1: The computational cost of 1 CG step is comparable to 1 inner GD step with the MAML algorithm, since both require 1 hessianvector product (see section C for discussion).
Sentence 2: the computational cost as well as memory of iMAML with 100 inner GD steps is significantly smaller than MAML with 100 GD steps.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: This straightforward application of the Laplace mechanism was mentioned at least by Dwork and Smith [2009] and has been widely applied since by several authors [e.g. Zhang et al., 2016, Foulds et al., 2016, Park et al., 2016, Honkela et al., 2018, Bernstein and Sheldon, 2018.
Sentence 2:  Foulds et al. [2016]  show that the sufficient statistics perturbation is more efficient than OPS for models where both are applicable.
Relationship: entailment
Domain: NeurIPS
Sentence 1: We compare LIIR with a number of state-of-the-art MARL methods on battle games in StarCraft II.
Sentence 2: 4(b) and (c), at time step 6, all the three agents focus fire on one of the enemy Marine, while agent 1 has the lowest HP; after that, agent 1 still keeps firing instead of running away from the enemies and the intrinsic reward function predicts a low r in 1 , indicating that u 1 = attack is not a good action at that time; finally, agent 1 dies at time step 9 and the corresponding intrinsic reward is very low.
Relationship: neutral
Domain: NeurIPS
Sentence 1: For HPNet, semantic clustering and decoding accuracy improves progressively as one moves up the hierarchy, from 26% in the first layer (module) to 63% for the top module of the 4-module network.
Sentence 2: the better performance of HPNet in long range video predictions might be attributed to its ability to learn semantically meaningful hierarchical spatiotemporal feature representations and movement to movement relationships (see also [51]).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: What is novel and unanticipated about our finding is that V2 neurons' receptive fields are very local, yet this "memory effect" depends on the presence of the global context of the entire movie -reducing the movie aperture from 8 o to 3 o , barely larger than the receptive fields of the individual neurons, would annihilate the predictive suppression effects.
Sentence 2: this predictive suppression effects were not due to adaptation of local receptive field features, but reflected a sensitivity to the memory of the global context of the movies or movement.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: We ask for algorithms that enforce well-studied statistical fairness constraints across two protected populations (we focus on the "equal opportunity" constraint of [18], which enforces equalized false positive rates or false negative rates, but our techniques also apply to other statistical fairness constraints like "statistical parity" [12]).
Sentence 2: we ask for algorithms that satisfy these constraints (with respect to the unknown underlying distribution) at every round of the learning procedure.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Based on this result, we manage to achieve variance-based regret bounds without the need for variance or moment penalization [15,20,9] used in prior work and which can render a computationally tractable policy learning problem, non-convex.
Sentence 2: our method provides a computationally efficient alternative to the variance penalization when the original ERM problem is convex.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Notably, our proposed approach here is able to learn provably stable systems without solving this (generally hard) problem.
Sentence 2: while it is difficult to find a Lyapunov function that certifies the stability of some known system, we exploit the fact that it is relatively much easier to enforce some function to behave in a stable manner according to a Lyapunov function.
Relationship: entailment
Domain: NeurIPS
Sentence 1: The intuition of the approach we propose in this paper is straightforward: instead of learning a dynamics function and attempting to separately verify its stability via a Lyapunov function, we propose to jointly learn a dynamics model and Lyapunov function, where the dynamics is inherently constrained to be stable (everywhere in the state space) according to the Lyapunov function.
Sentence 2: a damped, rigid n-link pendulum’s state x can be described by the angular position θi and angular velocity θi of each link i.
Relationship: neutral
Domain: NeurIPS
Sentence 1: For comparison, we also include an example trajectory and resulting frames when the dynamics are modelled without the stability constraint  (i.e. letting f in the above loss be a generic neural network).
Sentence 2: lyapunov functions in deep learning Finally, there has been a small set of recent work exploring the intersection of deep learning and Lyapunov analysis [5,17,19].
Relationship: neutral
Domain: NeurIPS
Sentence 1: However, because the input polyhedron P 1-ReLU is a hyperrectangle (when projected to X ), it does not capture relational constraints between different x i 's in X (meaning it typically has to substantially over-approximate the set S).
Sentence 2: as expected, the computed result S 1-ReLU of the 1-ReLU method will incur significant imprecision when compared with the S best result.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: We can observe that the volume of the output from 3-ReLU and 2-ReLU is respectively 9 and 7 orders of magnitude smaller than from 1-ReLU.
Sentence 2: as expected, the computed result S 1-ReLU of the 1-ReLU method will incur significant imprecision when compared with the S best result.
Relationship: neutral
Domain: NeurIPS
Sentence 1: Since our low-level policy training is related to goal-conditioned HRL, we can benefit from algorithmic advances in multi-goal reinforcement learning [29,55,4,50].
Sentence 2: we extend the recently popularized goal relabeling strategy [29,4] to instructions, allowing us to relabel based on achieving a language statement that describes a region of state space, rather than relabeling based on reaching an individual state.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Finally, in the comparison to a bag-of-words representation (BOW), we observe that, while the BOW agent's return increases at a faster rate than that of the language agent at the beginning of traininglikely due to the difficulty of optimizing recurrent neural network in language agent -the language agent achieves significantly better final performance.
Sentence 2: the performance of the BOW agent plateaus at around 8 instructions per episode.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: We can clearly see that increasing the size of the orientation set in OMP results in a larger angular distance since more dissimilar orientations are included.
Sentence 2: the angular distance of candidate sets chosen by GreedyOrientation decreases fast and then stabilized, which indicates that GreedyOrientation forward selection criterion is defined well so that the best candidate orientations approximate the ground truth are among the immediate ones.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: First of all, we notice that our method in general provides slightly less accurate solutions than the method in [DSSW18], i.e., r e > r e in this case.
Sentence 2:  comparing to the brute force algorithm, our method still generates relatively accurate solutions, especially when m is large, e.g., the relative residual percentage w.r.t. the optimal solution is around 1% when m ≈ 16000. 
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For the Kronecker Product Low-Rank Approximation (LRA) problem, we give an input sparsity O( q i=1 nnz(A i ) + poly(dk/ ))-time algorithm which computes a rank-k matrix B such that down the Kronecker Product A, which is also the running time of state-of-the-art general purpose LRA algorithms [CW13, MM13, NN13].
Sentence 2: our results demonstrate that substantially faster algorithms for approximate LRA are possible for inputs with a Kronecker product structure.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Our algorithm draws from a large number of disparate sketching techniques, such as the dyadic trick for quickly finding heavy hitters [CM05, KNPW11, LNNT16, NS19], and the precision sampling framework from the streaming literature [AKO11].
Sentence 2: compressing a matrix A to a low-trank approximation yields many of the same benefits as LRA, such as compact representation, fast matrix-vector product, and fast matrix multiplication, and thus is applicable in many of the settings where LRA is used.
Relationship: neutral
Domain: NeurIPS
Sentence 1: In medical trials [11], to identify the best drug for a disease, one can conduct tests in rounds, such that each round involves testing multiple candidate drugs on multiple clinical subjects (e.g., mice) simultaneously.
Sentence 2: after each round of testing, there is typically a waiting time (e.g., days) before the effects of drugs become observable to guide the design of the next round of testing.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Then encoding them with the same representation h a = h b can later help the agent extrapolate that if m a is associated with a previously unseen sensory state s a , then m b will also be.
Sentence 2: we propose to train a neural network to perform sensorimotor prediction, and to analyze how it learns to encode motor states depending on the type of exploration that generates the sensorimotor data.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The resulting model trained using LLR degrades gracefully in terms of adversarial accuracy when we increase the strength of attack, as shown in Fig 3 .
Sentence 2: fig 3a shows that, for CIFAR-10, when the attack changes from Untargeted to Multi-Targeted, the LLR's accuracy remains similar with only a 2.18% drop in accuracy.
Relationship: entailment
Domain: NeurIPS
Sentence 1: The activity of sensory neurons is noisy -a central goal of systems neuroscience has therefore been to devise probabilistic models that allow to model the stimulus-response relationship of such neurons while capturing their variability [1].
Sentence 2: linear-nonlinear (LN) models and their generalizations have been used extensively to describe neural activity in the retina [2,3].
Relationship: entailment