sadat2307/MSciNLI huggingface dataset:
Domain: NeurIPS
Sentence 1: Drawing inspiration from simpler dynamic Bayesian networks (DBNs) such as HMMs and Kalman filters, the proposed variational recurrent neural network (VRNN) explicitly models the dependencies between latent random variables across subsequent timesteps.
Sentence 2: unlike these simpler DBN models, the VRNN retains the flexibility to model highly non-linear dynamics.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: [26] showed global convergence and asymptotic superlinear convergence for Prox-QN methods under the assumption that the loss function is strongly convex.
Sentence 2: this assumption is known to fail to hold in high-dimensional sampling settings, where the Hessian is typically rank-deficient, or indeed even in low-dimensional settings where there are redundant features.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Note that, larger mini-batch sizes produce lower errors here because it becomes closer to full-batch gradient and thus lower bias in SGD.
Sentence 2: when the mini-batch size is as small as 10, the high error rate of 83.09% is close to a guess by majority rule -predicting the character (space) that has a largest proportion in the train set, i.e., 25, 499/153, 221 = 83.37%.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For comparison with [20,19] we evaluate our model following the set up described above and similarly, we have created dummy variables for those features that are categorical forming a total of 19 student-dependent features and 8 school-dependent features.
Sentence 2: we note that school-descriptor features such as the percentage of students eligible for free school meals and the percentage of students in VR band 1 actually depend on the year the particular sample was taken.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Immediately from its definition, this function has the following properties: The first property is due to the facts that: (i) * is a non-negative cost function; (ii) T π B ,d+ * is a contraction mapping, which by the fixed point theorem [7] implies For the second property, from the above inequality one concludes that the Lyapunov function L * is a uniform upper-bound to the constraint cost, i.e., L * (x) ≥ D π B (x), because the constraint cost D π B (x) w.r.t.
Sentence 2: policy π B is the unique solution to the fixed-point equation by construction, * (x) is an upper-bound of the cost-shaping term (x).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Meanwhile, OG collects the Boolean signals (h k ) from MaxOracle during the greedy process (Line 5), and update estimators X(•) and N (•) according to those signals (Line 10).
Sentence 2: maxOracle takes accessible arms A, estimators X(•), N (•), and counted time t , and returns an arm from A and signal h k ∈ {true, false} to instruct OG whether to update estimators for the following phase.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Also note that the most expensive step is in the first update for computing Σ t+1 yi , which if naively implemented would require an inversion of an L × L matrix.
Sentence 2: this inversion can be computed easily using Sherman-Morrison-Woodbury formula, which in turn reduces the complexity of the update to O(K 3 + K 2 L).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In fact, it has been shown in [1,2,11] that simple ad-hoc methods, and the greedy, SCoTLASS and SPCA algorithms, often underperform DSPCA.
Sentence 2: the first-order algorithm for solving DSPCA, as developed in [1], has a computational complexity of O(n 4 √ log n), with n the number of features, which is too high for many large-scale data sets.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For pred 1 ( (i)) = f 2 ( ) = i 1.1 , f 2 (i 1) f 2 (i) is monotonically increasing with i.
Sentence 2: for pred 1 ( (i)) = f 1 ( ) = 1/i, f 1 (i 1) f 1 (i) is monotonically decreasing with i. Lemma 4 shows that the mistake bound (in terms of L NDCG ) of Predtron is better when pred 1 function is selected to be f 2 ( (i)) = i 1.1 (as well as for f 3 ( (i)) = i 2 ) instead of f 1 ( (i)) = 1/i.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: On one  MovingMnist [33] DFN [19]& SVG [7] 10 inputs for 10 outputs 64x64 RobotPush [10] CDNA [10]& SV2P [3] 2 inputs for 8 outputs 64x64 Human3.6M [17] DrNet [8]& MCNet [34] 5 inputs for 10 outputs 64x64 Because the gradients back-propagated from them are to optimize different networks, which gets rid of balancing between these two loss functions.
Sentence 2: sub-networks are designed dedicated for different objectives, e.g., the sampler is required to produce high quality proposals without requirement of motion accuracy, while the selector should be able to fully capture the motion information of previous inputs and select out proposals with high motion accuracy.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: To interpret regression results we used the relative mean absolute deviation (Rmad), which is the quotient between the mean absolute distance of the function learned by the regression method and the mean absolute distance of the constant predictor that returns the mean value in all cases.
Sentence 2: we can obtain some preference judgments from the ratings of the sessions comparing the rating of each product with the rest, one by one, and constructing the corresponding pair.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Performing a first-order Taylor expansion of J( w + z) yields: In expectation, this becomes the gradient times a (diagonal) covariance matrix, and reduces to an unbiased estimate of the gradient, scaled by the learning rate and σ 2 , the variance of the perturbation.
Sentence 2: this unbiasedness comes with a very high variance, as the direction of an update is uniformly distributed.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Let us define the weights of the output layer of the original classification network as W c , and the weights of the output layer of the adapted detection network as W d . We know that for a category i ∈ B, the final detection weights should be computed as W d i = W c i + δB i .
Sentence 2: since there is no detection data for categories in A, we can not directly learn a corresponding δA layer during fine-tuning.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: On IU X-Ray dataset, HRGR-Agent achieves slightly lower BLEU-1,4 and ROUGE score than that of CoAtt [16].
Sentence 2: coAtt uses different pre-processing of reports and visual features, jointly predicts 'impression' and 'findings', and uses single-image input while our method focuses on 'findings' and use combined frontal and lateral view of patients.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Contemporaneous results due to [25] show that for A 0 distributed according to the random orthogonal model, and E 0 with Bernoulli support, correct recovery occurs with high probability provided This is an interesting result, especially since it makes no assumption on the signs of the errors.
Sentence 2: even for constant rank r it guarantees correction of only a vanishing fraction o(m 1.5 ) m 2 of errors.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: This is not a surprising result, since sequences with three motions generally have more (inlying) point trajectories than sequences with two motions, thus the outlier rates for sequences with three motions are lower (recall that a fixed number of 100 false trajectories are added).
Sentence 2: the KM method (column 3) is completely overwhelmed by the outliers-for all the sequences with outliers it returned a constant "3" as the number of motions.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The only known practical and theoretical way to achieve optimal rates in infinite Reproducing Kernel Hilbert Space (RKHS) is to use some form of cross-validation to select the step size that corresponds to a form of model selection [26,Chapter 7.4].
Sentence 2: cross-validation techniques would result in a slower training procedure partially neglecting the advantage of the stochastic training.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Observe that y t v k−1 x t ≥ 0 implies r k ≥ 0 if and only if ρ k ≤ 1/(1 + y t v k−1 x t ).
Sentence 2: if y t v k−1 x t < 0 then, in order for r k to be nonnegative, it suffices to pick ρ k ≤ 1.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Below, we examine the amount of reduction in sample complexity as a function of I s . For simplicity, we focus on a setting in which θ = 0 and γ = 1 2 , i.e., (1) is equal to 2n log n. We first consider the case of n 2 I s < 2n log n − 2m log m, i.e., (2) being equal to 2n log n − n 2 I s . In this case, sample complexity is reduced by n 2 I s . That being said, there is no asymptotical gain unless n 2 I s = Ω(n log n).
Sentence 2: when n 2 I s = Ω(n log n), this reduction can be as significant as resulting in an order-wise reduction in sample complexity.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Therefore, the development of a tractable algorithm for NSG is important, and this is one of the goals of our work.
Sentence 2: it was proven that the metric of NPG learning, F a (θ), accounts for the infinite time-steps joint distribution in the Markov chain M(θ) [19,1], while the metric of NSG learning, F s,a (θ) accounts only for the single time-step distribution, which is the stationary state-action joint distribution d θ (s)π(a|s;θ).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: As shown in [20], FoBa requires that n = Ω(log(p)/(ρ 3 k+ β 2 min )) for high-dimensional consistent support recovery, where the choice of , which is greater than k, depends on the correlations in the matrix X.
Sentence 2: the condition in (4.1), which reduces to n = Ω(log(p − k)/(ρ 2k β 2 min )), is weaker since 1/ρ 3 k+ < 1/ρ 2k for > k and p − k < p. This shows that if a sparse recovery algorithm can accurately estimate the true support, then SWAP does not introduce any false positives and also outputs the true support.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Such a definition makes sense when the player by definition must select a strategy from some bounded set, for example a probability from the n-dimensional simplex, or a distribution on paths in a graph.
Sentence 2: in contexts such as machine learning where any x 2 R n corresponds to a valid model, such a hard constraint is difficult to justify; while any x 2 R n is technically feasible, in order to prove regret bounds we compare to a much more restrictive set.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Similar to the case of single-task learning, each training point creates a depression on the σ 2 T (x * , ρ) surface [9,13].
Sentence 2: while each training point for task T creates a "full" depression that reaches the prior noise variance (horizontal dotted line at 0.05), the depression created by each training point for task S depends on ρ, "deeper" depressions for larger ρ 2 .
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Importance sampling (IS) based approaches [16,23,8,5,10,22] are typically unbiased and strongly consistent, but despite recent progress tend to have high variance-especially if the evaluation policy is deterministic, as evaluating deterministic policies requires finding in the data sequences where the actions exactly match the evaluation policy.
Sentence 2: in most real-world applications deterministic evaluation policies are more common-policies are typically to either amputate or not, rather than a policy that that flips a biased coin (to sample randomness) to decide whether to amputate.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In the discounted setting, several works have improved the bounds with respect to the number of states (Szita and Szepesvári, 2010) and the discount factor (Lattimore and Hutter, 2012).
Sentence 2: these analyses focus on worst case bounds that do not scale with the hardness of the MDP, missing an opportunity to help bridge the gap between theory and practice.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: As one would expect, the amount of information transmitted per second about the stimulus details, or s, depends on the ensemble parameter 0": larger velocities allow a higher SNR for velocity estimation, and the system is able to transmit more information.
Sentence 2: when we convert the information rate to bits/spike, we find that the system is transmitting at a constant efficiency of around 1.4 bits/spike.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: This phenomenon is due to the fact that we are using the Lipschitz assumption on F, which is a data-independent constraint on the scale of the trade-off between f (z 0 ) f (z) and d Z (z 0 , z).
Sentence 2: for p > 1, one may also think of a similar data-independent (or, worst-case) constraint this holds only if f is constant, even in the simplest case Z ✓ R. The illustrative example presented in Section 2.2 implies that the minimax learning might be possible even when the functions in F are not uniformly Lipschitz, but there exists at least one smooth hypothesis (at least, except for the regime % !
Relationship: contrasting
Domain: NeurIPS
Sentence 1: If L is the squared loss, for G = {g 1 , g 2 } with g 1 (s) = s and g 2 (s) = s and H = {h} with h(s) = s, for odd T , sup g2G disc(s) 1/2.
Sentence 2: the bound in terms of disc(s) is much finer and helps us select g such that disc(s) = 0 for that g. This example shows that even for simple deterministic dynamics our learning bounds are finer than existing ones.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Covariance kernels have to satisfy the property of positive definiteness 5 , and the MI score I does not.
Sentence 2: applying a standard transformation (called exponential embedding (EE) here), we arrive at EE becomes familiar if we note that it transforms the standard inner product x(1)' X(2) into the well-known Radial Basis Function (RBF) kernel 6 (4) 4When comparing X ( l) , X ( 2) via the inner product, we are not interested in correlating their features uniformly, but rather focus on regions of high volume under Pm e d . 5 K is positive definite if the matrix (K(X(k ll , X(k 2 »)hl ,k2 is positive definite for every set {x(1 ), ... , X (K ) } of distinct points.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: SKI works by providing accurate and fast matrix vector multiplies (MVMs) with kernel matrices, which can then be used in iterative solvers such as linear conjugate gradients for scalable GP inference.
Sentence 2: evaluating the marginal likelihood and its derivatives, for kernel learning, has followed a scaled eigenvalue approach [12,13] instead of iterative MVM approaches.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For any edge e ∈ E, we define N S,e as the number of nodes in V \ S it is relevant to, and define P S,e as the conditional probability that e is observed given S, ( Notice that N S,e only depends on the topology of G, while P S,e depends on both the topology of G and the probability weight w. The maximum observed relevance C * is defined as the maximum (over S) 2-norm of N S,e 's weighted by P S,e 's, As is detailed in the proof of Lemma 1 in Appendix A, C * arises in the step where Cauchy-Schwarz inequality is applied.
Sentence 2: note that C * also depends on both the topology of G and the probability weight w. C * can be bounded from above only based on the topology of G or the size of the problem, i.e., L = |V| and |E|.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Equation ( 1) is computationally highly intractable and its dual form is usually utilized [25]: where the supremum is taken over all Lipschitz functions f : Ξ → R, i.e., functions that assign a value to a sequence of events (points) and satisfy |f (ξ) − f (ρ)| ≤ ξ − ρ for all ξ and ρ.
Sentence 2: solving the dual form is still highly nontrivial.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: As shown in Figure 2, bottom n ′ th margin is a multimodal function, this algorithm with ǫ-relaxation is very sensitive to the choice of n ′ , and it usually gets stuck in a bad coordinatewise point without using ǫ-relaxation.
Sentence 2: an impressive advantage is that this method is tolerant to noise, which will be shown in Section 3.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: With noise vectors {η t }, {ξ t }, the system's time evolution is governed by the following equations: If the dynamics A, B, C, D are known, then the Kalman filter [Kal60] is known to estimate the hidden state optimally under Gaussian noise, thereby producing optimal predictions of the system's response to any given input.
Sentence 2: this is rarely the case -indeed, real-world systems are seldom purely linear, and rarely are their evolution matrices known.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For ReLU networks, p(q ⇤ ) = 1/2, and we see that max and 2 JJ T grow linearly with depth and dynamical isometry is unachievable in ReLU networks, even for critical orthogonal weights.
Sentence 2: for hard tanh networks, p(q ⇤ ) = erf(1/ p 2q ⇤ ).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: It is known that the greedy algorithm leads to a 1/2-approximation for the submodular social welfare maximization problem (SWM) [11], and this problem admits a 1 − 1/e-approximation in the offline setting [24], which is tight [19].
Sentence 2: for the online setting, the problem does not admit a better than 1/2-approximation algorithm unless P= NP [12].
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Traditionally, the multilayer perceptron is used to optimize such hierarchical models based on a discriminative criterion that models P (y|x) using a error backpropagating gradient descent [2,3].
Sentence 2: when the architecture is deep, it is challenging to train the entire network through supervised learning due to the large number of parameters, the non-convex optimization problem and the dilution of the error signal through the layers.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: [35] develops a novel way to split each word into sub-word units which is widely used in neural machine translation.
Sentence 2: the low-frequency sub-word units are still difficult to train: [31] provides a comprehensive study which shows that the rare (sub)words are usually under-estimated in neural machine translation: during inference step, the model tends to choose popular words over their rare alternatives.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Figure 1: Graphical Model for the sticky HDP-HMM (setting κ = 0 recovers the HDP-HMM) The hyperparameter α governs the variability of the prior mean across the rows of the transition matrix and γ controls how concentrated or diffuse the probability mass of β will be over the states of the transition matrix.
Sentence 2: in the HDP-HMM we have each row of the transition matrix is drawn as π j ∼ DP(α, β).
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For example, if a dataset has one billion features (D = 10 9 ), we can simply generate a "permutation vector" of length D = 10 9 , the memory cost of which (i.e., 4GB) is not significant.
Sentence 2: it would not be realistic to store a "permutation matrix" of size D × k if D = 10 9 and k = 500; instead, one usually has to resort to approximations such as universal hashing [5].
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Our exact global VB solver (EGVBS) provides the global solution of a non-convex minimization problem by using the homotopy method, which solves the stationary condition written as a polynomial system.
Sentence 2: our approximate global VB solver (AGVBS) finds the roots of a polynomial equation with a single unknown variable, and provides the global solution of an approximate problem.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Many machine learning algorithms iteratively transform some global state (e.g., model parameters or variable assignment) giving the illusion of serial dependencies between each operation.
Sentence 2: due to sparsity, exchangeability, and other symmetries, it is often the case that many, but not all, of the state-transforming operations can be computed concurrently while still preserving serializability: the equivalence to some serial execution where individual operations have been reordered.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: In practice, this means a discriminator might quickly learn to reject all samples that don't lie on V T n (sequences of one-hot vectors) and give meaningless gradients to the generator.
Sentence 2: it is easily seen that the conditions of Theorem 1 and Corollary 1 of [2] are satisfied even on this non-standard learning scenario with X = T n . This means that W (P r , P g ) is still well defined, continuous everywhere and differentiable almost everywhere, and we can optimize it just like in any other continuous variable setting.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: A VAE imposes a prior density p(z), such as z ∼ N (0, I), and uses a KL-divergence regularizer to align both posteriors p E (z|x 1 , y 1 ) and p E (z|x 2 , y 2 ) to it, The overall objective is to minimize L rec + L KL , whose opposite is the variational lower bound of data likelihood.
Sentence 2: as we have argued in the previous section, restricting z to a simple and even distribution and pushing most complexity to the decoder may not be a good strategy for non-parallel style transfer.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: It is worth noting that when α = (s log d/n) 1/4 , we still have (n 3 s log d) 1/4 uncorrupted labels.
Sentence 2: our lower bound (along with the upper bound shown in Theorem 3.2) indicates that the information contained in these uncorrupted labels are buried in the noise, and cannot essentially improve the detection quality compared with the unsupervised setting.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The prior work [21] achieved some success in entropy estimation using symmetric stable random projections.
Sentence 2: even after aggressively exploiting the bias-variance tradeoff, they still need to a large number of samples, e.g., 10 6 , which is prohibitive in both time and space, especially considering that in streaming applications the pseudo-random numbers have to be re-generated on the fly, the cost of which is directly proportional to the sample size.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: When we design the detail structure of the GAN, we adopt Gated Recurrent Unit (GRU) [10], a state-of-the-art RNN cell, as the basic network of G and D. It is worth noting that, others RNN variants can also be used in this work, such as the Long Short-Term Memory (LSTM) [21] cell.
Sentence 2: the time lags between two consecutive valid observations vary a lot due to data incompleteness, which makes traditional GRU cell or LSTM cell not applicable to our senario.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: The rate of decay is related to the variance of this Gaussian.
Sentence 2: if we take one standard deviation as the effective receptive field (ERF) size which is roughly the radius of the ERF, then this size is as we stack more convolutional layers, the theoretical receptive field grows linearly, therefore relative to the theoretical receptive field, the ERF actually shrinks at a rate of O(1/ √ n), which we found surprising.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: Define r b0 = log 2 1 + 1 2+3L/b0 and we have When there are multiple functions and the median partition is performed on pooled data, the partition might not happen at the empirical median on each subset.
Sentence 2: as long as the partition quantiles are upper and lower bounded by α and 1 − α for some α ∈ [1/2, 1), we can establish results similar to Theorem 2.
Relationship: contrasting
Domain: NeurIPS
Sentence 1: For example, if P X1 is a uniform distribution over a unit cube in q dimensional Euclidean space, then h(r) = min(1, r) q , and our error bound becomes meaningful for n ≥ (L 2 /ρ) q/2δ .
Sentence 2: if P X1 is supported over finitely many points, then h(r) = min x∈supp(P X 1 ) P X1 (x) is a positive constant, and the role of the latent dimension becomes irrelevant.
Relationship: contrasting