sadat2307/MSciNLI huggingface dataset:
Domain: NeurIPS
Sentence 1: When the Gaussian likelihood p(y|x) is replaced with a Bernoulli distribution (which leads to the logistic data fit term above), it is no longer possible to compute the marginalization (4) or the posterior distribution p(x|y; γ), which is used both for optimization purposes and to make predictive statements about test data.
Sentence 2: a heuristic Laplace approximation is adopted, which requires a second-order Newton inner-loop to fit a Gaussian about the mode of p(x|y; γ).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Given three real numbers γ init 0 , γ escap 0 and γ fin 0 we set down φ 1 .
Sentence 2: we can map M 0 onto the intervals ]γ escap 0 , γ init 0 [ and ]γ escap 0 , γ fin 0 [ respectively: if γ0 refers to the sigmoid function, φ 1 0 • γ0 will be a logistic curve, growing from γ escap 0 to γ init 0 .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: We first explain the reason the required preview window width W is proportional to the hidden vector size D. Figure 2 shows the singular value distribution of WikiText-2 LM softmax weights.
Sentence 2: we observed that the distributions are similar for all three cases when the singular value indices are scaled with D. it is important to preserve the ratio between W and D. The ratio of singular values in a D/8 window over the total sum of singular values for 256, 512, and 1,024 hidden vector dimensions is 0.42, 0.38, and 0.34, respectively.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: receptive fields) with more invariance gained by the pooling layers, akin to classical convolutional neural networks.
Sentence 2: the multilayer scheme produces a sequence of maps (I k ) k≥0 , where each vector I k (z) encodes a point-say f k (z)-in the linear subspace F k of H k . we implicitly represent an image at layer k as a spatial map As mentioned previously, the mapping to the RKHS is a key to the multilayer construction.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: We generated spike trains by using a multivariate Poisson distribution with only a thirdorder correlation whose Poisson mean depends on periods as: (a)  ), but they only differ in the third-order correlation.
Sentence 2: classical Poisson-HMMs that employ an independent Poisson assumption [1,2,7] are not able to segment them into distinct states.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: When λ is small, the Gaussian likelihood is highly restrictive, constraining most of its relative mass to a very localized region of x space.
Sentence 2: a tighter prior more closely resembling the 0 quasi-norm can be used without the risk of local minima, which occur when the spines of a sparse prior overlap non-negligible portions of the likelihood (see Figure 6 in [15] for a good 2D visual of a sparse prior with characteristic spines running alone the coordinate axis).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Using ( 13) in (10), we see that where m is the stacked means of the UBM.
Sentence 2: the GLDS kernel inner product is Note that Γ in ( 12) is almost the UBM covariance matrix, but is not quite the same because of a squaring of the p(j|z i ) in the diagonal.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: When x tm = 0, the power consumption of device m at time t is zero (P m 0 = 0), and when x tm ∈ {1, . . . , Q − 1} its average power consumption is given by P m xtm .
Sentence 2: the total power consumption is given by y t = M+ m=1 P m xtm + n t , where n t ∼ N (0, 0.5) represents the additive Gaussian noise.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Note that a network which minimizes this loss function is maximizing its information content, even if the number of neurons, N , far exceeds the input dimension K, so that N K. As becomes clear from inspecting the loss function, the network is trying to code an N + K dimensional signal with only N neurons.
Sentence 2: just as in the undercomplete autoencoder, all of its information capacity will be used.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The main idea promoted by Remix-UCB is to divide the time available in epochs 1, . . . , s max (the outer loop of the algorithm), such that at the end of each epoch s, for all the remaining arms k the following holds, , where τ k identifies the time instants up to current time t when arm k was selected.
Sentence 2: using (4), this means that, for all k, with high probability: at the end of epoch s we have, with high probability, a uniform control of the uncertainty with which the empirical rewards µ τ k k approximate their corresponding rewards µ k . Based on this, the algorithm eliminates the arms that appear significantly suboptimal (step 2 of the update of Remix-UCB).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: expected number of boxes × expected volume of a box) in X sampled from a RBP is a constant τ The expected length of the interval in u According to the definition of the RBP (Poisson distribution in Step 1), we have an expected number of τ • D d=1 1 + λL (d) bounding boxes in X .
Sentence 2: the expected total volume of the bounding boxes for a given budget τ and a domain X is τ • D d=1 L (d) .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Then: If S = A, i.e., using the augmented Lagrangian smoother, γ 0 := √ Lg = 1, and c k := 0, then the (1P2D) update satisfies: for all k ≥ 0.
Sentence 2: the worst-case analytical complexity of Algorithm 1 to achieve an ε-solution xk is O( √ ε).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Moreover, we have the same constraints as in the name-face problem [6]: a person can be assigned to at most one name-verb pair, and vice-versa.
Sentence 2: given a document with P i persons and W i name-verb pairs, the number of possible assignments is j , where j is the number of persons assigned to a name-verb pair instead of null.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Temporal correlations in the high-dimensional neural population are modelled via a lower number of shared latent processes x k (•), k = 1, . . . , K, which linearly relate to a high-dimensional signal h h h(•) ∈ R N in neural space.
Sentence 2: inter-trial variability in neural firing that is shared across the population is modelled via the evolution of the latent processes on a given trial.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The inner minimization problem is a convex conic program on K. Also, as the pointwise minimum of a family of concave quadratic functions of α, the solution to the inner problem is a concave function of α, and hence the outer optimization problem is also convex (see [15] for further details).
Sentence 2: ( 3) is a concave maximization problem subject to linear constraints and is therefore a convex problem in α.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Denoting the linear regression coefficient of log(v) versus log(κ) by β and the linear regression coefficient of log(α) versus log(κ) by ξ, it can be easily shown that (5) entails: Hence, if log(α) and log(κ) are statistically uncorrelated, the linear regression coefficient between them, which we termed ξ, would be 0, and thus from (6) the linear regression coefficient of log(v) versus log(κ), which we named β, would be exactly − 1 3 .
Sentence 2: any trajectory that produces log(α) and log(κ) that are statistically uncorrelated would precisely conform to the power-law in (1) 3 .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: However, r only the n = 1 arc cosine kernel preserves the norm of its inputs: the n = 0 kernel maps onto a unit hypersphere in feature space, while higher-order (n > 1) kernels may induc spaces with severely distorted dynamic ranges.
Sentence 2: we hypothesize that only n = 1 a kernels preserve sufficient information about the magnitude of their inputs to work effe composition with other kernels.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Use B to denote the set of all balanced strategies and Ḃ to denote the set of all balanced gain distributions.
Sentence 2: interestingly, as demonstrated in [9], a balanced adversary π inflicts the same regret on every learner: (See Lemma 10) given an adversary strategy π, we can define the value-to-go V π t0 (s) associated with π from time t 0 in state s, , s t+1 ∼ P (.|s t , π(s t , t), s t0 = s).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: If a graphical constant-sum game admits a fully-mixed NE x * and all players employ MW updates, then Before proving the lemma, we note that V i (t) is indeed the variance of the following random variable, and thus is always non-negative: the random variable realizes the value Proof.
Sentence 2: we first expand , and noting that in this interval the function ln(1 + y) + y 2 /4 is concave but the function ln(1 + y) + y 2 is convex, by the Jensen's inequality, we have By following the proof of Theorem 3 in [23], one can show that the above double summation is zero.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Our approach is based on continuous extensions that are linear operators on the class of set functions, namely, linear continuous extensions.
Sentence 2: as a specific example, considering the multilinear extension, we can write where F denotes the extension of f . the value of F (x), when ⇠ , is an unbiased estimator for F (x) and unbiased estimates of the (sub)gradients may be obtained analogously.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: As we discuss below, the model is trained conditionally, and therefore encodes a joint distribution over the rating variables, conditional on all of the voxel measurements.
Sentence 2: there will be no free parameters corresponding to the voxel nodes due to the use of a conditional model, while rating nodes R s (j, •) have an associated node potential parameter Q node (s, j).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: We investigate the application of the proposed approximation scheme using these subsets.
Sentence 2: consider the information projection of an arbitrary probability measure P with density 3 p to the set D = F S i given by: Applying Theorem 3, we can compute that p S = p(x)δ S (x)/Z, where Z is a normalization factor: the normalization factor is a marginal density at x S = c S . We may now compute the restriction explicitly: In other words, the information projection to a sparse support domain is the density of x S conditioned on x S = c S . The resulting gap is: for a given target sparsity k, we solve: , where J(s) = log p(x S = c S ).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: For scheme (b), ψ k corresponds to 1 n (α (by ( 128)) ( 25) and by setting β to 0 in (126), we get L 2 2 ≥ n 5 similar to (23).
Sentence 2: l 2 1 saves a factor of n compared with L 2 2 .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: From Equations (4.1), (4.2) we have 3) It is easy to check that under our assumptions, w ≥ nk 2 and m ≥ k 2 .
Sentence 2: (1 − nk/w) k ≥ 1/e and (1 − k/m) −k ≥ 1/2e.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: While there may be many different approaches to define what is a good clustering (and, in general, deciding what is a good clustering is a difficult problem), for the problem of classifying time-series samples there is a natural choice, proposed in [21]: those samples should be put together that were generated by the same distribution.
Sentence 2: define target clustering as the partitioning in which those and only those samples that were generated by the same distribution are placed in the same cluster.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: A cut (S 1 , S c 1 ) defines a 1-local minimum if and only if it has lower energy than all cuts that do not intersect it.
Sentence 2: if a 1-local minimum is not a global minimum then the cut (S 1 , S c 1 ) necessarily intersects all of the cuts defined by the global minimizers.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: [22] (when all elements of F have cardinality two).
Sentence 2: equation ( 13) then becomes and the inequalities ( 12) defining the polytope of concavity are we define By Theorem 2, the set C is the region of concavity for the Bethe entropy ( 14) within [0, 1] |F | .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Furthermore, suppose that K ⇤ = p p d U U T with U 2 R p⇥d is a generic (dense) orthogonal matrix with unit norm columns.
Sentence 2: the factor p p d is simply the scaling needed so that the average magnitude of the entries in K ⇤ is a constant, independent of the dimensions p and d. In this case, rank(K ⇤ ) = d and kK ⇤ k F = trace(U T U ) = p. These two facts imply that the tightest bound on the nuclear norm of K ⇤ is kK ⇤ k ⇤  p p d. we take = p p d for the nuclear norm constraint.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: By setting the derivative of Q(p, g) with respect to g 1 and g 2 to zero and compare them 3 , we have, After some algebra, we obtain, According to Proposition 5, we have ∂φ(g,y) ∂g1 due to the order preserving of the top-k subgroup probability space.
Sentence 2: the above equation cannot hold since at least one of components in the sum is negative according to Definition 3.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Finally, for SVM-PERF's training and score prediction we used the binary executables downloaded from the URL http://www.cs.cornell.edu/people/tj and used the SVMPERF's settings that are the closest to our optimization formulation.
Sentence 2: we used L1-norm for slack variables and allowed the constraint cache and the tolerance for termination criterion to grow in order to control the algorithm's convergence, especially for larger values of the regularization constant.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: We emphasize that all our results hold for any fixed input, which need not be random; we average only over the weights and the biases.
Sentence 2: fM1 cannot be directly solved by batch normalization [11], which renormalizes by averaging over inputs to N , rather than averaging over initializations for N . Theorem 1 (FM1 for fully connected networks (informal)).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Therefore, we can compute a Bayesian predictive score for a frame, and then can compute a phoneme sequence score by using the Viterbi algorithm.
Sentence 2: we can construct a VBEC framework for speech recognition by selecting an appropriate model structure and estimating posterior distributions with the VB approach, and then obtaining a recognition result based on the Bayesian prediction classification.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Denote by Q ⇤ the last query that the learner submits in the sub-interval I ⇤ in Phase 2, and by R ⇤ its response.
Sentence 2: it follows by construction that either R ⇤ = 0 and the learner can produce the estimator by setting b X to the mid point of either , depending on the value of R ⇤ , and this guarantees of an additive error of at most ✏/2.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: It takes the same time as the aggregation stage.
Sentence 2: the total GST for a BCube(n,k) network is thus 2 * compared with the theoretical GST of 2 * (N −1) N * T F when running PS algorithm on a Fat-Tree network, BML algorithm on a BCube(n,k) network theoretically uses only 1 k of GST, with less network cost.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: However, training a single scholar model while referring to the recent copy of the network is equivalent to training a sequence of scholar models (H i ) N i=1 where the n-th scholar H n (n > 1) learns the current task T n and the knowledge of previous scholar H n−1 .
Sentence 2: we describe our full training procedure as in Figure 1(a).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Intuitively, the expected number of oracle calls performed by a given AVG node s is proportional to: (1/ε 2 ) × (the product of the inverted squared gaps of the set of MAX nodes in the path from the root to s) × (the probability of reaching s by following a policy which always tries to reach s).
Sentence 2: a near-optimal path with a larger number of small MAX node gaps can be considered difficult.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: However, the PAC-Bayes-Bernstein inequality is expressed in terms of the true expected variance, which is rarely accessible in practice.
Sentence 2: in order to apply the PAC-Bayes-Bernstein inequality we need an upper bound on the expected variance (or, more precisely, on the average of the expected variances of losses of each hypothesis h ∈ H weighted according to the randomized prediction rule ρ).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Then there exists a linear combination of K rows of F that will always have 0 at the j-th column index and it is not possible to generate any row of the given matrix A.
Sentence 2: k is no less than 1 + M ax N o. of 0s in any column of F . Since, maximum value is always greater than average, A slight re-arrangement establishes the aforementioned lower bound.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: First, it is not hard to reformulate the objective of the (k, z)-median problem as minimizing sup L≥0 p∈P d L (p, C) − zL , where d L is obtained from d by truncating all distances at L. By discretization, we can construct a set L of O log(∆n/ ) interesting values that the L under the superior operator can take.
Sentence 2: our goal becomes to find a set C, that is simultaneously good for every k-median instance defined by d L , L ∈ L. Since now we are handling k-median instances (without outliers), we can use the communication-efficient algorithm of [2] to construct an -coreset Q L with weights w L for every L ∈ L. Roughly speaking, the coreset Q L is similar to the set P for the task of solving the k-median problem under metric d L . The size of each -coreset Q L is at most Φ, implying the communication cost stated in the theorem.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: However, in these approaches, the inverse map is not incorporated in the estimation algorithm because it is not considered in the problem definition and the forward model is usually more complex, making inference and learning more difficult.
Sentence 2: the key algorithmic problems are that of estimating the specialized domains and functions in an optimal way (taking into account the form of the specialized functions), and using the knowledge of the inverse function to formulate efficient infer-I( is a computer graphics rendering, in general called forward kinematics ence and learning algorithms.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: We can see from Equation 6 that this error signal is proportional to the fraction of p(π|X) log p(π|X) minus the fraction of p(π|X) for all feasible paths that go through symbol k at time t. Notice that these two fractions are zero when p(π|X) = 0, but p(π|X) log p(π|X) decreases more rapidly around zero and reaches its minimum when p(π|X) = 1/e.
Sentence 2: paths with probability between 0 and 1/e, i.e.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: As discussed above, the intrinsic group additive structure G * can achieve the goodness of fit represented by the first two terms in (3) and the penalty on the structural complexity represented by the last term.
Sentence 2: by properly choosing the tuning parameters, we expect that Ĝ is consistent in that the probability Ĝ = G * increases to one as n increases (see the Theory Section below).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The multinomial naive Bayes model assumes that the label Y is chosen from some prior distribution P (Y = •), the length N is drawn from some distribution P (N = •) independently of the label, and each word X i is drawn independently from some distribution P (W = •|Y ) over the dictionary.
Sentence 2: we have: Since the length n of the document does not depend on the label and therefore does not play a significant role, we leave it out of our subsequent derivations.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The "diffusion term", ⟨F 2 a ⟩, which moves p A away from p match A depends on p A but not on the magnitude of Ja 's.
Sentence 2: within the order parameter set satisfying p A = p match A , the larger the magnitudes of J a 's are, the weaker is the repulsive effect.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: CIFAR-100 has the same set of images as CIFAR-10 but with finer category distinction in the labels, with each "superclass" of 20 containing five finer divisions, and a 100 categories in total.
Sentence 2: we expect (and observe) that Maximum-Entropy training provides stronger gains on CIFAR-100 as compared to CIFAR-10 across models (Table 2).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The equation (4) follows from the observation that ∇g where I is the identity matrix, and ∆x t = x t+1 − x t . Then equation ( 4) can be re-written as: (5) Now, since x t minimizes the convex function F t over the convex set K, a standard inequality of convex optimization (see [BV04]) states that for any point y ∈ K, we have ∇F t (x t ) • (y − x t ) ≥ 0.
Sentence 2: for y = x t+1 , we get that ∇F t (x t ) • (x t+1 − x t ) ≥ 0.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Since we use the perplexity as the evaluation metric, we need to avoid probabilities equal to zero in the output of our models (which would result in infinite perplexity).
Sentence 2: we always interpolate the probability distributions of the various models with the uniform distribution over the full vocabulary: This is a standard technique, which was previously used to compare language models trained on datasets with different vocabularies [9].
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Given these sufficient statistics, the mean/moment parameters are given by the moments µ s;j := E ✓ I[X s = j] = P(X s = j; ✓), and µ st;jk , which precisely correspond to nodewise and pairwise marginals of the discrete graphical model.
Sentence 2: the forward mapping A : ✓ 7 !
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The new formulation of inverted Wishart is consistent under marginalization.
Sentence 2: similar to the way of deriving GPs from Gaussian distributions, we define a distribution of infinite-dimensional kernel functions, denoted by Σ ∼ IW ∞ (δ, Σ • ), such that any sub kernel matrix of size m × m follows Σ ∼ IW m (δ, Σ • ), where both Σ and Σ • are positive definite kernel functions.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Now fix any P * ∈ M . Since P * ∈ M , it must have some density p * .
Sentence 2: applying (8), and then (16), and then (8) again, we find that For i appearing in the second sum, with t j < i ≤ t j+1 , we have R i (P * , Pkj ) ≤ sup i ′ ≥tj+1 R i ′ (P * , Pkj ) = sup i ′ ≥tj+1 R i ′ (P * , Pδt j (x t j ) ) ≤ h(t j + 1), so that R i (P * , Pkj ) ≤ 1 t j + 1 • (t j + 1)h(t j + 1) ≤ 1 t j + 1 • ih(i) ≤ t j+1 t j + 1 h(i) ≤ αh(i), where the middle inequality follows because nh(n) is increasing (condition (b) of the theorem).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: By equivalence to the Barker acceptance function, this response rule defines a Markov chain with stationary distribution π(x) ∝ p(x|c) γ .
Sentence 2: (6) using the weaker assumptions of Equation 5 as a model of human behavior, we can estimate the category distribution p(x|c) up to a constant exponent.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Moreover, the number of times a node k has been selected at a given iteration t > 0 follows a binomial distribution with parameters t and p k . Let us define m k (t) such that m k (0) = 0 and for t > 0: otherwise.
Sentence 2: (11) For any k ∈ [n] and any t > 0, one has E[m k (t)] = t × p k × 1/p k = t. given that every node knows its degree and the total number of edges in the network, the iteration estimates are unbiased.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: When the field is rotated, however, two things happen: the vector at r −1 x is moved to a new (rotated) position x, and each vector is itself rotated by a 3 × 3 rotation matrix ρ(r).
Sentence 2: the rotation operator π(r) for vector fields is defined as [π(r)f ](x) := ρ(r)f (r −1 x).
Relationship: reasoning