sadat2307/MSciNLI huggingface dataset:
Domain: NeurIPS
Sentence 1: we are only allowed to go in a direction which is a linear function of g).
Sentence 2: n , we have The matrix M * which minimizes the probability of v T g to be positive satisfies The numerator of the derivative of this quantity is g g T M T CM g g T − 2 CM g g T M g g T . The first term is in the span of g and the second one is in the span of CM g. for this derivative to be 0 for all g, one must have M ∝ C −1 and we obtain the same result as in the Bayesian case: the natural gradient represents the direction minimizing the probability of increasing the generalization error.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: 12, we alternating between optimization of messages {λ k } and energy functions {f α }.
Sentence 2: optimization with respect to λ k for fixed F decomposes into minimizing A(λ k , θ k F ) independently for each y k , which can be done by running message-passing updates as in Section 5 using the parameter vector θ k F . the rest of this section is concerned with how to optimize with respect to F for fixed messages.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: We use ideas from the Markov Chain Monte Carlo (MCMC) literature to satisfy the first condition and guarantee that {⇡ t ✓ } 1 t=0 will asymptotically converge to p d . Specifically, we require the transition operator T ✓ (•|x) to satisfy the detailed balance condition: for all x and x 0 .
Sentence 2: this condition can be satisfied using Metropolis-Hastings (MH), where a sample x 0 is first obtained from a proposal distribution g ✓ (x 0 |x) and accepted with the following probability: the resulting MH transition kernel can be expressed as , and it can be shown that p d is stationary for T ✓ (•|x) [25].
Relationship: reasoning
Domain: NeurIPS
Sentence 1: However, unlike previous color constancy methods that explicitly model statistical co-dependencies between pixels-for example, by modeling spatial derivatives [4,5,6], or learning functions on whole-image histograms [21]-we retain the overall parametric "form" by which we compute the illuminant in (4).
Sentence 2: even though L[x, y] itself is learned through knowledge of co-occurence of chromaticities in natural images, estimation of the illuminant during inference is still achieved through a simple aggregation of per-pixel distributions.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: On the other hand, if Assumption 2 holds, then the features depend both on x i and class label y; therefore, generally speaking, distribution over those features does not factorize by solely conditioning on x i -there will be remnant dependencies due to y.
Sentence 2: if Assumption 2 is true, then Assumption 1 cannot be true in general, unless the data is generated according to a very specific model shown in the rightmost model in Fig.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Since for any (q, q 0 ) 2 [q min , q max ] 2 , s q and s q 0 differ only by a set of measure zero, the particular choice of q in that interval has no significant consequence.
Sentence 2: in what follows, when it is not unique, we will choose the quantile value to be the maximum, q max .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The eigenvalues of the Kronecker product S ⊗ X are of the form µ i (S)µ j (X) (see [8,Theorem 4.2.12]).
Sentence 2: the spectrum of the Hessian is determined by the eigenvalues of X: A difficulty in the analysis of the unregularized case is that the objective function is only convex, but not strictly convex.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: 1: for all t ≤ H − 1 do 2: ξ t ← GENSAMPLES(T, R) 3: for all s ∈ S do 4: creg H (s) ← 0 5: while t >= 0 do 6: for all s ∈ S do 7: for all ξ t q ∈ ξ t , a ∈ A do 8: It can be noted that MINIMIZECER() makes only H • |S| calls to the LP in GETCER() function, each of which has only |A| continuous variables and at most [1 + max t |ξ t |] number of constraints.
Sentence 2: the overall complexity of MinimizeCER() is polynomial in the number of samples given fixed values of other attributes.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: For any teaching sequence of length 2, the individual intervals of the version space are of size shows that for all h 1θ11 ∈ V 1 (2), the decision boundary is parallel to the true decision boundary and the test error is E (x,y)∼p(x,y) , the decision boundary is orthogonal to the true decision boundary and the test error is 1/2.
Sentence 2: we have R . The intuition is that a pair of teaching instances lead to a version space V (2) consisting of one interval per dimension.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: If the decision maker has the perfect observation model P (Z|S), she could compute P (S|Z) by estimating P (S) and P (Z) beforehand by counting the total number of occurrences of each state without considering any observations, and the total number of occurrences of observation z, respectively.
Sentence 2: accuracy and confidence are equal if the decision maker has the true model for observations.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: We then denote the symmetric part of W by so that the eigenvalues of W k are all real.
Sentence 2: note that the effect of W on the decay properties of the network is determined by the associated quadratic form, which is equivalent to that of the symmetric part of W , i.e., W . the eigenvalues of W k , especially those with the greatest magnitudes (absolute values), would describe the characteristics of the weights in nonlocal blocks.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In fact, in the results section we will show that a learning rule with G xyy 3 (s, s ) ≥ 0 for all s, s > 0 and F xy 2 (s) ≤ 0 for s > 0 and all other terms set to zero is sufficient to explain the results from protocols P1 and P2.
Sentence 2: in our learning rule an isolated pair of spikes in configuration post-before-pre will lead to depression.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: If the conditions of Lemma 1 hold for X, β * , then for m large enough they will hold for X m , β * .
Sentence 2: furthermore, as m → ∞ where the stochasticity on the left is due to W m , w m and on the right is due to w. with respect to the bounds ratio λ u /λ l , the construction of Eq.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: For every i = 1, . . . , k, we have a matrix X i ∈ R T ×m where each column of X i contains a week of energy usage (measured every hour) for a particular house and for this particular type of device.
Sentence 2: for example, the jth column of X 1 , which we denote x (j) 1 , may contain weekly energy consumption for a refrigerator (for a single week in a single house) and x (j) 2 could contain weekly energy consumption of a heater (for this same week in the same house).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: ImageNet is a notoriously large data set which can be quite a feat of engineering and infrastructure to run experiments upon it, requiring many resources.
Sentence 2: as well as using the full ImageNet data set, we devised a new data set -miniImageNet -consisting of 60, 000 colour images of size 84 × 84 with 100 classes, each having 600 examples.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: As in the Dirichlet-multinomial case, it is easily seen that if for each clique C ∈ C, the marginal distribution is defined by (11), then these distributions are consistent as they yield the same distribution (11) over the separators.
Sentence 2: the unique Markov distribution over G with Beta-Bernoulli distribution over the cliques is defined by [8] Pr(Z where Pr(Z B ) given by (11) for each complete set B ⊆ G. The prior over ξ 1:n is thus given, for ξ 1:n ∈ E K , by where for each complete set B ⊆ G, n j,B is the number of items having feature j, j = 1, . . . , K in the set B and n B is the whole set of objects in set B.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Thus, we implement a Rao-Blackwelized particle filter [6]: each particle corresponds to one sampled γ vector and contains the analytically computed state estimates for each of the objects, conditioned on that sampled γ vector.
Sentence 2: taken together, the particles used for tracking (in our case we use 50, but see Section 3.4 for discussion) approximate the joint probability distribution over γ and S. In practice, we sample γ with the following iterative procedure.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In this setting the goal is to identify a vector w with low generalization error (E[ (w ) − (u)]).
Sentence 2: we can solve this by running an OCO algorithm for T rounds and setting w to be the average value of w t . By online-to-batch conversion results [3,4], the generalization error is bounded by the expectation of the regret over the t divided by T . OCO algorithms can be used to solve stochastic optimization problems while also performing well in non-i.i.d.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: There is also physiological evidence on how these features could be extracted with separate sets of dedicated detectors working in parallel across the entire space [6].
Sentence 2: in schematic diagrams as well as computational models on visual saliency [13,3,12], image segmentation [5], object recognition [10,4,20], and scene classification [12,11], it is routinely assumed that features at all scales, colors, and orientations are processed and available simultaneously.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Proof: Choose an arbitrary index set χ ⊂ {1, 2, . . . , P } such that |χ| = K. Let F χ be the sub-matrix formed by chosen K rows of F indexed by χ.
Sentence 2: then, each row of Ã is generated by the chosen K rows of F . In the next lemma, we show how the row sparsity of F can be constrained to be at most N P (P −K+M ) by appropriately choosing the appended vectors z 1 , .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The sum-to-one constraint in (5) that is associated with leaf l can therefore be written as Furthermore, for all k ∈ K l the q B k(l) , as defined in (9), is a function of the same λ l . Accordingly, Now, consider l's leaf-node sibling, l . For example, in Figure 1, node l = a and l = b.
Sentence 2: the two leaves share the same path from their parent to the root in T . Hence, using Property 1, it must be the case that B k(l) = B k(l ) for k ∈ K l . The two sum-to-one constraints-one for each leaf-therefore imply that q l = q l . Using (11), it now follows that we can replace λ l with f (λ l ) in all q B k expressions.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: (The form of the linear EPSR update equation is identical to the form of the update in linear PSRs with s-tests given in Equation 3).
Sentence 2: a linear EPSR model is defined by Q V and the set of weight vectors, m aoe for all a ∈ A, o ∈ O, e ∈ {Q V ∪ φ}, used in Equation 5.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In this section we describe experiments we performed that compared PRank with two other online learning algorithms applied to ranking: a multiclass generalization of the perceptron algorithm [2], denoted MCP, and the Widrow-Hoff [9] algorithm for online regression learning which we denote by WHo For WH we fixed its learning rate to a constant value.
Sentence 2: the hypotheses the three algorithms maintain share similarities but are different in their complexity: PRank maintains a vector w of dimension n and a vector of k -1 modifiable thresholds b, totaling n + k -1 parameters; MCP maintains k prototypes which are vectors of size n, yielding kn parameters; WH maintains a single vector w of size n. MCP builds the most complex hypothesis of the three while WH builds the simplest.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Also, observe that the minimizer of f is x * = 1 H and f (x * ) = 0, while the minimizer of g is any vector x : i x i = n 1/2+δ and g(x ) = n 1+δ − n 1/2+2δ = Θ(n 1+δ ).
Sentence 2: the ratio between these two functions is unbounded.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: We now compare the proposed sampling distribution pk with the optimal sampling solution in hindsight.
Sentence 2: we know that if the true (gradient) vector c ∈ C k would be given to us, then the corresponding optimal probability distribution would be p (c) = for this c we can now analyze the ratio V ( pk ,c) V (p (c),c) .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: A sufficient number M of random projections is determined by a nonlinear procedure (i.e., sequential computation of Isomap residual variance) so that conventional In this manner, ML-RP helps determine the number of rows that Φ requires in order to act as an operator that preserves metric structure.
Sentence 2: mL-RP can be viewed as an adaptive method for linear reduction of data dimensionality.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: (b), the decision boundaries for class 1 and 2 have a large inter-class angular margin with each other due to the additionally injected class 5 (which doesn't take part in the final recognition evaluation) and the constrained region of class 1 is much tighter than before (other classes have the same phenomenon), yielding more compact and separable features which are highly beneficial to the original 4-class recognition.
Sentence 2: artificially injected virtual classes (e.g.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Also, once the consistency limits in (b) and (c) hold, the SE equations in Algorithm 2 reduce to the SE equations for the non-adaptive GAMP method running with the true parameters.
Sentence 2: we conclude there is asymptotically no performance loss between the adaptive GAMP algorithm and a corresponding oracle GAMP algorithm that knows the correct parameters in the sense that the empirical distributions of the algorithm outputs are described by the same SE equations.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Note that X = GS where G(: C(v,:) 1 D(v,:) 1 .
Sentence 2: we have 1 T S = 1 T if W ≥ 0, meaning the columns of X all lie on the simplex spanned by the columns of G, and the vertices of the simplex correspond to the anchor words.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: ( Since the theory is short scale insensitive, we can generate random probability densities chosen from the prior by replacing ¢ with its Fourier series and truncating the latter at some  3) enforces the amplitude of the k'th mode to be distributed a priori normally with the standard deviation Coded in such a way, the simulations are extremely computationally intensive.
Sentence 2: monte Carlo averagings given here are only over 500 runs, fluctuation determinants are calculated according to Eq.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: UoI-based methods leverage stochastic data resampling and a range of sparsity-inducing regularization parameters/dimensions to build families of potential features, and they then average nearly unbiased parameter estimates of selected features to maximize predictive accuracy.
Sentence 2: uoI separates model selection with intersection operations from model estimation with union operations: the limitations of selection by intersection are counteracted by the union of estimates, and vice versa.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The relation between contrast and Bayes risk becomes more convenient when we slightly modify the above definition (3) by a unit upper bound and by adding a lower bound on the -scaled density differences: with scale factor ¡ 9 A@ 5B DC .
Sentence 2: for an infinite scale factor the (expected) contrast F ¡ ¥ ¡ £ ¦¥ F ¡ ¤£ § G ¥ © approaches the negative Bayes risk up to constant shift and scaling: Thus the scale factor defines a subset of the input-space, which includes the decision boundary and which becomes increasingly focused in their vicinity as & .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: After T iterations, since φ 0 is a convex combination of functions in Φ, and since all the f (t) are linear functions, we have Since A is a no-regret algorithm, assume that A ensures that Regret Φ (T ) = O(T 1−c ) for some constant c > 0.
Sentence 2: when T = Ω( 1 ε 1/c ) the lower bound (2) on the regret cannot hold unless we have already found an ε-approximate fixed point of φ 0 .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: However, they act on individual parameters.
Sentence 2: similarly to [Jaderberg et al., 2014b, Denton et al., 2014 and to other parameter regularization techniques [Krogh andHertz, 1992, Bartlett, 1996], these methods do not perform model selection; the number of layers and neurons per layer is determined manually and won't be affected by learning.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The key advantage is that the acceptance probability in (2) only depends on d(y j , C) 2 and d(x j 1 , C) 2 since min ✓ p(y j ) p(x j 1 ) q(x j 1 ) q(y j ) , 1 Critically, in each of the k 1 iterations, the algorithm does not require a full pass through the data, but only needs to compute the distances between m points and up to k 1 centers.
Sentence 2: the complexity of K-MC 2 is O mk 2 d compared to O(nkd) for k-means++ seeding.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Although [26] has recently proposed an approximate iterative method, we provide an algorithm in Appendix K that is able to efficiently compute the exact solution.
Sentence 2: by combining this result with Propositions 2 and 3 we are able to efficiently compute the polar Ω • p and hence apply Algorithm 1 to solving (23) with respect to W . To investigate the effectiveness of these computational schemes we considered three applications: group lasso, path coding, and latent fused lasso.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Note that A diag(π) is an arbitrary d × d matrix whose entries sum to 1, which has d 2 − 1 degrees of freedom, whereas µ(θ) is a symmetric matrix whose entries sum to 1, which has d+1 2 − 1 degrees of freedom.
Sentence 2: s Θ (θ) has dimension d 2 and therefore the model is non-identifiable.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The standard procedure is to (re)generate entries of R on-demand [7] using pseudo-random numbers [16].
Sentence 2: we only need to store X ∈ R k . When a stream element a t = (i t , I t ) arrives, one updates the entries of X: ( By property of stable distributions, the samples x j , j = 1 to k, are also i.i.d.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The circuits were fabricated on the AMS 0.6µm 2P3M CMOS process, which allows a power supply voltage of five volts.
Sentence 2: the states of neurons {s i } and the corresponding weights {w ij } are designed to be represented by voltage in [1.5, 3.5]V and [0,5]V respectively, with both arithmetical zeros at 2.5V.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: For example, consider the alternate objective function f (w) min( w 0 , N ), leading to the optimization problem min While the global minimum remains unchanged, we observe that all local minima occurring at non-degenerate basic feasible solutions have been effectively removed.
Sentence 2: 6 In other words, at any solution w * with N nonzero entries, we can always add a small component αw ′ ∈ Null(Φ) without increasing f (w), since f (w) can never be greater than N . we are free to move from basic feasible solution to basic feasible solution without increasing f (w).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Let X = {x i } n i=1 be the data set, F be the feature space, and φ : X → F be the associated feature map of K. Then, the degree of smoothness of φ on the data graph can be captured by (Zhou et al., 2004): where w ij is the similarity of x i and x j , d ii = n j=1 w ij , and • F is the distance metric in F. The smaller the value S(φ), the smoother is the feature map φ.
Sentence 2: the SDP in (5-9) seeks the smoothest feature map that embeds the data on the unit hypersphere and at the same time respects the pairwise constraints.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: As mentioned before, μP is the "best" possible estimator to estimate µ P if nothing is known about P. However, depending on the information that is available about P, one can construct various estimators of µ P that perform "better" than µ P . Usually, the performance measure that is used for comparison is the mean-squared error though alternate measures can be used.
Sentence 2: our main objective is to improve upon KME in terms of the mean-squared error, i.e., construct μP such that E P μP − µ P 2 ≤ E P μP − µ P 2 for all P ∈ P with strict inequality holding for at least one element in P where P is a suitably large class of Borel probability measures on X .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: This is because the proposition is true for all values of µ satisfying µ L 8 n , so we can choose the smallest possible value of µ = 8L n . We have observed in practice that the IAG method with a step size of α k = 1 2nµ may diverge, even under these assumptions.
Sentence 2: for certain problems the SAG iterations can tolerate a much larger step size, which leads to increased robustness to the selection of the step size.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: As before, we make the approximation that the posterior is Gaussian, q(f |m, C) = N (f |m, C) where m ∈ R N is the mean posterior latent function, and C ∈ R N ×N is the posterior covariance.
Sentence 2: since the likelihood is isotropic and factorizes over the N observations we have the following expectation under our variational inference framework: the linearization is one-dimensional, that is g(f n ) ≈ a n f n + b n . Using this we can derive the approximate gradients, where A = diag([a 1 , .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: 1 Equivalently, min  In contrast with optimal control and planning tasks, softened distance functions, d soft (s), and costto-go functions, h soft (s), in predictive IOC are based on many paths rather than a single (best) one.
Sentence 2: unlike in A* search, each sub-optimal path cannot simply be ignored; its influence must instead be incorporated into the softened distance calculation (4).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Consider the typical situation in forward selection where we have a current working set I and we are interested in choosing the next basis vector, x i .
Sentence 2: the method of Smola and Bartlett (2001) evaluates each given x i / ∈ I by trying its complete inclusion, i.e., set I = I ∪ {x i } and optimize π(α) their selection criterion for the instance x i / ∈ I is the decrease in π(α) that can be obtained by allowing both α I and α i as variables to be non-zero.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Because the approximated posterior of µ 2 does not depend on the sequence X, the sequence variational lower bound L(θ, φ; X) can be decomposed into the sum of L(θ, φ; x (n) |μ 2 ), the conditional segment variational lower bounds, over segments, plus the log prior probability of μ2 and a constant.
Sentence 2: instead of sampling a batch at the sequence level to maximize the sequence variational lower bound, we can sample a batch at the segment level to maximize the segment variational lower bound: This approach provides better scalability when the sequences are extremely long, such that computing an entire sequence for a batched update is too computationally expensive.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Specifically, we have the likelihood of "observing" the topic assignments z d for document d 1 as .
Sentence 2: following Homes & Held [8], the likelihood for η d k conditioned on dn is the number of words assigned to topic k in document d. we have the conditional distribution where This is a posterior distribution of a Bayesian logistic model with a Gaussian prior, where z k dn are binary response variables.
Relationship: reasoning