sadat2307/MSciNLI huggingface dataset:
Domain: NeurIPS
Sentence 1: With its derivations deferred to Section 2, LAG resembles (2), given by where each ∇L m ( θk m ) is either ∇L m (θ k ), when θk m = θ k , or an outdated gradient that has been computed using an old copy θk m ̸ = θ k . Instead of requesting fresh gradient from every worker in (2), the twist is to obtain ∇ k by refining the previous aggregated gradient ∇ k−1 ; that is, using only the new gradients from the selected workers in M k , while reusing the outdated gradients from the rest of workers.
Sentence 2: with where is the difference between two evaluations of ∇L m at the current iterate θ k and the old copy θk−1 m . If ∇ k−1 is stored in the server, this simple modification scales down the per-iteration communication rounds from GD's M to LAG's |M k |.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The rejections could be incorporated into the importance weights (which would require estimating the corresponding partition function, or assuming it to be constant across rounds), but as these rejections do not depend on the data x o , we interpret them as modifying the prior: from an initially specified prior p(θ), we obtain a modified prior excluding those parameters which likely will lead to nonsensical simulations.
Sentence 2: the predictive model ĝ(θ) does not only lead to more efficient inference (especially in strongly under-constrained scenarios), but is also useful in identifying an effective prior -the space of parameters deemed plausible a priori intersected with the space of parameters for which the simulator is well-behaved.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: This new coarsened graph can then be used as input to another GNN layer, and this whole process can be repeated L times, generating a model with L GNN layers that operate on a series of coarser and coarser versions of the input graph (Figure 1).
Sentence 2: our goal is to learn how to cluster or pool together nodes using the output of a GNN, so that we can use this coarsened graph as input to another GNN layer.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In our case, given the current labeling f , and a particular α ∈ 1 . . . K, each segment has the following three options: (1a) a segment with the non-zero label can retain it (1b) a segment with zero label can change it to the current non-zero label of its ancestor (if any), (2) label f i can be changed to α, (3) label f i can be changed to 0 (or kept at 0 if already there).
Sentence 2: each step results in the 2-class inference task, where U and V potentials of the 2-class inference are induced by the U and V potentials of the multi-label problem (in fact, some boundary terms then become asymmetric if one of the adjacent segments have the current label α.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: When one minimizes FH(P) with respect to P under the constraint of normalization L x P(X) = 1, one obtains PH.
Sentence 2: computing marginals of PH such as PH(Xi) or PH(Xi, Xj) involves sums over all states, which is intractable for large n. one needs tractable approximations to PH.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In recurrent neural network (RNN) language modeling, given a corpus of tokens Y = (Y  This theorem shows that the length of the hidden vector in the output layer determines the representational power of RNN with softmax.
Sentence 2: in language modeling, the rank of A can be extremely high since contexts can vary and vocabulary size M is much larger than d. the softmax can be the bottleneck of the representational power.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: By the Eckart-Young theorem X k = XV k V T k , where V k ∈ R d×k is the matrix whose columns are the top-k right singular vectors of X (see, for example, e-Chapter 9 of [14]).
Sentence 2: the optimal linear encoder is H opt = V k , and the top-k PCA-features are Z pca = XV k . Since its early beginings, [19], PCA has evolved into a classic tool for data analysis.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Since the boundaries are Lipschitz, the expected volume that is incorrectly assigned to any decision set is > c 1 n −1/d , where c 1 > 0 is a constant.
Sentence 2: if the expected excess risk at a point that is incorrectly assigned to a decision set can be greater than a constant c 2 > 0, then the overall expected excess risk is > c 1 c 2 n −1/d .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Suppose that a single machine (m = 1) receives n i.i.d.
Sentence 2: observations X i according to P . Since the packing entropy is lower bounded as log M Θ (δ) ≥ log(1/δ), the lower bound (6) implies setting B = 1 2 log n yields the lower bound M ind (θ, P([0, 1]), B) ≥ e −2 4n .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Since the truncated SVD requires s to be set beforehand, we need to efficiently estimate the number of singular values such that must meet the condition of σ i ( Ŵ (τ ) hh ) > 2 − δ.
Sentence 2: we compute upper bounds of the singular values that meet the condition on the basis of the following lemma: Lemma 6.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: We can add any unit clause of the form ∀p i ∈ ∆ pi , q j ∈ ∆ qj : S(p i , q j ) or ∀p i ∈ ∆ pi , q j ∈ ∆ qj : ¬S(p i , q j ) to the theory α(S) in Lemma 1 and the theory is still domain-liftable.
Sentence 2: theories in S 2 FO 2 and S 2 RU are domain-liftable.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: As input for our models, we consider a collection of I documents, each document i having J i event mentions.
Sentence 2: each event mention is characterized by L feature types, FT, and each feature type is represented by a finite number of feature values, f v. we can represent the observable properties of an event mention, em, as a vector of pairs (FT 1 : f v 1i ), .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The main interest of the first and third points is that we obtain a final estimator of θ 0 which is at least provably consistent: indeed, although our criteria are obtained from an assumption of independence, the generalization performance result also holds for "weakly" dependent observations and thus ensures the consistency of our approach.
Sentence 2: as opposed to most previous active learning heuristics, our estimator will always converge (in probability) to the ML estimator.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: To make the resulting optimal control computations numerically tractable, future observations need to be assumed to coincide with their most likely values (see e.g., [19,5]).
Sentence 2: at every time step, the agent solves a constrained nonlinear optimization problem min which returns an optimal sequence of controls u 0:N −1 minimizing the objective function J.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Note that during normal play, we do not have access to y t . We can however utilize past games for training as replaying them in StarCraft provides both o t and y t . We note that humans generally make forward predictions on a high level at a long time-scale -humans will not make fine predictions but instead predict a general composition of units over the whole game map.
Sentence 2: we propose to model a low-resolution game state by considering accumulated counts of units by type, downsampled onto a coarse spatial grid.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: On the other hand, a logistic and probit approach both 0 and 1 at the same rate, and therefore cannot model the sparsity/skewness of the label matrix like the Bernoulli-Poisson link.
Sentence 2: in contrast to multilabel learning models based on logistic/probit likelihood function or standard loss functions such as the hinge-loss [25,14] for the binary labels, our proposed model provides better robustness against label imbalance.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Given a gradient at step t, i , y We assume that only one data instance of one party changes in neighbouring datasets D and D . Hence the sensitivity bound, ∇J(θ, D) − ∇J(θ, D ) ≤ 2G mn (1) .
Sentence 2: using Lemma 2.1, (1) σ 2 .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Consequently, we can equate the output current of the bump circuit with the probability of the input under a distribution parameterized by mean µ: ( ) In addition, increasing the similarity between V in and µ is equivalent to increasing P(V in |µ).
Sentence 2: the adaptive bump circuit adapts to maximize the likelihood of the present input under the circuit's probability distribution.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Given the restrictions imposed on µ and σ, one can show that the normalizing quantity Z = 1 0 N (x; µ, σ 2 ) is always greater than one-third.
Sentence 2: to sample from the posterior we simply draw from the original, closed form posterior distribution, discarding (and re-sampling) if µ ∈ [0, 1], σ ∈ [0, 1], or with probability 1 − (3Z) −1 .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Therefore, the maximum value is observed in every 2 k−1 ≤ 2 sd samples.
Sentence 2: a direct application of LearnBool yields poly(n, 2 k−1 ) time complexity, which improves the O(n d ) bound for small s and d. Improving further, we provide a more efficient algorithm tailored for the hypergraph sketching problem, which makes use of the unique sign property and some other properties of the cut function.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: For this neuron, incorporating non-Poisson renewal properties into a model with spike history dependent intensity (light gray) provides only a modest (<1%) increase in cross-validation performance.
Sentence 2: in addition to being more tractable for estimation, it appears that the generalized linear modeling framework captures spike-train dependencies more accurately than a non-Poisson renewal process (at least for this neuron).
Relationship: reasoning
Domain: NeurIPS
Sentence 1: This guarantees that linear constraints of P(X N ) converge to the quadratic constraint of P, and hence the two problems match.
Sentence 2: now since S ⊆ T N for all N , it is easy to see that S ⊆ lim N →∞ T N . To prove the converse, let t 0 ∈ lim N →∞ T N but t 0 ∈ S. d(t 0 , S) > 0.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Let A be any (deterministic) clustering algorithm which, given the full sample X m+u , can cluster this sample into any desired number of clusters.
Sentence 2: we use A to cluster X m+u into 2, 3 . . . , c clusters where c ≤ m. the algorithm generates a collection of partitions of X m+u into τ = 2, 3, . . . , c clusters, where each partition is denoted by C τ .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: The following result has an alternate interpretation for λ: Result: The sparsegen formulation (Eq.1) is equivalent to the following, when γ = 1 1−λ (where The above result says that scaling g(z) by γ = 1 1−λ is equivalent to applying the negative L-2 norm with λ coefficient when considering projection of g(z) onto the simplex.
Sentence 2: we can write: This equivalence helps us borrow results from sparsemax to establish various properties for sparsegen.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: On the other hand, the approximately convex function fw (x) for f w (x) we consider will be such that fw (x) = G(x) except in a very small cone around w. Picking w at random, no algorithm with small number of queries will, with high probability, every query a point in this cone.
Sentence 2: the algorithm will proceed as if the function is G(x) and fail to optimize fw .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: When K < M , the subproblem with respect to s has solution s = (B B + σ 2 Φ) −1 x.
Sentence 2: lower dimensional random vector s has less variables among which each variable is a linear combination of the original variables of x with the combination weights stored in W = (B B + σ 2 Φ) −1 .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: A typical iterative minimization process based on the alternating direction method of multipliers (ADMM) [15,16] can be written explicitly as However, minimizing the augmented Lagrangian function with respect to M i , L and E using ADMM is expensive in practice, and moreover, the global convergence can not be guaranteed.
Sentence 2: we propose to solve the above problem by taking one proximal gradient step.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Nevertheless, in our FESL problem, on rounds T 1 + 1, . . . , T 1 + T 2 , w 2,t cannot satisfy the requirement in the beginning due to insufficient training data and w 1,t may become worse when more and more data come causing a cumulation of recovered error.
Sentence 2: it may not be appropriate to combine the two models all the time, whereas dynamically selecting the best single may be a better choice.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: (3) Third, the lack of sufficient evidence or an insignificant improvement of the metric may make it undesirable from a practical or financial perspective to replace the default.
Sentence 2: when a company runs hundreds to thousands of A/B tests within a year, ideally the number of statistically insignificant changes that it made should be small relative to the total number of changes made.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: For SNR estimation, the input signal is transformed into spectro-temporal input features, which are neurophysiologically-motivated: experiments on amplitude modulation processing in higher stages of the auditory system in mammals show that modulations are represented in "periodotopical" gradients, which are almost orthogonal to the tonotopical organization of center frequencies [5].
Sentence 2: both spectral and temporal information is represented in two-dimensional maps.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: We know from Theorem 2.1 that Ψ n ≤ Ψ n−1 + β n − Z n , where β n is non-stochastic and Z n is a quantity of positive expected value.
Sentence 2: in expectation, and modulo a small additive term, Ψ n decreases monotonically.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: A necessary condition for identification is that the total number of spikes generated in response to all N trials is larger than Proof: The equivalent representation of the t-transform in equations ( 4) and (7) implies that the decoding of the stimulus u m nm (in Theorem 1) and the identification of the filter projections Ph m nm encountered here are dual problems.
Sentence 2: the receptive field identification problem is equivalent to a neural encoding problem: the projections Ph m nm , m = 1, ..., M , are encoded with an mTEM comprised of N neurons and receptive fields u im nm , i = 1, ..., N , m = 1, ..., M . The algorithm for finding the coefficients h m l1...ln m is analogous to the one for u m l1...ln m in Theorem 1.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Let κ(Σ) be the condition number of the covariance matrix Σ, i.e., the ratio of the maximum and minimum eigenvalues; refer to Section 2 for other relevant definitions. .
Sentence 2: there exist two covariance matrices Σ and Σ that are good entrywise approximations: but the corresponding distributions are as far as they can get, i.e., given a black-box algorithm that provides an entrywise ε-approximation to the true covariance matrix Σ, if ε ≥ 2 κ(Σ)+1 , it might output Σ, which does not approximate Σ in KL divergence or total variation distance.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Generally speaking, user U i gives item For example, if the ground-truth score of item I j is 4.8 for user U i , then user U i makes a noisy rating by giving I j score 5, yet makes an attack by giving I j score −3.
Sentence 2: we assume that Z F ≤ δ, where δ is a small parameter.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: From the form of the truncated ΓP, thanks to the additive property of independent gamma random variables, the total sum of λ k over countably infinite atoms follows a gamma distribution as ∞ k=1 λ k ∼ Gamma(γ 0 , c 0 ), wherein the intensity function of the ΓP has a finite expectation as E[ .
Sentence 2: the ΓP has a regularization mechanism that automatically shrinks the number of atoms according to given observations.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: This priming mechanism yields priors on average that match the presentation probabilities in the task, e.g., .17 and .83 for the two responses in the 17:83 condition of the Jones and Braver experiment.
Sentence 2: when we report results for overall error rate and reaction time in a condition, we make the assumption of rationality that the model's priors correspond to the true priors of the environment.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: We have Let i be the total weights at time t. For each t, similarly to (3), we have For each i such that err Di (g (t) ) ≤ 12 , by Lemma 5, know that Pr[i ∈ Z (t) ] ≤ 1 100 .
Sentence 2: if we take the expectation over the randomness of WEAKTEST at time t, we have, When χ (t) = 0, similarly to the proof of Theorem 3, we have Pr i∼p (t) (•) err Di (g (t) ) > 12 ≤ .1, and Combining ( 7), (8), and ( 9), we have (when Together with (6), we have E W (t+1) W (0) , .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In round 0, pattern nodes are initialized by a pattern x, sampled from dataset X , perturbed by external errors z, i.e., x(0) = x + z.
Sentence 2: for cluster we have x ( ) (0) = x( ) + z ( ) , where z ( ) is the realization of errors on subpattern x ( ) .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Briefly, CG ensures that z after iteration k is a maximizer of q over a (Krylow) subspace of dimension k. For details about CG and many other approximate linear solvers, see [15].
Sentence 2: z "converges" to p (the unconstrained maximizer of q) after n steps, but intermediate z can be used as approximate solutions.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: At each iteration, the computational cost is dominated by the update of Z (−n) (n = n), which can be obtained by where (−n, −n) denotes every mode except n and n, which greatly reduces the computation.
Sentence 2: when n = n, the updates of Z (−n) , e, Z (−n) e and Diag(Z  We provide convergence analysis for our stagewise algorithm in this section.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Our setup in (12) introduces a projection operator before Ψ γ a γ in the basis (11) and therefore it can be viewed as the unique hybrid parametrization, which confines the function modeled by γ to be orthogonal to the span the β basis.
Sentence 2: there is no correlation between optimizing a γ and a β , making the problem more well-conditioned.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: By Hoeffding's inequality and union bounds, we have that with a probability of at least It can be easily seen that ŷi = y i for all i ∈ J.
Sentence 2: for all x such that κ(x, U(S a (t))) ∈ J, h nn S (x) = h nn Sa(t) (x), and hence err( The second term is at most ν(t), and it remains to bound the first term, on the condition that E holds.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: A Bayesian network is a triple (X, G, θ), where G = (N, A) is a directed acyclic graph (DAG) whose nodes are in one-to-one correspondence with variables in X, and θ = {θ i (x i , x Gi )} is a set of numerical parameters specifying (conditional) probabilities θ i (x i , x Gi ) = Pr(x i |x Gi ), for every node i in G, value x i of X i and assignment x Gi to the parents G i of X i in G. The structure G of the network represents a set of stochastic independence assessments among variables in X called graphical Markov conditions: every variable X i is conditionally independent of its nondescendant nonparents given its parents.
Sentence 2: a Bayesian network uniquely defines a joint probability distribution over X as the product of its parameters.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Recall that the loss function is defined as follows: weight of the edge ((n + 1) a , ⊥) is y t , weight of the edge ((n + 1) b , ⊥) is 1 − y t ; other edges have loss 0.
Sentence 2: the loss suffered by the policy π S is 1 if PAR S (x t ) = y t and 0 otherwise.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In practice, however, we do not know which of the two events is correct.
Sentence 2: the ideal choice in either case can be expressed using indicators as follows: Taking the expected value of A ij (X) with respect to p(x|C i (x) ∨ C j (x)) yields This results in a single likelihood correction S ij (x) = Lij (x) + α ij that is obtained simply by integrating in Equation 5 with respect to the average distribution over class c i and c j , i.e., With these assumptions, and supposing the zero set is Lebesgue measurable with measure µ, we get: where As can be seen, α ij is composed of two factors, the integral and the 1/Ψ(P i , P j ) factor.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: A group's mixing weights are randomly chosen from w 1 = [0.33, 0.33, 0.33] or w 2 = [0.84, 0.08, 0.08].
Sentence 2: a group is normal if its points are sampled from these three Gaussians, and their mixing weights are close to either w 1 or w 2 .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: As defined in (5), our regularizer has a parameter, β i , associated with each training point, which can be used to encode the information of the marginal or input distribution, P (x), by setting where λ is a tradeoff or regularization parameter.
Sentence 2: the use of β i would make the regularization take effect only in dense regions although our experiments reported were carried out by setting β i = 1 2 ; i.e., we were using a weak assumption that data are scattered uniformly throughout the whole space.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: With probability Pr(d L ) • Φ(0 | µ t , σ t ), the hidden state x is less than 0, making A R an incorrect decision and resulting in a penalty and earns a reward of R P . The inverse is true for A L . When A S is selected, the animal simply receives an observation at a cost of R S . Computing the value function defined in ( 5) involves an expectation with respect to future belief.
Sentence 2: we need to compute the transition probabilities over belief states, T (b t+1 |b t , a), for each action.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: In order to get an expression for the weight change in a specific time step t k we write the probabilities occurring in (7) as products over individual time bins, i.e., P ( ), according to the chain rule of information theory [5].
Sentence 2: we rewrite L 12 as a sum over the contributions of the individual time bins, The weight change ∆ wk 1j is then proportional to the gradient of this expression with respect to the weights w 1j , i.e., ∆ wk 1j = α(∂∆L k 12 /∂w 1j ), with some learning rate α > 0.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: By the union bound, this event happens with probability 1 − tδ = 1 − δ/4.
Sentence 2: consider the set of players i / ∈ I (r) ∪ G r . These are the players for whom the classifier of the round performed well but FASTTEST made a mistake and did not include them in the set G r . By linearity of expectation: the expected value of the potential function in round r conditioned on its value in the previous round is bounded by 1) .
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Theorem 1 provides an upper bound on the number of measurements that is equal to a constant factor times the statistical dimension of the descent cone, d(T L λ (x 0 )).
Sentence 2: even though our analysis is not exact in this section, it leads us to the important observation that our proposed method is order-wise optimal in terms of the required sample complexity for perfect signal reconstruction.
Relationship: reasoning
Domain: NeurIPS
Sentence 1: Specifically, v(•) is called a monotone operator [21] if the following holds (with equality if and only if x = x): If v(•) is monotone, the game admits a unique Nash equilibrium x * which (per the property of a Nash equilibrium) satisfies v(x * ), x − x * ≤ 0.
Sentence 2: if v(•) is a monotone operator, it follows that v(x), x − x * ≤ v(x * ), x − x * ≤ 0, where equality is achieved if and only if x = x * .
Relationship: reasoning