sadat2307/MSciNLI huggingface dataset:
Domain: NeurIPS
Sentence 1: The "full-exponential-family-ness" is formally captured by the statistical curvature originally defined by Efron (1975), and is a measure of the minimum loss of Fisher information when summarizing the data using first order efficient estimators (e.g., Fisher, 1925, Rao, 1963.
Sentence 2: we show that arbitrary combinations of the local MLEs on the local datasets can approximate the global MLE on the whole dataset at most up to an asymptotic error rate proportional to the square of the statistical curvature.
Relationship: entailment
Domain: NeurIPS
Sentence 1: We need such an orthogonal transformation, because cos ⇥ E r , U s 2 F can be expressed as Since the sequence {z sh , U s } 1 s=1 forms a discrete Markov process, we can apply diffusion approximations to establish global convergence of SGD.
Sentence 2: by a continuous time interpolation, we construct continuous time processes U ⌘ (t) and X ⌘ (t) such that U ⌘ (t) = U bt/⌘c+1 and X ⌘ (t) = X bt/⌘c+1 .
Relationship: entailment
Domain: NeurIPS
Sentence 1: Exact maximum likelihood learning in this model is intractable, but efficient approximate learning can be carried out by using mean-field inference to estimate data-dependent expectations, and an MCMC based stochastic approximation procedure to approximate the model's expected sufficient statistics [1].
Sentence 2: during the inference step, we approximate the true posterior P (h|v; θ), where v = {v m , v t }, with a fully factorized approximating distribution over the five sets of hidden units {h m , h t , h (3) }: tl |v) where µ = {µ m , µ t , µ t , µ (3) } are the mean-field parameters with q(h Learning proceeds by finding the value of µ that maximizes the variational lower bound for the current value of model parameters θ, which results in a set of the mean-field fixed-point equations.
Relationship: entailment
Domain: NeurIPS
Sentence 1: The crux of our proof argument lies in showing that t − t+1 ≥ J t+1 (w t+1 ) − J t (w t ) (see Theorem 4) is sufficiently bounded away from 0.
Sentence 2: since t bounds the distance from the optimality, at every step Algorithm 1 makes sufficient progress towards the optimum.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Our heuristic deal with this is to compute a quantity ↵ 0 i,j,p which is analogous to the ↵ i,j defined above, but is defined for every pixel p. Then we compute ↵ i,j for a given patch by averaging ↵ 0 i,j,p over all pixels in the patch.
Sentence 2: we compute ↵ i,j for patch i as the mean over all pixels p in that patch of the following quantity: Where Ov(p) denotes the set of features for positive patches that contain the pixel p. It is admittedly difficult to analyze how well these heuristics approximate the adaptive bandwidth approach of [4], and even there the setting of the bandwidth for each datapoint has heuristic aspects.
Relationship: entailment
Domain: NeurIPS
Sentence 1: where {ψ i , ψ α } are (given) non-negative functions, the so-called factors; F is a collection of subsets (each α j is a subset of {1, 2, . . . , n} with |α j | ≥ 2); z α is the projection of z onto dimensions included in α.
Sentence 2: 2 ψ i is called a variable factor.
Relationship: entailment
Domain: NeurIPS
Sentence 1: As a consequence, consists of conjunctions and disjunctions of linear (in)equalities; standard tools exist for checking whether such formulas are satisfiable [12].
Sentence 2: the controller is correct if and only if ¬ is unsatisfiable, since a satisfying assignment to ¬ is a counterexample showing that does not always hold.
Relationship: entailment
Domain: NeurIPS
Sentence 1: In this work we analyze Stochastic Gradient Descent (SGD) with k-sparsification or compression (for instance top-k or random-k) and show that this scheme converges at the same rate as vanilla SGD when equipped with error compensation (keeping track of accumulated errors in memory).
Sentence 2: communication can be reduced by a factor of the dimension of the problem (sometimes even more) whilst still converging at the same rate.
Relationship: entailment
Domain: NeurIPS
Sentence 1: We show a general reduction from the problem of searching for a violated comparison S 2 C to the problem of agnostic learning over the corresponding family of boolean functions.
Sentence 2: recall that for a collection of comparisons C ✓ 2 X ⇥X , we can also think of C as a family of boolean concepts, where for each S 2 C, there is an associated boolean function c We frame this search problem as an agnostic learning problem, where we design a set of "labels" for each pair (x, x 0 ) 2 X ⇥ X such that any function that is highly correlated with these labels encodes a way to update the parameters towards feasibility.
Relationship: entailment
Domain: NeurIPS
Sentence 1: The idea for choosing these partial sums is pictured in Figure 1: For a function s(t) that returns an integer smaller than t, we take f t = f s(t)+1:t + f s(s(t))+1:s(t) + • • • + f 0:0 .
Sentence 2: s(t) is determined by writing t in binary, then flipping the rightmost "one" bit to zero.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Although ARC-I enjoys the flexibility brought by the convolutional sentence model, it suffers from a drawback inherited from the Siamese architecture: it defers the interaction between two sentences (in the final MLP) to until their individual representation matures (in the convolution model), therefore runs at the risk of losing details (e.g., a city name) important for the matching task in representing the sentences.
Sentence 2: in the forward phase (prediction), the representation of each sentence is formed without knowledge of each other.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Our results can be generalized to all functions that are "close" to a mixture of log-concave distributions.
Sentence 2: more precisely, assume the function 𝑓 satisfies the following properties: where ∇𝑓 0 (0) = 0, and 𝑓 is within a 𝑒 Δ multiplicative factor of an (unknown) mixture of log-concave distributions.
Relationship: entailment
Domain: NeurIPS
Sentence 1: With the above definitions, we show in the following theorem that when X and Y are both full row rank, (G 0 , E 0 ) is the unique solution to Eq.
Sentence 2: ( 2) with high probability as long as there are O(r log N ) observed components in F. with a sampling rate of O(r log N ), our method can fully recover both E 0 and G 0 with a high probability when X and Y are full row rank.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Below we describe an alternate mapping in place of pred * MAP which can be computed efficiently, and show that under certain conditions on the probability distribution, the surrogate ψ * MAP together with this mapping is still calibrated for � MAP .
Sentence 2: define pred MAP : R r(r+1)/2 →S r as follows: Clearly, pred MAP (u) can be implemented efficiently by simply sorting the 'diagonal' elements u ii for i ∈ [r].
Relationship: entailment
Domain: NeurIPS
Sentence 1: The variational distribution q φ (z t |z t−1 , d t:T , x t:T ) needs to approximate the dependence of the true posterior p θ (z t |z t−1 , d t:T , x t:T ) on d t:T and x t:T , and as alluded to in (7), this is done by running a RNN with inputs d t:T and x t:T backwards in time.
Sentence 2: we initialize the hidden state of the backward-recursive RNN in Figure 2b as a T +1 = 0, and recursively compute a t = g φa (a t+1 , [ d t , x t ]).
Relationship: entailment
Domain: NeurIPS
Sentence 1: (that is, we do alternating optimization of the bits over the blocks).
Sentence 2: to optimize over the bits in block B, ignoring the constants, we can rewrite equation ( 6) in the standard form for the GraphCut algorithm as: where v nm = a nm , u nm = 2 m ∈B a nm z mi − µh i (x n ).
Relationship: entailment
Domain: NeurIPS
Sentence 1: There are two shortcomings of SMC: (i) it does not handle inference over hyper-parameters; (ii) despite the fact that the sampler targets the joint smoothing distribution, it does in general not provide an accurate approximation of the full joint distribution due to path degeneracy.
Sentence 2: the successive resampling steps cause the particle diversity to be very low for time points t far from the final time instant T . To address these issues, we propose to use a particle Markov chain Monte Carlo (PMCMC, [3,13]) sampler.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Substituting yields Note that 1) ρ is the empirical distribution over the observations, and so converges almost surely (a.s.) to the true generating distribution, and 2) the convergence properties of P n are determined by the quantity between the square brackets in (2).
Sentence 2: let L ∞ is commonly referred to as the likelihood ratio, familiar from classical binary hypothesis testing.
Relationship: entailment
Domain: NeurIPS
Sentence 1: A particularly remarkable feature is that the choice of H k is much easier for the problems that we consider than in the binary classification problem.
Sentence 2: if (for some fixed k) the classifier that achieves the minimal (Bayes) error for the classification problem is not in H k , then obviously the error of an empirical risk minimizer will not tend to zero, no matter how much data we have.
Relationship: entailment
Domain: NeurIPS
Sentence 1: By generalizing linear response methods from statistical physics [9][10][11][12] to exponential family variational posteriors, we develop a methodology that augments MFVB to deliver accurate uncertainty estimates for model variables-both for individual variables and coherently across variables.
Sentence 2: as we elaborate in Section 2, when the approximating posterior in MFVB is in the exponential family, MFVB defines a fixed-point equation in the means of the approximating posterior, and our approach yields a covariance estimate by perturbing this fixed point.
Relationship: entailment
Domain: NeurIPS
Sentence 1: ( 2) to be robust against variations of the mean and covariance matrix within these sets: inf Pr{aTy::; b} 2 a V(y,~y) E y.
Sentence 2: we would like to guarantee a worst-case misclassification probability for all distributions which have unknown-but-bounded mean and covariance matrix, but which are otherwise arbitrary.
Relationship: entailment
Domain: NeurIPS
Sentence 1: . . , d K (i, j)] for all training samples 4: Construct Pareto fronts on set of all dyads until each dyad is in a front Testing phase: Calculate dissimilarities between test sample X and all training samples in criterion l 4: nb ← [nb, nb l ] {append neighbors to list} 6: Create s new dyads D new i between X and training samples in nb 7: . . , D new s }, corresponding to the connections between X and the union of the k i nearest neighbors in each criterion i.
Sentence 2: we create a dyad between X and X j if X j is among the k i nearest neighbors 1 of X in any criterion i.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Then, using the right singular vectors V n of H , the next step consists of computing a weighted automaton A Z = α, β, {A a } as follows: The fact that the spectral method is based on a singular value decomposition justifies in part the use of a Schatten p-norm as a regularizer in (HMC-H).
Sentence 2: two very natural choices are p = 1 and p = 2.
Relationship: entailment
Domain: NeurIPS
Sentence 1: To this end, we establish formal connections between the PR framework with a broad set of algorithms in the control and reinforcement learning (RL) domains, and, based on the connections, transfer well-developed RL techniques for constraint learning in PR.
Sentence 2: though the PR framework and the RL are apparently distinct paradigms applied in different context, we show mathematical correspondence between the model and constraints in PR with the policy and reward in entropyregularized policy optimization [43,45,1], respectively.
Relationship: entailment
Domain: NeurIPS
Sentence 1: The Generalized Mallows model (GMM) of Fligner and Verducci (1986) extends the Mallows model by introducing a parameter for each of the elements in E and decomposes the inversion distance into a per element dis-tance 1 .
Sentence 2: we define v j (π, π 0 ) to be the number of inversions for element j in π with respect to π 0 is v j (π, π 0 ) = i>π 0 j D ij (π, π 0 ).
Relationship: entailment
Domain: NeurIPS
Sentence 1: We show that there is a trade off between computational complexity and query complexity; We prove that for the case of k-means clustering (i.e., when the expert conforms to a solution of k-means), having access to relatively few such queries allows efficient solutions to otherwise NP hard problems.
Sentence 2: we provide a probabilistic polynomial-time (BPP) algorithm for clustering in this setting that asks O k 2 log k + k log n) same-cluster queries and runs with time complexity O kn log n) (where k is the number of clusters and n is the number of instances).
Relationship: entailment
Domain: NeurIPS
Sentence 1: Equivalently, we may drop this effort() term from (2) altogether while requiring the search space D to consist of teaching sets of size exactly n. In this paper, we consider test distributions p(x, y) whose marginal on x has a special form.
Sentence 2: we assume that p(x) is a uniform distribution over m distinct test stimuli z 1 , .
Relationship: entailment
Domain: NeurIPS
Sentence 1: The second result in [1] is for the regime where the connectivity matrix scales as ln(n)Q/n, Q independent of n, where it is shown that exact recovery has a sharp threshold characterized by the divergence function named the CH-divergence in [1].
Sentence 2: if all pairs of columns in diag(p)Q are at D + -distance at least 1 from each other, then exact recovery is solvable in the general SBM.
Relationship: entailment
Domain: NeurIPS
Sentence 1: To the reader not familiar with this model, it should be considered as nothing more but an idealization of the well-known fixed design regression problem Y i = s(i/n) + ε i for i = 1, . . . , n, where ε i ∼ N (0, 1), where the goal is to recover the regression function s. (The white noise model is actually simpler to study from a mathematical point of view).
Sentence 2: the least square criterion is defined as Given a Mercer kernel k on [0, 1]×[0, 1], the regularization least square procedure proposes to minimize where (ζ n ) is a conveniently chosen sequence and H k denotes the RKHS induced by k. This procedure can indeed be viewed as a model selection procedure since minimizing regularization aims at selecting the "best" RKHS ball {f, f ≤ R} to represent our data.
Relationship: entailment
Domain: NeurIPS
Sentence 1: To deliver robust features, we implement two kinds of high-level pooling frameworks: Bag-of-Systems (BoS) [11] and Spatial-Temporal-Pyramid-Matching (STPM) [16] 3 .
Sentence 2: 1) BoS is performed with the clustering methods, i.e., MDS, Align and PGD.
Relationship: entailment
Domain: NeurIPS
Sentence 1: We used the logistic loss for a model matrix W ∈ R D×C .
Sentence 2: for each training example x i with label y i ∈ {1, .., C}, we defined an individual loss L i (W ) as where for any class c, Then L(W ) is defined as the average of L i (W ) over the whole training set.
Relationship: entailment
Domain: NeurIPS
Sentence 1: The main contributions of this paper are highlighted as follows: • Our result for ASYSG-CON generalizes and improves earlier analysis of ASYSG-CON for convex optimization in [Agarwal and Duchi, 2011].
Sentence 2: we improve the upper bound of the maximal number of workers to ensure the linear speedup from O(K 1/4 M −3/4 ) to O(K 1/2 M −1/2 ) by a factor K 1/4 M 1/4 ; • The proposed ASYSG-INCON algorithm provides a more accurate description than HOGWILD!
Relationship: entailment
Domain: NeurIPS
Sentence 1: Kernel ridge regression considers a feature mapping Φ : X → F before performing regularized linear regression.
Sentence 2: according to Φ, the kernel function k(x, x ) = Φ(x) T Φ(x ) is defined as the inner product in the space F. When applying kernel ridge regression with a regularization parameter λ to map from X to ZV, if Φ(x) can be explicitly computed, it is known that the closed-form solution is [22]  where Φ is the matrix containing Φ(x n ) T as rows, and K is the matrix with K = ΦΦ T and is called the kernel matrix of X.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Building on this observation, it is reasonable to devise algorithms that can detect and separate out the generalized gradient components corresponding to mistakenly estimated signs a T i zt |a T i zt| along the iterates {z t }.
Sentence 2: if z t and x lie in different sides of the hyperplane a T i z = 0, then the sign of a T i z t will be different than that of a T i x; that is, where h := z − x.
Relationship: entailment
Domain: NeurIPS
Sentence 1: [19] proved EEB of g pu is tighter than EEB of g pn when π p / (3) and is Lipschitz continuous; (b) the Rademacher complexity of G decays in O(1/ √ n) for data of size n drawn from p(x), p p (x) or p n (x).
Sentence 2: 4 under mild conditions, PU learning is likely to outperform PN learning when π p / √ n p + 1/ √ n u < π n / √ n n . This phenomenon has been observed in experiments [19] and is illustrated in Figure 1(a).
Relationship: entailment
Domain: NeurIPS
Sentence 1: In this section, we present numerical results on both synthetic and real datasets to verify the theoretical properties of our algorithm, and compare it with the state-of-the-art methods.
Sentence 2: we compare our method, denoted by AltGD, with two convex relaxation based methods for estimating LVGGM: (1) LogdetPPA [9,32] for solving log-determinant semidefinite programs, denoted by PPA, and (2) the alternating direction method of multipliers in [22,24], denoted by ADMM.
Relationship: entailment
Domain: NeurIPS
Sentence 1: When BN is applied after a linear layer, it is well known that the output is invariant to the channel weight vector norm.
Sentence 2: denoting a channel weight vector with w and ŵ = w/ w 2 , channel input as x and BN for batch-norm, we have This invariance to the weight vector norm means that a BN applied after a layer renders its norm irrelevant to the inputs of consecutive layers.
Relationship: entailment
Domain: NeurIPS
Sentence 1: In the most general formulation, the constraints, Q : R IK ×R KJ → R NQ and R : R IK ×R KJ → R NR , are biaffine maps, that define N Q inequality and N R equality constraints jointly in a and b.
Sentence 2: each inequality constraint has the form By rearranging terms and combining the N Q constraints in matrix notation, we may write from which it is clear that the constraints are linear in a.
Relationship: entailment
Domain: NeurIPS
Sentence 1: The result says that, subject to widely believed conjectures in computational complexity, no efficient algorithm exists for approximating the partition function of certain hard-core models.
Sentence 2: recall that the hard-core model with fugacity is given by (2.1) with Assuming NP 6 = RP, there exists no FPRAS for computing the partition function of the hard-core model with fugacity on regular graphs of degree d. no FPRAS exists when = 1 and d 5.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Unsupervised feature learning algorithms have recently attracted much attention, with the promise of letting the data guide the discovery of good representations.
Sentence 2: unsupervised feature learning is an important component of many Deep Learning algorithms (Bengio, 2009), such as those based on auto-encoders (Bengio et al., 2007) and Restricted Boltzmann Machines or RBMs (Hinton et al., 2006).
Relationship: entailment
Domain: NeurIPS
Sentence 1: This gives the variational lower bound L(q): where H[•] denotes the Shannon entropy.
Sentence 2: from inspecting the VB lower bound ( 12) and ( 6) we arrive at the following induced factorizations without forcing further factorization upon (11): the approximate posterior on assignment matrices factorizes across time; and the approximate posterior on latent states factorizes across tracks.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Consider the support set J t = S t−1 ∪ supp ∇F (x t−1 ), r . We have for any 0 < θ ≤ 1/ρ + , where ν = √ s − r + 2.
Sentence 2: picking θ = 1/ρ + gives Note that the lemma also applies to the two-stage thresholding algorithms (e.g., CoSaMP [15]) whose first step is expanding the support set.
Relationship: entailment
Domain: NeurIPS
Sentence 1: In the second set of experiments, we evaluate the performance of the coordinate-update mfNMF algorithm based on the multiplicative updating algorithm of the SMS problem (Section 4.1).
Sentence 2: we consider the mfNMF problem that approximates a randomly generated target nonnegative matrix V of dimension m×n with the product of three stochastic factors, W 1 (m×m ), W 2 (m ×n ), and W 3 (n × n).
Relationship: entailment
Domain: NeurIPS
Sentence 1: As generator at stage I, we adopt a U-Net-like architecture [20], i.e., convolutional autoencoder with skip connections as is shown in Figure 2.
Sentence 2: we first use several stacked convolutional layers to integrate I A and P B from small local neighborhoods to larger ones so that appearance information can be integrated and transferred to neighboring body parts.
Relationship: entailment
Domain: NeurIPS
Sentence 1: Despite the apparent complexity of this objective, we find that approximate inference using an alternating minimization algorithm, like in [15], works well in practice.
Sentence 2: we create explicit auxiliary variables w i (n) for the coefficients, and solve the following modified optimization problem: Note that the second term above forces coefficients of y(n) to be equal to the corresponding auxiliary variables w i (n), as β → ∞.
Relationship: entailment
Domain: NeurIPS
Sentence 1: When the algorithm knows the size of each cluster, we modify the definition of ē by replacing the terms |A (t−1) j |'s with the actual sizes |A j |'s.
Sentence 2: when the algorithm knows that the clusters are of equal size, we obtain the following refinement rule: we put user i to A (t) j * , where j * = arg max j∈[k] c 1 • e {i}, A (t−1) j + Π i (u (j) ) for some hyperparameter c 1 > 0.
Relationship: entailment
Domain: NeurIPS
Sentence 1: In both settings, we allow our algorithm to be improper, that is, the learned functions need not belong to F. We compare the sample complexity of our algorithms to their PAC counterparts in the realizable setting.
Sentence 2: in the traditional realizable PAC setting, m ,δ denotes the number of samples needed for ( , δ)-learning F. m ,δ is the total number of samples drawn from a realizable distribution D, such that, with probability 1 − δ, any classifier f ∈ F that is consistent with the sample set satisfies err D (f ) ≤ .
Relationship: entailment
Domain: NeurIPS
Sentence 1: Assume for instance that θ i = 0 for all i ≥ 3.
Sentence 2: it follows from (1) that for any x ∈ {−1, 0, 1} n , with H(x) defined as in the proof of Proposition 1: two parameters θ, θ ′ such that θ 1 θ 2 = θ ′ 1 θ ′ 2 and θ i = θ ′ i = 0 for all i ≥ 3 cannot be distinguished.
Relationship: entailment
Domain: NeurIPS
Sentence 1: 5 The motivation for the specific choice of the problem formulation (6-8) is that projecting S onto § or § f can be done inexpensively.
Sentence 2: the first projection step S ¡ 6 B 3 C ¢¡ §£ 8 ¢ B DB S ¥¤ E S B B f ¦ ¡ S 4¤ @) 5 § ¡ involves minimizing a quadratic objective subject to a single linear constraint; the solution to this is easily found by solving (in V !
Relationship: entailment
Domain: NeurIPS
Sentence 1: 500 of these input pairs and targets were used as training data, and presented one per step to the LSNN during the episode, where each step lasted 20 ms. LSNN parameters were updated using BPTT to minimize the mean squared error between the LSNN output and the target in the training set, using gradients computed over batches of 10 such episodes, which formed one iteration of the outer loop.
Sentence 2: each weight update included gradients calculated on the input/target pairs from 10 different TNs.
Relationship: entailment