dataesr/scientific-paragraphs-categorization huggingface dataset:
With some abuse of notation, the proposal kernel R can be written as a transition kernel on the alphabet {0,. , N -n + 1}. It is defined in this example as: R(i; j) = 1 i =j ω exp (-λ|j -i|) j≤| Ūn|, j =i exp (-λ|j -i|) + (1 -ω) 1 | Ūn | -1. ( 39 ) The rationale is to propose a new subset through a mixture of two distributions: the first gives higher weight to local moves and the latter allows jumps to remote sections of the time series. In this example, we have used ω = 0. 9 and λ = 0. 1. We study the efficiency of ISS-MCMC in function of n, and S. For any subsample Y U , U ∈ Ūn , we have set the summary statistics S(Y U ) to the solution of the AR(2) Yule-Walker equations for the dataset Y U. As shown in Figure  4 , this choice of summary statistics satisfies (graphically) A. 4 with γ ≈ 5 10 6.
e TV(t) = 1 d d j=1 π (j) - π(j) t , where π (j) and π(j) t are respectively the true j-th marginal and the j-th marginal of the chain distribution after a runtime of t seconds. The true marginals were estimated from a long Metropolis-Hastings chain, at stationarity. π(j) t was estimated using 500 independent chains starting from the prior. The Matlab function ksdensity in default settings was applied to estimate π (j) -π(j) t TV from the chains samples, hence the variability. On the one hand, when = 0, there is no informed search for subsamples which makes the algorithm much faster than the other setups but yields a significant bias (larger than 0. 5). On the other hand, setting > 0 adds to the computational burden but allows to reduce the bias. In fact, for n = 5, 000 and a computational budget of t = 1, 000 seconds, the bias of ISS-MCMC is similar to that of Metropolis Hastings but converges 100 times as fast.
Logistic Regression Example X i = (X i,1 ,. , X i,d ) ∼ N (0, (1/d) 2 ) • simulate Y i given θ and X i as Y i = 1 w. p. 1/ 1 + e -θX T , 0 otherwise. ( 40 ) We have simulated N = 10 6 observations Y 1 , Y 2 ,. under the true parameter θ * = (1, 2, -1) (d = 3). The summary statistics were set as the maximum likelihood estimator returned by the Matlab routine glmfit and were graphically validated, as in Figure  4. The tolerance parameter was consequently set = 5. 0 10 6. We study the influence of n on the Informed Sub-Sampling chain marginal distributions in Figure  10. We note that as soon as n ≥ 5, 000, the bias vanishes and that when random subsampling is used (i. e = 0), the bias is much larger. Of course, Figure  10  only gives information about the marginal distributions. To complement the study, we consider estimating the probability π(D) where D is the domain defined as: D ⊂ Θ = {θ 1 ∈ (0.
Binary Classification Example 5. A training dataset consisting of N = 10 7 labeled observations Y = {Y k , k ≤ N } from a 2 dimensional Gaussian mixture model is simulated with Y k | I k = i ∼ N (µ i , Γ i ) , I k ∼ Bernoulli(1/2) , where µ 1 = [θ 1 , 0], µ 2 = [θ 2 , 0], Γ 1 = diag([θ 3 /2 , θ 3 ]) and Γ 2 = diag([θ 4 /2 , θ 4 ]). We define θ = (θ 1 , θ 2 ) with θ i ∈ R × R + * for each model i ∈ {1, 2}. A prior distribution (θ 1 , θ 2 ) ∼ i. i. d. p := N (0, 1/2) ⊗ Γ(1, 2) (Γ(a, b) is the Gamma distribution with shape a and rate b) is assigned to θ. Consider an algorithm a that simulates a Markov chain   2017 )) and the Stochastic Gradient Langevin Dynamic (SGLD)  Welling and Teh (2011). θ a := θ (a) 1,k , θ (a The plots represent the marginal distribution πi , (after i = 1, 000 iterations) and different subset sizes n ∈ {1, 000 ; 5, 000 ; 10, 000}.
The true marginal π is in black. πi was estimated by simulating 1,000 iid copies of the Markov chain generated by the five algorithms. SubLhd1 and SubLhd2 implemented with n ∈ {1, 000 ; 5, 000 ; 10, 000}. The Monte Carlo estimation of π(D) was carried out using those algorithms for 500 seconds. Each estimation was replicated 100 times and started from θ * for each algorithm. Note that for SubLhd1 and SubLhd2, n corresponds to the initial subset size and not to the actual subsample size that was actually used in each iteration, a parameter which is adaptively tuned by the algorithms. targeting the posterior distribution of θ given Y , perhaps approximately. We consider the real-time supervised classifier I * a (t), driven by θ a , for the test dataset Y * = {Y * k , k ≤ N test } (N test = 10 4 ) and defined as: I * a (t) = (I * a,1 (t),.
0812 classifying Y * k in class 1 if Y * k,1 < 0 and in class 2 alternatively. Unsurprisingly, Metropolis-Hastings is penalized because it evaluates the norm of a N = 10 7 dimensional vector at each iteration. Subsampled Likelihoods does slightly better than M-H but suffers from the fact that close to stationary regime, the algorithm ends up drawing the quasi-entire dataset with high probability, a fact which was explained in  Bardenet et al. (2014).
Additional details for the handwritten digit inference (Example 1) In the handwritten digit example (Example 1), we have used batches of n = 100 data. Since the initial dataset comprises 2, 000 observations per digit, the summary statistics were defined in a way that any subsample contains 20 observations from each class. More precisely, we have set for any subsample Y U , S(Y U ) = 0 if for at least one class i ∈ {1,. , 5}, (1/n) k∈U 1 {J(k)=i} = 20 and otherwise. We set the bandwidth to = 10 5. The proposal kernels of the Informed Sub-Sampling chain and M-H were defined as the same Random Walk kernel. In particular, at each iteration only a bloc of the template parameter of one of the five classes is updated. The variance parameter of the Random Walk is adapted according to the past trajectory of the chain, so as to maintain an acceptance rate of.
Conclusion When the available computational budget is limited, inferring a statistical model based on a tall dataset in the Bayesian paradigm using the Metropolis-Hastings (M-H) algorithm is not computationally efficient. Several variants of the M-H algorithm have been proposed to address this computational issue  (Bardenet et al. , 2014; Banterle et al. , 2015; Korattikara et al. , 2014; Maclaurin and Adams, 2015). However, (i) they often lose the original simplicity of M-H, (ii) they are only applicable in situations where the data are independent and (iii) the computational cost of one iteration is stochastic which can potentially compromise any computational saving. Informed Sub-Sampling MCMC pushes the approximation one step forward: the computational cost of one iteration is deterministic and is controlled through a user specified parameter, the size of the subsamples.
The aforementioned methods rely on subsampling the whole dataset uniformly at random, at each iteration. Using such subsamples in lieu of the whole dataset in the original M-H chain leads to an algorithm whose approximation of π comes with no guarantee when the subsamples size is fixed. Our main contribution is to show that assigning a distribution to the subsamples that reflects their fidelity to the whole dataset allows one to control the L1 approximation error, even when the subsample size is fixed, which is of computational interest. The algorithm we propose to achieve this task, Informed Sub-Sampling MCMC, offers an alternative to situations where other scalable Metropolis-Hastings variants cannot be implemented (because the model does not satisfy the assumptions e. g. independence of the data, existence of a concentration inequality for the model or a cheap lower bound of the likelihood) or are inefficient (because the method ends up using nearly the whole dataset at each iteration).
We leave the design and evaluation of such a method for future research. Similarly to the other noisy subsampling methods that approximate MH (e. g. Welling and Teh (2011) ,  Korattikara et al. (2014) ,  Bardenet et al. (2017) , etc. ), our theoretical results address the convergence in distribution of the ISS-MCMC chain to the posterior distribution. Questions of interest to practitioners encompass establishing Law of Large Numbers and Central limit theorems for those types of algorithms. Addressing those questions would allow a better understanding of approximate MCMC applied to large dataset contexts, from a practical perspective. In the noisy Monte Carlo literature, the closest result is perhaps Theorem 2. 5 of  Johndrow et al. (2015)  which comes in the form of a non-asymptotic error bound in the L2 norm between πf and its MCMC estimate.
Acknowledgements The  Insight Centre for Data Analytics  is supported by  Science Foundation Ireland  under Grant Number  SFI/12/RC/2289. Nial Friel 's research was also supported by an  Science Foundation Ireland  grant:  12/IP/1424. Pierre Alquier 's research was funded by  Labex ECODEC  ( ANR -11-LABEX-0047 ) and by the research programme New Challenges for New Data from LCL and GENES, hosted by the  Fondation du Risque. We thank the Associate Editor and two anonymous Referees for their contribution to this work.
(2) This raises the question of how to deal with the unknown probabilities. Adjusted quantum data. Quantum data can be adjusted to handle missing values in several ways. Three cases are examined here. Way 1. We use the same quantum observable operator H as in the classical model. The missing unknown probabilities are thus considered as 0. Way 2. The values 2d and 2u are not eigenvalues of the observables. This requires to derive a different Hamiltonian. Way 3. We consider as only possible jumps either 0 or 1-step jumps d, u, d + u. A new Hamiltonian is then obtained. Data and simulation. We have assumed that each claim has only two possible values d and u. Nevertheless, the data obtained by simulation are observed values tainted by errors. For example, a simple dataset such as {4, 7, 2, 11, 3, 6} can be treated as generated both by (1) or (  2 ) with (u = 3, d = 7) observed with errors.
Data Analysis We want to analyze the claims data by the non-traditional quantum representation (3)-(  5 ) of the models (1) and (  2 ). This can be done by applying the supervised machine learning method  (Bishop 2006; Hastie and Tibshirani 1996; Hastie et al. 2009). The method uses the cross-validation technique, which is based on dividing the dataset into test data (also known as validation data) and training data. The nearest neighbors algorithms are applied to classify the data, then the maximum likelihood estimate and the error risk calculation are performed to find the optimal parameters. Finally, the part of the training information obtained is applied to analyze the test data. In the k-fold cross-validation, all data is divided into k subsets of equal size. In each iteration, a subset is chosen as training data and the remaining subsets are used as test data.
Estimation Procedure Our goal is to estimate the values (u, d) of the claim amounts and their probabilities (p, q = 1p). The dataset V = {v 1 , v 2 ,. , v n } consists of claim amounts in successive time intervals ∆t = (t -1, t] (t = 1,. , m say). We assume that the likelihood function is defined by one of the functions given in (  11 )-(  13 ). The estimation method proposed is slightly similar to the EM algorithm and its successive steps are as follows. - Choose an initial estimate (u 0 , d 0 ). -Classify and label the data with respect to (u = u 0 , d = d 0 ) by using the nearest neighbour algorithm. This leads to the classes G 2u , G 2d , G u+d , G u , G d , G 0 for the representation (3),(4), and G u+d , G u , G d , G 0 for the representation (  5 ). -Given u, d, estimate p, q = 1p by maximizing the appropriate likelihood function L(p, q) given in (  11 )-(  13 ).
-Estimate u, d by minimizing the corresponding error risk function F(u, d) defined below in (  14 )-(  16 ), for all possible u > d > 0. - Loop it until |F(u i+1 , d i+1 ) -F(u i , d i )| < M small enough. For the k-fold cross-validation strategy, the steps are first applied to the training data and yield an estimate for (u, d) which is then used in the test data as the initial estimate (u 0 , d 0 ). Estimating (u, d) by risk functions. Consider a dataset {β i }. The target set to estimate is a set of eigenvalues µ with observed probabilities and clusters p µ and G µ. This can be done by minimizing the weighted L 1 -norm risk function F(µ) = µ -β = ∑ µ p µ ∑ β i ∈G µ |β i -µ|. Note that a L 2 -norm risk function is, of course, a possible alternative. For the Maxwell-Boltzmann statistic, this risk function is F(u, d) = p 2 δ 2 ∑ β i ∈G 2u |β i -2u| + q 2 δ 2 ∑ β i ∈G 2d |β i -2d| + 2pqδ 2 ∑ β i ∈G u+d |β i -(u + d)| + pδ 1 ∑ β i ∈G u |β i -u| + qδ 1 ∑ β i ∈G d |β i -d| + δ 0 ∑ β i ∈G 0 |β i -0|.
Numerical example. Consider the following dataset V = {20 V 1 = {20, 8, 1, 7, 15}, V 2 = {17, 11, 0, 19, 1}. The successive steps of the procedure described in Section 5. 1 are applied from the estimate (u 0 , d 0 ) = (40, 25). (1) Maxwell-Boltzmann likelihood (11) with risk function (  14 ). We obtain the following results (Table  1 ). Choosing M = 0. 01, we see that (u, d) = (15, 9) and (p, q) = (0. 1, 0. 9). The associated minimum risk is 2. 987181 and the maximum likelihood value is 2. 198608 × 10 -7. The loop takes 8 steps, i. e. , it works very fast for a small data set. To reduce overfitting, we apply the k-fold cross-validation method with k = 2. This gives the results below (Table  2 ). 13 , 8. 5) and ( p, q) = (1/2)(p 1 + p 2 , q 1 + q 2 ) = (0. 3, 0. 7), with F(13, 8. 5) = 1. 9884.
The results are in the following table  (Table 5 ), again for M = 0. 01. The results here are somewhat different since (u, d) = (17, 9) and (p, q) = (0. 57, 0. 43). The loop now takes only six steps. For this dataset, the model which fits best, i. e. , with the smallest risk function, is using Bose-Einstein statistics. We also performed several numerical experiments with simulated data. In the examples (4)-(  7 ) below, the simulations yield datasets of size n = 100 (n = 1000 was used too), and the calculations are made with M = 0. 1. (4) Uniform random data (Table  6 ). As in the examples (1), (2), we apply the usual Maxwell-Boltzmann and Bose-Einstein statistics. As expected, a small value of m does not affect the results very much. What is a little surprising is that for a relatively large value m = 20 (20%), estimates of probabilities change slightly (2%) but estimates for claim amounts are significantly modified (20%).
In such a situation, the non-ruin probability when t n > t f is given by P(T > t n |x 0 ) = E P 1 (T > t f |x 0 ) P 2 (T > t nt f |R(t f )) , where P 1 (resp. P 2 ) means that the computation is made with the parameters (u 1 , d 1 ), (p 1 , q 1 ) (resp. (u 2 , d 2 ), (p 2 , q 2 )). (8) Change of risk. Consider the dataset of example (1), i. e. , V = {20, 8, 1, 7, 15, 17, 11, 0, 19, 1}, and take λ = 0. 1, c = 1, ∆t = 1 and M = 0. 05. When the analysis is by the Maxwell-Boltzmann statistics, we find (u, d) = (15, 8) and (p, q) = (0. 2, 0. 8), with L = 6. 1720e -14 and F = 2. 1151. Given an initial capital x 0 = 5, we compute the probability of non-ruin until time 30 and obtain P(T > 30|5) = 0. 4021. Suppose that, as in example (1), V is divided in two subsets V 1 , V 2. With the dataset V 1 , we find similarly (u 1 , d 1 ) = (15, 8) and (p 1 , q 1 ) = (0.
Section 4 discusses the empirical results. Section 5 concludes and discusses some policy implications. The consumption expenditure is considered as an indicator of a household's income. Source: published reports and papers; see for instance  (IMF 2007; Delaunay 2012). These ratios correspond to the number of students formally registered in primary school. Among the studies using the ESPS datasets, we can cite  Boccanfuso et al. (2008) ;  Boccanfuso et al. (2009) ;  Diawara (2012) , among others, and the national and institutional reports  : DSRP 2005; IMF 2007; ANSD 2007. 8  The standard  (Mincer 1974 ) earnings equation linearly regresses the log of wage on the year of education and the quadratic function of labor market experience.
Thus we solve the question raised by  [11]. Variational Bayes is a method for computing intractable posteriors in Bayesian statistics and machine learning. Markov Chain Monte Carlo (MCMC) algorithms remain the most widely used methods in computational Bayesian statistics. Nevertheless, they are often too slow for practical uses when the dataset is very large. A more and more popular alternative consists in finding a deterministic approximation of the target distribution called Variational Bayes approximation. The idea is to minimize the Kullback-Leibler divergence of a tractable distribution ρ with respect to the posterior, which is also equivalent to maximizing the ELBO. This optimization procedure is much faster and efficient than MCMC sampling with numerous applications in different fields: matrix completion for collaborative filtering  [2] , computer vision  [40] , computational biology  [13]  and natural language processing  [27] , to name a few prominent examples.
Algorithm 1 Coordinate Descent Variational Bayes for mixtures 1: Input: a dataset (X 1 ,. , X n ), priors π p ,{π j } K j=1 and a family {q θ /θ ∈ Θ} 2: Output: a variational approximation ρ p (p) set w i j = exp log(p j )ρ p (dp) + log(q θj (X i ))ρ j (dθ j ) 8: end for 9: normalize (w i j ) 1≤j≤K 10: end for 11: set ρ p (dp) ∝ exp α n i=1 K j=1 ω i j log(p j ) π p (dp) 12: for j = 1,. , K do 13: set ρ j (dθ j ) ∝ exp α n i=1 ω i j log(q θj (X i )) π j (dθ j ) 14: end for.
& Estimation Steps and Prior Information The structure of the models used for estimating the simulated datasets was the same as the model used for the simulation step. The population PK parameters (typical values and variances) for both drugs were considered as known from previous clinical studies and were fixed while the empirical Bayes estimates were estimated. In order to reduce MBAOD runtime, only drug S drug effect (Slope drugS ), its associated interindividual variability, and the residual error component (Ɛ) were estimated in the myelosuppression model. To bridge the gap between preclinical and clinical settings regarding the toxicological aspect and to use all the information available, information from preclinical species on drug effect scaled up to human (considering species differences in protein binding and drug sensitivity) was used for drug S as prior information  (25).
Tools Used MBAOD processes were simulated using the MBAOD R-package  (27). This package currently uses PopED  (28, 29) , NONMEM  (26) , PsN  (30) , and R (31) to handle the various tasks inherent to simulating and evaluating the MBAOD process. Parallel computational processes available in the MBAOD R-package and compilation in Clanguage of the scripts were also used to shorten runtimes. A schematic for the examined MBAOD process is shown in Fig. 4.
ACKNOWLEDGEMENTS This work was supported by  Servier Laboratories. Open Access This article is distributed under the terms of the Creative Commons Attribution 4. 0 International License (http://creativecommons. org/licenses/by/4. 0/),which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Introduction With the aim of using an improved method to study the dynamics of galaxy pairs, we constructed two new pair catalogs using well-defined criteria and improved observational data. The first is a catalog of ≈1000 pairs with high accuracy radial velocities  (Chamaraux & Nottale 2016)  extracted from Nilson's Uppsala Galaxy Catalog (UGC;  Nilson 1973) , which has the advantage that it is complete in apparent diameter. The second is a catalog of ≈13 000 pairs  (Nottale & Chamaraux 2018 , the largest presently available to our knowledge), which makes use of the huge recent increase in astronomical data. We used the Hyper-LEDA database  (2016 ( , Makarov et al. 2014) )  to identify galaxy pairs and extract their parameters from the large surveys (e. g. , SDSS;  Alam et al. 2015 and 2MASS).
contacts which are very frequent in the hospital context. More recently, electronic wireless devices started being used to record close-proximity interactions (CPIs), notably in schools or hospitals  [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17]  , providing accurate and detailed data on social interactions in these settings. To this date, hospital CPIs have mostly been recorded at the ward level, over short time periods; in order to fully understand the risk of HAI spread at the hospital level, the impact of different ward characteristics on contact networks, as well as inter-individual interactions across wards also need to be taken into account. In addition, most of previously published hospital CPI data was collected in acute-care settings. However, long-term care facilities (LTCF) have been shown to play a major part in the global spread of HAI, due notably to long patient lengths of stay  18, 19.
Here, we report data collected using Radio-Frequency Identification Devices (RFID) as wearable sensors to measure proximities with a high spatio-temporal resolution in a 200-bed LTCF, over a 4-month period. The objectives of this study are twofold. First, to provide a dynamic description of inter-individual contacts in different individual categories (patients and healthcare workers) and wards in the LTCF setting. Second, to identify factors associated with high contact levels in either patients or staff in order to inform future control programs in LTCF settings.
However, it also presents several limitations which will be discussed. First, although a good correlation between RFID data collection and direct observation has been reported  11  , this technology has some limits, in particular regarding its pertinence in terms of potential pathogen spread. In our study, every CPI at less than 1. 5 m was recorded, without providing any information about the actual distance between individuals or potential physical contact. Hence, many brief interactions that do not actually present any potential for pathogen transmission may have been recorded. However, a recent analysis showed, using the same data, that S. aureus transmission within the hospital was consistent with contacts defined by these electronically collected CPIs  16. More generally, the implications of measured CPIs in terms of infectious risk is bound to depend on pathogen-specific transmission modes -for instance, some of the CPIs may only play a part in epidemic dynamics if airborne transmission is possible.
Data collection. The overall participation rate was 90. 1%. All participating staff and patients wore a small wireless sensor recording Close-Proximity Interactions (CPIs, typically at less than 1. 5 m) over their entire presence in the hospital. Because RFID devices only exchange packets when individuals are face-to-face, as the human body acts as a shield at the frequency used for communication, only front-facing CPIs were recorded. CPIs were recorded every 30 seconds, along with the time, date and anonymous identifiers of the receiving and transmitting sensors. Over the 6-month duration of the study, sensor batteries had to be replaced on 3 occasions, during which recording was discontinued for 2 days. Descriptive analysis of CPIs. The patterns of CPIs within the hospital were analyzed using several quantities related to frequency and duration of contacts.
Acknowledgements We thank  Thomas Sorel  for his work on the database and  Judith Legrand  for statistical advice. This study was supported by the  European Commission  under the  Life Science Health Priority of the 6th Framework Program (MOSAR network  contract  LSHP-CT-2007-037941 ). Funding was also received from the  French Government  through the  National Clinical Research Program  and the  Investissement d'Avenir program ,  Laboratoire d'Excellence "Integrative Biology of Emerging Infectious Diseases " (grant no. ANR-10-LABX-62-IBEID ).