dataesr/scientific-paragraphs-categorization huggingface dataset:
We comment on the potential benefits and downsides, which open promising avenues for future research. In the proposed use-case, we consider comparing the practice adoption rates of a team against other teams in the collected data set. Engineering ability can then be assessed as, e. g. , the percentile into which a given participant falls. Moreover, instead of considering the difficulty of moving from ''Partially'' to ''Mostly'', or from ''Mostly'' to ''Completely'' as equal for each practice, we would like to estimate the difficulty individually for each practice-and-answer combination using the data. To define such a benchmark, we borrow concepts from the theory of measurement -a field of study concerned with the objective measurement of knowledge and abilities from test data. In particular, we use item response theory (IRT), which is a method for scoring tests and measuring an individuals' ability on the test based on the ability of other test respondents  (Embretson and Reise, 2013).
We experimentally demonstrate the advantages of our methods by performing extensive imputations on synthetic and real clinical data. We also develop quantum circuits for implementing determinantal point processes, since such quantum algorithms provide a computational advantage with respect to classical ones. We demonstrate competitive results with up to ten qubits for small-scale imputation tasks on a state-of-the-art IBM quantum processor. Our classical and quantum methods improve the effectiveness and robustness of clinical data prediction modeling by providing better and more reliable data imputations. These improvements can add significant value in settings where where high precision is critical, such as in pharmaceutical drug trials where our approach can provide higher confidence in the predictions made.
Missing data is a recurring problem in machine learning and in particular for clinical datasets, where it is common that numerous feature values are not present for reasons including incomplete data collection, discrepancies in data formats and data corruption  [8 , 16 , 20 , 21 ]. Machine learning is routinely used in life science and clinical research for prediction tasks, such as diagnostics  [22 ]  and prognostics  [2 ] , as well as estimation tasks, such as biomarker proxies  [28 ]  and digital biomarkers  [23 ]. Beyond the research setting, machine learning is becoming more commonplace as regulated Software as a Medical Device, where machine learning models are influencing -or making -clinical decisions that affect patient care. Machine learning algorithms typically require complete data sets and missing values can significantly affect the quality of the machine learning models trained on such data.
This is in large part due to the fact that there can be different underlying reasons for the missingness: for example, feature values can be missing completely at random (MCAR), missing at random (MAR) and missing not at random (MNAR), each one with their own characteristics. Despite its importance for clinical trials, there is no canonical approach for dealing with missingness and finding appropriate, effective and reproducible methods remains a challenge. A common way to deal with missing clinical data is to exclude subjects that do not have the complete set of feature values present. A drawback of this approach is that excluding subjects can in fact introduce significant biases in the final model. For example, it can result in the model being trained to be more effective for the type of subjects that are likely to have complete data than for those that do not.
Moreover, the effectiveness and reliability of clinical trials is reduced when subjects with missing feature values are excluded from the clinical trial. Data imputation is an alternative to the complete dataset approach, where subjects with missing feature values are not excluded. Instead, missing values are imputed to create a complete dataset. There are different ways to achieve this, including "filling" missing values with zeros, or with the mean value of the feature across all subjects that have such a value present. These methods provide consistent imputation results, but there are important caveats for using such simple methods, since they ignore possible correlations between features and can make the dataset appear more homogeneous than it really is. More advanced data imputation methods have been proposed in the literature: iterative methods include the multivariate imputation by chained equations (MICE)  [27 ]  and  MissForest [25 ]  algorithms, and deep learning methods include GAIN (generative adversarial imputation nets)  [29 ]  and MIWAE (missing data importanceweighted autoencoder)  [17 ].
In practical terms, the performance of the downstream classifier is usually of highest importance for clinical data sets: for example, in one of our datasets, the classifier denominates a binary outcome of a critical care unit stay (e. g. survival) for each patient. Accordingly, we have used AUC for the classification task here to assess the performance of our novel methods. An inherent feature of standard Random Forest and determinantal sampling algorithms is randomness that produces data imputations that vary from one run of the algorithm to the next. This is often undesirable, since the downstream classification performance can also be affected. This motivated us to develop a deterministic version of determinantal sampling within the.
We also describe a computationally advantageous way to perform the determinantal sampling using efficient quantum circuits. In order to benchmark the different imputation methods, we used two types of datasets with a categorical outcome variable. First, a synthetic dataset, created using the scikit-learn method make classification. It consists of 2000 rows with 25 informative features. This is useful to study the imputation quality where features have equal importance. Second, the MIMIC-III dataset [11 ]:.
DPP-MICE, and detDPP-MICE outperform MICE We present the performance results of MICE-based methods in terms of the AUC of the downstream classification task using an XGBoost classifier, which has been shown to be the strongest classifier for such datasets  [24 ]. In each case, the original dataset with induced missing values is imputed using MICE, DPP-MICE or detDPP-MICE, then it is divided into 3 folds of Development/Holdout sets. The downstream classifier is then trained on each development set and its performance is measured by the AUC for the corresponding holdout set. The results appear in Table  I  and in the figures in Table  II. The imputation procedure is performed for a total of 10 iterations over the whole columns and for each column, a (DPP) Random Forest regressor is trained using 10 trees.
For each Random Forest training, the dataset is divided into batches of 150 points each and DPPs are used to sample from every batch. The results show that across the twelve in total dataset experiments DPP-MICE outperforms MICE on expectation in ten of them, while detDPP-MICE provides a single deterministic imputation which outperforms the expected result from MICE in all twelve datasets and from DPP-MICE eleven out of twelve times.
DPP-MissForest and detDPP-MissForest outperform MissForest Here we present the performance results of MissForest-based methods in terms of the AUC of the downstream classification task using again an XGBoost classifier. In each case, the original dataset with induced missing values is imputed using MissForest, DPP-MissForest or detDPP-MissForest, then it is divided into 3 folds of Development/Holdout sets. The downstream classifier is again then trained on each development set and its performance is measured by the AUC for the corresponding holdout set. The results appear in Table  I  and in the figures in Table  III. The specifics of the Random Forest training are the same as in the case of MICE. The results show that across all experiments, DPP-MissForest outperforms MissForest in all twelve of them, while detDPP-MissForest provides a single deterministic imputation which outperforms the expected result from MissForest in all twelve datasets and from DPP-MissForest in eleven out of twelve times.
Quantum hardware implementation of DPP-MissForest results in competitive downstream classification As we describe in Methods, quantum computers can in principle be used to offer a computational advantage in determinantal sampling. In order to better understand the state-of-the-art of current quantum hardware, we used a currently available quantum computer to perform determinantal sampling within a DPP-MissForest imputation method for scaled-down versions of the synthetic and MIMIC datasets. Reduced synthetic dataset: 100 points and 3 features, created using the sklearn method make classification. Reduced MIMIC dataset: 200 points and 3 features. The three features were chosen from the original dataset features based on low degree of missingness and their predictiveness of the downstream classifier and they were: "Oxygen saturation std", "Oxygen saturation mean" and "Diastolic blood pressure mean".
For the purposes of our experiments, we used the "ibm hanoi" 27-qubit quantum processor shown in Figure  3. We implemented quantum circuits with up to 10 qubits. The decision trees of the DPP-Random Forests used by the imputation models are trained using batches of decreasing sizes (see Table  IV  ). For example, for the algorithm with batch size equal to 10, the algorithm first samples two out of the ten data points to use for the first decision tree, then from the remaining eight datapoints it picks another two for the second tree, then two from the remaining six, and last two from the remaining four. In other words, we train four different trees, and each time we use a quantum circuit with number of qubits equal to 10, 8, 6, and 4, to perform the respective determinantal sampling. In the figures of Table  V  and in Table  VI  we provide for the different dataset experiments the AUC for MissForest, the simulated results of the quantum version of DPP-MissForest, and the actual hardware experimental results of running the quantum version of DPP-MissForest.
Even for these very small datasets, when simulating the quantum version of DPP-MissForest, we demonstrate an increase in the AUC compared to the MissForest algorithm. This further highlights the potential advantages of determinantal sampling within imputation methods. Of note, running our algorithms on current hardware introduces variance in the downstream classifier. Importantly, this variance is due to noise in the hardware rather than inherent to the algorithm. Our quantum hardware results are competitive with standard methods and in many cases close to the values expected from the simulation. In some cases, we observed a clear deterioration of the AUC due to the noise and errors in the quantum hardware. The results are closer to the simulations when using MCAR missingness with larger batch sizes that use more trees both for synthetic and the MIMIC datasets.
Determinantal sampling is an important tool not only for Random Forest models, but also for linear regression, where data diversity results in more robust and fair models  [18 ]. Moreover, such sampling methods based on determinantal point processes are computationally intensive and quantum computers are expected to be useful in this case: quantum computers offer an asymptotic speedup for performing this sampling and it is expected that next generation quantum computers will provide a speedup in practice. We show that, as expected, the quantum version of detDPP-MissForest does not introduce any variance in the downstream classifier when simulated in the absence of hardware noise. However, when implemented on quantum hardware, we observed variance that is caused by the noise in the hardware itself.
Algorithm 1 Deterministic k-DPP algorithm The DPP-Random Forest algorithm (see Figure  4  ) replaces the uniform sampling with DPP sampling without replacement. Determinantal sampling for regression and classification tasks with full data has been proposed previously for Linear Regressors  [18 ]  and for Random Forest training for a financial data classification use case where it outperformed the standard Random Forest model  [26 ]. We can also use the deterministic version of DPP sampling for the Random Forest algorithm. This requires removing the sample used at each step (which is the one with the highest probability according to the determinantal distribution) in order to create a smaller dataset from which to sample for the next decision tree (see Figure  5  ). We call this new model detDPP-Random Forest.
Let us note that the distributions of the in-bag DPP samples, which are biased towards diversity, and the out-of-bag (OOB) samples, which reflect the original dataset's distribution, may be different. This could lead to an inaccuratecalculation of the OOB error that can be in fact overestimated  [9 ]. In the DPP-Random Forest case, the batches are stratified and according to the output variable that follows the same distribution as the original dataset. Thus, sampling from different batches could bridge the gap between the in-bag and the out-of-bag distributions. We leave these considerations for future work.
I wonder if there could be some interaction between the imputation and the classification methods, that leads to overfitting the data; in particular when the deterministic DPP is used. In its current state, I do not think this article brings not very much value to the community that could benefit from it. I did not find an implementation of the method available to other scientists, nor the data used to evaluate it (while one data set is public, the simulated data is not available). This not only hinders the use of the method by the scientific community, but also makes it impossible to reproduce the results or test the algorithm in similar databases.
Reviewer #2 (Public Review): In this work, the authors address the problem of missing data imputation in the life sciences domain and propose several new algorithms which improve on the current state-of-the-art. In particular (i) they modify two existing Random Forest-based imputation methods --MissForest and miceRanger --to use either determinantal sampling or deterministic determinantal sampling, and show slightly improved classification performance on two datasets (one synthetic, one real); in addition, (ii) the authors present a quantum circuit for performing the determinantal sampling which scales asymptotically better than the bestknown classical methods, and perform small scale experiments using both a (noiseless) quantum simulator as well as a 10 qubit IBM quantum computer to validate that the approach works in principle.
The problem of data imputation is important in practice, and results that improve on existing methods should be of interest to those in the field. The use of determinantal sampling for applications beyond data imputation should also be of broader interest, and the connection to quantum computing warrants further investigation and analysis. The use of classification accuracy (as measured by AUC) as a measure of success is wellmotivated, and the authors use both real and synthetic datasets to evaluate their methods, which consistently (if only marginally) outperform the existing state-of-the-art. The results obtained here motivate the further study of this approach to a wider class of datasets, and to areas beyond life sciences. As it stands, in my opinion, two points need addressing.
Additional analysis is needed to support the claims of potential for quantum advantage: The authors claim that the quantum algorithm for implementing determinantal point processes provides a computational advantage over classical ones, in that the quantum circuits scale linearly in the number of features compared with cubic scaling classically. While this may be true asymptotically, in my opinion, more discussion is required about the utility and feasibility of this method in practice, as well as the realistic prospects of this being a potential area of application for quantum computing. For example, the authors mention that a quantum computer of 150 qubits capable of running circuits of depth 400 is needed to perform the determinantal sampling for the MIMIC-III dataset considered, and say "while [such hardware is] not available right now, it seems quite possible that they will be available in the not so far future".
The authors also state "This suggests that with the advent of next-generation quantum computers. one could expect a computational speedup in performing determinantal sampling" and "it is expected that nextgeneration quantum computers will provide a speedup in practice". These are strong assertions (even if 'next generation' is not clearly defined), and in my opinion, are not sufficiently backed by evidence to date. Given that datasets of the size of MIMIC-III (and presumably much larger) can be handled efficiently classically, the authors should clarify whether one expects a quantum advantage by this approach in the "NISQ" (pre-errorcorrected) era of quantum computing. This seems unlikely, and any argument that this is the case should include an analysis accounting for the absolute operation speeds and absolute times required to perform such computations, including any time required for inputting data, resetting quantum circuits etc.
The determinantal sampling takes time Nd^2, how does this compare with the original methods? For what dataset and feature sizes are the determinantal methods feasible (which will determine the scale at which other approaches, e. g. those based on quantum computing may be required). A discussion (or at least mention) of the algorithmic complexity of the classical deterministic determinantal sampling (which seems to also be Nd^2) in the main body of the text would be welcome. The final paragraph of the Methods section discusses sampling many times from the quantum circuits to estimate the most probable outcome, and hence perform the deterministic determinantal sampling. A more careful analysis of the number of samples needed (for bounded variance/error) and the impact on the running time (and whether one still expects an advantage over classical (although one must define some bounded error version of the deterministic algorithm to do so) or performance of the algorithm would be welcome.
A discussion on the absolute running time required for the quantum experiments performed (and how they compare to classical) would be interesting. A mention of which quantum simulator was used would be welcome. In the introduction, three kinds of data missingness (MCAR, MAR, MNAR) are mentioned, although experiments are only performed for MCAR and MNAR. Can some explanation for excluding MAR be given? Reference 24 (Shadbar et al. , the study that demonstrated the effectiveness of miceRanger and MissForest) used 4 datasets: MIMIC-III, Simulated, Breast Cancer, and NHSX COVID-19. Of these, MIMIC-III is used in the current paper, and Simulated appears similar (although with 1000 instead of 2000 rows) to the synthetic dataset of the current paper. An analysis of the determinantal sampling methods applied to the Breast Cancer and NHSX COVID-19 datasets (which have naturally occurring missingness), and a comparison to the results of Shadbar et al.
The cause and effect relationship between sea weather and inland seismic noise in the mid-frequency band is clear: ocean waves cause varying pressure at the bottom of the sea, which propagates to distant points as seismic waves in the Earth mantel. The seismic waves generated near each buoy take between a few tens of seconds and a few minutes to reach the LSBB, depending on the distance. Given the time granularity of the buoy data, we assume instantaneous dependence. We remove the intervals which clearly contain transient signals covering or competing with the targeted effect in the mid-frequency band from the datasets, namely seismic events and glitches in the seismic records due to human activities. They are typically characterized by much faster variations than ocean wave generated signals, and can thus be easily filtered out.
The intervals identified with our algorithm are cross-validated using earthquake databases (See Appendix A). We next compute Pearson's correlation ρ between the variables measured at each buoy and the seismic noise in each spectral bin, to pinpoint the most significant dependencies. Of the 27 available buoys we selected a representative subset of 3 in the Mediterranean Sea (#01305 Marseille -Le Planier; #61001 Nice; #61002 Golfe du Lion) and 3 in the Atlantic Ocean (#03302 Cap Ferret; #62001 Gascogne Ouest Arcachon; #62163 Bretagne), for their proximity to the LSBB site (see Fig. 1 ). The correlation matrix shown in Fig. 2  shows three distinctive features: (i) the measurements at the Mediterranean buoys corre-late strongly with the seismic noise between 0. 15 Hz (∼7 s) and 2 Hz, especially the wave height, period and wind force (positive correlation) and the water and air temperature (negative correlation); (ii) the correlation with the Atlantic buoys is generally less pronounced, but more extended at low frequency and significant down to 0.
The seismic data are downloaded monthly using a web harvesting robot, which calculates the vertical acceleration spectrum at every half hour and averaging over an hour time interval from the available continuous velocity records. The sampling frequency for the seismic signal is 250 Hz, hence the spectra are binned to reduce the required computation resource for the later steps: the frequency interval between 0. 01 Hz and 10 Hz is divided in 60 bins, by adopting an exponential progression (the boundary frequencies are defined by the multiplication factor 20 √ 10). The time intervals characterized by the presence of earthquakes and other glitches are removed from the datasets, because they mask the targeted dependence between sea weather parameters and the inland seismic background noise. To identify these intervals we look for substantial sudden changes of the seismic noise power spectral density; we set the time integral parameters and the threshold for the variation so as to have a good agreement between the identified intervals and the earthquakes listed in the seismic databases.
Seismogram Atlases with UMAP UMAP is a manifold learning technique, which has been introduced in the work of  McInnes et al. (2018). Similar to ICA, UMAP is a tool to reduce the dimensions of a highdimensional dataset for downstream tasks such as visualization. Since we are interested in a visualization of the high dimensional scattering coefficient matrix, we restrict the number of dimensions to two. Any dimensionality reduction technique comes with a loss of information and the loss depends on the objective of the dimensionality reduction technique. Because ICA performs a linear mapping, it preserves well the pair-wise distances, but it loses information about local structures. UMAP learns the manifold of the given data and, thus, performs better in preserving local structures at the price of distorting the global structure.
Hence, the distances between neighboring points are more reliable than distances between clusters of data points or the area of a cluster. Without going into further details, the inner workings of UMAP are based on topological data analysis and Riemannian Geometry, providing a complex but safe and sound mathematical background (see the original work of  McInnes et al. , 2018 , for more details). It shares similarities with the t-distributed Stochastic Neighbor Embedding (t-SNE), which has been used extensively for visualizations since its appearance in the 2000s (  Van der Maaten & Hinton, 2008). However, compared to UMAP, t-SNE performs poorly in preserving global structures and its computation time is much slower  (McInnes et al. , 2018; Becht et al. , 2019). Despite its relatively recent introduction, UMAP has been already utilized in many scientific domains to create a two-dimensional representations, simplifying the visualization of large and high-dimensional datasets.