dataesr/scientific-paragraphs-categorization huggingface dataset:
4. Pipes 2. 5 m in length and 1. 5 cm in internal diameter are connected down to the targeted injection depth. This pipeline ends with a screened cap that ensures horizontal diffusion of the tracer at the outlet. 5. The tracer solution (typically 5 g of uranine diluted in 2 L of water) is injected, followed by a water flush volume of 40 L. The total injection duration, including flushing, is always less than 3 min. 6. The concentration at the outlet of the pumped well is monitored with a flow-through fluorometer (Albillia GGUN) connected to a bypass of the discharge pipeline. The BTC dataset adopted in the present study corresponds to 50 tracer tests conducted between wells located in the area covered by seismic surveys. Four pumping wells were employed in the tracer experiments: M06, M07, M22, and MP6.
The main strength of the approach we propose is to supplement the information content of geophysical surveys with the results of tracer BTC analysis. Part of the ambiguity in the identification of karst conduits based on geophysical data can be alleviated by building on the prior estimation of the number of conduits between pairs of point locations. The first step of the method is therefore to analyze each BTC using a regularized multiflow inversion method, as implemented in the MFIT software, to determine the minimum number of distinct paths used by the tracer between injection and monitoring points. The second step consists of implementing a graph based on the 3D geophysical model of subsurface heterogeneity and calculating the previously enumerated paths via the application of an alternative SP algorithm.
With the use of HES tracer test data, 3D seismic data, and borehole flowmeter logs to illustrate our approach, we have demonstrated the feasibility of delineating discrete karst conduit networks. The reliability of the identified network was locally verified through new tracer experiments performed after completion of the computations. To our knowledge, the present paper is the first to combine tracer and geophysical data to identify the discrete geometry of a karst network. Another conceivable approach would be the joint inversion of both types of data, but inversion methods based on a discrete approach to flow and/or transport paths are still at an early stage of development (e. g. ,  Somogyvári et al. , 2017; Fischer et al. , 2020). The main limitations of the proposed approach are (i) the spatial density of the tracer tests required to constrain the number of paths between the nodes of the network, and (ii) the sensitivity and/or spatial resolution problems inherent to geophysical methods.
Acknowledgements. The authors thank  Junfeng Zhu  and an anonymous referee for their valuable comments and feedback, as well as  Konstantinos Chalikakis , who reviewed an earlier version of this paper. Their comments led to significant improvements to this article. Financial support. This research was supported by the  French National Observatory H+ , the  European Union (ERDF) , and " Région Nouvelle Aquitaine ". Review statement. This paper was edited by  Philippe Ackerer  and reviewed by two anonymous referees.
Code availability. The MFIT program is available from https://doi. org/10. 5281/zenodo. 3470751 (Bodin, 2020b)  under the terms of the CeCILL Free Software License Agreement v2. 1. The kPOP program used for graph generation and computation of the alternative paths between pairs of tracer injection and monitoring points was written in C++. The source code of the kPOP program is available from https://doi. org/10. 5281/zenodo. 4487305 (Bodin, 2021)  under the terms of the CeCILL Free Software License Agreement v2. 1. Data availability. The HES tracer test data, seismic data, and borehole flowmeter logs processed in Sect. 4 of this study are available from the H+ database (http://hplus. ore. fr/en/poitiers/data-poitiers; French National Observatory H+, 2022) with registration of a free account.
Physically Unclonable Functions (PUFs) have become ubiquitous as part of the emerging cryptographic algorithms. Strong PUFs are also predominantly addressed as the suitable variant for lightweight device authentication and strong single-use key generation protocols. This variant of PUF can produce a very large number of device-specific unique identifiers (CRPs). Consequently, it is infeasible to store the entire CRP space of a strong PUF into a database. However, it is potential to use Machine Learning to provide an estimated model of strong PUF for enrollment. An estimated model of PUF is a compact solution for the designer's community, which can provide access to the full CRP space of the PUF with some probability of erroneous behavior. To use this solution for enrollment, it is crucial on one hand to ensure that PUF is safe against a model-building attack.
On the other hand, it is important to ensure that the ML-based enrollment will be performed efficiently. In this work, we discuss these factors, and we present a formalized procedure of ML-based modeling of PUF for enrollment. We first define a secure sketch which allows modelability of PUF only for a trusted party. We then highlight important parameters which constitute the cost of enrollment. We show how an ML-based enrollment procedure should use these parameters to evaluate the enrollment cost prior to enrolling a large group of PUF-enabled devices. We introduce several parameters as well to control ML-based modeling in favor of PUF enrollment with minimum cost. Our proposed ML-based enrollment procedure can be considered a starting point to develop enrollment solutions for protocols which use an estimated model of PUF instead of a CRP database.
However, practically, it depends strongly on the level of the PUF structure's complexity  [12, 13, 22]. Therefore, to increase security of the PUF, it can be suggested to employ PUF with increased design complexity. However, for enrollment of the PUF with ML-based techniques, it is then crucial to make sure, on the other hand, that the modelability of the PUF on the server side during the enrollment is facilitated. For instance, it is suggested in the Slender PUF protocol design to build an equivalent model of an XOR Arbiter PUF during the enrollment via having access to each individual Arbiter PUF block. It can be assumed here that, for increased security, the XOR size of the PUF is large-while, with the eased access to the individual Arbiter PUF, it is still viable to accurately model the XOR PUF for enrollment using a compact CRP dataset.
Machine Learning Modeling of Strong PUF The goal of strong PUF enrollment with ML-based modeling is to replace the conventional CRP database with an estimated model of the PUF. Let us denote the estimated model of PUF as h PUF. The enrollment of PUF with ML-based modeling means that the verifier server will own h PUF , which provides access to the full CRP space of the PUF circuit with some miss-prediction error that is tolerable (see Figure  1a ). This should in turn mean that the h PUF and the PUF circuit itself should respond similarly to any given challenge from the CRP space. Let us consider c i as a challenge input to the PUF circuit. If we observe the PUF circuit as a function f PUF of c i , then its estimation can be defined as a function g PUF of c i and a set of internal values θ of the model.
Optimization Phase The training of the estimated model is done in this phase. It comprises multiple functions for training and evaluation of the estimated model. At the beginning of the phase, the estimated model is given a set of challenges from the CRP tr dataset. The model then predicts the corresponding responses. The loss function will take the predicted responses and the actual responses from the CRP tr and compute the prediction loss δ. Then, the loss value δ is given to the optimizer function which propagates adjustments to the internal parameters θ of the estimated model. Then, the Eval function computes the prediction accuracy of the updated estimated model, using the CRP val dataset. This entire process is called an epoch, which is counted as epoch in the procedure. The procedure undergoes several epochs of training until the desired value of loss δ g or prediction accuracy val_ g is observed or epoch reaches max value epoch max.
Evaluation Phase The evaluation phase performs the final prediction accuracy assessment of the updated estimated model over the CRP te dataset. Here, the same eval function measures the prediction accuracy of the model over CRP te. This evaluation in turn tries to emulate the scenario where the estimated model is invoked during mission mode to communicate with the PUF circuit. In such case, it is justified to have the CRP te set size be considerably larger than that of CRP val. The control sequence in this phase compares the prediction accuracy of the model with the test_ g. If greater, then the the enrollment yields the model as an estimated model with desired accuracy for enrollment. If the prediction value is less than test_ g , however, the control sequence sets to redo the training from the initialization phase where the model's internal parameters are initialized.
Specifications We considered using XOR Arbiter PUF as our target PUF model family in our evaluation code. We conduct our experiments on data generated from a Python based Arbiter PUF and XOR Arbiter PUF simulation. We elaborate on XOR Arbiter PUF and its simulation in Python in the following. XOR Arbiter PUF is considered as a variant of the Arbiter PUF family. Arbiter PUF was first introduced in 2002 by Gassend et al. in  [23]. The idea of Arbiter PUF is based on the delay difference between two racing paths that are structurally similar, but, due to minor process variations, they differ in time of passing a signal given to them at the same time. The structure of XOR Arbiter PUF is based on multiple Arbiter PUFs whose input (challenge) is of the same size, and triggered by a global input.
Thus, we stored 100 CRP datasets with 35,000 CRPs in each set. Note that the datasets generated from the simulated PUF instances do not simulate the instability that is inevitably present in real PUFs. We intentionally chose the instability-free condition, since the presence of instability is a new fold of complexity that can affect the modeling results, and thus it needs to be discussed thoroughly in a separate set of experimental work. To assure the reliability of the simulated instances and the generated CRPs, we measured the randomness, uniqueness, and diffuseness, using the formulations proposed in  [26, 27]. A collective use-case of Hori's Uniqueness and Maiti's Uniqueness, as well as the randomness and the diffuseness, can be found in  [14]. Our measurements of uniqueness, randomness, and diffuseness over 100 instances of 128-bit 2-XOR Arbiter PUF are brought in Table  2.
Experimental Observations We did an exploration to find the ideal values for css tr and T. The choice of the lower bound of css tr and the upper bound was also arbitrary. We inferred from the previous that value for tr for a 2-XOR 128-it XOR Arbiter PUF is a of about 3000 CRPs. Therefore, we choose that as the lower bound. For the upper bound, we chose 10,000 CRPs. We speculated that the characteristic we observe around this number of CRPs can be interpolated for larger sizes of css tr as well. We used Pytorch in Python 3. 7 to build and train our ANNs. We conducted our experiments on a PC running windows 10 with an Intel core i7 8th Gen CPU and 16 GB of memory. We developed our experimental Python codes using Spyder 4. 0. 1 on Anaconda Navigator 1. 9. 12. The results on the performance of training with various training set sizes are reflected in Figure  7.
If the cost of re-querying is tolerable, then the first option could potentially be the cost-efficient choice. Choosing the second option, however, could yield the overall increased CRP read-out cost. We also saw that training time T can also be minimized if the prediction accuracy is maximized with increasing css. However, the chance of yielding h PUF with sub-optimal is negligible on the other hand. This option could be a potential choice for cost-efficiency if the cost of re-querying the PUF is high. We also measured the size of the trained estimated models. The MLP used in this study comprises 282 trainable parameters in total. Each parameter is 32-bit floating point, yielding in total a model size of 1128 bytes. Note that we do not need to save any metadata regarding the internal connectivity of the MLP, since all layers in the MLP model are Fully-Connected.
We refer to it as measuring the uniqueness of the estimated models. Uniqueness here is observed as the prediction accuracy of each trained estimated model over a given CRP dataset coming from each PUF circuit. Since we have 100 PUF circuits and their corresponding 100 estimated models, we therefore mea-sured 10,000 cases for the uniqueness. We considered two cases, one which is training the models with css tr = 3000, and one with css tr = 8000 CRPs. The results of this measurement are brought in Figure  9. We expect for css tr = 3000 that the models have low similarity to their corresponding PUF. Looking at Figure  9a , we see that only a selective number of cases have above 0. 70. Nonetheless, for all cases, it is apparent that no similarity with higher than 0. 6 is achieved. We also observe on Figure  9b  that estimated models trained with css tr = 8000 show similarity uniquely only to their corresponding PUF circuit with high accuracy.
It is expected of course that both the model and the PUF device generate the same response value. The response value of course is prone to variations due to device instability and model miss-prediction, which is why the authors also propose using helper data and error correction codes to recover the original key generated on the TTP server for the PUF device. Similar to the idea of PUF authenticating in  [18] , here only the challenge values are exchanged and no responses, in order to avoid model-building attacks. The protocol proposed here also refreshes the key after a certain period. This feature of course needs both the device and server to have access to a large CRP space, which is again for the verifier server, provided using the model of the PUF. This way, the users will be able to refresh keys frequently and each time guarantee that a new value is generated.
Recently, various methods have been proposed to integrate different heterogeneous highdimensional genomic data sources to predict cancer survival, often in addition to widely available and highly predictive clinical data. Although clinical applications of survival models have high sparsity requirements, most state-of-the-art models do not naturally exhibit this sparsity, as they are based on random forests or deep learning. We propose to use ℓ 1 -penalized linear student models within a knowledge distillation framework to sparsify underlying multi-omics black-box teachers. We show that by excluding clinical variables from our ℓ 1 penalty, we can effectively guide the knowledge distillation, reaching virtually identical discriminative performance to the teachers while using on average 140 features or less across the 17 cancer datasets from The Cancer Genome Atlas (TCGA) considered in our study.
Related work Black-box multi-omics cancer survival methods. In recent years, there has been a considerable amount of work on survival methods that aim to integrate multiple genomic data sources, so-called multi-omics data. Hornung & Wright (2019b)  investigated multiple random forest adaptations, all of which changed the split-point selection of the standard random survival forest model  (Ishwaran et al. , 2008). The best performing new method introduced by the authors, BlockForest, was shown to significantly outperform a standard random survival forest in terms of Harrell's concordance across 20 TCGA datasets when integrating multiple omics modalities, in addition to clinical data. Herrmann et al. (2021)  further validated BlockForest, showing that it performed the best out of various considered statistical multi-omics models across 18 TCGA datasets and was one of the only models that (marginally) outperformed the baseline of a clinical-only Cox PH model when set to integrate clinical data in addition to multiple omics modalities.
, mean-pooling) did not matter, with all investigated techniques performing similarly. Meanwhile, when set to integrate all available modalities, an autoencoder outperformed other integration methods for fusing modality-specific representations, seemingly due to its ability to focus on more informative modalities. In addition, the authors showed that using a hierarchical autoencoder for multi-omics integration performed comparably to a variant of BlockForest, RandomBlock favoring clincal variables. Sparse linear multi-omics cancer survival methods. Boulesteix et al. (2017)  proposed a modified ℓ 1 penalized Cox PH model which used separate regularization hyperparameters λ 1 ,. , λ m per modality which allowed the model to focus more strongly on some modalities. The authors showed that their newly proposed model, termed IPF-LASSO, outperformed the standard Cox PH Lasso in simulations and two cancer data sets from TCGA and the Gene Expression Omnibus (GEO), each containing clinical data and one or more molecular modalities, in terms of the integrated Brier score.
Klau et al. (2018)  proposed a sequential approach to integrate multiple modalities in linear sparse cancer survival models. Their proposed method, prioritylasso, fitted each modality using an ℓ 1 -regularized Cox PH model sequentially and at each step used the predictions from the previous round as an unregularized offset. The authors showed that prioritylasso performed as well or better than a Cox PH Lasso model on a cancer survival dataset obtained from GEO that contained both clinical data and molecular modalities. In addition, prioritylasso provided clinicians the ability to influence the model by assigning their preferred modality ordering. The model was in turn more likely to include variables belonging to a block that had been assigned a higher priority. Knowledge distillation. Knowledge distillation was coined by  Hinton et al.
Datasets We considered 17 out of the 33 available cancer datasets from TCGA, following the dataset selection of  Herrmann et al. (2021). We excluded datasets that contained less than 100 samples after preprocessing, datasets that did not have a minimum event ratio of 5%, and datasets that did not have all required molecular modalities. We also excluded Acute Myeloid Leukemia (LAML) relative to the work of  Herrmann et al. (2021)  since it contained only 35 samples after preprocessing in their study. We used the same clinical variables as Herrmann et al. (2021) except that we excluded clinical variables that were missing for more than five patients on a particular dataset. 2 2 Presumably, using fewer clinical variables relative to Her- We excluded any patients that were still missing any of the thus chosen clinical variables.
Categorical clinical variables were one-hot encoded. We used all molecular information broadly available on TCGA, except whole slides images. We thus used, in addition to clinical data, gene expression (GEX), mutation, DNA methylation, copy number variation (CNV), reverse phase protein array (RPPA), and miRNA. We excluded patients that did not have all required genomic modalities (that is, patients that were, for example, completely missing the RPPA modality). Molecular features that were missing for more than one patient were excluded to conserve as many patients as possible. We used log-transformed values of miRNA and gene expression. All other molecular input data were used without further preprocessing. We provide a link to our complete datasets within our code repository.
Teacher models For the considered teacher models, we restricted ourselves to methods that were originally designed for individual cancer datasets, that is, we excluded methods fit on pan-cancer data, both since clinical data are often cancer-specific and since this would have made us deviate from our main research question. We considered BlockForest  (Hornung & Wright, 2019b)  (BF), a random forest method adapted to multi-omics data. In addition, we included RandomBlock, a variant of Block-Forest, which has been shown to outperform BlockForest if clinical variables were favored (mandatorily considered at every split point selection within each tree)  (Hornung & Wright, 2019b; Wissel et al. , 2022). For RandomBlock, we favored clinical features and will thus refer to it as Ran-domBlock favoring (RBF).
Validation For validation, on each dataset, we used five-fold crossvalidation, stratified for the censoring indicator, twice repeated, yielding 10 test splits per dataset. Although pooling performance measures across datasets has been described as questionable  (Demšar, 2006) , depending on the setting, pooling test concordances across cancers on TCGA is somewhat standard, for both models training on multiple cancers and models trained only on individual cancers  (Herrmann et al. , 2021; Vale-Silva & Rohr, 2021; Cheerla & Gevaert, 2019; Kim et al. , 2020). Thus, for convenience of exposition, we will, at times also pool performance metrics across datasets. When pooling test concordances across datasets, we included Wilcoxon ranked sign significance tests that have been recommended to compare models across datasets  (Demšar, 2006).
For the Wilcoxon ranked sign tests, to summarise the performance of each model on each dataset, we calculated the mean concordance across test splits on each dataset and used these means per dataset as the performances to be tested for differences, following the approach of  (Hornung & Wright, 2019b). We emphasize that a higher pooled mean concordance does not necessarily correspond to a more significant performance difference as measured by the p-value of the respective Wilcoxon ranked sign test. We did not test for significant performance differences on individual cancer datasets, as tests adjusting for the repeated cross-validation that we used, such as the corrected repeated k-fold cv test  (Bouckaert & Frank, 2004) , would have been underpowered giving our relatively low sample size of 10 splits per cancer.
Knowledge distillation was effective for sparsifying high-dimensional survival models when guided by clinical variables We found that, when performing knowledge distillation as outlined, standard ℓ 1 -regularized linear students (Equation (2)) were able to match their teachers well, achieving reasonably high train and test fidelity as measured by Spearman's correlation coefficient ρ between the teacher predictions and the student predictions (Table  1 ). However, in terms of prediction accuracy, only the student of HSAE did not perform significantly worse than its teacher across datasets, with the students of BF and RBF both performing significantly worse than their teachers. All student models were reasonably sparse, with the students of BF and RBF both using an average of roughly 50 features, while the student of HSAE was considerably less sparse at around 130 features.
Of course, both types of students were orders of magnitude more sparse than their teachers, which did not perform explicit feature selection and thus used all available features (Table  1 ). Various factors had an impact on the discriminative performance of our clinically guided students, relative to their teachers. First and foremost, we saw the clinically guided students outperform their teachers on datasets on which the clinical-only model performed particularly well (e. g. , COAD, OV, and LUSC) (Figure  1 ). However, we also found that our clinically guided students tended to slightly outperform their teachers even on cancers on which the clinical-only model underperformed, such as LGG or LIHC. Overall, we rarely found the clinically guided students to match their teachers exactly in terms of discriminative performance, instead often but not always underperforming (e.