dataesr/scientific-paragraphs-categorization huggingface dataset:
2. For comparing ENLIGHT to other biomarkers on ICB datasets (Figure  3D ), we identified a threshold for each of the presented biomarkers based only on Anti-PD1. The difference in threshold identification is intended to make an appropriate comparison to ENLIGHT's recall which naturally varies between the two cases. The identified thresholds were then used to calculate OR based on the test datasets for Figure  3. In order to avoid artifacts arising from varying mRNA quantification platform (i. e. RNAseq vs. microarrays), for example the difference in the dynamic range of expression between platforms, all biomarkers that do not have explicit means for calculation (i. e. all markers except SELECT, TIDE and CD8 + abundance which have available code for calculation) were calculated based on gene-wise ranking of expression across samples, after applying proper normalizations to account for library size and gene lengths (if needed).
QUANTIFICATION AND STATISTICAL ANALYSIS Differences between distributions were analyzed using a one-sided Mann-Whitney test using the scipy. stats. mannwhitneyu function in python. Test of OR > 1 was done using Fisher's exact test. Difference between OR was done using a one-sided test for difference in OR. Difference between PPV and overall response rates was done using a one-sided proportion test. Difference between OR of two methods was done using Chi square test. All statistical results were FDR corrected for multiple hypotheses throughout the paper. Throughout the paper, we denoted p < 0. 1 with a single asterix (*) and p < 0. 05 with two asterix (**). Throughout the paper, brackets denote 95% CI and N denotes the sample size of the dataset.
Software and algorithms EMS web service This manuscript https://ems. pangeabiomed. com/ Deposited data Bevacizumab (Watanabe et al. , 2011) 40 https://github. com/PangeaResearch/ enlight-data/blob/main/Data/Bevacizumab. csv Sorafenib (Pinyol et al. , 2019) 39 https://github. com/PangeaResearch/ enlight-data/blob/main/Data/Sorafenib. csv Lapatinib (Dieci et al. , 2016) 34 https://github. com/PangeaResearch/ enlight-data/blob/main/Data/Lapatinib. csv Trastuzumab (Prat et al. , 2014) 41 https://github. com/PangeaResearch/ enlight-data/blob/main/Data/Trastuzumab. csv BRAFi_1 (Guarneri et al. , 2015) 35 https://github. com/PangeaResearch/ enlight-data/blob/main/Data/BRAFi_1. csv BRAFi_2 (Kakavand et al. , 2017) 36 https://github. com/PangeaResearch/ enlight-data/blob/main/Data/BRAFi_2. csv BRAFi_3 (Rizos et al.
, 2014) 37 https://github. com/PangeaResearch/ enlight-data/blob/main/Data/BRAFi_3. csv Anti-PD1 (Riaz et al. , 2017) 38 https://github. com/PangeaResearch/ enlight-data/blob/main/Data/Anti-PD1. csv Bevacizumab 2 (Pentheroudakis et al. , 2014) 45 https://github. com/PangeaResearch/ enlight-data/blob/main/Data/Bevacizumab_2. csv Bevacizumab 3 (Birkbak et al. , 2018) 46 https://github. com/PangeaResearch/ enlight-data/blob/main/Data/Bevacizumab_3. csv Bevacizumab 4 (Verstraete et al. , 2015) 47 https://github. com/PangeaResearch/ enlight-data/blob/main/Data/Bevacizumab_4. csv Sorafenib 2 (Byers et al. , 2013) 48 https://github. com/PangeaResearch/ enlight-data/blob/main/Data/Sorafenib_2. csv Trastuzumab 2 (Dieci et al. , 2016) 34 https://github. com/PangeaResearch/ enlight-data/blob/main/Data/Trastuzumab_2.
csv Trastuzumab 3 (Liu et al. , 2012) 49 https://github. com/PangeaResearch/ enlight-data/blob/main/Data/Trastuzumab_3. csv Trastuzumab 4 (Shen et al. , 2012) 50 https://github. com/PangeaResearch/ enlight-data/blob/main/Data/Trastuzumab_4. csv Trastuzumab 5 (Sammut et al. , 2022) 20 https://github. com/PangeaResearch/ enlight-data/blob/main/Data/Trastuzumab_5. csv Cetuximab (Bossi et al. , 2016) 51 https://github. com/PangeaResearch/ enlight-data/blob/main/Data/Cetuximab. csv Selinexor (Lassman et al. , 2022) 52 https://github. com/PangeaResearch/ enlight-data/blob/main/Data/Selinexor. csv MK2206 (Magbanua et al. , 2021) 53 https://github. com/PangeaResearch/ enlight-data/blob/main/Data/MK2206. csv Tipifarnib 1 (Raponi et al. , 2007) 54 https://github. com/PangeaResearch/ enlight-data/blob/main/Data/Tipifarnib_1.
csv Tipifarnib 2 (Raponi et al. , 2008) 55 https://github. com/PangeaResearch/ enlight-data/blob/main/Data/Tipifarnib_2. csv Rituximab (Foaet al. , 2014) 56 https://github. com/PangeaResearch/ enlight-data/blob/main/Data/Rituximab. csv Alpelisib/Ribociclib This manuscript https://github. com/PangeaResearch/ enlight-data/blob/main/Data/MGH_Alpelisib. csv, https://github. com/PangeaResearch/ enlight-data/blob/main/Data/MGH_Ribociclib. csv Anti-PD1 2 (Zhao et al. , 2019) 57 https://github. com/PangeaResearch/ enlight-data/blob/main/Data/Anti-PD1_2. csv Anti-PD1 +-Anti-CTLA4 (Hsu et al. , 2021) 58 https://github. com/PangeaResearch/ enlight-data/blob/main/Data/Anti-PD1%20%2B-%20Anti-CTLA4. csv (Continued on next page).
METHOD DETAILS Data collection We surveyed the public domain for available cohorts of patients receiving targeted therapies or immunotherapies, containing both pre-treatment transcriptomics and response information (either RECIST or a binary classification of response). We identified a total of 23 real world datasets which were not previously analyzed by either SELECT or ENLIGHT, and can hence serve as unseen datasets: 22 datasets 20,24,34,40,45-61 from GEO, ArrayExpress, CTRDB or the broader literature published by February 2022, and one dataset that was obtained as part of a collaboration with Massachusetts General Hospital (MGH) which we publish here for the first time. We selected six datasets  [34] [35] [36] [37] [38] [39]  already analyzed in Lee et al. 29  along with two 40,41 of these 23 unseen sets to serve as tuning sets.
These eight tuning datasets were selected as they span a range of different treatments, therapeutic classes, response rates and sample sizes, reflecting diverse real-world data, covering five targeted therapies and one immune checkpoint blockade (ICB). These datasets were used to tune the parameters of ENLIGHT, including the GI network size and a decision threshold on the ENLIGHT Matching Score (EMS) that is used for predicting response (see below). We set aside the remaining 21 unseen datasets -the evaluation sets -as unseen data for evaluation.
In a nutshell, the PATE protocol labels a subset of a public dataset and uses this partially labeled dataset to train a student model in a semi-supervised way. The labelization is achieved by aggregating (usually by means of a majority voting) the labels -considered as votes -provided by a set of teacher models trained on private datasets. Since the teachers' labels would leak information on their training data, the PATE protocol makes use of differential privacy (DP). To get a reasonable privacy-utility trade-off, the aggregation of votes is performed on a trusted independent server. The SPEED approach  [20]  builds upon the work from  [33]  and uses Fully Homomorphic Encryption (FHE) to blind the server by having it perform the aggregation directly over encrypted votes, therefore protecting the data against an honest-but-curious server.
Differential privacy Differential privacy  [14]  is a gold standard concept in privacypreserving data analysis. It provides a guarantee that under a reasonable privacy budget (𝜖, 𝛿), two adjacent databases produce statistically indistinguishable results. In the context of SPEED, we consider, as in  [20] , that a database is the concatenation of the data owners' (teachers) datasets and that two databases are adjacent if they differ by one teacher. In the whole paper, P(𝐴) denotes the probability of event 𝐴. Definition 3. 1. A randomized mechanism M with output range R satisfies (𝜖, 𝛿)-DP if for any two adjacent databases 𝑑 and 𝑑 ′ and for any subset of outputs 𝑆 ⊂ R one has P [M (𝑑) ∈ 𝑆] ≤ 𝑒 𝜖 P M (𝑑 ′ ) ∈ 𝑆 + 𝛿. (𝜖, 𝛿) constitutes the privacy cost of the mechanism -the lower 𝜖 and 𝛿, the more private the mechanism.
Definition 3. 1 implies that a differentially private mechanism is necessarily probabilistic. Most often, a deterministic mechanism is turned differentially private by adding random noise at a certain moment in the computation. Since the privacy cost increases with the number of queries to the mechanism, we determine the privacy cost of our protocol via a two-fold approach. First of all, we determine the privacy cost per query and then we compose the privacy costs of each query to get the overall cost. The classical composition theorem (see e. g. [15] ) states that the guarantees 𝜖 of sequential queries add up. Nevertheless, training a deep neural network, even with a collaborative framework as presented in this paper, requires a large amount of calls to the databases, precluding the use of this classical composition.
Multi-degree SHIELD We can imagine a parameter 𝑝 that decreases as the iterations run, as if it adapted to the vote distribution. Indeed, on one hand, a high 𝑝 for the first iterations ensures (with high probability) that we get the most frequent vector if getting a non-null vector is easy (i. e. probable), which happens if a vast majority of the vectors code for the same class (i. e. a vast majority of voters agree on one candidate). On the other hand, if the first iterations failed, which suggests that getting a non-null vector is not so probable, the number 𝑝 of multiplications decreases, making the production of a non-null vector easier. In this framework, our SHIELD operator can be represented by a polynomial 𝐷 𝑝=1 𝑎 𝑝 𝑋 𝑝 with positive integer coefficients, where 𝐷 is the highest value taken by 𝑝, 𝐷 𝑝=1 𝑎 𝑝 = 𝑎 and some coefficients 𝑎 𝑝 which may be null.
Offset parameter The SHIELD operator as defined above cannot always provide finite DP guarantees. Let us consider two adjacent databases 𝑑 and 𝑑 ′ such that, in 𝑑, a class 𝑐 was chosen by no voter and, in 𝑑 ′ , 𝑐 was chosen by one voter. Then, with input 𝑑, SHIELD will never output 𝑐 because it cannot pick a one-hot encoding for 𝑐, the probability of outputting 𝑐 is then null. On the contrary, with input 𝑑 ′ , there is a non-null probability (even if it is small) of outputting 𝑐. Hence, the ratio of probabilities of outputting 𝑐 is not bounded and we get an infinite privacy cost  4. To avoid this problem, we force all the classes to have at least one vote by creating a dummy one-hot encoding for each class. More generally, 𝜔 dummy one-hot encodings can be created for each class, where 𝜔 is another parameter of SHIELD, called the offset.
Algorithm 2 gives the pseudocode of the multi-degree version of SHIELD with the offset parameter. In our experiments, we fixed 𝜔 to 1, letting the optimization of this parameter for further work. It is nevertheless intuitive that the greater 𝜔, the worse the accuracy because, when 𝜔 is large, the distribution of the votes is flattened and the probability of outputting the true argmax is lower.
FHE IMPLEMENTATION OF SHIELD Algorithm 2 is a generic version of SHIELD that actually needs to be adapted for an implementation using an HE cryptosystem. We consider two kinds of encodings, depending on the encryption scheme that we use: • With the BFV cryptosystem, we use batching i. e. , encode a number of values together in a single polynomial which is then encrypted. A single operation on a ciphertext thus leads to the same operation applied to all values encoded (slots) inside the ciphertext in a Single Instruction, Multiple Data (SIMD) fashion. On the downside, BFV performances are very sensitive to multiplicative depth and inter-slot operations are costly. • With the TFHE cryptosystem  [12] , we use a single ciphertext to encrypt a single value i. e. proceed in a Single Instruction, Single Data (SISD) fashion.
AN APPLICATION CASE: SPEED 6. 1 PATE workflow Our SHIELD operator is very well adapted to a learning protocol called SPEED, from  [20] , itself inspired from PATE  [33]. SPEED method is illustrated by Figure  1 , inspired from  [20]. Assuming the existence of a public unlabeled database Δ (we will keep this notation throughout the paper), PATE enables several data-owners, called teachers, to collaboratively train a classification model without outsourcing their data that are considered private. The idea is to label Δ and use it to train the final classification model, called the student model or simply the student. To do so, each teacher is asked to train a model beforehand for the same task as the student's target task with its own data only. For each sample of Δ to label, every teacher infers a label through its model and sends this label to an aggregation server.
A priori accuracy metrics The ultimate accuracy that we want to maximize in SPEED application case is obviously the testing accuracy of the student model. Nevertheless, it could be interesting to measure the accuracy of the argmax operator itself, independently of the student training. Also, even if this depends on the teachers' votes and thus on the used dataset, this enables us to evaluate polynomial parameterizations without performing the student training, which is much faster and allows to test much more parameterizations. We call such an accuracy an a priori accuracy. The most straightforward way to define the argmax accuracy is to consider the probability of getting the exact argmax. Nevertheless, this approach treats any mistake the same way. It could be argued that outputting, say, the class that received the second greatest number of votes is better than outputting the least preferred class.
Differential privacy analysis Since the student model training requires many requests to the teachers and, indirectly, to their private datasets, we use, as in  [20] , the moments accountant technique  [1]  to get a better privacy cost over composition. We here consider that two databases 𝑑 and 𝑑 ′ are adjacent if they are the concatenations of the datasets from the same number of teachers and only one teacher differs from one database to the other. This implies that either all the 𝑛 ′ 𝑘 , counts for database 𝑑 ′ , for 𝑘 ∈ [𝑀], are equal to the 𝑛 𝑘 , counts for database 𝑑, in which case the corresponding moments accountant is null, or the 𝑛 ′ 𝑘 differ from the 𝑛 𝑘 only for two values of 𝑘, say 𝑘 1 and 𝑘 2 , such that 𝑛 ′ Unlike most DP mechanisms, the stochastic behavior of our operator does not come from an additional random noise, since the operator is inherently probabilistic.
Computing the probability distribution of the output We compute the probability distribution of the output of the algorithm SHIELD with a given polynomial parameterization in a recursive manner. For a sample 𝑥 of Δ, let M 𝑃,𝑥 be the mechanism that takes the whole database (concatenation of the teachers' datasets) as input and outputs the class sent to the student i. e. the output of SHIELD, with the polynomial parameterization 𝑃 ∈ N[𝑋 ]. Let 𝑑 be the database composed of the teachers' data. Let 𝑘 be a class of the problem. If 𝑃 = 𝑋 𝑝 , 𝑝 ∈ N * , P[M 𝑃,𝑥 (𝑑) = 𝑘] = 𝑛 𝑘 𝑛 𝑝. If 𝑃 = 𝑋 𝑝 + 𝑄 (𝑋 ) , where 𝑄 ∈ N[X] and 𝑝 ∈ N * is greater or equal than the degree of 𝑄, P[M 𝑃,𝑥 (𝑑) = 𝑘] = 𝑛 𝑘 𝑛 𝑝 + 1 - 𝐾 ∑︁ 𝑗=1 𝑛 𝑗 𝑛 𝑝 P[M 𝑄,𝑥 (𝑑) = 𝑘]. Using these expressions, we simply compute the moments accountant for each query by taking the maximum over all pairs (𝑑, 𝑑 ′ ) such that 𝑑 is the database constituted by the concatenation of the teachers' database and 𝑑 ′ is a database adjacent to 𝑑.
We then derive the overall privacy cost using the moments accountant's composition properties. Note that the obtained DP guarantees are data-dependent since we explored only the pairs of adjacent databases such that one of them is the actual database given by our application. The very values 𝜖 and 𝛿 of these guarantees then reveal some information about the training data. In a real-life scenario, these values should be sanitized before being published, as in  [34]  for instance, but this is beyond the scope of this work.
Choice of the polynomial parameterization We tested SPEED with SHIELD on MNIST dataset  [27]. While the offset parameter has been set to 1, a key aspect of our experiments is the choice of a polynomial parameterization that realizes a good trade-off between model accuracy, DP guarantees and computational efficiency. Since the computational time overall depends on the sum of coefficients and the degree of the polynomial parameterization, we proceeded by constraining the maximum degree and the maximum value for the sum of coefficients of the polynomials. We fixed the maximum degree to 4 to achieve a reasonable balance between the multiplicative depth, the number of tries and the argmax accuracy. For several integer values  (6, 12, 17, 32) , we considered all the polynomials of degree at most 4 whose sum of coefficients is less than this value.
SIMD SHIELD with BFV For our implementation of the SIMD SHIELD algorithm, we use the BFV cryptosystem in the openFHE library  [4]. The parameters we choose are the following: log 2 (𝑞) = 540 ; 𝑝 = 65537 ; 𝑚 = 65536 ; 𝑁 = 32768. These parameters achieve a security level of 𝜆 = 128 bits with a standard deviation of 3. 2. Our implementation was tested on a machine with an AMD Opteron(tm) Processor 6172 using a single thread. We achieve performances presented in Table  2  for a set of different polynomial parameterizations. Although we tested using the MNIST data set, the performance of an HE algorithm does not depend on the underlying data by construction. Otherwise one could infer something on the data from seeing the computation happen in the encrypted domain. For our implementation, we need to run the SHIELD algorithm over 100 samples.
Testing SHIELD on more difficult datasets and especially datasets with numerous classes could reveal its full potential. Besides, a more thorough theoretical study to get results that may lead us through the choice of the parameters (polynomial, offset) is desirable. Other versions including sampling without replacement (Appendix A) or an exponential version of SHIELD (Section 4. 4) would also deserve theoretical and experimental analyses. Studying SHIELD in terms of strategy-proofness and fairness could be interesting too and would extend the added value of SHIELD to the area of computational social choice and voting rules  [11]. term 𝑋 , then the server will know that the class sent to the student is its victim's vote. Supposing that the server knows the votes of all the teachers except its victim's (classical assumption in DP), it will be able to recover its victim's votes in many cases.
B ON COUNTER-PRODUCTIVE NOISE FOR DATA-DEPENDENT DIFFERENTIAL PRIVACY GUARANTEES Null data-dependent privacy cost of the exact argmax: While doing experiments on a subset of MNIST with polynomial parameterizations that yield better and better accuracies (up to the probability of getting the true argmax being more than 99,99%) we remarked that the value epsilon of the privacy cost did not increase much and did not seem to approach infinity. This surprising result suggested that the exact argmax operator had a finite privacy cost. Actually, on the subset we were working on, for every sample, the dominant class had at least two more votes than the second dominant class. We will say in the following that the distribution has a highly dominant argmax. This implies that, any database which is adjacent (i.
e. differs from at most one teacher) to the database 𝑑 we were working on has the same dominant class as 𝑑 for every sample. As a consequence, the output of the argmax does not leak any information about which of two adjacent databases was used as input. In other words, the privacy cost of the exact argmax operator is null in this case. Counter-productivity of the noise regarding privacy: On the contrary, the so-called private argmax operator (noised by an additional random noise as in PATE  [33, 34]  and SPEED  [20]  or intrinsically stochastic as in SHIELD) may output any class and the probabilities of outputting a class depends on the frequencies of the votes for all classes. As a consequence, even changing only one teacher will change the probabilities of outputting a certain class, even if the effect is mild.
Therefore, since the output of the DP argmax operator gives information on the probability of outputting a class, it also gives information on the frequencies of the classes in the votes. We end up in a (particular) situation where applying noise is counter-productive in the sense that it increases the privacy cost of revealing the output (by an infinite factor actually). Note, however, that this was not the case for the entire MNIST training set but only for a certain subset of it. The case of data-independent DP guarantees: This consideration only applies to data-dependent DP guarantees. In the dataindependent case, the privacy cost of the exact argmax would be infinite because we would consider the maximum over all the pairs of adjacent databases i. e. all the possible pairs of distributions of 𝑛 votes among 𝐾 classes that differ by one vote on two classes.
ACKNOWLEDGMENTS We thank  Victor Berger  early discussions on the idea of ground truth accuracy as well as the anonymous referees for their comments which lead to improvements of this paper. This work was supported by the  France 2030 ANR  Project  ANR-22-PECY-003  SecureCompute and the  European Union 's  Horizon Europe Research and Innovation Programme  under Grant Agreement No. 101070670  ( ENCRYPT -A Scalable and Practical Privacy-preserving Framework ).
We incorporate modules included in previously developed simulators  (Ma et al. , 2018; Vuolo et al. , 2009; Hodzic et al. , 2004)  into the community tool Cloud Feedback Model Intercomparison Project (CFMIP) Observation Simulator Package version 2 (COSPv2) to create a simple base on which each group can build up its own analysis. The goal is to facilitate the comparison between general circulation models (GCMs) and space lidar aerosol data. Besides CALIPSO operating at 532 and 1064 nm, the Atmospheric Lidar (ATLID) instrument of the EarthCARE mission is expected to become operational in 2023. In synergy with other instruments, it will provide vertical profiles of aerosol and thin clouds, operating at 355 nm with a high-spectral-resolution (HSR) receiver and depolarization channel. Moreover another HSR Lidar operating at 532 and 1064 nm is expected to be launched in the future.
The COSPv2 lidar simulator will thus be a useful tool for the exploitation of these new datasets and the comparison with GCMs of several modeling groups. We have chosen to implement the lidar aerosol simulator within the COSPv2 software package to leverage all the simulator capabilities available in COSPv2. Moreover, COSPv2 is already implemented in several GCMs  (Swales et al. , 2018) , so the addition of the aerosol lidar simulator module should only require a small amount of effort for the modeling groups.
2. 1. The GCM vertical profile of cloud fraction is also passed to the Subgrid Cloud Overlap Profile Sampler (SCOPS)  (Klein and Jakob, 1999)  procedure in COSPv2 to generate sub-grid columns within a grid cell in accordance with the simulated cloud fraction and the vertical overlap assumption. 1  for the correspondence between the names of the variables in the code and in the present paper. input variables α a and β a and the variables α m β m and AMB calculated in step 1. Because the GCM does not consider sub-grid variability of aerosols, we compute the ATB and SR for each grid cell. 3. Vertical regridding. The total extinction (α a + α m ), ATB and SR profiles are vertically re-gridded over a standard vertical grid having N equidistant levels to obtain profiles of total extinction (EXT_initial), attenuated total backscatter (ATB_initial) and backscatter ratio (SR_initial) at the vertical resolution of the space lidar observations that would be observed in absence of instrumental noise.