dataesr/scientific-paragraphs-categorization huggingface dataset:
Group fairness metrics, such as disparate impact and equalized odds constrain the probability of the positive prediction of the classifier to be (almost) equal among different sensitive groups  (Dwork et al. 2012; Feldman et al. 2015). On the other hand, causal fairness metrics assess the difference in positive predictions if every feature in the causal relation remains identical except the sensitive feature  (Nabi and Shpitser 2018; Zhang and Bareinboim 2018). The early works on fairness verification focused on measuring fairness metrics of a classifier for a given dataset  (Bellamy et al. 2018). Naturally, such techniques were limited in enhancing confidence of users for wide deployment. Consequently, recent verifiers seek to achieve verification beyond finite dataset and in turn focus on the probability distribution of features  (Albarghouthi et al.
Background In this section, we define different fairness metrics proposed for classification. Following that, we state basics of stochastic subset sum and Bayesian networks that are the main components of our methodology. Fairness in ML. We consider 1  a dataset D as a collection of triples (X, A, Y ) generated from an underlying distribution D. X {X 1 ,. , X m1 } are non-sensitive features whereas A {A 1 ,. , A m2 } are categorical sensitive features. Y ∈ {0, 1} is the binary label (or class) of (X, A). Each non-sensitive feature X i is sampled from a continuous probability distribution X i , and each sensitive feature A j ∈ {0,. , N j } is sampled from a discrete probability distribution A j. We use (x, a) to denote the feature-values of (X, A). For sensitive features, a valuation vector a = [a 1 ,.
, a m2 ] is called a compound sensitive group. For example, consider A = {race, sex} where race ∈ {Asian, Color, White} and sex ∈ {female, male}. Thus a = [Asian, female] is a compound sensitive group. We represent a binary classifier trained on the dataset D as M : (X, A) → Ŷ. Here, Ŷ ∈ {0, 1} is the predicted class of (X, A). We now discuss different fairness metrics in the literature based on the prediction of a classifier  (Feldman et al. 2015; Hardt, Price, and Srebro 2016; Nabi and Shpitser 2018). In this paper, FVGM verifies two families of fairness metrics: group fairness (first three in the following) and path-specific causal fairness.
Empirical Performance Analysis In this section, we empirically evaluate the performance of FVGM. We first present the experimental setup and the objective of our experiments, followed by experimental results. time taken on the Y -axis; a point (x, y) implies that a verifier takes less than or equal to y seconds to compute fairness metrics of x many benchmarks. We consider 100 benchmarks generated from 5 real-world datasets using 5-fold cross-validation. In each fold, we consider {25, 50, 75, 100} percent of non-sensitive features. Experimental Setup. We implement a prototype of FVGM in Python (version 3. 8). We deploy the Scikit-learn library for learning linear classifiers such as Logistic Regression (LR) and Support Vector Machine (SVM) with linear kernels. We perform five-fold cross-validation.
Scalability Analysis Benchmarks. We perform the scalability analysis on five real-world datasets studied in fair ML literature: UCI Adult, Germancredit (Doshi-Velez and Kim 2017), COMPAS  (Angwin et al. 2016 ), Ricci (McGinley 2010) , and Titanic (https://www. kaggle. com/c/titanic). We consider 100 benchmarks generated from 5 real-world datasets and report the computation times (for DI and SP) of different verifiers. Results. In Figure  3 , we present the scalability results of different verifiers. First, we observe that FairSquare often times out (= 900 seconds) and can solve ≤ 5 benchmarks. This indicates that SMT-based reduction for linear classifiers cannot scale. Similarly, SSAT-based verifier Justicia that performs pseudo-Boolean to CNF translation for linear classifiers, times out for around 20 out of 100 benchmarks.
In most instances for Adult dataset, reweighing (RW)  (Kamiran and Calders 2012)  and optimized pre-processing (OP)  (Calmon et al. 2017 ) algorithms 3 To analytically compute DI, let the coefficients of the classifier be wi for Xi and wA for A, and bias be τ. Since all non-sensitive features are from Gaussian distributions, we compute the probability of the predicted class Pr [ Ŷ |A = 1] ∼ N ( n-1 i=1 wiµi, σ 2 Ŷ ) and Pr[ Ŷ |A = 0] ∼ N ( n-1 i=1 wiµ i , σ 2 Ŷ ) with σ 2 Ŷ = σ 2 ( n-1 i=1 w 2 i ). Hence, the PPV of the classifier is 1 -CDF Ŷ |A=1 (τ -wA) for A = 1 and 1 -CDF Ŷ |A=0 (τ ) for A = 0, where CDF is the cumulative distribution function. Finally, we compute DI by taking the ratio of the minimum and the maximum of the two PPVs. are successful in improving fairness. Except for the unfairness regarding the sensitive feature 'sex', where OP algorithm fails.
Fairness Influence Function (FIF): Tracing Sources of Unfairness. Another application of FVGM as a fairness verifier is to quantify the effect of a subset of features on fairness. Thus, we define fairness influence function (FIF) that computes the effect of a subset of features S on the PPV of a classifier given a specific sensitive group A = a, FIF(S) Pr[ Ŷ = 1|A = a, D] -Pr[ Ŷ = 1|A = a, D -S ]. FIF allows us to explain the sources of unfairness in the classifier. In practice, we compute FIF of S by replacing the probability distribution of S with a uniformly random distribution, referred to as D -S , and reporting differences in the conditional PPVs. Further details of FIF are in Appendix. In Figure  6 , we compute FIF for all features in COMPAS dataset, separately for two sensitive groups: Female ('sex' = 1) and Male.
This dataset concerns the likelihood of a person of re-offending crimes within the next two years. We first observe that the base PPV values are different for the two groups (0. 46 vs 0. 61 for Female and Male), thereby showing Male as more probable to re-offend crimes than Female. Moreover, FIF of the feature 'age' is comparatively higher in magnitude for Male than Female. This implies that while deciding recidivism the algorithm assumes that Female individuals across different ages re-offend crimes with almost the same probability and the probability of re-offending for Male individuals highly depends on age. Thus, applying FVGM to compute FIF provides us insights about sources of bias and thus, indicators to improve fairness.
D. 3 Verifying Fairness Algorithms on Multiple Fairness Metrics We show extended results on verifying fairness attack in Figure  9  for two fairness metrics: disparate impact (DI) and statistical parity (SP). We observe that FVGM can detect poisoning attack for both metrics. In Figure  10  we show verification results on compound sensitive groups with respect to multiple fairness metrics. In this figure, we observe that with an increase in the number of groups, fairness metrics worsens-disparate impact decreases and other three metrics increases. Additionally, in Table  2 , we show extended results for German and COMPAS dataset on verifying fairness algorithms: reweighing and optimized pre-processing algorithms. 1: 0. 12 income < 0. 24 2: 0. 24 income < 0. 37 3: 0. 37 income < 0. 49 4: 0. 49 income < 0.
In contrast, no guarantee is given for functions that target the CL model, which is harder to verify; Program Counter (PC) Conditionals leak their guards Baseline (BL) PC + Memory R/W leak addresses Cache line (CL) PC + Mem. accesses leak cache lines Time-Variable (TV) BL + TV arithmetic operators leak TV + CL TV arithmetic operators leak + CL • a library is verified for constant-time in a weaker leakage model than intended, because of inherent limitations in the verification technology, and as a consequence, it may still leak in the intended leakage model. For instance, some crypto routines in (earlier versions of) OpenSSL are provably secure in the BL leakage model but are insecure in the TV model and vulnerable to practical timing attacks-such as Lucky13  [1]. These two scenarios reflect the existence of a potentially dangerous gap in computer-aided cryptography.
The second and main technical contribution is a mechanized proof in the Coq proof assistant that the Jasmin compiler preserves a class of fine-grained constant-time policies. This class encompasses the BL policy as well as policies that capture time-variable instructions and fine-grained leakage models of memory accesses. The proof takes the form of an instrumented correctness theorem  [15]  and shows that leakage of assembly programs can be computed deterministically from leakage of source programs. Our third contribution is a set of formal proofs that previously unverified cryptographic code is constant-time in a (non-baseline) leakage model. This includes Langley's patch to Lucky13 and our own fix of OpenSSL. The proofs are carried via an embedding of Jasmin source code into EasyCrypt and using EasyCrypt's implementation of relational Hoare logic.
Although the proofs are carried for Jasmin programs, our certified compiler carries the guarantees to the generated code. Overall, our work expands tool support for the constant-time policy, which has been very beneficial and has contributed to making cryptographic libraries more robust against timing attacks, to a rich set of policies. Supplementary material. The full development is provided as supplementary material at https://github. com/jasmin-lang/jasmin/ tree/constant-time-op.
Background on Jasmin Jasmin is a framework for high-speed and high-assurance cryptography  [2, 4]. The Jasmin language smoothly combines high-level and low-level constructs so as to support "assembly in the head" programming. Programmers can control many low-level details that are performance-critical: instruction selection and scheduling, what registers to spill and when etc. They can also rely on high-level abstractions (variables, functions, arrays, loops, etc. ) to structure their code and make it more amenable to formal verification. Indeed, the Jasmin infrastructure has been used to implement cryptographic constructions that are as efficient as state-of-theart handwritten assembly and, nonetheless, formally verified for correctness and security with machine-checked proofs. These implementations notably include scalar multiplication on the Curve25519 elliptic curve  [2] , the ChaCha20 stream cipher  [4] , and the SHA-3 hash function  [5].
Background on EasyCrypt EasyCrypt  [11]  is a proof assistant used for reasoning about cryptographic primitives. EasyCrypt features an ambient higher-order logic with a backend to SMT solvers as well as several program logics to reason about (probabilistic) imperative programs written in a core language. EasyCrypt was originally used for proving security properties for several cryptographic primitives  [8, 12, 13]. More recently, it has been used to prove functional correctness and side-channel security of cryptographic primitives written in Jasmin. For the latter, proofs proceed by embedding Jasmin programs with their leakage into EasyCrypt, and by using relational Hoare logic to prove that leakage is independent of secrets. In prior work, focusing on the BL model, application of relational Hoare logic is fully automated.
e first cache line is touched first and then the second). We have reported this issue and provided a formally verified fix (presented in Section 7. 3. 2) in the intended leakage model to OpenSSL developers. The bug is acknowledged, and OpenSSL developers merged the fix into affected releases. Outline. In the next sections, we introduce a general setting to reason about fine-grained constant-time policies and prove that the above examples verify their policies using relational program logic. We also provide a generic proof that these policies are preserved by compilation. Finally, we implement our approach and evaluate its working on the illustrative and other examples.
IMPLEMENTATION We have have implemented our approach in the Jasmin framework. Our implementation consists of: • a Coq formalization of fine-grained leakage, and a formal proof that the Jasmin compiler preserves fine-grained constant-time policies; • an OCaml implementation that extracts an EasyCrypt program from a Jasmin program; • an OCaml implementation of an evaluator for testing constanttimeness. The first two components of the implementation preserve the workflow for Jasmin programs: • Jasmin programs are checked for safety and compiled; • proofs of constant-timeness are carried on source Jasmin programs via an embedding into EasyCrypt 5. Program instrumentation is performed during the embedding, in a way similar to what is presented in Section 5. The last component adds an additional functionality (testing for constant-timeness), which is extremely helpful when dealing with fine-grained policies, which have considerably more complex proofs.
out is a pointer that is public, which means its value (the address) will be equal on both sides, and it reveals nothing about the data stored at the address. The judgment requires no assumptions for the value of the variable rotate offset; hence it can be secret dependent. The proof in EasyCrypt is fully automatic and uses an automated tactic based on dependency analysis. Similarly, we prove that the function rotate offset BL is secure in the BL model. Next we prove that rotate offset TV is secure in the TV model. The pre-condition is: = {leak, md size, scan start} ∧ (0 ≤ mac start{1} -scan start{1} < 256) ∧ (0 ≤ mac start{2} -scan start{2} < 256) ∧ 16 ≤ md size{1} ≤ 64 and the post-condition is simply = {leak}. The notation x{1} refers to the value of the variable x in the left state while x{2} refers to its value in the right state.
Remark that the variable mac start is not required to be equal on both sides (i. e. , the variable can be private), the precondition simply requires that its distance to scan start is bounded (on both sides). The proof of this statement requires about 25 lines of EasyCrypt code. It requires basic results on non-linear arithmetic. We use a shift by 23 to match the OpenSSL implementation, but a shift by 8 would suffice. One other interesting remark is that the original code for the shift was of the form (md size >> 1) << 24. As md size is even, it is equivalent to md size << 23, but compilers can not infer it. The idea of writing it in this form by the OpenSSL developer was to prevent compiler from optimizing the code by removing introduced counter measure (i. e. , replace ((md size << 23) + rotate offset) % md size by (rotate offset) % md size, this replacement is functionally correct but does not preserve constant-time hyperproperty).
As the data will fit in two cache lines, there are two load operations within a loop. When trying to prove it against the CL 32 model, we realized that this is incorrect and it is dependent on secret rotate offset. We were able to easily create a counterexample using Jasmin evaluator. With a 64-byte aligned rotated mac buffer, rotate offset with value 31 touches the second cache line first and then the first. However, rotate offset with value 63 touches the first cache line first and then the second. Figure  12  shows the verified fix where we always access the first cache line and then the second cache line. Later, we select the correct value in constant-time. This incurs an overhead of 5. 9% at ssl3 cbc copy mac function granularity. However, the overhead is negligible in OpenSSL macro benchmarks.
, 2006)  methodological framework. Two main steps are at the core of this approach: Quantification of the masses of substances emitted into the environment through the Life Cycle Inventory (LCI). While it is possible to rely on databases that facilitate this inventory work for the background of the system under study, this task must nevertheless be carried out on a case-by-case basis to represent all the specificities of the foreground elements. This is the task of the LCA practitioner. Calculation of the impacts on ecosystems and human health of these emitted masses. Due to the complexity of environmental mechanisms, it is not possible to (re)model impact pathways on a case-bycase basis. Therefore, LCA uses characterization factors (CF) to assess the potential impacts of a compound. Concretely, if two compounds are emitted with the same mass, the one with the higher CFs will have the higher impact.
For example, in a recent study,  Aemig et al. (2021)  studied the potential impacts on Human health and aquatic environment of the release of 286 organic and inorganic micropollutants at the scale of France. One of their conclusion was that, due to a lack of characterization factors, these impacts could be assessed only for 1/3 of these molecules. That is why more computational models are needed to complement experimental approaches to decrease the experimental cost and determine the prioritization for those chemicals which may need further in vivo studies. Such models already exist, like QSAR models that are mostly linear models based on the chemical structure of compounds  (Danish QSAR database (DTU, 2015) , ECOSAR  (Mayo-Bean et al. , 2011) , VEGA  (Benfenati et al. , 2013) ) and are used to predict ecotoxicological data (LC50) needed for REACH for example.
TyPol database We recently developed TyPol (Typology of Pollutants), a classification method based on statistical analyses combining several environmental parameters (i. e. , sorption coefficient, degradation half-life, Henry constant) and an ecotoxicological parameter (bioconcentration factor BCF), and structural molecular descriptors (i. e. , number of atoms in the molecule, molecular surface, dipole moment, energy of orbitals). Molecular descriptors are calculated using an in silico approach (combining Austin Model1 and Dragon software). In the present paper, we only extracted and used the molecular descriptors from the TyPol database, as this information could be easily collected for any new compound. The 40 descriptors included in the TyPol database have been selected based on a literature review on QSAR equations used to predict the main environmental processes as degradation, sorption, volatilization.
SVM constructs a hyperplane or set of hyperplanes in a high-or infinite-dimensional space in which the problem is linearly separable. These choices allowed us to compare several ideas. The PLS is a simple linear method that will not exhibit good performances if the underlying relationship is not linear. The SVM and RF methods are wellknown non-linear machine learning algorithms that used to show good results in this kind of problem  (Hou et al. , 2020a). All the models were computed in the freeware R (R core team, 2019). The PLS has been computed using the package mixOmics  (Rohart et al. , 2017) , the random forests using the package randomForest  (Liaw et al. , 2002) , and the SVM using the package e1071  (Meyer et al. , 2019). These 3 modeling methods have some parameters that needed to be fixed: the number of latent components for the PLS (fixed using the tune.
, 2018). So we decided to use the clustering given by the TyPol application (more details in  Servien et al. , 2014). Note that the TyPol clustering has already been shown relevant on various occasion: in combination with mass spectrometry to categorize tebuconazole products in soil  (Storck et al. , 2016) , to explore the potential environmental behaviour of putative chlordecone transformation products  (Benoit et al. , 2017)  or to classify pesticides with similar environmental behaviors  (Traore et al. , 2018; Mamy et al. , 2021). So, the clustering procedure of TyPol was applied on the whole database of 526 compounds using the 40 molecular descriptors. This approach provided us a global clustering based on all the available information contained in the TyPol database. It is based on PLS, hierarchical clustering and an optimal choice of the number of clusters and is detailed in  Servien et al.
Comparison procedure of the models To assess the performances of the different models we used the following procedure: 1. Split each cluster (the whole dataset if the model is global, only the data lying in the dedicated cluster if that is a cluster-then-predict model) between a training set (85% of the dataset) and a test set (15%) (Pareto principle). The test set is not used for any step of the procedure (such as the imputation of the missing data, the calibration of the parameters …). 2. Imputation of the NA (Not Available, i. e. missing) values (less than 1%) in the descriptor matrix using the NIPALS algorithm  (Wold, 1985). 3. Tune the parameters and train the specific models by performing cross-validation on the training set. We have 3 global models to train (PLS, random forest, and SVM) and the clusterthen-test models (PLS, random forest and SVM for each cluster).
Predictions Then, the best model was calibrated and computed on the whole cluster. Finally, it was applied to the compounds, according to their clusters, with a CF ET (or a CF HT ) equals to NA to provide a prediction. For the compounds in cluster 5, this best model cannot be a cluster-then-predict one and, by consequence, is a global one. To assess the robustness of our prediction we derived a 95% prediction interval for each prediction. The type of model and its corresponding parameters were fixed during this process, according to the best model of the cluster. For example, if the best model of cluster 1 was the random forest approach, random forest models are used with the parameters optimized during the previous step. Then, we performed a leave-one-out bootstrap on the dataset that was used to compute the model (the whole dataset if the model is global, only the data lying in the dedicated cluster if that is a cluster-then-predict model) and a new model was computed on this leave-one-out sample.
Descriptive analysis of the intersection of the TyPol and the USEtox® databases As the objective of this proof-of-concept study was to predict USEtox® CF ET and CF HT using the molecular descriptors contained in TyPol, we could only use the compounds that are present in both databases. This resulted in 274 compounds that are detailed in Table  S1  in supplementary material and the range of their CF ET and CF HT values are summarized in the boxplots in Figures  2  and S3. Note that for the 274 common compounds there are 15 NA values for the CF ET and 102 for the CF HT. Figure  2  Boxplots of the CF HT for the USEtox® database and the common molecules between the USEtox® and the TyPol databases. This CF HT is equal to log 10 ((DALY+ε). kg -  ). The ε is needed as some values of the DALY are exactly equal to zero.
ε has been chosen equal to 1e-10 to be below the minimum of the USEtox® database (5e-9). We could see on these two figures that the common compounds present higher CF ET and CF HT values than the one of the complete USEtox® database: it focuses on the more dangerous compounds as their boxplots are above the USEtox® counterparts and they cover the whole order of magnitude of the CFs of the USEtox® database.
For the prediction of the CF ET The methodology described in the previous section was applied to our dataset and gave the results gathered in Figure  4  for each cluster. The performances were not similar from one cluster to another. For example, performances of all methods for cluster 1 were very poor (median absolute error above 1) whereas performances for cluster 4 seemed good despite its smallest size (median absolute error around 0. 6). Therefore, a future prediction of an unknown compound which lies in cluster 1 will be less reliable than in other clusters. Note that we could not test this in the next section as no NA value is present in this cluster 1. The cluster-then-predict methods seemed more appropriate in each cluster. The cluster-then-RF approach had the best performances (with a global median absolute error equals to 0.