dataesr/scientific-paragraphs-categorization huggingface dataset:
1) DIGIT CLASSIFICATION: MNIST-USPS-SVHN This is the most common benchmark for domain adaptation tasks and UDA specifically. Hence we follow the same TABLE 7. Digit Recognition Benchmark on the MNSIT-USPS-SVHN dataset for Unsupervised Domain Adaptation. Each experiment name follows source_domain -target_domain naming convention. ARL-sep is used to reference to our method + the seperability loss and ARL is used to reference our model without it. The ''-'' notation is used for experiments where the results have not been reported in previous works. experimental setup as  [10] ,  [24]. We compare our results against first the two baselines (Upper Bound UB, and Lower Bound LB) which represent the accuracy of training and testing on the target dataset, and the accuracy of training on the source dataset only without access to the target dataset (not even unlabeled images), respectively.
D. OVER-FITTING Over-fitting problems have been explored ever since the start of neural networks. Given the strong ability of neural nets to remember and memorize data samples. To evaluate the efficiency of our method on this problem we make the following setting, Train a model longer than it needs to force it to over fit, and then see if adding our loss can help bring it back from the over-fitting scenario, we refer to this model as (O-ARL). We compare our method against several baselines: (i) Over-fitted model (OF), (ii) Well trained model (WT), (iii) model trained with ARL only from the start (T-ARL), and (iv) model fine-tuned with ARL-sep (F-ARL-sep). We perform this experiment on several benchmarks for digit classification which are: MNIST, SVHN, USPS, MNISTR-0,. , MNISTR75. For each one of these experiments we used a different set of Hyper-parameters which are all mentioned in Table  10.
81% in accuracy on average. We also see that the T-ARL model which is trained from the beginning on the ARL loss was as rigid as O-ARL and even better than WT model in most of the cases. We also confirm our findings through Figure  7  where we show the behaviour of our different losses and how they influence the testing accuracy of the model on the MNIST dataset. We can easily notice that the over-fitted models always go up and then quickly decreases in performance as shown with OF chart, which is continued using the O-ARL chart which drops the performance in the first few epochs but then quickly starts giving positive outcome on the model's performance approaching results provided by the WT models. We can also notice that the WT models achieve better than our models in the first few epochs where as our models (F-ARL-Sep and T-ARL) improve slower but with enough epochs they exceed the WT performances.
The first math trails followed primarily the goal to popularize mathematics  [1]. The math trail approach is also transferred to the educational context and used in school contexts. One project that focuses on math trails in the school context is the MathCityMap project. It has been developed at Goethe University Frankfurt in cooperation with international partners since 2012. The context of this paper is the Erasmus+ project MaSCE 3 that aims at the further development, didactical enrichment, and dissemination of MathCityMap. MathCityMap is a two-component system including a web portal and a smartphone app (see Figure  2 ). The web portal is an international database of tasks and math trails. In September 2021, while this paper was being written, there were more than 27,000 tasks by more than 8800 authors in 12 different languages, available in more than 40 countries.
The former were used to allow participants to interact with each other in asynchronous mode, and the latter to allow instructors to monitor the learning progress of participants. Structure and Contents: The MOOC was organized in modules, with each module's respective data collection instruments defined as shown in Figure  5. Besides general information about the MOOC, its homepage allowed the participants to access the module's resources when they were made available. On the Introductive module, participants found the presentation and introduction resources; namely, a presentation video made by Prof. Matthias Ludwig (the MaSCE 3 project coordinator and one the co-author of this paper), another video introducing the instructors, a brief overview of the MOOC setup, and videos concerning outdoor learning and math trails.
This was the module wherein a pre-course questionnaire was administered. author of this paper), another video introducing the instructors, a brief overview of the MOOC setup, and videos concerning outdoor learning and math trails. This was the module wherein a pre-course questionnaire was administered. In module 1 "Outdoor Learning and Task Design", the following topics were covered: an overview of the MCM web portal; an explanation about math tasks' creation using different answering formats; the criteria for meaningful math tasks; and, finally, the math task publication process. In these two weeks, different data collecting instruments were used; namely, two discussion forums and the Excel review file used by the instructors to register the reviewing process for the two submitted math tasks, particularly multiplechoice and exact value.
In module 2 "Task Wizard and Subtasks", to explore the content task wizard (a template used to implement generic tasks), four resources were used, namely, three videos and a static tutorial: a video that explains the task wizard concept with an example; the tutorial made using Sway (https://sway. office. com/ (accessed on 1 October 2021)) to explore other task wizard topics; a video exploring the creation of one of the GPS tasks; and a video with other examples of GPS tasks. Moreover, to explore the subtasks content, a webinar was used to present this new feature of MCM. The instruments used to collect the data were the task wizard forum, the subtasks forum, and the Excel review file of the four requested math tasks. Module 3 "Interval" began with the exploration of interval tasks. The two resources available (a video and a pdf) teach strategies to set the bounds of an interval.
In these two weeks, different data collecting instruments were used; namely, two discussion forums and the Excel review file used by the instructors to register the reviewing process for the two submitted math tasks, particularly multiple-choice and exact value. In module 2 "Task Wizard and Subtasks", to explore the content task wizard (a template used to implement generic tasks), four resources were used, namely, three videos and a static tutorial: a video that explains the task wizard concept with an example; the tutorial made using Sway (https://sway. office. com/ (accessed on 1 October 2021)) to explore other task wizard topics; a video exploring the creation of one of the GPS tasks; and a video with other examples of GPS tasks. Moreover, to explore the subtasks content, a webinar was used to present this new feature of MCM.
The instruments used to collect the data were the task wizard forum, the subtasks forum, and the Excel review file of the four requested math tasks. Module 3 "Interval" began with the exploration of interval tasks. The two resources available (a video and a pdf) teach strategies to set the bounds of an interval. As in the previous modules, a forum and review file in Excel were available for the two tasks. Module 4 "Trail & App" included the following topics through nine resources (videos): How to create a math trail? How to modify it (if necessary)? How to request its publication? As well as an overview of the MCM app; the feature narrative that places math trails in some fiction context (e. g. , a pirates' adventure), so that the students can feel along the trail as characters of the story; and the digital classroom, a tool to monitor, interact, and follow the math trail with the students.
Acknowledgments: The  MaSCE 3 MOOC  was made possible with the support of the  Erasmus+ Programme of the European Union  ( 2019-1-DE03-KA201-060118 ). A special thanks go to all the MOOC instructors who contributed to the preparation of the resources in the various modules and who collaborated, together with the technicians of the  University of Catania , in monitoring the delivery of the MOOC. Therefore, in addition to the authors of this contribution, we would like to thank  Virginia Alberti ,  Roberta Ferro ,  Sara Labasin ,  Gerardo Maiorano ,  Gregor Milicic ,  Andrus Rinde , and  Angelo Sarra Fiore. E. T. and M. F. M were supported by the  research project "Programma Ricerca di Ateneo UNICT 2020-22 linea 2, EEEP&DLaD ". T. R. was partially supported by the grant  PID2020-113192GB-I00  from the  Spanish MICINN.
This explains that the errors for these coefficients are set to zero. To compute the numbers in the second table we built a Fortran program to perform the numerical evaluation of the different formulae presented in this article. The results of this program are cross checked by another program built in Mathematica  [31]. In addition, for each phase space point, the coefficients a -2 , a -1 and a 0 are also compared with the results of the SecDec program  [11]. Our results are in perfect agreement with those of SecDec within the error bars quoted by the three programs. Note that we differ by a global -1 factor with SecDec because of the use of a different measure for the loop integration. As it is, our Fortran program is, in average, not faster than SecDec for the same accuracy: there are less integrations and the number of dimensions per integration is also lower but the integrand is more complicated.
It is difficult to give quantitative results because there are lot of options in SecDec and we did not play with them and also our computations have been done without any optimisation and there is room for that. Indeed, our program is very fast for Euclidian phase space points and slower for phase points in the Minkowski region. In these last regions, the numerical integration is performed by brute force over the square, it could be improved by contour deformations or using Richardson extrapolation. We can also test with other integration packages. All these problems as well as quantitative comparisons with other programs will be postponed to a dedicated article when our program will become public. Topology T 22 0 2 1 (e111|e|) Kinem p 2 m 2 1 m 2 2 m 2 3 3. 3. 5 1. 6. 13. 0. 5 3. 6. 43. 0.
Both master topologies correspond to two-loop functions with five internal lines and two external legs. One can generate from the divergent topology nine other topologies, and from the finite one eight other topologies belonging to the same class. The topologies generated by the same master topology have the same formula with different coefficients for the polynomial F. We implemented the two-dimensional integral representation of nine of these topologies belonging to the four classes in two different codes, one is written in Fortran and the other one is written in Mathematica, to perform the numerical integration over the two remaining variables (ρ and ξ). We showed that our results are in good agreement with the public code SecDec, which is based on sector decomposition method. This paper is accompanied with many appendices which are dedicated to the fully analytical calculation of the divergent parts and some related finite contributions of these topologies.
However, the Boolean approach is purely qualitative and does not consider real time of cellular events (half time of proteins, triggering of apoptosis, etc. ). To cope with this issue, we developed the MaBoSS software to compute continuous Markov Chain simulations on the model state transition graph  (STG) , in which a model state is defined as a vector of nodes that are either active or inactive. In practice, MaBoSS associates transition rates for activation and inhibition of each node of the network, enabling it to account for different time scales of the processes described by the model. Given some initial conditions, MaBoSS applies a Monte-Carlo kinetic algorithm (or Gillespie algorithm) to the STG to produce time trajectories  (Stoll et al, 2012 (Stoll et al, , 2017) )  such that time evolution of the model state probabilities can be estimated.
Lastly, we developed a new methodology to simulate drug effects on these data-tailored Boolean models and present a list of viable drugs and regimes that could be used on these patient-and cell-line-specific models for optimal results. Experimental validations were performed on the LNCaP prostate cell line with two predicted targets, confirming the predictions of the model. Omnipath  (Türei et al, 2021)  and pathways identified with ROMA  (Martignetti et al, 2016)  on the TCGA data to build a prostate-specific prior knowledge network. This network was manually converted into a prostate Boolean model that could be stochastically simulated using MaBoSS  (Stoll et al, 2017)  and tailored to different TCGA and GDSC datasets using our PROFILE tool to have personalized Boolean models. Then, we studied all the possible single and double mutants on these tailored models using our logical pipeline of tools  (Montagud et al, 2017).
ROMA is applied on omics data, either transcriptomics or proteomics. In each pathway, the genes that contribute the most to the overdispersion are selected. ROMA was applied to the TCGA transcriptomics data using gene sets from cancer pathway databases (Appendix File, Figure  S1 ). These results were used as guidelines to extend the network to fully cover the alterations found in prostate cancer patients. OmniPath was used to complete our network finding connections between the proteins of interest known to play a role in prostate and the ones identified with ROMA, and the list of genes already present in the model. The final network includes pathways such as androgen receptor, MAPK, Wnt, NFkB, PI3K/AKT, MAPK, mTOR, SHH, the cell cycle, the epithelial-mesenchymal transition (EMT), apoptosis and DNA damage pathways.
First, a prostate cancer Boolean model that encompasses relevant signalling pathways in cancer was constructed based on already published models, experimental data analyses and pathway databases (Figure  2 ). The influence network and the assignment of logical rules for each node of this network were obtained from known interactions described in the literature (Figure  3 ). This model describes the regulation of invasion, migration, cell cycle, apoptosis, androgen and growth factors signalling in prostate cancer (Appendix file). Second, from this generic Boolean model, we constructed personalised models using the different datasets, i. e. 488 patients from TCGA and eight cell lines from GDSC. We obtained Gleason-score-specific behaviours for TCGA's patients when studying their Proliferation and Apoptosis scores, observing that high Proliferation scores are higher in high Gleason groups (Figure  4 ).
We therefore based our analysis on clinical grouping defined by the Gleason grades, but some works have emphasized the difficulty to properly assess them  (Chen & Zhou, 2016)  and as a result may not be the perfect predictor for the patient subgrouping in this analysis, even though it is the only available one for these datasets. The lack of subgrouping that stratifies patients adequately may undermine the analysis of our results and could explain the Proliferation and Apoptosis scores of high-grade and low-grade Gleason patients. Moreover, the behaviours observed in the simulations of the cell-lines-specific models do not always correspond to what is reported in the literature. The differences between simulation results and biological characteristics could be addressed in further studies by including other pathways, for example better describing the DNA repair mechanisms, or by tailoring the model with different sets of data, as the data used to personalise these models do not allow to cluster these cell lines according to their different characteristics (Appendix File, Figure  S21  and S22 ).
Data acquisition Publicly available data of 489 human prostate cancer patients from TCGA described in  (Hoadley et al, 2018)  were used in the present work. We gathered mutations, CNA, RNA and clinical data from cBioPortal (https://www. cbioportal. org/study/summary?id=prad_tcga_pan_can_atlas_2018) for all of these samples resulting in 488 with complete omics datasets. Publicly available data of cell lines used in the present work were obtained from Genomics of Drug Sensitivity in Cancer database (GDSC)  (Iorio et al, 2016). Mutations, CNA and RNA data, as well as cell lines descriptors, were downloaded from (https://www. cancerrxgene. org/downloads). All these data were used to personalise Boolean models using our PROFILE method  (Béal et al, 2019).
Prior knowledge network construction Several sources were used in building this prostate Boolean model and in particular the model published by  Fumiã and Martins (2013). This model includes several signalling pathways such as the ones involving receptor tyrosine kinase (RTKs), phosphatidylinositol 3-kinase (PI3K)/AKT, WNT/b-Catenin, transforming growth factor-b (TGF-b)/Smads, cyclins, retinoblastoma protein (Rb), hypoxia-inducible transcription factor (HIF-1), p53 and ataxiatelangiectasia mutated (ATM)/ataxia-telangiectasia and Rad3-related (ATR) protein kinases. The model includes these pathways as well as the substantial cross-talks among them. For a complete description of the process of construction, see Appendix File. The model also includes several pathways that have a relevant role in our datasets identified by ROMA  (Martignetti et al, 2016) , a software that uses the first principal component of a PCA analysis to summarise the coexpression of a group of genes in the gene set, identifying significantly overdispersed pathways with a relevant role in a given set of samples.
This software was applied on the TCGA transcriptomics data using the gene sets described in the Atlas of Cancer Signaling Networks, ACSN  (Kuperstein et al, 2015)  (www. acsn. curie. fr) and in Hallmarks  (Liberzon et al, 2015)  (Appendix File, Figure  S1 ) and highlighted the signalling pathways that show high variance across all samples, suggesting candidate pathways and genes. Additionally, OmniPath  (Türei et al, 2021)  was used to extend the model and complete it connecting the nodes from Fumiã and Martins and the ones from ROMA analysis. OmniPath is a comprehensive collection of literature-curated human signalling pathways, which includes several databases such as Signor  (Perfetto et al, 2016)  or Reactome  (Fabregat et al, 2018)  and that can be queried using pypath, a Python module for molecular networks and pathways analyses.
Boolean model construction We converted the network to a Boolean model by defining a regulatory graph, where each node is associated with discrete levels of activity (0 or 1). Each edge represents a regulatory interaction between the source and target nodes and is labelled with a threshold and a sign (positive or negative). The model is completed by logical rules (or functions), which assign a target value to each node for each regulator level combination  (Chaouiya et al, 2012; Abou-Jaoudé et al, 2016). The regulatory graph was constructed using GINsim software  (Chaouiya et al, 2012)  and then exported in a format readable by MaBoSS software (see below) in order to perform stochastic simulations on the Boolean model. The final model has a total of 133 nodes and 449 edges (SuppFile 1) and includes pathways such as androgen receptor and growth factor signalling, several signalling pathways (Wnt, NFkB, PI3K/AKT, MAPK, mTOR, SHH), cell cycle, epithelial-mesenchymal transition (EMT), Apoptosis, DNA damage, etc.
This model has 9 inputs (EGF, FGF, TGF beta, Nutrients, Hypoxia, Acidosis, Androgen, TNF alpha and Carcinogen presence) and 6 outputs (Proliferation, Apoptosis, Invasion, Migration, (bone) Metastasis and DNA repair). This model was deposited in the GINsim Database with identifier 252 (http://ginsim. org/model/signallingprostate-cancer) and in BioModels (Malik-Sheriff et al, 2019) with identifier MODEL2106070001 (https://www. ebi. ac. uk/biomodels/MODEL2106070001). SuppFile 1 is provided as a zipped folder with the model in several formats: MaBoSS, GINsim, SBML as well as images of the networks and its annotations.
Stochastic Boolean model simulation MaBoSS  (Stoll et al, 2012 (Stoll et al, , 2017 ) is a C++ software for stochastically simulating continuous/discrete-time Markov processes defined on the state transition graph (STG) describing the dynamics of a Boolean model (for more details, see  (Chaouiya et al, 2012; Abou-Jaoudé et al, 2016) ). MaBoSS associates transition rates to each node's activation and inhibition, enabling it to account for different time scales of the processes described by the model. Probabilities to reach a phenotype (to have value ON) are thus computed by simulating random walks on the probabilistic STG. Since a state in the STG can combine the activation of several phenotypic variables, not all phenotype probabilities are mutually exclusive (like the ones in Appendix File, Figure  S25 ).
Data tailoring the Boolean model Logical models were tailored to a dataset using PROFILE to obtain personalised models that capture the particularities of a set of patients  (Béal et al, 2019)  and cell lines  (Béal et al, 2021). Proteomics, transcriptomics, mutations and CNA data can be used to modify different variables of the MaBoSS framework such as node activity status, transition rates and initial conditions. The resulting ensemble of models is a set of personalised variants of the original model that can show great phenotypic differences. Different recipes (use of a given data type to modify a given MaBoSS variable) can be tested to find the combination that better correlates to a given clinical or otherwise descriptive data. In the present case, TCGA-patient-specific models were built using mutations, CNA and/or RNA expression data.
High-throughput mutant analysis of Boolean models MaBoSS allows the study of knock-out or loss-of-function (node forced to 0) and gain-offunction (node forced to 1) mutants as genetic perturbations and of initial conditions as environmental perturbations. Phenotypes' stabilities against perturbations can be studied and allow to determine driver mutations that promote phenotypic transitions  (Montagud et al, 2017). Genetic interactions were thoroughly studied using our pipeline of computational methods for Boolean modelling of biological networks (available at https://github. com/sysbiocurie/Logical_modelling_pipeline). LNCaP-specific Boolean model was used to perform single and double knock-out (node forced to 0) and gain-of-function (node forced to 1) mutants for each one of the 133 nodes, resulting in a total of 32258 models.
Identification of drugs associated with proposed targets To identify drugs that could act as potential inhibitors of the genes identified with our models (Table  1 ), we explored the drug-target associations in DrugBank  (Wishart et al, 2018). For those genes with multiple drug-target links, only those drugs that are selective and known to have relevance in various forms of cancer are considered here. In addition to DrugBank searches, we also conducted exhaustive searches in ChEMBL  (Gaulton et al, 2017)  (http://doi. org/10. 6019/CHEMBL. database. 23) to suggest potential candidates for genes, whose information is not well documented in Drug Bank. From the large number of bioactivities extracted from ChEMBL, we filtered human data and considered only those compounds, whose bioactivities fall within a specific threshold (IC50/Kd/ Ki<100 nM).