dataesr/scientific-paragraphs-categorization huggingface dataset:
Colorectal and Cirrhosis datasets are benchmarked with all previous methods whereas the Simulated dataset is used to evaluate M2V-MIL efficiency and interpretability. Models used for BoK and M2V-VR methods are tuned with random search using 100 different sets of parameters. M2V-MIL is tuned with approaches more adapted for model composed by a lot of hyper parameters like neural network. Thus, Bayesian optimization is applied for continuous parameters and bandit optimization for discrete parameters. The table 2 summarize our results.
65 (0. 071) 0. 67 (0. 077) / 0. 73 (0. 062) / 0. 75 (0. 043) 0. 61 (0. 032) / 0. 66 (0. 040) / 0. 63 (0. 057) BoK precision 0. 81 / 0. 88 / 0. 84 0. 66 / 0. 76 / 0. 77 0. 61 / 0. 66 / 0. 71 k=3 / k=6 / k=9 recall 0. 57 / 0. 61 / 0. 60 0. 77 / 0. 69 / 0. 74 0. 56 / 0. 62 / 0. 55 F1-score 0. 55 / 0. 59 / 0. 56 0. 70 / 0. 72 / 0. 74 0. 59 / 0. 64 / 0. 65 AUC 0. 66 / 0. 73 / 0. 67 0. 71 / 0. 77 / 0. 77 0. 65 / 0. 69 / 0. 67 Accuracy 0. 72 (0. 058) 0. 79 (0. 044) 0. 85 (0. 047) M2V-VR precision recall 0. 77 0. 64 0. 78 0. 67 0. 87 0. 81 F1-score 0. 69 0. 70 0. 84 AUC 0. 794 0. 81 0. 85 Accuracy 0. 81 (0. 065) 0. 82 (0. 056) 0. 92 (0. 070) M2V-MIL precision recall 0. 78 0. 76 0. 83 0. 84 0. 94 0. 91 F1-score 0. 76 0. 83 0. 92 AUC 0. 81 0. 83 0. 92 Classification metrics for three databases Colorectal, Cirrhosis and Simulated.
Data collection The monitoring campaign was conducted during the main upstream migration months from April to August in 2017. Fish migration records were collected from the left riverbank in the monitoring site, 5 km downstream from the dam (Fig. 1 , right). A DIDSON was installed on an iron pole, and the pole was fixed at the bank to prevent the echo image from shaking due to the movement of the acoustic camera caused by waves or swells. Because Schizothorax are midwater and demersal fishes, the DIDSON was always deployed 0. 5 m below the surface of the water and the water depth changed approximately from 2. 5e3. 5 m during the monitoring period. The sensor of the DIDSON pointed perpendicularly to face the opposite bank of the river to monitor the fish passing through the cross section of the river.
The DIDSON was operated in low-frequency (1. 8 MHz) mode with a window length of 40 m. The river width at the measurement site was approximately 80 m, and the observation range covered half of the cross section. Fish migration records were collected on three to five days in each monitoring month. The monitoring dates are displayed in Table  1. Due to the harsh natural environment and limited power supply, the monitoring campaigns were mainly during the daytime, and full 24-h measurements were not conducted. Data were collected continuously only from 9:00 to 18:00 on each collection day. An additional nighttime measurement was conducted from 18:00 to 21:00 in May, July, and August. The sonar was set up to record ten acoustic images every second, and the data were saved hourly in a single file to facilitate data processing and statistics.
During the monitoring periods, the water temperature was also recorded by DIDSON and river discharge was collected from the hydrological station downstream of the dam (Fig. 1 , right). These data were used to compare with the measured number and length of the upstream migrated fishes. For comparison with the DIDSON results, fishing was synchronously conducted in a 20-km reach that stretched from the dam to Yajiang County (where the hydrological station is located in Fig. 1 ) using three layers of drift gill nets (120 m long, 1. 5 m high, 5. 0 cm mesh size). The fishing time covered two periods: the first period was from April 30 to May 5, which was close to the time when the DIDSON records were collected in May, and the second period was from August 16 and August 23, which overlapped with the DIDSON monitoring period in August.
Data analysis The DIDSON data were processed using Echoview software (version 6. 1, SonarData Pty. , Ltd. , Australia). The acoustic image data (Fig. 2a ) were imported into the software, and two variables were created from each acoustic file. The first variable was proportional to the received power (in dB), and the second variable was the mean volume of backscatter (in dB). The fish size, abundance, and behavior were estimated using a semiautomated approach proposed by  Boswell et al. (2008). The approach was based on the mean volume of backscatter and was conducted in Echoview by linking the defined variables and operations as an analysis pathway. First, a sample statistic subtract operator was implemented to generate a synthetic ping that represents the background signal, and the synthetic ping was subsequently subtracted from each real ping.
After background subtraction (Fig. 2b ), a 3Â3 median filter method was applied to enhance each image without significantly affecting the target shape. Then, a multibeam target detection operator was employed to generate multibeam targets from groups of adjoining data points. Finally, the multiple targets were converted to a single target track, and several target-related properties (e. g. , fish length, speed, orientation, and surface area) were obtained and exported from Echoview. With multibeam data, the fish length was estimated using the maximum distance between any two above-threshold samples from the target, which were measured based on the angle and range from the transducer. The fish swimming orientation is the angle between the line joining the two most distant samples for the target and the line perpendicular to the transducer axis.
The DIDSON images were played back at 10 frames/s as videos in the DIDSON software. As the object in the water identified by the sonar would have a high brightness in the image, any object that had an obvious swimming stance and a fish shape was recognized as a fish species. The swimming direction of the fish were carefully checked, and the NFU within a time period was counted. However, this method can be extremely time consuming as it involves large amounts of measured data. Manual counting was only conducted for the sampling periods of 10:00e11:00, a The NFU percentage was calculated by dividing the total number of counted fish that migrate upstream and downstream by the NFU. 13:00e14:00, and 19:00e20:00 on each measurement day. The manually counted NFUs were used to verify the data processed by Echoview software.
Validation results Fig. 3  displays the comparison of the NFU identified from software processing and from manual counting. The results from the two methods generally agreed with each other, and the correlation coefficient (R 2 ) was 0. 95 (n ¼ 68, p < 0. 05). The mean absolute error and root mean square error were 5. 4 and 6. 4, respectively. This finding reveals that the NFU from software processing tended to be lower than that from manual counting, which could be due to the error from the fish identification algorithms used in the Echoview system. The average TL and SL of the fish collected from field fishing were 22. 8 ± 11. 5 cm and18. 3 ± 9. 1 cm in May, and 25. 2 ± 7. 7 cm and 20. 4 ± 6. 1 cm in August, respectively. The measured average fish lengths from the DIDSON in May and August are shown in Table  1 , and they were compared with the TL and SL measured form field fishing in Fig.
Previous studies have shown acoustic cameras are currently mainly used to monitoring fish behavior at the entrance or passage abilities in related application and research of fish passage facilities  (Johnson et al. , 2012; Grote et al. , 2014; Nyqvist et al. , 2017). The monitoring results from these tools also have great potential in aiding design and operation of fish passage facilities. This study employed a fixed-point observation method to continuously monitoring the temporal changes in the upstream migration of fish species downstream from the under-construction LHK dam. The DIDSON-measured fish number and length showed reasonable agreement with the data from visual counting and fishing with nets before analyzing the response of fish migration to environmental factors. Although our study did not achieve a long term monitoring of fish behavior downstream of the underconstruction dam, the results were still comparable and meaningful in terms of understanding the upstream migration behavior of fishes in the studied river reaches, providing useful data to help for the operation of future fish passage facilities.
Acknowledgments This study was supported financially by the  National Key Research and Development Program of China  (No. 2019YFC0408901 ), the  National Natural Science Foundation of China  (No. 51709187  and No. 51609155 ), the  Key Projects in Technology Innovation of Hubei Province, China  (No. 2019ACA154 ) and the  China Scholarship Council (CSC). We thank three anonymous reviewers for their constructive suggestions that greatly improved our work.
during its interaction with the surrounding world. Because of that, an important part of A. S. representation is its behaviour. Through ROS, the system accepts as an input a program that has the role to move the A. S. during the simulation. In the context of rover ArcTurius, the simplest version of such program moves the virtual rover within a virtual environment applying a force to the junctions that connect the wheels with the chassis of the rover. The movement decisions can be based on the data acquired during the simulation. Even though sensors are logically part of an A. S. , they have a different nature. For this reasons, we treat them in section 2. 2.
S. behaviour can take decisions starting from data produced by sensors. As already mentioned, ROS handles this data exchange. However, the deepening of all the algorithms that rely on sensor data is not part of this paper. Regardless of the usage that A. S. modules make of data produced by virtual sensors, engineers in charge to find the best configuration of an A. S. upon a disaster can rely to a 3D sensor date representation that enhance the comprehension of the simulation. Thus, the second stated goal is facilitating the understanding of the telemetry results checking how the A. S. is seeing at each simulation step. This can lead to a faster decision making, reducing the effort in the interpretation of telemetry results whenever possible. In this respect, Figures  3  and 4  shows the capturing of laser data through RViz  [13] , a 3D visualization tool for ROS.
SRTM30+ provides coarse data (900 meters of resolution) and they are used to the general contours of the land. USGS NED provides precise elevation data (up to 10 meters of resolution) for zones close to United States geography. Terrains generated from DEMs files do not usually contain external components that can be useful for simulation purposes, such as rocks, rubble, fails, etc. However, they can be integrated in the ground, according to the extended process present in Figure  7. A possibility is to use a Drone that runs a low altitude to map them in a precise way. However, they can be generated and integrated in a realistic way. We can use mathematical functions, common in remote sensing applications, in order to estimate the roughness of the terrain. After, the physical properties of obstacles (such as density, bounciness, etc.
g. , sensors nodes that send their value to a 3D-visualizer node) or they can occur one-off (e. g. , a master node that express a request and wait for a reply to a slave node). In the first case, the nodes exchange data through ROS messages sent over ROS topics, with a reference paradigm similar to the Publisher/Subscriber. In the second case ROS messages are sent over artefacts named ROS services, with a reference model Client/Server. In our respect, ROS is used to connect all the elements intrinsic to the simulation (such as rover, environmental actors, sensors, etc. ) and the data analysis tools that we use for the evaluation of the measurements. However, ROS is more than a meta operating system that provides a standard connection way. As already mentioned, it includes many other modules that can be useful to robot simulation.
sent through ROS topics 2. It is able to receive customized visual markers obtained by sending primitive shapes such as coloured cubes, arrows, etc. A combination of two methods is accepted too. In fact, it is the most common method among algorithms in ROS navigation package  [19]. Gazebo-3D: Many graphical engines are available in the domain. For several reasons, only hinted in this paper, we chose Gazebo-3D  [8]. Gazebo-3D is a graphic engine that includes a physical engine such as ODE  [20]. It is able to perform 3D physical simulations while displacing a rover within a surrounding virtual world. Moreover, it can be used to rapidly evaluate different kinds of algorithm, to design robots and to simulate their behaviour in realistic scenarios. For that, it is necessary to realise realistic models of A.
Example 28 (Forking strategies). In Unix-like operating systems there are standard mechanisms to create a new child process from its parent process. This is done through specific system calls typically named fork() or clone(). Both the parent and child processes run exactly the same code. They differ by the return code of the system call. Actually there are many variants of such system calls (see  [33]  for concrete examples). The variants differ in the way the address space of parent and child processes are separated or shared. For instance it is possible to create a child process with an address space completely different from the one of its parent process. It is also possible that both the parent and the child processes share parts of their execution context. P E s o o t G G AS A in o o L tl f f sl y y (18) In this example we show how such subtleties can be modeled with AGREE rules.
This paper considers the problems of modeling and predicting a long-term and "blurry" relapse that occurs after a medical act, such as a surgery. We do not consider a short-term complication related to the act itself, but a long-term relapse that clinicians cannot explain easily, since it depends on unknown sets or sequences of past events that occurred before the act. The relapse is observed only indirectly, in a "blurry" fashion, through longitudinal prescriptions of drugs over a long period of time after the medical act. We introduce a new model, called ZiMM (Zero-inflated Mixture of Multinomial distributions) in order to capture long-term and blurry relapses. On top of it, we build an end-to-end deep-learning architecture called ZiMM Encoder-Decoder (ZiMM ED) that can learn from the complex, irregular, highly heterogeneous and sparse patterns of health events that are observed through a claims-only database.
ZiMM ED is applied on a "non-clinical" claims database, that contains only timestamped reimbursement codes for drug purchases, medical procedures and hospital diagnoses, the only available clinical feature being the age of the patient. This setting is more challenging than a setting where bedside clinical signals are available. Our motivation for using such a non-clinical claims database is its exhaustivity population-wise, compared to clinical electronic health records coming from a single or a small set of hospitals. Indeed, we consider a dataset containing the claims of almost all French citizens who had surgery for prostatic problems, with a history between 1. 5 and 5 years. We consider a long-term (18 months) relapse (urination problems still occur despite surgery), which is blurry since it is observed only through the reimbursement of a specific set of drugs for urination problems.
Aim of this paper and contributions This paper aims at the construction of a predictive model for long-term and "blurry" relapses that occur after a medical act. We do not consider a short-term complication related to the act itself, but a long-term relapse which is observed only indirectly. In the example considered here, the medical act is a precise surgery, and the relapse is observed through longitudinal prescriptions of a specific set of drugs for urinary problems, over a period of 18 months after surgery. We use all the data available in SNIIRAM (French national health insurance information system, a huge database containing health reimbursements claims of almost all French citizens since 2015, see Section 3. 1 for details) for patients on which such a surgery has been performed (TURP surgery, see Section 3.
Our end-to-end architecture addresses several challenges, such as variable-size inputs, confounding interactions between medical codes, long-term predictions. Finally, we do not inject any prior knowledge: the embeddings of patient pathways are fully trained in an end-to-end fashion, we do not use any strong preprocessing or simplifying aggregations, the data is used almost in its raw form. We keep the smallest granularity possible both on the codes (we use the actual medical codes instead of encompassing categories) and on time, namely we work on the original 1-day time scale. Organization of the paper. The ZiMM model is described in Section 2. 1 while the ZiMM ED architecture is described in Sections 2. 3 to 2. 5. Some details concerning data preprocessing are provided in Section 2. 2. Section 3 provides results and a comparison with strong baselines, together with some variations of the architecture to fine-tune the final performance and we conclude in Section 4.
This corresponds to Steps 1,2 and 3 in Figure  1. This leads to an embedding vector x i ∈ R d of the full pathway of patient i prior to T i. • The vector x i ∈ R d is concatenated with other static features of the patient, and used as the input of the Decoder, which is based on several recurrent layers, in order to produce the parameters of the ZiMM model. This corresponds to Step 4 in Figure  1. Precise descriptions of all steps are provided below, starting with the ZiMM model, followed by some details concerning data preprocessing, and the Encoder-Decoder architecture.
The ZiMM Encoder The ZiMM Encoder corresponds to Steps (2) to (4) in Figure  1 , a more detailed illustration is shown in Figure  3. Let us provide now details about each step, following the flow of the data. Medical codes and timestamps embeddings. The following is applied separately for each type of event code (drugs, medical procedures and diagnoses). Codes are first tokenized, and each unique token is individually mapped to an embedding vector in R d E which is learned during training, where d E is a hyper-parameter to be tuned (see Section 3. 7). We consider only tokens that occur at least 50 times in the training dataset. To address time irregularity between two asynchronously timestamped events, we do the following: for any event occurring at time t ≤ T i in the observation period of patient i, we compute the "time horizon" as T i -t, namely the distance (in days) between the event and medical act.
Training The full ZiMM Encoder-Decoder architecture is trained in an end-to-end fashion by minimizing the average negative-log likelihood over all patients i = 1,. , n: i (Θ) = log(π 0 (x i ))1 n i =0 + B b=1 log(π b (x i ))+log b! Π B b =1 y i,b ! + B b =1 y i,b log(p b,b (x i )) 1 n i =b , (9) where Θ stands for the concatenation of all the trainable parameters involved in the layers described in Sections 2. 3 and 2. 4. The choices of optimizer, learning rate schedule and other hyperparameters are described in Section 3. 3 Application: prediction of post-surgical relapse of urinary problems In this section, we apply the ZiMM ED architecture to predict the blurry relapse of urinary problems after a TURP surgery for patients with benign prostatic hyperplasia (see Section 3. 2 below) using a cohort constructed from the French SNIIRAM database (see Section 3.
SNIIRAM: A non-clinical claims dataset SNIIRAM (French national health insurance information system) contains health reimbursements claims of almost all French citizens since 2015 (more than 65 million) and has been used for health research on many topics, to cite but a few  [61, 2, 31]. Since, in France, most health-cares are at least partially reimbursed by the administration, this database contains records corresponding to very various information going from hospital stays to drug purchases in city pharmacies, all coded with different systems  [67]. Absence of clinical information and "forced" normalization makes this data highly complex and heterogeneous. A consequence is that it requires a lot of domain expertise about the way healthcare is reimbursed in France in order to prepare it as training data for machine learning algorithms.
In this paper, we exploit medication, procedure and diagnosis codes only. Diagnoses are primarily coded with ICD-10, while procedures are coded with CCAM, a French medical classification of clinical procedures. Medications codes use the French pharmaceutical categories CIP13  [11]  that we map to the international ATC (Anatomical Therapeutic Chemical Classification) classification system. Let us stress that while this data is extremely rich and almost population-wide over France, it is not clinical data, but only claims data, clinical information being only latent in the codes appearing in reimbursement information. No vital bedside information, lab tests results or natural language notes from clinicians is available. An important amount of data preprocessing and domain expertise is required for data extraction, in particular in order to clean existing inaccuracies in codes and timestamps.
This is achieved through our SCALPEL3 library  [4] , which is a separate topic of research that this paper builds upon. Furthermore, the identification of a disease state (indication, concomitant disease or outcome) does not rely only on a single source of information but on the convergence of information from different sources. This might include, for instance, the presence of a chronic disease registration, hospital diagnoses, tracer drugs, and procedures or lab tests  [11] , that are all observed in the data through claims. We believe that these downsides are counter-balanced by the exhaustive nature of this data. It is exhaustive both in terms of population (it includes nearly all the population living in France) and in terms of healthcare events. Indeed, in France, almost all healthcare spendings are at least partially reimbursed by the government, so almost each healthcare of each individual is associated to at least one event in the database.
This makes it a very rich and powerful database, with a relatively small bias, and many works have already proved successful in improving population-level health  [54, 61, 2, 31]. Though most EHR studies are dealing with clinical EHR's  [23] , it is clearly crucial to exploit such claims-only EHR, and we hope that our work is a first step towards broader use, in healthcare and machine learning research community, of non-clinical claims datasets. Let us note that, since 2016, the access to this database, for public interest research, is possible through the SNDS access pipeline  [65]. It mainly offers access through the SAS software. An access using classical open source big data and AI frameworks (e. g. R, Python, Spark) including the SCALPEL3 library will be possible, by the end of 2020, through the Health Data Hub  [34] , a 80m$-funded national project which aims to be the national unique gateway to most French health data for operating public interest research (operated by both public or private entities) on modern infrastructures.