dataesr/scientific-paragraphs-categorization huggingface dataset:
The FCAE were re-initialised and retrained 8 times per iteration, and the one receiving the lowest validation A 2 error was selected to be evaluated on the testing dataset. Dataset splitting was performed patient-wise, assigning all of the data from a patient into the same fold, implying that clinical records for one patient at two different times could not appear in both training and testing simultaneously. The validation dataset is corrupted in the same manner as the testing dataset to ensure that the validation loss represents both A 1 and A 2 testing error. The A 1 and A 2 errors are shown in Figure  4. As expected, the A 1 error decreased monotonically with IR size for all methods (p < 0. 01). As hypothesised, A 2 did show an optimal IR size for the PCA approaches with both methods monotonically decreasing until IR = 4 (p < 0.
(As with the cumulative information in Figure  2 , Figure  7  is an indication of the amount of data available, although in terms of the sheer volume of data rather than number of datapoints. That is, datapoints with half their values missing have half the volume of complete datapoints, reflected in     2 : Average per sample compression and decompression time for our proposed autoencoder and the two baselines, with an Intel Xeon E5-1620 v4 CPU at 3. 50GHz. All values are expressed in microseconds. this figure, but are given equal weight in Figure  2. ) There was another swift increase between 60% and 80%, which can be seen in the improvements in the FCAE's A 1 and A 2 errors. This provides some evidence that training with incomplete data is beneficial, even if a sizeable portion of the dataset is largely incomplete.
Discussion and Future Work We compared our method to two state-of-the-art versions of PCA. To the best of our knowledge, PCA is the only commonly-used method that combines both imputation and compression and provides some optimality guarantees as a linear baseline. The first point of discussion is the relatively good performance of the two variants of PCA, indicating that a large amount of the variation can be addressed by a linear model equipped with simple missing-value handling. The difference between the two approaches to constructing linear models (PPCA and IPCA) was smaller than initially expected, with the more complex IPCA underperforming PPCA, illustrating that the linear approximation of the dataset's underlying structure is only that: an approximation. The issue with using these linear models for reconstruction was expected: a sufficiently large internal representation allows PCA to "remember" the missing variables as their mean value rather than impute them.
The deterioration of A 2 after a certain point in PCA is visible even at the relatively low IR sizes shown in Figure  4  (bottom). This trade-off is much less pronounced in the FCAE likely due to the application of modality masking noise which implies that the loss metric encodes both A 1 and A 2. This combination of losses means that the FCAE must take a more complex imputation strategy even at higher IR sizes. We observed the distribution of the output prediction of each item, as seen in Figure  8. In this figure, we compare the output distribution of the first question of the part 3 (motor examination) of the MDS-UPDRS questionnaire as reconstructed using FCAE and PPCA on a test set against the true distribution as recorded in the database. 40 iterations have been performed, with a corruption ratio of 10% and IR size of 4.
We can notice that there is an important loss of variance in the distributions, especially with PPCA reconstruction. We can explain that by the reconstruction algorithms being designed to minimize a variant of the mean squared error, thus biasing predictions values towards the mean of the distribution and penalising extrema. This behaviour is seen in both reconstruction methods although less distinctly in FCAE indicating that it has a higher capability of reconstructing these extreme values. We have found that this reduction in variance is common across all questions in the database, possibly due to the inherent uncertainty and subjectivity of the variables encouraging some regression-towards-the-mean. In addition, the variables investigated in Figure  8  show two distinct underlying distributes, with the first question of UPDRS part 3 having something similar to a binomial distribution with the seventh question of UPDRS part 1 having something closer to a geometric distribution.
Unsurprisingly, the linear baseline's reconstruction for both is more binomial-shaped, given that binomial distributions are more similar to discretely-sampled Gaussian distributions, the continuous version being explicitly optimized for by linear methods. Perhaps unsurprisingly, the proposed auto-encoder method shows a similar tendency, which indicates that the likely cause is the L2-style loss function similar to it and   the baseline methods. One possible expansion for the auto-encoder framework is the design of a heterogeneous loss function that is more cognisant of these distributional differences rather than the use of an equivalent L2 loss across all the questions in the database regardless of their distributional characteristics. These distributional differences would also play an important role in evaluating any imputation method, especially model-based methods such as MICE  [19] , as it implies that the L2 loss is not a representative metric for error.
In future, we would like to extend our analysis of medical questionnaire imputation and compression for a particular downstream analysis goal, such as the classification of different Parkinsonian patient groups or the stratification of the disorder. At the moment, diagnosis and stratification are done by a neurologist using some of the data provided by the medical questionnaire, indicating the utility of making the analysis thereof more objective and robust to heterogeneous, incomplete data. One fact that complicates this study is the relatively low number of data points, especially complete data points, given the relatively high dimensionality and lack of simplifying structure in each point. These issues are further complicated by the majority of tests in the PPMI database used having a middling number of ordinal values (most variables took on values in {0.
Ordinal and categorical data handling It is important to note that, with this database, all the variables were numerical or ordinal. Thus, we did not test the framework with categorical variables, which can be present in other medical questionnaires. We would suggest the use of one-hot encoding for this type of variable in addition to a custom output layer capable of handling different loss functions for the different variable types. Imputing ordinal variables is problematic. Although ordered, the leap between two successive categories is not consistent, nor quantified. Thus, an error in imputation for an ordinal variable can be more or less significant in certain parts of the answers range. In medical questionnaires, some questions suffer greatly from this problem, as a '0' could mean complete absence of a symptom, and '1' to '4' could quantify its severity.
Conclusions This paper presents an autoencoder specifically designed for data imputation and compression of medical questionnaires, in which entire modalities may be missing. This represents some of the initial steps into performing deep learning on specific medical tasks that are challenging due to the dataset size compared to the high number of features, the data types provided and the complexity of the learning problem. We have shown that significant and consistent improvements can be made over linear methods, especially when a lower number of features is missing. We have also shown that their is an interest in learning on incomplete data vector, and that the imputation performance of our framework is not negatively impacted by a high internal representation size, making the downstream choice of the compression rate easier.
Application of DEB model to A. cordatus Parameter estimation. An individual mechanistic DEB model was developed for A. cordatus  (Guillaumot, 2019). Estimated DEB parameters are reported in Table  1. The DEB model considers a larval growth accelerated compared to the adult stage  (Schatt, 1985) , so-called 'abj' type model. The model was constructed using data from the literature (Table  2 ). The goodness of fit of the DEB model to the data was evaluated by calculating the Mean Relative Error (MRE) of each dataset, which is the sum of the absolute differences between observed and expected values, divided by the expected values. MRE values are contained in the interval [0, infinity). The MRE is considered to be a reference method to assess DEB modelling performance  (Lika et al. , 2011b) , for which the closer to 0, the better model predictions match the data.
Each individual is a component of the modelled population, which is itself affected by population death rate and density-dependant processes. The IBM was built with the software Netlogo version 6. 0. 4 (Wilensky, 1999), using the DEB-IBM model developed by  Martin et al. (2010)  Table  1  Parameters estimated for the DEB model developed for Abatus cordatus. Values are given for the reference temperature of 20 • C. The MRE of the model is 0. 121.
Symbol Value Unit Basic DEB parameters Volume of specific somatic maintenance 1  Somatic maintenance rate coefficient 1 Fraction of energy allocated to somatic maintenance and growth 1 Volume-specific cost of structure 1 Energy of maturity at birth 1 Energy of maturity at metamorphosis 1 Energy of maturity at puberty 1 Arrhenius temperature 3 [p˙M] k˙M κ [EG] [E b H ] [E j H ] [E p H ] TA 13. 84 5. 10 -3 0. 78 2395 0. 5693 8. 325 1638 9000 Gompertz stress coefficient 3 ḧa sG 5. 02. 10 -6 0. 0001 J. d -1. cm -3 d -1 - J. cm -3 J. cm -3 J. cm -3 J. cm -3 K DEB compound d -2 - for the species Daphnia magna. The NetLogo code is available at http://modelingcommons. org/browse/one_model/6201. It contains the script to run the model, the input files of monthly food resources and temperatures for the three stations and a detailed description of the model following the ODD protocol from  Grimm et al.
The correction is applied to the metabolic rates v˙, k˙M, k˙J, ḧ a (Table  1 ). Temperatures recorded since 1993 at Port aux Français, another site in the Gulf of Morbihan, show a clear 6-year cycle of increasing and decreasing temperatures (Appendix A). The  [2012] [2013] [2014] [2015] [2016] [2017] [2018]  temperature dataset selected as input forcing variable in the model therefore constitutes an interesting proxy of temperature conditions at Anse du Halage which includes a complete overview of the environmental variability at the station. However, it is important to take this choice into consideration during interpretation of results, as it needs to be differentiated from a cycle that would be inherent to the biology of the species.
The arrhenius function and the optimal temperature range In DEB theory, the Arrhenius function determines the optimal temperature range of the organism's metabolism as a response to enzymatic tolerance  (Kooijman, 2010; Thomas and Bacher 2018). In the present work, calculation of the Arrhenius function relies on fragmental datasets. The ascending part of the Arrhenius curve that is, the temperature range in which faster metabolic rates are determined by higher temperatures was estimated, but values are still missing for the descending slope (i. e. the temperature range beyond the optimal temperatures in which the metabolic rates slow down with higher temperatures)  (Kooijman, 2010). The present model assumes that higher temperatures favour more suitable conditions with no limit (Appendix F), which has to be corrected arbitrarily using our personal field and experimental observations on the echinoid ecology (Fig.
Although  Magniez (1983)  observed a decrease in gonadal material after the reproduction period, it is very small in females (-0. 3%) and slightly bigger in males (-1. 6%), and the exact cause has not been studied. It is not known whether this decrease in gonadal material can be directly attributed to a reabsorption for survival purposes or some other mechanism. The use of previously stored energy in the different compartments to sustain the maintenance of structure is assumed to be non-existent in our model. Such starvation processes could be tested in future implementation, provided sufficient data is obtained through experimental setups observing the phenomenon. The two scenarios of future food availability were based on coarse IPCC and NOAA projection models for the region. These simulations and associated outputs are here considered as conceivable scenarios of the influence of food and temperature changes on population dynamics.
Acknowledgements The authors would like to thank  Yoann Thomas  and  Cédric Bacher  for their valuable advice during model construction, and  Pierre Magniez  for his contribution with data on A. cordatus reproduction strategies. The present work is a contribution to the  IPEV  programme  PROTEKER  (No. 1044 ) and the  French LTER Zone ATelier Antarctique (ZATA). It is also contribution no. 39  to the vERSO project (www. versoproject. be),funded by the  Belgian Science Policy Office (BELSPO , contract n •  BR/132/A1/ vERSO ) and contribution no. 13  to the " Refugia and Ecosystem Tolerance in the Southern Ocean " project (RECTO;  BR/154/A1/RECTO ) funded by the  Belgian Science Policy Office (BELSPO). M A-P was supported by the  French Foundation for Research on Biodiversity  and its partners (FRB -www.
PHYLOGENOMICS Phylogenomic analyses compared SIMO Bins 19-2 and 31-1 (ref. 25)  to previouslyreported Ca. Poseidoniales MAGs, including those binned from TARA Oceans, Mediterranean Sea, Red Sea, Gulf of Mexico, Guaymas basin, and Puget Sound metagenomes  [11, 13, [15] [16] [17] [27] [28] [29] [30] [31] [32]  (Table  S1 ). Average nucleotide identity (ANI) was calculated using fastANI  [33]  to compare non-redundant MAGs from Tully  [17]  to 15 Port Hacking MAGs  [16]  and the two SIMO MAGs; any with ANI <98. 5% were added to the non-redundant set. Phylogenomic analysis used sixteen ribosomal proteins  [34]  within anvi'o v4 (ref. 35). All genomes were converted to contig databases and proteins were identified using HMMER  [36] , concatenated, aligned using MUSCLE  [37] , and used to build a phylogenomic tree using FastTree  [38]  within anvi'o.
[23] , while "dark incubation" analyses included only Vorobev et al. samples (T0, which were processed immediately after collection, and T24, which were processed following 24 hours of in situ incubation in dark containers; NCBI BioProject PRJNA419903). Temperature, salinity, dissolved oxygen, pH, and turbidity data corresponding to metatranscriptome sampling times were downloaded from the NOAA National Estuarine Research Reserve System website (http://cdmo. baruch. sc. edu; last accessed 16 July 2020). Contigs from all MAGs used for phylogenomics (Table  S1 ) were used as a read mapping database using Bowtie2 v. 2. 2. 9 (ref. 40)  with the "very-sensitive" flag. Samtools v. 1. 3. 1 (ref. 41)  was used to index BAM files, which were profiled and summarized in anvi'o. Contig genus identity was imported to the anvi'o contig database as an external collection.
The number of transcripts L -1 was calculated by scaling the number of mapped reads by the volume of water filtered and the recovery of internal standards (reported in  [23, 26] ) as previously described  [39]. Seasonal transcript abundances were compared using a one-way ANOVA test in R  [42]  with data log-transformed as necessary to improve normality. When ANOVA results were significant, groupings were defined post-hoc with Tukey's Honest Significant Difference (HSD) test using the agricolae R package  [43]. Non-metric multidimensional scaling (NMDS) analysis of metatranscriptome hits was conducted using the vegan R package  [44]. NMDS input was a distance matrix constructed by Hellinger-transforming the table of transcript hits and calculating Euclidean distance between samples  [45]. Genus vectors were calculated using the envfit command.
MAG-SPECIFIC ANNOTATION AND TRANSCRIPT ANALYSIS Gene-specific analyses focused on three MAGs: two from the SIMO collection (SIMO Bin 19-2, Genbank: VMDE00000000; SIMO Bin 31-1, VMBU00000000;  [25] ) and one previously binned from Red Sea metagenomes (RS440, PBUZ00000000;  [27] ). These MAGs represented genera O1, O3, and M, respectively, which were highly abundant in metatranscriptomes (see Fig. 1 ). RS440 was selected due to a high number of transcripts recruited when genus M was abundant (data not shown). MAGs were annotated using the archaeal database in Prokka v. 1. 13 (ref. 46) , using DIAMOND  [47]  to search against all orthologous groups in eggNOG-mapper v. 1 (refs. 45,46) , and using the BlastKOALA portal (https://www. kegg. jp/blastkoala/, last accessed 6 March 2019)  [48]. Putative genes for carbohydrate-active enzymes, peptidases, and membrane transport proteins were identified using HMMER searches of dbCAN2 (HMMdb v.
7)  [49, 50] , MEROPS v. 12. 0 (ref. 51) , and the Transporter Classification Database  [52] , respectively. Transcript reads were mapped to MAGs (combined into a single database such that each read mapped to only one MAG) to identify Ca. Poseidoniales genes that were highly or differentially transcribed. Coverage was calculated by profiling BAM files in anvi'o and normalized to coverage per million reads (CPM) by dividing by the total number of reads per sample. For each MAG, "highly transcribed" genes were the 5% of putative genes with the highest median CPM across metatranscriptomes (SIMO Bin 19-2: 63 genes, SIMO Bin 31-1: 70 genes, RS440: 77 genes). DESeq2 v. 3. 11 (ref. 53)  was used to identify genes from each MAG that were differentially transcribed when each genus was highly transcriptionally active.
Abundance of Ca. Poseidoniales 16S rRNA genes was compared to published bacterial 16S rRNA gene abundance from the same samples  [24, 54]. Regional variability of gene abundance was assessed using a one-way ANOVA and a post-hoc HSD test as described above. Model II regressions of log-transformed qPCR data were estimated using the lmodel2 R package  [58]  as previously described  [54]. All plots were constructed with anvi'o or the ggplot2 R package  [59].
ACKNOWLEDGMENTS We thank  Qian Liu ,  Bradley Tolar , the  Georgia Coastal Ecosystems LTER  field team, the  University of Georgia Marine Institute at Sapelo Island  staff, and the captain and crew of the  R/V Savannah  for assistance with original field sampling. This work was funded in part by  NSF   OCE grant   1538677  and  OPP  grant  1643466  (to JTH),  NSF  Grant  IOS1656311  (to MAM),  NSF   OCE grant   1832178  (to the  Georgia Coastal Ecosystems LTER) , as well as by resources and expertise from the  Georgia Advanced Computing Resource Center , a partnership between the  University of Georgia's Office of the Vice President for Research and Office of the Vice President for Information Technology.
Lately, Quan et al. [37]  proposed a technique of Spatio-Temporal Lacunarity Spectrum (STLS) to capture lacunarity-based features based on lacunarity analysis for local binary patterns in DT slices. In another aspect of DT analysis, Baktashmotlagh et al. [38]  utilized Stationary Subspace Analysis (SSA) in order to investigate stationary components for video description. In terms of effectiveness in DT recognition, experiments have shown that the geometry-based methods principally have good performances on simple datasets, e. g. , UCLA  [27] , but not on the more challenging ones, e. g. , DynTex  [39]  and DynTex++  [40]. It may be due to lack of temporal information involved in their encodings. Optical-flow-based methods: Most of optical-flow-based methods have represented DTs based on magnitudes and directions of normal flow.
In other works, Hong et al. [53]  introduced a learning concept of ''key frames'' and ''key segments'' to construct a deep dual descriptor based on static and dynamic learned features. Hadji et al. [54]  composed a new challenging large scale dataset (DTDB). They then implemented some deep-learning methods for learning DTs on DTDB: Convolutional 3D (C3D)  [55] , RGB/Flow Stream  [56] , Marginalized Spatio-temporal Oriented Energy (MSOE) in two learning streams (MSOE-two-Stream)  [54]. The second trend concerns with dictionary learning approaches. Quan et al. [57]  considered patches of a given video as atoms fed into a sparse coding method to learn a dictionary for DT representation, while Quan et al. [58]  proposed equiangular kernel to learn a dictionary in reasonable dimension. In regard to efficiency of the learning-based methods in DT recognition, just the deep models have outperformed the others.
Datasets and protocols The benchmark datasets for evaluating our proposed descriptors in DT classification are expressed in this section. A brief of those is shown in Table  1  for a quick reference. UCLA dataset  [27]  consists of 200 videos recorded in 110 × 160 × 75 dimension to capture textural motions (see Fig. 6(a)  for several instances of DT videos). The following protocols are usually addressed for DT recognition. 20 ), ''smoke'' (4), ''water''  (12) , and ''waterfall''  (16) , where the numbers in parentheses indicate their quantities. Due to the dominance of ''plants'' class, it is removed to form 8-class with more challenges  [73]. Following settings in  [40, 72] , a half of videos in each group is randomly selected for training and the rest for testing. The trial is repeated 20 times and the final rate is obtained from the average of those.
DynTex dataset  [39]  consists of 650 high-quality sequences recorded in various conditional environments (see Fig. 6 (b) for several instances of DT videos). Following settings in  [21, 65] , DynTex's challenging schemes are often addressed for DT recognition using leave-one-out protocol. • DynTex35 is composed as follows. Each of 35 selected videos is split into 10 sub-videos subject to its spacial axes to correspondingly form 35 categories. • Alpha includes three categories as ''Sea''  (20) , ''Trees''  (20) , and ''Grass''  (20). Herein, the numbers in parentheses indicate cardinality of corresponding categories. DynTex++ dataset  [40]  consists of 36 categories with 100 subvideos of 50 × 50 × 50 dimension for each, i. e. , 3600 sequences in total. These sub-videos are composed by capturing the major turbulent DTs from 345 raw videos of DynTex.
Following settings in  [21, 40] , a half number of videos in each category is randomly addressed for training and the rest for testing. The final result is then reported by the average of 10 trials. DTDB dataset  [54]  is recently a large scale dataset of DT videos for principally evaluating effectiveness of CNN-based proposals. It consists of over 10000 DT videos with a total of ∼3. 5M frames captured from different sources: websites, handled cameras, etc. (see Fig. 6 (c) for some samples). Two challenging schemes are addressed for DT recognition as • Dynamics scheme is arranged into 18 categories so that its DT videos just include features of dynamics, i. e. , independent of spatial appearance. Note: LOO and 4fold are leave-one-out and four cross-fold validation respectively. 50%/50% is 50% random samples for training and the remain (50%) for testing.
(refer to those works for more detail of computation). ) are implemented on the alike computing system: a 64-bit Linux desktop of CPU Core i7 3. 4 GHz 16G RAM. This is to make an impartial evaluation with other LBPbased ones done in  [19]. Table  3  shows that runtime of encoding the DoDG-based descriptors of a 50 × 50 × 50 video is nearly the same as that of other LBP-based ones. In addition, it should be noted that all runtimes in Table  3  are reported using the CPU in only one thread for running their raw MATLAB codes. In the case of addressing 4 multithreads, it takes about 0. 26s and 0. 29s for encoding DoDGF 2 , ′ ,1st and DoDGF 3 , ′ ,1st respectively.