dataesr/scientific-paragraphs-categorization huggingface dataset:
This was 2. 49 for the DsNM3F and 12. 3 for sNM3F, thus showing the task discrimination gain obtained when applying the DsNM3F algorithm to the complete dataset. This discrimination gain was also observed in the remaining five datasets we tested [24 ± 2. 6 for DsNM3F vs. 36 ± 7. 8 for sNM3F (mean±SEM)] and was achieved because activation coefficients were better clustered together for each task but also because the task centroids were moved toward more optimal locations in the space of activation coefficients. Applying LDA to the coefficients of the two algorithms gave 0. 8 correct classification for sNM3F vs. 1 (i. e. , perfect classification) for DsNM3F for the example subject and 0. 675 ± 0. 06 0. 933 ± 0. 03 on (±SEM) across subjects. Hence, by modifying the single-sample activation coefficients, DsNM3F achieved significantly higher task separability.
g. , small R-values for S 2 × T 1 and S 4 × T 1 ). independent samples corresponding to unknown tasks. This problem requires the computation of a dedicated projection mapping from the EMG data onto the task-dependent activation coefficients computed by DsNM3F. It is likely that this mapping cannot be made by means of a simple projection onto the extracted modules because, as we showed above, they do not really carry the extra task information introduced by the task discrimination objectives. Nevertheless, as there are slight differences in the extracted temporal modules, we can still test how DsNM3F compares with sNM3F when no task information is available. To do this, we randomly split our data into two sets of samples. We exploited task information only in the first set (training samples) and used the second set (test samples) to predict the task of each sample (see Materials and Methods for a detailed explanation of the procedure).
, 2011; Roh et al. , 2011)  but did not explicitly assess task separability. Yet another approach proposed an algorithm for extracting task-specific modules  (Cheung et al. , 2009). This approach is indeed useful when the examined dataset contains tasks whose execution relies on synergies that are not shared with other tasks, however, it tends to increase the dimensionality of the extracted set of modules. Finally, in our previous work that motivated this study, we employed a single-sample task decoding metric for selecting the dimensionality of the set of modules and comparing alternative decomposition models  (Delis et al. , 2013a (Delis et al. , ,b, 2014)). On a similar vein, novel approaches have been proposed to critically test the functional role of modularity in task space. Recent studies examine the use of modularity for control and its ability to explain motor behavior  (Kuppuswamy and Harris, 2014).
Insights about the space-by-time Decomposition We applied DsNM3F to simulated data with known underlying structure and to an EMG dataset collected during an arm pointing experiment that we studied before. For both datasets, we were able to compare with the standard sNM3F algorithm and gain valuable insights concerning the functionality and usability of the new method. Indeed, the DsNM3F algorithm validated previous results regarding the spatial and temporal structure of muscle activity. Interestingly, adding the task discrimination objective had little or no effect on the extracted spatial and temporal modules on condition that data approximation had to be preserved as much as possible. A possible explanation for this similarity was that even for DsNM3F the data approximation objective prevailed over the task discrimination objective, so it did not allow the emergence of alternative modules.
fact, we tested this by relaxing the data approximation constraint and found that it led to the identification of relatively different modules (data not shown). However, these new modules accounted for significantly lower variance and had lower task discrimination performance. Hence, we suggest as a more plausible explanation that the standard modules (extracted by either sNM3F or DsNM3F) constitute the lowest-dimensional representation that achieves the best VAF-DEC trade-off for this dataset. In this respect, our results support effectiveness of the space-by-time decomposition of muscle activity in finding robustly and consistently modules that carry taskrelevant information, independently of the details of module computation. Regarding module activations, our simulations suggested that DsNM3F retrieves the task dependence of the activation coefficients more accurately than sNM3F, which results in better task categorization.
To exploit this distribution, we need to learn a (possibly non-linear) projective mapping between the EMG data and the extracted activation coefficients. The standard NMF projection we used here is simply based on data approximation criteria and depends only on the shape of the spatial and temporal modules, thus it ignores the distribution of activation coefficients. In future work, it would be interesting to develop methods that perform this projection efficiently in order to enhance the generalization power of our approach. For example, neural networks or Gaussian processes could be used to learn how to map EMG data (or parts of such data) to task-discriminant activation coefficients. Knowing this mapping, new EMG data could be mapped onto the identified activation coefficients, which, in turn, would give both the task being performed and a predictive reconstruction of the full EMG pattern that should be used to execute that task.
This type of approach may be useful in body-machine interfaces and myoelectric control devices for inferring user's intention when performing different goal-directed movements. We believe that learning such a mapping constitutes an additional interesting research question, which may be application-dependent and therefore will be treated in future work. We suggested before that the space-by-time decomposition of muscle activations is not only task-relevant but also lowdimensional. We explicitly demonstrate this point using the dimensions of our experimental dataset as an example. To specify the activity of M muscles (M = 9 in our data) over T time frames (this value could be considered smaller as a result of filtering or larger to account for arbitrary waveforms in continuous time) for the execution of S movements (S = 640 = 16 tasks ×40 repetitions), the number of parameters to be determined for the whole dataset is MTS = 288, 000.
When applying the spaceby-time decomposition to this dataset, the model parameters comprise (a) the activation coefficients combining the N spatial with P temporal modules to perform each movement S, i. e. , NPS = 3 × 4 = 7680 parameters and (b) the spatial and temporal modules that need to be stored and reused across movements, i. e. , NM + TP = 27 + 150 = 177 parameters, which makes 7857 parameters in total. Hence, for this example the space-bytime model of modularity reduces considerably the number of dimensions from 288,000 to 7857 (see also  Delis et al. , 2014). At last, to investigate the above questions further, we also plan to apply the space-by-time decomposition using both proposed algorithms to other EMG datasets recorded during more complex motor tasks, in which task decoding will perhaps be more challenging, thus more differences between DsNM3F and sNM3F may be revealed.
e. which dynamically change over time or can be either incomplete or partially available. In this paper, we draw the path towards this ambitious goal. Our approach leverages Data Life Cycle as a key concept to provide MapReduce for distributed and dynamic data sets on heterogeneous and distributed infrastructures. We first report on our attempts at implementing the MapReduce programming model for Hybrid Distributed Computing Infrastructures (Hybrid DCIs). We present the architecture of the prototype based on BitDew, a middleware for large scale data management, and Active Data, a programming model for data life cycle management. Second, we outline the challenges in term of methodology and present our approaches based on simulation and emulation on the Grid'5000 experimental testbed. We conduct performance evaluations and compare our prototype with Hadoop, the industry reference MapReduce implementation.
I. INTRODUCTION Corresponding author: Haiwu He, Computer Network Information Center, Chinese Academy of Sciences, 4 Zhongguancun Nansijie, Haidian District, Beijing 100190, China Increasingly the next scientific discoveries and the next industrial innovative breakthroughs will depend on the capacity to extract knowledge and sense from gigantic amount of information. Examples vary from processing data provided by scientific instruments such as the CERNs Large Hadron Collider (LHC); collecting data from large-scale sensor networks such as the OOI ocean underwater observatory; grabbing, indexing and nearly instantaneously mining and searching the Web; building and traversing the billion-edges social network graphs; anticipating market and customer trends through multiple channels of information.
Collecting information from various sources, recognizing patterns and distilling insights constitute what is called the Big Data challenge. Since its introduction in 2004 by Google  [1] , MapReduce has become the programming model of choice for processing large data sets. Two factors can explain the success of MapReduce. First, expression of the parallelism is very simple, almost hidden to the programmer. MapReduce borrows from functional programming, where a programmer can define both a Map task that maps a data set into another data set, and a Reduce task that combines intermediate outputs into a final result. The second factor of success is the wide availability of solid open-source solutions, as well as a mature technological ecosystem, which has been built around this paradigm: high level query languages  [2] , iterative computing  [3] , NoSQL databases, machine learning environments and many more.
However, one of the biggest challenges is to enable infrastructure operability, i. e. being able to have several infrastructures and systems with different usage paradigms that collaborate to the processing and the management of the dataset. Active Data  [6]  is a programming model for Data Life Cycle Management (DLCM). Active Data allows to have a unified view of the data-sets when handled by heterogeneous software and infrastructures. We present our implementation of MapReduce with Active Data which also allows to process dynamic data-sets. The rest of this paper is organized as follows; Section II reviews our background work on data life cycle management and MapReduce. We present the design choices, the requirements and the system architecture of our new prototype in section III. We evaluate our prototype on the Grid'5000 experimental platform in Section IV, and we also give preliminary results of our incremental MapReduce implementation.
Conceptually, data management systems report on their internal activity, and users specify actions to be conducted when specific data state transitions are reported, making it easy to implement decisions and optimizations. Active Data provides a non-invasive method of improving coordination and obtaining feedback from loosely coupled systems. Specifically, we can use the Active Datas life cycle model to individually represent and expose the internal life cycle of the systems across the various infrastructures. Thus, this approach can provide interoperability both between the systems and the infrastructures. As shown in Figure  1 , the model of a system's life cycle is composed of places, represented by circles and transitions, represented by rectangles. Places represent all the possible states of data, and transitions represent all the legal changes of states.
A. Hybrid MapReduce The implementation of MapReduce on top of BitDew is very flexible and allows the deployment of the processing and storage agents on many kinds of infrastructures: • On Desktop PCs and mobile devices, the MapReduce/BitDew agents can directly fetch the data from Cloud storage servers, perform the Map and Reduce computation and upload the results to the stable storage. On Cloud and Grid nodes, two approaches are feasible: • The first one consists in deploying MapReduce/BitDew agents on the Grids and Clouds nodes following a Pilot Job approach. In this case, we can provision Cloud virtual instances that embed the MapReduce/BitDew agent, or we can schedule the MapReduce/BitDew using the Grid Resource Management software. • The second approach consists in relying on Hadoop already installed on the Cloud or the Grid.
In this case, we consider the usage of two different MapReduce middleware at the same time, MapReduce/BitDew and Hadoop. Thus, a first stage consists in splitting the dataset in two parts and transferring the data to the two middleware. Then both middleware process their respective data sets independently, which are then aggregated at the end of the computation. Thanks to these advances, we can now execute MapReduce applications, which use any combination of Grids, Desktop Grids, and Clouds, even using legacy Hadoop installations. However, when considering a Hybrid DCI, it is necessary that a data scheduler splits the data-set and distribute the data according to the processing capabilities of each infrastructure. We are working now on defining the best strategies to distribute the data. The hybrid infrastructure enables the use of highly heterogeneous machines, with stable and volatile storage to avoid data loss.
B. Global MapReduce Dataflow Surveillance However to implement efficient Hybrid MapReduce, it is necessary that the infrastructures and the systems report the progress of the MapReduce computation, for instance to enable redistribution of the data if some bottleneck appears. To observe the life cycle of the data involved in a MapReduce/BitDew and Hadoop computation, we use Active Data. Programmers supply code to a special service in BitDew called "Active Data", specifying at what step of the data life cycle the code must be executed. At the same time, the BitDew and Hadoop runtime must inform the Active Data service when it uses or alters data. Any data item has a corresponding life cycle model that documents everything that can be done with it. After making Hadoop, HDFS and MapReduce/BitDew "Active Data compliant", we are now able to have several MapReduce applications that share a unified data life cycle and that are able to cooperate on a unique MapReduce application execution.
A. Evaluation Against an Emulated Desktop Grid Environment We have performed all of our experiments in the GdX and NetGdX clusters which are part of the Grid'5000 infrastructure. The two clusters are composed of 356 IBM eServer nodes featuring one 2-core 2. 0GHz AMD Opteron CPU and 2GB RAM, running Debian with kernel 2. 6. 18, and interconnected by Gigabit Ethernet network. All results described in this paper were obtained using Hadoop version 0. 21. 0, while the Fig. 4 : Scalability evaluation on the WordCount application: the y axis presents the throughput in MB/s and the x axis the number of nodes varying from 1 to 512. &'(#)* $%&& "$% $ %+",* &&&& %"&&&&& &+'$ %&'!+!! ! " #$ %& $! #&" &'$ '#& ( & (( !(( $(( "(( #((( #&(( #!(( ) data was stored with 2 replicas per each block in Hadoop Distributed File System.
We perform our experiments by repeatedly executing the wordcount benchmark, with 50GB dataset generated by Hadoop RandomTextWriter application, and set chunk size to 64MB, map slot number to 2, reduce slot number to 1 and total reduce number to 10. We emulate a desktop grid environment using several artifacts to modify the machine and network characteristics of the platform. The goal of these modifications is to study several aspects such as scalability, fault tolerance, host churn, network connectivity, error computing and sabotage, CPU heterogeneity, stragglers and network failure tolerance. The rest of this section gives an overview of the how we emulated each aspect of a Desktop Grid platform and some key results we obtained.
B. Evaluation of the incremental MapReduce/BitDew To evaluate the performance of incremental MapReduce, we compare the time to process the full data set compared with the time to update the result after modifying a part of the dataset. The experiment is configured as follows; the benchmark is the WordCount application running with 10 mappers and 5 reducers, the data set is 3. 2 GB split in 200 chuncks. Table  III  presents the time to update the result with respect to the original computation time when a varying fraction of the dataset is modified. As expected, the less the dataset is modified, the less time it takes to update the result: it takes 27% of the original computation time to update the result when 20% of the data chunks are modified. However, there is an overhead due to the fact that the shuffle and the reduce phase are fully executed in our implementation.
In addition, the modified chunks are not evenly distributed amongst the nodes, which provokes a load imbalance. Further optimizations would possibly decrease the overhead but would require significant modification of the MapReduce runtime. However, thanks to Active Data, we demonstrate that we can reach significant speedup with a patch that impacts less than 2% of the MapReduce runtime source code. Of course, many systems that allow massive processing of distributed and dynamic data sets have been designed. To cite a few works that give a taste of research results in this direction: incremental processing of data sets dynamically updated  [10] [11] , data streams parallel processing  [12]  [13], widely distributed data sets  [14]. Our approach contrasts from the previous work in the sense that we aim at adapting the MapReduce programming model to these emerging constraints on data sets.
This approach would allow people to continue to "think" MapReduce, while avoiding the constraint of processing static data sets. There have been several attempts at providing MapReduce runtime execution environments for Desktop Grids  [15]  and the Internet  [16] ,  [7]. In previous works, we investigated MapReduce on Hybrid DCIs  [17] ,  [18] ,  [9] , combining Desktop Grid and Cloud infrastructures. In this work, we extend those results by considering dynamic datasets thanks to Active Data. In addition, Active Data can be used to provide additional important features, such as monitoring and optimizing the execution of MapReduce applications on Hybrid DCIs.
ACKNOWLEDGEMENTS This work is partially supported by  International Science & Technology Cooperation Program of China  under grant No. 2015DFE12860 , and  NSFC  under grant No. 61370104 , by the  French National Research Agency  ( MapReduce   ANR-10-SEGI-001 ) and by the  Chinese Academy of Sciences President's International Fellowship Initiative (PIFI)   2015  Grant No. 2015VTB064.
Likelihood searches using RAxML were improved by data partitioning by gene and codon position. Finally, the PhyloBayes software, which allows different substitution processes for amino acid replacement at various sites, produced a tree that best matched known higher level taxa and defined basal relationships in Coleoptera. After rooting with Neuropterida outgroups, suborder relationships were resolved as (Polyphaga (Myxophaga (Archostemata + Adephaga))). The infraorder relationships in Polyphaga were (Scirtiformia (Elateriformia ((Staphyliniformia + Scarabaeiformia) (Bostrichiformia (Cucujiformia))))). Polyphagan superfamilies were recovered as monophyla except Staphylinoidea (paraphyletic for Scarabaeiformia) and Cucujoidea, which can no longer be considered a valid taxon. The study shows that, although compositional heterogeneity is not universal, it cannot be eliminated for some mitochondrial genes, but dense taxon sampling and the use of appropriate Bayesian analyses can still produce robust phylogenetic trees.
Primers used are described in  Timmermans et al. (2010). Sequence reads were assembled using the MIRA or Newbler software as described previously  (Timmermans et al. 2010; Haran et al. 2013 ) and the longest contig obtained with either assembler was retained. tRNA genes were annotated with COVE using beetle-specific covariance models  (Timmermans and Vogler 2012). Protein-coding gene sequences were annotated using existing Coleoptera mitochondrial genomes as reference in Geneious (http://www. geneious. com/, last accessed December 17, 2015). For the rRNA genes, sequences were extracted from the newly generated and previously published mitogenome sequences, using BLAST searches on a fasta formatted database with methods described in  Bocak et al. (2014). The taxonomic classification, voucher ID, GenBank accession numbers, and geographic origin for each specimen are given in supplementary table S1, Supplementary Material online.
Phylogenetic Inference The 13 protein-coding genes were aligned with ClustalW using the transAlign wrapper (Bininda-Emonds 2005). The cox1 gene was split into the 5 0 "barcode" region  (Hebert et al. 2003 ) and the 3 0 region widely used in Coleoptera systematics usually amplified with the Pat and Jerry primers  (Simon et al. 1994). This was to account for the fact that the two PCR fragments with different amplification success are confined to the 5 0 or 3 0 ends for the short and long fragment, respectively. The two rRNA genes were aligned using MAFFT v. 7  (Katoh et al. 2009)  under default parameters on the server http://mafft. cbrc. jp/ alignment/software/, last accessed December 17, 2015. Protein-coding alignments were edited, trimmed, and translated with Mesquite v. 2. 75  (Maddison WP and Maddison DR 2014).
Twelve partitioning schemes for the protein-coding genes were compared, ranging from unpartitioned to a maximum of 42 partitions (by gene + codon position). Likelihood scores were compared with reference to the complexity of the partitioning schemes using the Akaike Information Criterion (AIC). Bayes Factors and Relative Bayes Factors (RBF) were calculated according to  Castoe et al. (2005). Phylogenetic trees were generated using ML and Bayesian methods for partitioned and unpartitioned data sets. All RAxML trees were generated at the CIPRES webserver,  (Miller et al. 2010 ; https://www. phylo. org/, last accessed December 17, 2015) under the GTRCAT model of nucleotide substitution, which approximates a GTR + À model with a reduced computational cost  (Stamatakis 2006). Where relevant, node support was assessed using a rapid bootstrap algorithm implemented in RAxML with 500 replicates.
PhyML  (Galtier and Gouy 1998; Guindon and Gascuel 2003 ) was run on the ATGC webserver  (Guindon et al. 2010; www. atgc-montpellier. fr, last accessed December 17, 2015)  and used a GTR substitution model using eight rate categories. The gamma shape parameter and the proportion of invariable sites were estimated from the data. To infer relationships under the nonhomogeneous model of  Galtier and Gouy (1998) , nhPhyML  (Boussau and Gouy 2006)  was used, again using eight rate categories. Topology, gamma shape parameter, and transition/transversion rates were evaluated, but no final optimization of parameters such as branch lengths was performed (setting: -quick = y). As starting tree for tree searches in PhyML and nhPhyML, we used the RAxML tree of the complete, partitioned data set rooted on the Archostemata.
Both analyses used the Nearest Neighbor Interchange algorithm. Finally, the translated data matrix was subjected to Bayesian analysis with PhyloBayes 3 under the CAT-Poisson model  (Lartillot et al. 2009). Two Markov chain Monte Carlo chains were run after the removal of constant sites from the alignment. This Bayesian analysis was repeated with outgroups included. These outgroups were from three orders of Neuropterida, the presumed sister lineage of Coleoptera, and were obtained from GenBank (Accession numbers: NC_011277, NC_011278, NC_013257, NC_015 093, NC_021415, NC_023362, NC_024825, NC_024826). PhyloBayes tree searches were also conducted on the CIPRES webserver. The R package "ape"  (Paradis et al. 2004 ) was used to obtain root-to-tip branch lengths from the RY-coded ML and the Bayesian amino acid trees.