dataesr/scientific-paragraphs-categorization huggingface dataset:
Low error rate, medium coverage Here, we present results on the E. coli 60x (low), S. cerevisiae 60x (low) and C. elegans 60x (low) datasets of Table  3. For these datasets, when possible, we evaluated the trimmed corrected long reads. When trimmed reads were not available, we evaluated the complete corrected long reads. These results are summarized in Table  10  and commented below. Moreover, for these experiments, we excluded the following tools from the comparison, for the ensuing reasons: • ECTools, because of its unsatisfying results on previous experiments; • Hercules, because of its unsatisfying results and large runtimes on previous experiments; • LSC, because of its unsatisfying results on previous experiments; • Nanocorr, because of its large runtimes on previous experiments; • NaS, because of its large runtimes on previous experiments; On these datasets, both hybrid and self-correction methods did manage to reduce the error rate to around 1% or less, on all datasets.
More precisely, only the hybrid methods FMLRC and LoRDEC reached an error rate of more than 1% on the C. elegans dataset. In comparison, on this dataset, all self-correction tools managed to reduce the error rate well below 1%, the highest error rate being that of Canu on the C. elegans dataset, and only reaching 0. 79%. This observation is in accordance with that previously made in Section 3. 7, but also further shows that self-correction methods provide better correction as the sequencing depth grows. Moreover, it is also interesting to note that, like in Section 3. 7, most self-correction tools reported as many corrected bases as hybrid methods, although LoRMA still performed poorly regarding this metric, and MECAT still reported slightly less corrected bases than other methods. Overall, hybrid correction and self-correction methods were thus highly comparable in terms of quality of the results on these three datasets, and self-correction methods even performed better than in Section 3.
7. This underlines the fact that, given the error rate of the reads reaches around 12%, self-correction methods perform well on low coverage dataset, but also benefit from larger sequencing depths, providing higher quality correction as coverage grows higher. In terms of runtime, all hybrid correction tools except Jabba once again performed slower than self-correction methods. More precisely, Jabba even was the fastest tool on all these datasets, requiring at most 49 min to perform on the C. elegans dataset, while MECAT , the fastest self-correction tool, required 2 h 43 min. Jabba excluded, however, self-correction methods performed faster than most hybrid correction methods. This can especially be seen on the C. elegans dataset, where hybrid correction tools such as CoLoRMap, HG-CoLoR and Proovred were not run due to their unreasonable runtimes.
In comparison, LoRMA and CONSENT , the slowest self-correction tools on this dataset, required respectively 31 hours and 27 hours to perform, while other self-correction tools required 10 hours or less. In terms of memory consumption, self-correction tools were once again more demanding than most hybrid correction tools on all datasets, although hybrid tools such as CoLoRMap and Proovread required more resources than all other tools except LoRMA and Daccord. These observations are in accordance with conclusions drawn in Section 3. 7, indicating that, when error rates reach around 12% or less, self-correction methods perform as well as hybrid correction methods, or even outperform them on complex datasets. However, this results also show that self-correction methods can also benefit from higher sequencing depth, and manage to perform better as it grows larger.
Moreover, these results further show that self-correction methods perform faster than hybrid correction methods, even more so when the sequencing depth is large. As a result, this suggests that self-correction is best suited, both in terms on quality of the results and in terms of resource consumption, and should thus be preferred when analyzing low error rates datasets, especially when the sequencing depth grows large. This corresponds to the E. coli 60x (low), S. cerevisiae 60x (low) and C. elegans 60x (low) datasets of Table  3. 1 CoLoRMap was not launched on the C. elegans dataset due to its large runtimes. 2 HG-CoLoR was not launched on the C. elegans dataset due to its large runtimes. 3 Proovread was not launched on the C. elegans dataset due to its large runtimes. 4 Daccord could not perform correction on the C.
Ultra-long reads Here, we present results on the H. sapiens dataset of Table  3. For this dataset, when possible, we evaluated the trimmed corrected long reads. When trimmed reads were not available, we evaluated the complete corrected long reads. These results are summarized in Table  11  and commented below. Moreover, for these experiments, we excluded the following tools from the comparison, for the ensuing reasons: • ECTools, because of its unsatisfying results on previous experiments; • Hercules, because of its unsatisfying results and large runtimes on previous experiments; • LSC, because of its unsatisfying results on previous experiments; • Nanocorr, because of its large runtimes on previous experiments; • NaS, because of its large runtimes on previous experiments; • Proovread, because it could not be installed on the computer we performed these experiments on; • Daccord , because it could not perform correction, since the alignment step with DALIGNER required more than 128 GB of memory.
On this dataset, most tools, both hybrid and non-hybrid, failed to efficiently reduce the error rate of the reads to 1% or less. More precisely, the two best performing tools were Jabba and HG-CoLoR, which respectively reached an error rate of 0. 1% and 1. 2%. However, it is worth noting that Jabba output two times less bases than HG-CoLoR, and that the average length of its corrected reads only reached 3,265, indicating that it split a large proportion of reads, and thus generated high quality, but short fragments. In the same fashion, CoLoRMap also produced corrected reads reaching a short average length, despite the fact it did not trim or split the reads. This tends to indicate CoLoRMap mostly managed to correct the shortest reads of the dataset. The fact that hybrid correction methods did not manage to perform well on this dataset is consistent with previous observations indicating that they tend to suffer from performance drops when applied to large and complex genomes.
However, it is still interesting to note that HG-CoLoR performed well on this dataset, managing to output a large number of high-quality corrected reads. As for self-correction, it worth noting that FLAS and MECAT could not perform correction on these ultra-long reads, and that they had to be filtered out in order to allow the methods to run. In terms of quality of the results, LoRMA reached the lowest error rate, of only 2. 2%, but reported reads with an average length of 183 bp, and thus aggressively trimmed the reads once again. Other self-correction tools globally performed worse than hybrid correction tools in terms of reduction of the error rate, the best performing tool being CONSENT , and only reducing the error rate to 7%. It is however worth noting that the average length of the corrected reads was larger for self-correction than for hybrid correction methods, which indicates that self-correction methods trimmed a smaller proportion of the reads, and thus explains the differences in terms of error rates.
Indeed, while most assemblies yielded from reads corrected by hybrid methods covered less than 50% of the genome, self-correction methods managed to produce corrected reads yielding assemblies covering around 88% of the genome. Once again, this can be explained by the performance drops hybrid methods suffer from when applied to large and complex datasets, and by the fact that self-correction methods benefit from the whole long-range information of the reads when performing correction. In terms of runtime, most hybrid correction tools performed slower than self-correction methods, although Jabba was faster than all self-correction methods but FLAS and MECAT , and LoRDEC was faster than Canu and LoRMA. The fastest tool on this dataset was MECAT , which managed to perform correction in under 2 hours.
5. 1, 3. 6 and 3. 7, and indicating that hybrid correction methods suffer from performance drops when the complexity of the studied organism grows. This can especially be seen on the assembly metrics of this dataset, where self-correction methods largely outperformed hybrid correction tools. Despite this fact, it is nonetheless worth noting that, in pure terms of reduction of the error rate, hybrid tool HG-CoLoR outperform all other methods. Moreover, it is also interesting to note that, while all hybrid tools could directly be applied to this ultra-long reads dataset, self-correction methods FLAS and MECAT required these ultra-long reads to be filtered out in order to run, although this did not impact their performances. As a result, a hybrid tool such a HG-CoLoR could be preferred when the aim is to lower the error rate as much as possible, even though it is much more time consuming than self-correction tools, which are not only faster, but should also be privileged when downstream analyses include assembly.
For each of these thirty tools, we precisely describe the way they perform correction, the manner in which they employ the aforementioned methodologies, the algorithmic specificities they rely on, as well as the specificities of the corrected reads they output, that is to say, whether they report corrected bases in a different case, or perform trimming or splitting of uncorrected reads regions. As a result, we present the most comprehensive survey on long-read error correction up to date. Additionally, we also gathered a total of 19 different datasets, composed of both simulated and real data, both PacBio and ONT data, and displaying various error rates, sequencing depths, read lengths, and ranging from small bacterial to large mammal genomes, in order to precisely assess the efficiency of all the tools we described.
This way, we not only provide a complete description of existing tools, but also the most comprehensive benchmark on long-read error correction up to date. These experiments allowed us to study the precise behavior of the different tools and methodologies according to the characteristics of the long-reads datasets. For instance, they show that de Bruijn graphs based methods tend to split a large proportion of the reads, when they do not manage to find paths, but can also extend the reads beyond their initial extremities when non-branching paths can be followed. More practically, these experiments also allow us to provide a few guidelines regarding the best performing tools according to the datasets characteristics. More precisely, these results show that, when the error rate of the reads is high, above 30%, hybrid correction methods largely outperform self-correction methods, even when the sequencing depth is high.
However, when the error rate is moderately high, around 18%, our experiments show that hybrid correction outperforms self-correction when the sequencing depth is low, but that self-correction tends to be more efficient as the sequencing depth grows. Finally, when the error rates are much lower, and reach 12% or less, our experiments show that self-correction performs best, even on low coverage datasets. Additionally, the complexity of the studied organisms was shown to have drastic impacts on the quality of the correction. More precisely, hybrid correction methods suffered from severe performance drops when applied to more complex and repeat-rich organisms, while the impact on self-correction methods was more balanced. Our experiments show that this especially impacts assembly results, and metrics reported on a H.
sapiens dataset showed that all self-correction methods largely outperformed hybrid correction in terms of assembly quality. It is also worth noting that all our experiments were performed using default parameters. Fine tuning of the parameters for each tool might thus slightly improve their performances. However, our experiments and results nonetheless give a reliable trend as to which tool or type of tool performs best. Although our work represents the most comprehensive survey and benchmark on long-read error correction up to date, further work shall focus on the study of additional datasets. In particular, the error rates of the datasets we studied here no longer properly reflect the error rates of both PacBio and ONT latest sequencing experiments, which now reach around 8% or less. However, since results on 12% error rates datasets indicate that self-correction is best suited, the further reduction of the error rates should only further confirm our conclusions.
Virtual reality (VR) head-mounted displays (HMD) appear to be effective research tools, which may address the problem of ecological validity in neuropsychological testing. However, their widespread implementation is hindered by VR induced symptoms and effects (VRISE) and the lack of skills in VR software development. This study offers guidelines for the development of VR software in cognitive neuroscience and neuropsychology, by describing and discussing the stages of the development of Virtual Reality Everyday Assessment Lab (VR-EAL), the first neuropsychological battery in immersive VR. Techniques for evaluating cognitive functions within a realistic storyline are discussed. The utility of various assets in Unity, software development kits, and other software are described so that cognitive scientists can overcome challenges pertinent to VRISE and the quality of the VR software.
In addition, this pilot study attempts to evaluate VR-EAL in accordance with the necessary criteria for VR software for research purposes. The VR neuroscience questionnaire (VRNQ; Kourtesis et al. , 2019b)  was implemented to appraise the quality of the three versions of VR-EAL in terms of user experience, game mechanics, in-game assistance, and VRISE. Twenty-five participants aged between 20 and 45 years with 12-16 years of full-time education evaluated various versions of VR-EAL. The final version of VR-EAL achieved high scores in every sub-score of the VRNQ and exceeded its parsimonious cut-offs. It also appeared to have better in-game assistance and game mechanics, while its improved graphics substantially increased the quality of the user experience and almost eradicated VRISE. The results substantially support the feasibility of the development of effective VR research and clinical software without the presence of VRISE during a 60-minute VR session.
In our previous work  (Kourtesis et al. , 2019a) , we have highlighted a number of suggested minimum hardware and software features which appraise the suitability of VR hardware and software. Firstly, interactions with the VE should be ergonomic in order to elude or alleviate the presence of VRISE. Also, the utilization of 6 degrees of freedom (DoF) wands (i. e. , controllers) facilitates ergonomic interactions and provides highly accurate motion tracking. Lastly, the two types of HMD that exceed the minimum standards and support 6DoF controllers are the HTC Vive and Oculus Rift; hence, the target HMD should have hardware characteristics equal to or greater than these high-end HMDs  (Kourtesis et al. , 2019a). VR-EAL is developed to be compatible with HTC Vive, HTC Vive Pro, Oculus Rift, and Oculus Rift-S.
Finally, the developer of VR-EAL attained intermediate programming skills in C# and software development skills in Unity. This was predominantly achieved by attending online specializations and tutorials on websites such as Coursera, Udemy, CodeAcademy, SoloLearn, and EdX. Also, a developer may consider established textbooks such as the "The VR book: Human-centered design for virtual reality"  (Jerald, 2015) , "3D user interfaces: theory and practice" (LaViola et al, 2017), and "Understanding virtual reality: Interface, application, and design"  (Sherman & Craig, 2018). To sum it up, the acquisition of these skills enabled progression to the next stage of the development of VR-EAL, which is the writing of the scenarios/scripts.
Development of VR software in Unity The scenario provides the main framework for developing the VR application. VR-EAL was developed to be compatible with the HTC Vive, HTC Vive Pro, Oculus Rift, and Oculus Rift-S to be aligned with the minimum hardware technological specifications for guaranteeing health and safety standards and the reliability of the data  (Kourtesis et al. , 2019a). The quality of VR-EAL was assessed in terms of user experience, game mechanics, in-game assistance, and VRISE using the Virtual Reality Neuroscience Questionnaire (VRNQ;  Kourtesis et al. , 2019b). The total duration for the VR neuropsychological assessment is approximately 60 minutes, which falls within the suggested maximum duration range for VR sessions  (Kourtesis et al. , 2019b). Long VR sessions appear more susceptible to VRISE, though, long (50-70 minutes) VR sessions which exceed the parsimonious cutoffs from the VRNQ do not induce VRISE  (Kourtesis et al.
, 2019b). For this reason, the parsimonious cut-offs for the VRNQ (see Table  2 ) will be used to ensure that VR-EAL users do not suffer from VRISE  (Kourtesis et al. , 2019b). Table  2. VRNQ minimum and parsimonious cut-offs The median of each sub-score and total score should meet the suggested cut-offs to determine that the evaluated VR software is of adequate quality without any significant VRISE. The utilization of the parsimonious cut-offs more robustly supports the suitability of the VR software. Derived from  Kourtesis et al. (2019b)  The development of VR-EAL should be proximal to commercial VR applications. The first step of the development is to select Unity's settings to support the development of VR software. For the development of VR-EAL, Unity version 2017. 4. 8f1 was used. Unity supports VR software development kits (SDK).
Navigation and interactions VR software for the cognitive sciences may require intensive movement and interactions. However, the development of such interactions demands highly advanced programming skills in C# and expertise in VR software development in Unity. Nonetheless, on Unity's asset store and GitHub's website, there are some effective alternatives that facilitate the implementation of intensive interactions without the requirement of highly advanced software development skills. The utilization of the SteamVR SDK, Oculus SDK, Virtual Reality Toolkit (VRTK) or similar toolkits and assets are options which should be considered. For the development of VR-EAL, the SteamVR SDK and VRTK were selected to develop accurate interactions compatible with the capabilities of the 6DoF controllers of HTC Vive and Oculus Rift.
The advantage of SteamVR SDK, which was developed based on OpenVR SDK, is that is compatible with both the HTC Vive and Oculus Rift, though, it does not offer a wide variety of interactions or good quality physics. Nonetheless, the VRTK mounts the SteamVR SDK and offers better quality physics and plenty of interactions that support the development of VR research software for cognitive sciences. A fundamental interaction in the VE is navigation. HTC Vive and Oculus Rift offer a play area of an acceptable size, which permits ecologically valid scenarios and interactions to be developed (Porcino   , 2017; Borrego et al. , 2018). However, the VR play area is restricted to the limits of the physical space and tracking area; hence, it does not allow navigation which is based on physically walking  (Porcino et al.
, 2017; Borrego et al. , 2018). A suitable solution is the implementation of a navigation system based on teleportation. Teleportation enables navigation exceeding the boundaries of the VR play area and delivers high-level immersion, a pleasant user experience, and decreases the frequency of VRISE. Typically, a navigation system of a VR software which depends on a touchpad, keyboard, or joystick, substantially increasing the frequency and intensity of VRISE  (Bozgeyikli et al. , 2016; Frommel et al. , 2017; Porcino et al. , 2017). In VR-EAL, a combination of teleportation and physical movement (i. e. , free movement of the upper limbs and walking in small-restricted area) is used (see  Figures 1, 2, 3,  and 6 ). The VRTK provides scripts and tools that aid the developer to build a teleportation system.
Graphics A strong placement illusion relies on the quality of the graphics and 3D objects  (Slater, 2009; Slater et al. , 2010). Correspondingly, the quality of the graphics principally depends on the rendering  (Lavoué & Mantiuk, 2015). The rendering comprises the in-game quality of the image (i. e. , perceptual quality), and the omission of unnecessary visual information (i. e. , occlusion culling)  (Lavoué & Mantiuk, 2015). The advancement of these rendering aspects ameliorates both the quality of graphics and the performance of the VR software  (Brennesholtz, 2018). Likewise, the amplified image refresh rate and resolution decrease the frequency and intensity of VRISE  (Brennesholtz, 2018). However, the rendering pipeline and shaders in Unity are not optimized to meet VR standards. The VR software developer should select different rendering options, so the quality of graphics is good and the image's refresh rate is equal to or above 90Hz, which is the minimum for high-end HMDs like the HTC Vive and Oculus Rift.
For example, the "Lab renderer" is an asset that allows VR optimized rendering and replaces the common shaders with VR optimized ones. Additionally, the "Lab renderer" supports an extensive number of light sources (i. e. , up to 15), which otherwise would not be feasible in VR. However, the developer needs to build a global illumination map (i. e. , lightmap), which substantially alleviates the cost of lights and shadows on the software's performance  (Jerald, 2015; LaViola et al. , 2017; Sherman & Craig, 2018). Usually, the lightmapping process is the final step in the development process. The acquisition of 3D objects may be expensive or time-consuming. However, there are several free 3D objects on Unity's asset store and webpages, such as TurboSquid and Cgtrader, which can be used for the of VR research software.
Importantly, the license for these 3D objects obliges the developer not to use them for commercial purposes. However, research VR software like VR-EAL is free, and research software developers usually do not commercialize their products. Although there are several free 3D objects on the websites mentioned above, it is likely that these 3D objects are not compatible with VR standards. In VR, the 3D objects should comprise a low number of polygons  (Jerald, 2015; LaViola et al. , 2017; Sherman & Craig, 2018). A decrease in polygons may be achieved using software like 3DS Max. The optimization of the 3D objects (to meet VR standards) may be achieved by simply importing the 3D objects, optimizing them, and then exporting them with a low number of polygons in a Unity compatible format (i. e. , fbx and obj).
Nevertheless, developers often aim to create large VEs such as cities, towns, shops, and neighborhoods. Each 3D object, whether it be small (e. g. , a pen), medium (e. g. , a chair), or large (e. g. , a building), may comprise several mesh renderers. Unity requires one batch (i. e. , draw call) for each mesh renderer. In large environments, the batching may significantly lower the image's refresh rate and the overall performance of the software  (Jerald, 2015; LaViola et al. , 2017; Sherman & Craig, 2018). However, assets like MeshBaker are designed to solve this problem. MeshBaker merges all the selected textures and meshes into a clone game object with a small number of meshes and textures. For example, the town that was designed for VR-EAL required >1000 draw calls. After the implementation of MeshBaker, the draw calls were decreased to 16.