dataesr/scientific-paragraphs-categorization huggingface dataset:
Related work In this section we report on previous work that has inspired us to build our system. Although no single prior system provides the full set of features that we require nor the novel features we have developed, VETA takes advantage of existing gaze data visualisation and analytic tools and extends them with novel interactive features (e. g. interactive lenses, animations, bundling, small multiples). In traditional eye-tracking analysis one would segment the session into predefined TWI, and define AOI in the workspace. Typical metrics include the time spent on each AOI during each TWI, or the number of transitions between AOIs. Current eyetracking analysis software provides tools to define these TWIs and AOIs, as well as overview visualisations such as heatmaps and gaze plots. Our system is not based on the exploration of static AOIs, but on the combination of recorded video and dynamic focus plus context (F+C) techniques such as interactive lenses to leverage the user's ability to gain a better understanding of recorded gaze datasets.
These gaze data analytics tools collect a large body of records consisting of fixation points linked by paths. These tools can visualise and extract relevant user patterns via heatmaps  (Bojko, 2009) , scanpaths  (Goldberg and Kotval, 1999; Holmqvist and Andersson, 2017) , AOIs  (Burch et al. , 2019b; Holmqvist et al. , 2003) , Sankey diagrams  (Fabian et al. , 2020)  or can automatically annotate interesting video sections  (Kurzhals et al. , 2020). Andrienko et al. used  geographical visual analytics approaches to explore gaze data based on AOI  (Andrienko et al. , 2012). Blasheck et al. investigated multiple source aggregation • • • • VETA • • • • among eye tracking data and think-aloud to evaluate an interactive system  (Blascheck et al. , 2016). Other systems use AOI and space-time cube data representations to investigate eye-tracking datasets  (Kurzhals and Weiskopf, 2013; Muthumanickam et al.
, 2006). Most latter work tended to focus on edges as nodes positions can encode data and thus represent information (which is the case for gaze trajectories). With Edgelens  (Wong et al. , 2003) ,  Wong et al. proposed  to curve the edges that went through the lens instead of removing them. The same concept was adapted to only non-local edges by  Schmidt et al. Schmidt et al. (2010). In EdgeAnalyzer  (Panagiotidis et al. , 2011) , Panagioditis et al. explored the use of multiple lenses in order to perform set operations (e. g. to filter the edges that went through two different lenses). The techniques also allow to filter inside the lens using different parameters including their direction and other non-geometrical data dimensions. Finally it allowed to bundle local edges inside the lenses.
We explore, in this paper, the use of these interactive AOIs to perform complex filtering operations and show contextual information regarding users'gaze behaviour (including their use of glances), and its spatial and temporal distribution. In addition to interactive lenses, we use other techniques to help the exploration and the understanding of gaze dataset using, for instance, animated transition  (Chevalier et al. , 2016)  and trajectories bundling  (Lhuillier et al. , 2017).
Data processing and implementation Three kinds of data are utilised: (1) eye tracking data of a sequence of fixations, each containing the time of recording, an X and Y coordinate in space, and a ∆time representing how long the subject fixated at that point. This data is stored as a CSV file, (2) a background image as a spatial reference to give the user context and (3) a video of the subjects' actions can be attached from the original eye-tracker recording. The eye tracking data can be explored without the mapped image or the video, however these provide further context for understanding the behaviour and verifying patterns and insights. Mapping gaze data onto a reference image of a real-world environment is an increasingly common technique included in eye tracking software. Tobii Pro and other eye-tracking providers allow for tracking data to be mapped onto a still image of the environment, and gaze data of computer screens will usually be calibrated to be recorded in terms of screen coordinates, while gaze data on a scrollable webpage might be given in page coordinates, accounting for scrolling.
This can be done manually, or using image processing algorithms to find matching regions on the reference image  (Bykowski and Kupiundefinedski, 2018). The mapped gaze coordinates produced by the eye tracking software can then be formatted for importing into VETA. We used Tobii Pro Studio for the initial mapping of the data and then exported to CSV files which can be imported into the VETA application. For the VETA application, we implemented the application in Processing 3. 5. 3 8  for speed of data processing as well as the flexibility of the graphical interface. The Processing Video library is used for accessing video files, the Java Swing library for file access and the Java OpenCL 9  library for accessing GPU power for implementing real-time dynamic edge-bundling. It was developed and run well on a standard computer.
Visual encoding and clutter reduction Immediately on top of the mapped image all fixations in the full dataset are displayed for context. This provides an overview of the activity to enable natural clusters to be identified instantaneously. The user has the ability to turn this context layer on and off during their exploration process. These fixations are shown on the canvas as transparent circles (with 20% transparency). The appearance of the circles can be altered to present them either equally sized or sized proportionally to the length of the fixation. The maximum circle size can also be adjusted manually. On top of the context layer, saccades between the fixations are presented as lines linking fixation locations on the canvas. We distinguish two specific subtypes of saccades that were identified during our design phase: Short-distance saccades and glances (Fig.
Exploring areas of interest Areas of interest (AOIs) are defined by the creation of interactive lenses that can be manipulated manually using the mouse. The user has the ability to create, reshape, select or delete them. They are shaped as superellipses, ranging from diamonds to rectangles, where the user has the flexibility to control the size, shape and aspect ratio. In each AOI the percentage of the fixation duration is displayed as a percentage value. As this is moved around the screen, this value recalculates on the fly. Gaze datasets are often dense with many fixations and saccades, which can create visually cluttered views. The use of interactive lenses in for eye tracking has been shown to be promising  (Krueger et al. , 2016). AOI can be used to reduce data clutter and filter the data for exploration in several ways.
Using AOI saccades can be filtered using multiple filter modes: within an AOI, around an AOI, from/to one AOI or between multiple AOIs. AOIs can also be used to remove fixations shown on the canvas to remove outliers or unnecessary data. Fixations can be removed from view using two options: within or outside an AOI. This action can be reset at anytime to return to the full dataset. All fixations within an AOI are highlighted on the canvas using white opaque circles. These are 30% of the size of the context circles shown in the background. This allows the user to identify visual clusters within the AOI easily, alleviating issues of occlusion due to overlapping fixations. Background context fixations can be turned off or on to aid comprehension and exploration at this time. When an AOI is activated the interactions between the AOI and surrounding space are highlighted in the foreground.
Provenance and sense-making To ensure that the system is useful in practice we incorporated a number of features to aid dataset comparison, user collaboration and report making. VETA provides the ability for users to explore multiple datasets in one view in juxtaposition. The screen can be divided into four small-multiple sections, and each small multiple can present subsets of the same dataset (i. e. different AOIs or TWIs) or different datasets (i. e. different participants). Selection and filtering within each small multiple can be manipulated separately or they can be explored all together. The user has the ability to zoom into one view for exploration and move back to the multi-view easily. At present VETA is limited to four juxtaposed small-multiple views but also this could be extended if necessary and if system resources allow.
The user has the ability to load different datasets and to save workspace settings, this includes all selections and filtering, including AOI, TWI, bundling and colour mode settings. Saved workspaces can be loaded into VETA at anytime and when saved or loaded they appear as a sequence of small thumbnails on the right side of the screen. This allows for a smooth exploration process being able to move between workspaces quickly. In addition to saving settings record making is essential in the data exploration process. VETA allows user-defined text notes to be created at any time, anywhere on the canvas or linked to the timeline. These notes appear as the top layer and can be edited or deleted at anytime. This not only allows the provenance of the exploration process to be recorded but allows for collaboration with others.
Analysis of day 1 Using VETA, we loaded the mapped data with the appropriate background image. We started by displaying only the short saccades (Saccades filtering), to see which area of the workspace the participant focused on the most (Fig. 4-a2 ). We could see that the participant mostly focused on 4 screens during the session (see Fig. 4 -a1 for screen number -Outcome 2). We then created an interactive AOI for each screen to get a more accurate idea of relative time spent across screens: Screen 2 (42%), Screen 7 (18%), Screen 5 (12%), and Screen 1 (11%) (Interactive AOIs). By looking at the timeline, we saw a 5 min period during which no data was recorded. By looking at the video, we can easily see that the participant took a coffee break during this pause (Video exploration). Considering the evident difference in screen focus between these times we decided to divide the session into two phases for analysis: Phase 1 before and Phase 2 after the break.
Analysis of day 2 We then loaded the data of the second day of recording and mapped this to the same background image, as the participant arranged their workspace in very similar ways on both days (Fig. 4-b2 ). When looking at the timeline, we can see there is a break similar to day 1 after the first 15 min. A quick look at the video showed us that again the participant took a coffee break (Video exploration). The overview of the short saccades seemed to be slightly different than in day 1 (Saccades filtering). We placed the same AOIs and observed different focus time proportions the participant spent around 32% of the total time on Screen 5, 17% on both Screen 2 and Screen 7 and only 6% on Screen 1 (Interactive AOIs -Outcome 2). We focused the timeline on Phase 1 (before the break), and observed a pattern similar to the previous day (Fig.
Experts' feedback To evaluate our system and gather feedback from other eyetracking data analysts, we ran 14 individual feedback sessions. Participants were all eye tracking data users (i. e. they had used eye tracking data for their own work) with different levels of expertise. All participants did one session. This study was approved by Monash University Human Research Ethics Committee (Project 23671). The purpose of the session was to show the functionalities of the tool with our dataset, understand the activities of the participant and get their feedback about the tool itself.
Procedure All the sessions were performed remotely (due to COVID-19 restrictions) using video conference software. The first stage of the session started with a brief introduction of the purpose of the tool and the study. Then, participants introduced themselves, their main activities, and how they use eye tracking (i. e. for what goal, which software tools they used, etc. ). In the second stage of the interview we showed a video of the functionality of VETA, pausing when necessary. Finally, the third stage consisted of a discussion around the different ways such a tool could help them analyse their own data. One experimenter shared their screen with the tool running to answer specific questions regarding any functionalities or demonstrate specific processes. In this session, we used data from the use case presented in Sections 2 and 5.
The same participants found our tool allows for an accurate definition of such time windows compared to current manufacturer tools. As some of the events this participant focused on can repeat several times in one session, P7 would like to be able to define several of these windows in the timeline (TWIs). Finally, 10/14 participants valued the ability to link the real video of the trial to explore their eye tracking datasets. Indeed, as P5 explained, lots of use cases imply a dynamic background, which is necessary to provide context information but hard to visualise with the data. In VETA, it is thus useful to have access to the raw video in order to ''help answer questions about specific behaviour" (P7) and to ''link the data with the precise stimuli" (P8). P7, who analysed individual pilot behaviour, also envisioned using it to first provide quick analysis of the user behaviour, before visualising the data.
The majority of participants expected to be able to get quantitative metrics about participants' behaviour (7/14). Eye tracking manufacturer software provides basic ones, but when needing more advanced metrics, participants had to develop their own tools. We asked which metrics participants would need, and we received a wide range of answers, including gaze distance inside AOIs (P2), number of transitions between AOIs (P4) and entropy (P7). Yet, they acknowledged it would be difficult to include all metrics in to one tool as people would always ask for more, and that other tools already provide these. To solve this problem, P7 suggested implementing a link from the visualisation tool to open source applications that provide eye tracking data metrics, such as Ogama. 11.
One suggested option is to link VETA to open software such as Ogama, which would allow users to define their own measures. It could also be interesting to implement more complex measures and provide sequence analysis to extract viewing patterns  (Wilson, 1998). Such measures have been shown to be beneficial in the context of air traffic control  (Westin et al. , 2019; Vrotsou and Nordman, 2020). Whilst in our use case comparing between participants was not anticipated, we did choose to implement small multiple views for comparing gaze patterns of one participant across multiple sessions and TWIs (e. g. Fig. 4 ). Through our evaluation we found that the small multiple comparisons were highly beneficial, and the ability to switch quickly between small multiple and specific context is also needed.
In addition to sharing the digital state of the system, the system has the ability to save a static image. For our motivational use case we needed to produce a report for non-expert users, so simple and good quality static imagery was needed. The ability of the system to produce simple visualisations of complex eye-tracking data was verified through our interviews. Finally, we acknowledge that there is a learning curve for the features of the system, and that some aspects are more intuitive than others, yet we found that the learning time is balanced by the time gained in the analysis of the data. In our use case (see Section 5) we found that the video link for verifying operator tasks meant this could be performed in a fraction of the time spent on similar activities in traditional software, and that many more insights could be identified through visual encoding and reducing visual clutter.
De-novo assembly methods have also been described to assemble different viral genomes present in one sample  14  but are computationally intensive. Herein, we used different mixed ratios of Delta and Omicron cell culture isolates in order to assess the performances of four different primer sets for the detection of Delta/Omicron co-infections. A coinfection score was determined to warn about probable co-infection. The prevalence of Delta/Omicron and BA. 1/BA. 2 co-infections were then estimated on a large data set of sequences obtained from random surveillance of out-patients and systematic sequencing of hospitalized patients.
Evaluation of 4 different primer sets to detect SARS-CoV-2 coinfections using WGS To simulate co-infections, Delta (B. 1. 617. 2) and Omicron (BA. 1) viral isolates were mixed using different Delta:Omicron ratios: 0:100, 10:90, 20:80, 30:70, 40:60, 50:50, 60:40, 70:30, 80:20, 90:10, and 100:0. Resulting mixes had fixed viral loads (median = 4. 2 log10 cp/ml; IQR = 0. 4, Supplementary Data 1). Four sets of primers (Artic V4 and V4. 1, Midnight V1 and V2) were used in duplicate on extracts to test the impact of PCR amplification prior to sequencing on co-infection characterization. All mixes were sequenced to 1 M paired-end reads, leading to SARS-CoV-2 genome covered > 98% with a median coverage of 2276X (IQR = 315X) (Supplementary Data 1). The evaluation of the primer sets was performed using a previously published method based on a curated list of mutations specific to Delta and to Omicron derived from co-variants  [5] [6] [7]  (Supplementary Data 1 and Fig.
1 ). More than 90% of the Delta-specific mutations were found in all mixes with the 4 primer sets. In contrast, the detection rate of Omicron-specific mutations ranged from 26% in the 90:10 mix using Midnight primers (V1 and V2) to >77% for mixes with the expected frequency of Omicron above 30% (Fig. 1A  and Supplementary Data 1). Lower detection rate of Omicron-specific mutations was associated with different primer bias preferentially amplifying Delta over Omicron (Suppl Results, Supplementary Data 2). Medians of covered allele frequency for the specific mutations were used to estimate viral frequency. Relations between measured and expected frequency were not linear (Fig. 1B ). Over-estimation of Delta was observed for all mixes with all primer sets, and especially in mixes with expected frequency of Delta under 30% and sequenced with Midnight primer sets.
1, respectively. These low ratios were associated with primer bias preferentially amplifying Delta over Omicron (Suppl Results, Supplementary Data 2). To test whether our pipeline may detect co-infection in mixes with a lower percentage of Delta at risk of leading to chimeric sequences, we performed additional mixes with Delta: Omicron ratios of 1:99 and 5:95 that were sequenced in 10 replicates (Fig. S6  and Supplementary Data 3). All chimeric sequences were detected by seqmet as co-infected, except one chimeric 1:99 mix sequenced with Midnight V1 primers classified as non co-infected (Suppl Results, Fig. S6 ). Altogether, the results of the unbiased co-infection detection scripts were consistent with the curated list approach with a better detection of low abundant BA. 1 with Artic primers.
3C ). All co-infections were confirmed by visual examination of vcf plots using the lists of specific mutations from our lineage variant database (Figs. S10-S12 ). Uniform frequencies of lineage-specific mutations along the genome were observed in each sample, except for three samples: "021228537801 (R1)" and "021229656701 (R3)" among the Delta/BA. 1 co-infections and "722000801801 (R2)" among the BA. 1/BA. 2 co-infections (Suppl Results, Fig  S13 ). In these samples, the distribution of lineage-specific allele frequencies suggests the presence of potential recombinants with different relative abundance within co-infected samples (Suppl Results, Figs. S14  and S15 ). The suspicion of a recombination event among co-infected samples was therefore established for 3 out of 53 samples (5. 6%). Delta/Omicron (BA.
61%  17. The low genetic diversity of SARS-CoV-2 observed during the pre-VOC period, the differences of genomic epidemiology depending on the geographical location, as well as the lack of standardization between the methods used may limit the interpretation of these results. More recently, a study conducted on out-patients in the USA during the same period than the present study (November 2021 to February 2022) identified 20 Delta/Omicron co-infections out of 16,386 samples sequenced. The resulting prevalence of about 0. 1% for Delta/Omicron is highly consistent with the prevalence found herein. However, no BA. 1/BA. 2 co-infection was reported by the authors as they specifically looked for Delta/Omicron co-infections using a curated list of mutations  7. In contrast, the unbiased co-infection detection script presented herein and validated on mixed Delta: Omicron isolates was defined to detect likely co-infections with genetically divergent lineages, without a priori on these lineages.
To date (2022-03-18), 2,253 Omicron sequences and 1,156 Delta sequences have been tagged as "Multiple Lineage" in GISAID, including Omicron sequences bearing S: L452R + M: I82T, and may represent Delta/Omicron co-infections, contaminations or genuine recombinants. The co-infection detection methodology presented herein highlights a key step in WGS data analysis that could be transposed for any pathogens analyzed by mapping on a single reference. The amount of work required to adapt the approach depends on the ploidy of the pathogen, its genetic diversity, and whether a pre-existing clade/lineage definition and a well-curated database of the currently circulating strains exist. Therefore, applying this approach to influenza virus, monkeypox, or respiratory syncytial viruses, for example, would be fairly straight-forward but would require experimental testing to adjust the lineage-specific mutation cutoff to the laboratory settings (level of contamination background) and the genetic distance between co-circulating lineages.
Bioinformatics Reads were processed using the in-house bioinformatic pipeline seqmet (available at https://github. com/genepii/seqmet). Paired reads were trimmed with cutadapt to remove sequencing adapters, and lowquality ends, only keeping reads longer than 30 bp  23. Alignment to the SARS-CoV-2 reference genome MN908947 was performed by Minimap2  24. Mapped reads were processed to remove duplicates tagged by picard, then realigned by abra2 to improve indel detection sensitivity and finally clipped with samtools ampliconclip to remove read ends containing primer sequences  25, 26. Variants present at frequencies of 5% or above were called using freebayes, decomposed and normalized with vt, and then filtered with bcftools to eliminate false positives  [26] [27] [28]. To detect co-infection, obtained vcf files were compared to a lineage variant database by a script, both developed internally to this end.
The database consists of vcf files listing variants found in 50% or more of 100 randomly selected sequences for a given pangolin lineage in the full GISAID data set available (extracted on 02 February 2022). The database is available at https://github. com/genepii/seqmetdb. The co-infection detection script searches for each lineage any major or minor variant matching expected variants of the putative main lineage and then searches for any minor variant matching any other lineage, excluding variants in common with the main lineage. Since the variants of the main lineage are excluded when searching the secondary lineage, variants for the secondary lineage are specific to this lineage, whereas variants for the main lineage are defining variants of the lineage. Variants were expected when they occurred on a position covered with at least 100 reads.