dataesr/scientific-paragraphs-categorization huggingface dataset:
EXPERIMENT DESIGN To evaluate the utility of VizNet as a resource for data scientists and visualization researchers, we conducted an experiment where we frst replicated the  Kim and Heer (2018)  prior study  [29]  using real-world datasets from the VizNet corpus to assess the infuence of user task and data distribution on visual encoding efectiveness. These datasets were sampled to match constraints from the prior study and ensure that participants only saw valid data. We then extended this experiment by including an additional task on outlier detection. Finally, we trained a machine learning model that learns the perceptual efectiveness of diferent visual designs and evaluated its predictive power across test datasets.
Datasets For this experiment, we sampled VizNet datasets according to a procedure that matched constraints from  Kim and Heer (2018)  and ensured that participants only saw valid data without missing values. This procedure was developed after an initial pilot study with a subset of the corpus in which all datasets were manually verifed. To begin, we identifed all datasets with more than one categorical feld and two quantitative felds (C≥1 and Q≥2). Then, we sampled all possible three column subsets with exactly one categorical and two quantitative felds (C=1, Q=2). Following this sampling, we fltered out datasets using a number of constraints. First, we rejected datasets containing any null values. Second, we required that the column names of all datasets must contain between 1 and 50 ASCII-encoded characters.
Third, we limited the cardinality (e. g. the number of unique groups) of the categorical columns between 3 and 30. Fourth, we restricted the group names between 3 and 30 characters, at least one of which is alphanumeric. Lastly, we required that each of the groups must contain 3 to 30 values. We chose these values to be consistent with the upper and lower constraints of  Kim and Heer (2018). Our sampling procedure resulted in 2,941 valid datasets from the Open Data Corpus (100,626 possible combinations), 6,090 valid datasets from Many Eyes (354,206 combinations), 1,368 from Plotly (347,387 combinations), and 82,150 from a subset of the Webtables corpus  (1,512,966 combinations). From this set of candidates, we randomly selected 200 candidates per visualization specifcation × task condition.
We use V to denote the number of visualization specifcations and T to denote the number of tasks, which leads to 60 such conditions (V × T = 12 × 5 = 60). The 200 number of datasets sampled from the VizNet corpus is consistent with the 192 datasets sampled in  Kim and Heer (2018). As a result, this sampling resulted in 200×12 = 2, 400 datasets per task, 2, 400 datasets per corpus, and 9, 600 = 2, 400 × 4 total datasets.
Procedure Identical to  Kim and Heer (2018) , we also employed a mixed design incorporating a within-subjects treatment for visual encodings and a between-subjects treatment for tasks. Each participant answered 9 questions (1 attention check and 8 real) for each of the 12 visual encodings, presented in a random order. Every participant was assigned to a specifc task. Unlike Kim and Heer (2018), we did not incorporate dataset conditions. Each dataset was selected randomly from the pool of 200 datasets per V × T condition. In order to ensure reliable human judgment, we followed the process from Kim and Heer (2018) and incorporated 12 evenly distributed gold standard tasks. The gold standard tasks presented a user with a real dataset encoded in the present visual encoding condition, and asked what information is presented in the visual channel that encodes the frst quantitative column (Q 1 ).
Extending with an Outlier Detection Task As suggested by  Kim and Heer (2018) , investigating additional task types is a promising direction of future research. In particular, tasks with more subjective defnitions, such as Cluster and Find Anomalies were not included in  Kim and Heer (2018). Nevertheless, as outlier detection is one of the most important data analysis tasks in practice, it warrants further empirical study. We extended the prior work by considering this latter task of identifying "which data cases in a set S of data cases have unexpected/exceptional values. " We generated 2,400 datasets using the sampling methodology described in the previous section. First, we presented users with a defnition of outliers as "observations that lie outside the overall pattern of distribution.
Learning a Model to Predict Efectiveness To characterize a dataset, we extracted 167 features: 60 per quantitative feld Q, 11 for the categorical feld C, 15 for the Q-Q pair, 6 for the two C-Q pairs, and 9 which consider all three felds. These features characterized summary statistics (e. g. coefcient of variance and kurtosis), statistical distributions (e. g. entropy and statistical fts), pairwise relationships (e. g. correlations and one-way ANOVA p-values), clusteredness and spatial autocorrelation. We frst decoded diversity within our space of datasets using these features. Using principal components analysis, we computed 32 principal components which collectively explain over 85% of the variance within our dataset. Then, we generated a two-dimensional t-SNE projection of these principal components, as shown in Figure  5.
It is important to note that the datasets used in  Kim and Heer (2018)   [29]  are highly clustered and separate from the datasets used within our replication. This observation is robust for diferent numbers of principal components and values of perplexity. To predict log completion time we use gradient boosted regression trees, a model with strong "of-the-shelf" performance. Training on 80% sample of the data, we were able to predict log completion times in a 20% hold-out test set with a 5-fold cross-validated R 2 of 0. 47, which strongly outperforms fit 5-fold CV R 2 = 0. 4687 baseline models such as K-nearest neighbors and simple linear regression. A scatter plot of observed vs. predicted values for the top performing model is shown in Figure  6. Learning curves in Figure  7  indicate that, despite the large number of features, our model does not overft on the training set, and that there are still gains from increasing the number of training samples.
Limitations Although we have successfully demonstrated the efectiveness of VizNet, it is important to acknowledge limitations. Replication and reproducibility are essential to advance research  [52]. In the experiment, we attempted to replicate  Kim and Heer (2018)  as closely as possible. However, due to practical constraints, we introduced clarifying modifcations to the question text and interface design. Due to variance between crowd workers, we were not able to recruit the same participants; nor do we control for question difculty, which is calibrated in  Kim and Heer (2018). Most of all, we did not exactly replicate the original conditions of the synthetic datasets, which would have limited the amount of real-world VizNet datasets available for sampling. Notwithstanding these limitations, our work provides an important direction to understand the opportunities and challenges faced in replicating prior work in human-computer interaction and visualization research.
With respect to extending the experiment to include an additional task, we note that outlier detection, unlike the other tasks, does not have a defned ground truth. Though we used a robust outlier detection method, there may be a limitation to any purely quantitative method that does not rely on human consensus. The lack of an objective notion of outliers and absence of a clear defnition thereof in the questions, reinforces the inconsistency between ground truth and crowdsourced labels presumably partially explaining the consistently high error rate. In the context of the machine learning model, while human judgments can play an important role in help predicting perceptual efectiveness, crowdsourced training data can be noisy. The current experiment was unable to analyze lower bound requirements of quality data, but VizNet's diverse dataset ofers such opportunity for future research.
DISCUSSION There are several important areas where VizNet makes important contributions. VizNet provides a noteworthy contribution to advance our knowledge of efective graphical perception by enabling scientifc community access to rich datasets for visualization learning, experimentation, replication, and benchmarking. VizNet ofers both the full corpus and the sampled corpus of one million datasets (VizNet 1M). It further described the dimensions, types, and statistical properties of these datasets. The voluminous collection of VizNet complements synthetically generated data. Moreover, the properties of the VizNet corpus can inform assessments of the ecological validity of other corpora from domains beyond VizNet. Implications of enabling the VizNet interface for the scientifc community. We envision that in the long run, adoption of a common corpus and benchmarks by the visualization community will facilitate the sharing and comparing of results at scale.
We have made VizNet publicly available at https://viznet. media. mit. edu. A taxonomy in VizNet is formed by splitting our corpus frst on the number of columns of a datasets, and then on the composition of column types. Therefore, we should design interactions to help users query, flter, sample datasets within this taxonomy (e. g. give me all datasets with one categorical, two quantitative, and one temporal feld). Moreover, this informs the need for supporting keyword search to allow fltering by domain, in addition to fltering on other dataset properties (e. g. give me highly correlated datasets with exactly two quantitative felds). Implications of VizNet for replication and experimentation. We replicate  Kim and Heer (2018)  to demonstrate the utility of using VizNet. Our results with real-world data are largely consistent with their fndings.
As a result of our more diverse backing datasets, however, there are statistically signifcant diferences in error rates for the complex tasks. We also note that task completion times with real data are consistently longer for all but one task. These discrepancies suggest that graphical perception studies must account for the variation found in real datasets. Kim and Heer (2018)  acknowledge this direction of future work by describing the need for investigating "all [data] distributions of potential interest. " The process of harvesting these diverse distributions would be facilitated by using VizNet. We further extend the original experiment by considering an additional "detect outliers" task, an important but subjective visual analysis task that is difcult to assess using synthetic data. Implications of VizNet for learning a metric of perceptual efectiveness.
While Kim and Heer (2018) employed a mixed efects model to analyze their results, we proposed to conceive the harvested data as a collection of (data, visualization, task) triplets, each of which is associated with efectiveness measures. Using machine learning models, we predicted the completion time with an R 2 value of 0. 47. Acknowledging the trade-of between completion time and error rate, we constructed a combined metric and achieved a top-3 prediction accuracy of 52. 48%. Despite the noise and skew of crowdsourced labels, and a relatively small sample size, these results out-perform both random chance and baseline classifers. In doing so, they illustrate the potential for learning a metric of perceptual efectiveness from experimental results. We plan to extend VizNet along three major directions: (1) incorporate and characterize more datasets, (2) harness the wisdom of the crowd, and (3) develop active learning algorithms for optimal experiment design.
Incorporate and characterize more datasets. VizNet currently centralizes four corpora of data from the web, open data portals, and online visualization galleries. We plan to expand the VizNet corpus with the 410,554 Microsoft Excel workbook fles (1,181,530 sheets)  [8]  extracted from the ClueWeb09 web crawl 1. Furthermore, Morton et. al. [50]  report 73, 000 Tableau workbooks and 107, 500 datasets from Tableau Public, which could be integrated into VizNet. Lastly, we plan to incorporate 10, 663 datasets from Kaggle 2 , 1, 161 datasets included alongside the R statistical environment 3 , and to leverage the Google Dataset Search 4 to source more open datasets. In the future work, we plan to characterize the semantic content within column and group names using natural language processing techniques such as language detection, named entity recognition, and word embeddings.
Moreover, as we describe the features of datasets within the VizNet corpus, we can characterize the bias between corpora in terms of dimensions, type composition, and statistical properties of columns. This will enable us to systematically study the extent to which these corpora difer. The existence of such bias between corpora is clear from the previous data section § 3. A clearer understanding of between-corpus bias could inform future techniques for sampling from the VizNet corpus. Harness the wisdom of the crowd. Domain specifc crowdsourcing platforms such as FoldIt, EteRNA, Galaxy-Zoo, and Game with Purpose, have incentivized citizen scientists to discover new forms of proteins  [14] , RNAs  [36] , galaxies  [40] , and artifcial intelligence algorithms  [72]. We envision VizNet will enable citizen scientists and visualization researchers to execute graphical perception experiments at scale.
In recent years, crowdsourcing has been pivotal in the creation of large-scale machine learning corpora. Daemo  [21] , a self-governed crowdsourcing marketplace, was instrumental in the creation of the Stanford Question Answering Dataset (SQuAD)  [55] , whereas MTurk was used to curate the ImageNet dataset  [17]. The efectiveness of the crowdsourcing has also been exemplifed in our experiment while collecting the human judgments for the critical evaluation of visual designs. It is interesting to note that some of the crowd workers enjoyed the intellectual aspect of the experiment, as illustrated by 1    [31] , crowd work quality improvements  [19, 35] , and incentive design  [22, 72]. Develop active learning for optimal experiment design. Although gathering human-judgment labels for each triplet is costly, it is possible to learn the efectiveness from labeled triplets to predict labels for unseen ones (see section § 5).
CONCLUSION Large-scale data collection eforts for facilitating research are common across sciences and engineering, from genomics to machine learning. Their success in accelerating the impact of research in respective felds is a testament to the importance of easy access to large-scale realistic data as well as benchmarking and performing research on shared databases. As the feld of data visualization research grows from its infancy, we expect the need for and utility of large-scale data and visualization repositories to signifcantly grow as well. VizNet is a step forward in addressing this need.
https://lemurproject. org/clueweb09. php 2 https://kaggle. com/datasets 3 https://github. com/vincentarelbundock/Rdatasets 4 https://toolbox. google. com/datasetsearch Performance curves obtained by semi-supervised active learning and supervised learning over 10 iterations. post experiment responses: (1) 'I found this survey entertaining, it makes you think and use your head' (2) 'It is a very interesting survey to carry out since it promotes the capacity of analysis I congratulate you for that'. A natural progression to harness crowdsourcing mechanisms for VizNet includes extension of literature on task design.
https://lemurproject. org/clueweb09. php 2 https://kaggle. com/datasets 3 https://github. com/vincentarelbundock/Rdatasets 4 https://toolbox. google. com/datasetsearch Performance curves obtained by semi-supervised active learning and supervised learning over 10 iterations. post experiment responses: (1) 'I found this survey entertaining, it makes you think and use your head' (2) 'It is a very interesting survey to carry out since it promotes the capacity of analysis I congratulate you for that'. A natural progression to harness crowdsourcing mechanisms for VizNet includes extension of literature on task design 0. 615 0. 610 Prediction Accuracy 0. 595 0. 600 0. 605 0. 585 0. 590 Semi-Supervised Active Sampling Supervised Learning 1 2 3 4 5 6 7 8 9 10 Iteration Figure 8:.
, 2015; Issarny et al. , 2016; Lefevre & Issarny, 2018). The Ambiciti app has been available for both iOS and Android devices since 2016. Among other features, the app monitors the personal exposure to noise measured in dB(A) (see Figure  2 ). The default app settings provide average noise levels from automatic 5 second-long measurements every 5 (Android) or 7. 5 (iOS) minutes, while the user may change the duration and frequency of the measurements in the parameter section. The heterogeneity of the embedded sensors requires the calibration of the app so as to ensure the sufficient accuracy of the reported measurements. We analyzed that the bias for the phones of the same model is almost identical  (Ventura et al. , 2017). This leads us to maintain the known biases in an Ambiciti database so that they can be reused as default bias values as far as possible.
MPS as an opportunity to enhance our knowledge of the acoustic environment MPS technologies can contribute to conceive locally-anchored and relevant strategies for a better understanding and management of urban noise. However, the social practices and motivations of the MPS users impact directly upon the quantity and quality of the noise data that is gathered. Specific and formally supported initiatives focused on noise sensing can increase the sensitization and involvement of urban citizens. Still, the characterization of the urban noise environment benefits from the aggregation of complementary datasets that are generated using a combination of technologies (spanning MPS, high-end sensors, simulation). The collection of noise measurements using MPS is one such data source, provided the users gather the data in the appropriate context.
The challenge of monitoring actual urban activity Several mathematical modeling software are able to produce urban noise maps, e. g. , SoundPlan 1  and CadnaA 2  , released under commercial licenses, and the free and opensource NoiseModeling 3  suite. These tools implement complex algorithms for computing the spatial and temporal propagation of noise from given sources. The underlying models use many global structural characteristics (roads, buildings, specific sites) and recurring activities (traffic flow and speed, the composition of the vehicle fleet) along with time period and associated meteorological conditions. The modeling provides estimated equivalent noise levels (LAeq) data, covering the whole urban area, with variable resolutions and accuracies  (Gille, Marquis-Favre, & Weber, 2017; Halim, Abdullah, Mohd.
Each user generates about 200 individual noise levels measurements along the route. All the participants share their own measurements, which are collected at the Ambiciti server. The server then processes (through filtering, assimilating, interpolating) the data to build a collective, optimized noise map of the walked area. Finally, the resulting enhanced noise map of the area (on the basis of the filtered and assimilated data) is presented so that the participants may witness and exchange about the contribution of their collective data sensing. As part of the Ambiciti workshops, we interviewed individual users, representatives of local civil organizations, and members of institutional bodies. Our aim was to get feedback about the actual relevance of MPS in specific situations and to evaluate the political, institutional and socio-economical characteristics of the local contexts.
This filtering resulted in keeping only 198,197 measurements (less than 10% collected) out of the initial dataset, to aggregate with the other types of noise data. This points out the rather low reliability, and hence usability, of MPS noise measurements if gathered passively in the background. It is interesting to further analyze the "quality" of the crowdsensed data in the (default) passive background versus proactive journey mode, provided in the two rightmost columns of Table  1. We see that while the Journey mode contributes with quantitatively less data that the background mode (23,469 out of the 2. 4 million measurements under scrutiny), the ratio of qualitatively relevant data is significantly higher with the Journey mode (11,669 data points that is about 50% of the overall data points in the Journey mode versus less than 1% for the automatic measurements).
Acknowledgements: The authors would like to thank : Ambiciti  for granting us access to the specific data as part of research collaboration; the  City of Helsinki, notably Forum Virium ; and  Professor Kirsi Pynnönen-Oudman  who is managing the experiment for the zoo. The authors also acknowledge the support of the  CityLab@Inria research program  on digital solutions contributing to socially and environmentally sustainable cities.
However, the temperature of formation of lizardite and chrysotile can range from 85 to 185 °C, while the temperature of formation of antigorite ranges from 220 to 460 °C44. Because high-salinity geochemical databases are not reliable beyond 250 °C, our study focuses on lizardite. Second, salt formation depends on temperature and pCO 2. Because serpentinization occurs at a large range of temperatures and the amount of CO 2 released by the mantle is not well constrained, we investigated the possible ranges of the two parameters. The results show that (1) salt and brines form in all cases, but their composition varies with temperature and pCO 2 , (2) low pCO 2 and low temperature enhance soluble salt precipitation, (3) a high temperature does not necessarily produce more salt (salts are generally more soluble at high temperatures) but does produce more concentrated brines, and (4) additional soluble salt formation occurs when seawater cools the brines close to the seafloor.
Methods Below, we summarize the methodology applied to obtain the results presented in this article. All calculations were performed at thermodynamic equilibrium. When available in the Thermoddem database  51  , missing phases or better temperature dependences were added or replaced in the Pitzer database. The seawater composition was taken from Nordstrom et al. 52. We checked the consistency of the phases with the temperature: bischofite precipitates below 117 °C, MgCl Database limit. The validity of thermodynamic databases used for geochemical modelling in high-salinity conditions does not exceed 250 °C. Therefore, serpentinization modelling cannot be performed beyond this temperature without introducing large uncertainties. To our knowledge, the Pitzer database is the largest and most consistent database that allows consideration of temperature and pressure variations in geochemical calculations at high ionic strength (>1.
5M). Some aqueous ion pairs and complexes, in particular those related to the amphoteric behaviour of carbonates and sulfates, are included in the database and therefore in the model. S1  and S3  in Supplementary Information 3 summarize the initial brine composition. The Pitzer coefficients are one of the main potential sources of uncertainty in this thermodynamic approach. For the sensitivity tests, we compared the results obtained with the Pitzer database  50  (distributed with PhreeqC) and those obtained with the PhreeSCALE code  55  and its associated thermodynamic database at 25 °C (data are not available at higher temperatures in the database associated with PhreeSCALE)  8, 55  but found no considerable difference. Furthermore, elements such as aluminium, ferric iron or sulfur were not considered at either ambient temperature or temperatures above ambient temperature.