dataesr/scientific-paragraphs-categorization huggingface dataset:
CRIS in the Cloud as a Buying Solution We cited above a verbatim from the OCLC survey about "a "Buy not Build" policy" which implies that "all applications need to be cloud hosted". There are good reasons for research institutions to opt for a cloud solution, and the general benefits of cloud computing mentioned before apply of course also to the special case of CRIS for universities and other research structures. The literature about CRIS provides some examples of CRIS as a cloud solution, from both sides, from system providers as well as from customer institutions. (Lawson & Herzog, 2014)  from UberResearch, a portfolio company from Digital Science, present a shared commercial cloud-based solution for research information management, including a global shared grant database. Other commercial CRIS that are available as a cloud solution are from Elsevier (Pure) and Clarivate Analytics (Converis).
Initially, both systems were implemented only as local, on campus applications but they are now also delivered as a cloud solution, as a response to the increasing demand for software and platforms as a service, especially from smaller institutional customers with limited IT expertise. Based on empirical data on CRIS implementations and projects in German universities and research institutions, a recent survey concludes that in the near future, "an increasing part of the (research information systems) may move into the cloud, as an infrastructure-, platform-or software-as-a-service solution, similar to other parts of the campus information system infrastructure"  (Azeroual et al. 2019 ). The Spanish SIGMA consortium provides an interesting case study of the development, benefits and requirements of a CRIS solution in the cloud.
More than twenty years ago, SIGMA was created by eight Spanish universities as a non-profit software provider for student and research information systems. When SIGMA launched its first fully functional CRIS, it was a kind of "shared in-house development", specifically designed for Spanish universities, and to be implemented as part of the universities" information systems and hosted on the campus, requiring staff, infrastructure, and licenses. However, over the years, SIGMA had to admit that its solutions "designed with a very wide functional coverage, (were) suitable for the needs of large universities, but we had a big drawback, anyone who would like to use SIGMA SIS or CRIS solutions should buy the necessary infrastructure and invest on licenses that were not cheap. This made difficult for us to enlarge our number of clients"  (Cuni, 2015).
For this reason, and "in order to offer to the markets a more competitive solution", SIGMA decided to develop a SaaS version of its software, especially for "relatively small private colleges but of great reputation". In a general way, the SIGMA approach illustrates how the development of a cloud solution can be considered as a contribution to strategic marketing with the purpose of gaining new market segments and increasing the number of customers. Regarding the CRIS software, the new approach required "a review of the organization of the software offered to universities, so as to define lighter suites to be used by higher education colleges and centres, more process-oriented and focused on the end user, who would have also a more specialized and nearer support from SIGMA"  (Cuni, 2015). Today, SIGMA offers its software in a complete SaaS model for new customers and an upgrade of the implemented software to a cloud solution for the former customers.
The success is there: "Several institutions uses our software as a service"  (Guillaumet, 2015) , and today, SIGMA proposes another commercial option for their software, i. e. a consortium model  (Guillaumet, 2018). Additionally, SIGMA provides for all customers a private cloud solution whenever asked for. A virtual private cloud: this is the architecture implemented by the German Fraunhofer Gesellschaft for its institutional CRIS, to cope with their organizational structure of 72 research institutes and units and more than 25,000 staff, and embedded in a complex infrastructure of research information, publications, administrative and research data, library services and so on  (Küsters & Klages, 2018). Following  (Houssos, 2012) , cloud computing in the field of research information management offers three general benefits:  "Minimal or no need for IT expertise by organisations to run the system, very good fit for small-to-medium size institutions (and not only).
g. OpenAIRE, DART Europe, BASE, Google Scholar, PeopleFinder, college and school webpages, Scopus, Web of Science, Scival, InCites, PubMed…) and different entry points (institutional repository, library system, web services linked to the CRIS). An integrated CRIS must be able to make use of cloud-based data on research activities, and its strength, so Brennan, is its data model, i. e. the pre-defined interlinkage between several elements, and its accountability guaranteed by the institution. (Dvorak et al. 2019) provide detailed insight in the "rich variety of other information systems and services" with which institutional CRIS "typically interoperate and interact", revealing "several types of heterogeneity: the heterogeneity in the underlying technology, in the data model, the semantic gaps, the different processes".
One part of these information systems is hosted on campus as in-house systems, such as human resources and finance information systems, identity or document management systems, or the institutional open access repository. But other systems are hosted externally, in particular article-level and citation-level bibliographic databases, journal-level citation metrics sources, the global researcher profile registry ORCID, and funder research information systems. (Dvorak et al. 2019) describe how, at the Czech Technical University in Prague, data are retrieved from these cloud sources and ingested into the local CRIS, through the webservice API the database providers offer, via ad-hoc background jobs set up to contact the external database or ad-hoc imports of MS Excel files. Especially in the case of funder information system, the authors state that they "are not aware of any established, automated information interchange between institutional and funder CRISs anywhere in the world".
In any case, they conclude that the application of the standard format CERIF would be helpful for the data ingestion from the cloud and that an institutional CRIS could increase the homogeneity and quality of research information retrieved from different, external sources. Similar to the Czech case study, a survey with 160 German institutions revealed that "among the external sources, the respondents mention the Web of Science, Scopus, PubMed and the Online Computer Library Center (OCLC) and German National Library catalogues"  (Azeroual et al. 2019 ). In the environment of networking institutions, the cloud includes data from other CRIS; in other words, ingesting external data may include the aggregation of data made available by the research information systems from other institutions. Shared standard data models and standard processing of data harvesting and exchange are required  (Ghirardi & Scipione, 2014).
But because they are "randomly distributed, heterogeneous, and typically unstructured", special attention and efforts are required. "Assuming that in the future highly individualised, heterogeneous web pages of researchers and institutions will be important sources of information about their activities, (the authors) propose to develop solid, easy-to-use software tools for the extraction and provision of structured research information. Such software tools can be used to crawl the web sites of universities and research institutions and to extract metadata from them, which is then complemented and consolidated with freely available databases and authority files"  (Blümel et al. 2014).  The ingestion from external data is a challenge for data quality in general and for CRIS quality in particular.
The quality of the source data has a direct influence on the quality of the CRIS. (Stempfhuber, 2008)  was the first to address data quality as an issue for CRIS; he required that the notion of quality should be made more explicit to support the connection of distributed sources of research information at the level of the European infrastructure cloud. "All data providing systems have their own, independent data models, describing publications, projects, institutions, people etc. , and their curation and maintenance need domain-specific knowledge which must be mobilized for the processing and curation of those data that are exported and ingested into the CRIS"  (Azeroual & Schöpfel, 2019). Standardization is one major solution to reduce the impact of heterogeneity and inconsistency, with identifiers (ORCID, DOI), standard data formats (CERIF) and terminology (CASRAI).
Service Models Universities and research institutions are increasingly interested in CRIS services in the cloud. However, the choice of the specific kind of service depends on the local needs and requirements. Based on the fact that software and IT infrastructures are operated by an external IT service provider and the user uses CRIS as a service, the combination of both technologies can also be described as CRIS as a Service (CRISaaS). The architecture of CRIS consists of three different levels, which are differentiated as follows  (Azeroual & Schöpfel, 2019) :  data collection (internal and external data sources from different systems are collected),  data integration / data storage (the integration of research information is filled and stored in the CRIS by means of an ETL process (extraction, transformation, load)) and  data presentation (research information is analyzed in CRIS and provided to the end-user).
Above, we have described three different cloud service levels. Also, we can distinguish three different cloud service models for CRIS. Infrastructure as a service. The scientific institutions have already outsourced part of their data sources to an external IT service provider, so the CRIS solution for data collection usually offers vendor-specific interfaces from the infrastructure level. The CRIS process steps of data integration and data storage can be considered platforms for CRIS. The CRIS platforms are then used in the data presentation, the application layer, to display the evaluated data (Figure  4 ).
Requirements Figure  7  illustrates a complete CRIS solution in the cloud. Provision of information: Depending on the requirements of the user, information must be provided in different contexts. As a rule, this takes the form of reports and analyzes with further options for formatting. For easy sharing of documents, their results should be exportable in Microsoft Office formats. Usability: CRIS must offer a different range of functions in terms of the tasks to be performed and the addressed user group. This ranges from viewing static reports for occasional users. Despite the proposed functions, the user tools must be intuitive and without time-consuming user training to operate. Data quality: Irrespective of the chosen delivery model of a CRIS solution, data quality is an essential prerequisite for generating decision-relevant information from operational data.
Risks and Barriers Unfortunately, despite the many benefits that cloud computing brings to CRIS users, there are also disadvantages to using cloud services. We already mentioned critics from (Dvorak et al. 2019) concerning limited flexibility and autonomy. Other reasons against using cloud computing are: Security: The topic of security is a controversial topic; on the one hand users think that it is not safe to leave important data in hand of the providers, because the probability of data theft, data manipulation or data loss is high. On the other hand, data centers of cloud providers often offer a higher level of security. Data security should also be handled with great care and attention. Because security concepts must implement measures that ensure the data at a high and reliable level.
Conclusion and Outlook In less than ten years, cloud computing has become an important and significant aspect of research information management, as part of academic networking and research infrastructures, as provider of essential data for research information systems, and as a solution for these systems themselves. Our paper shows that the choice of a cloud solution for a CRIS project is not only a technical question but must be considered in a complex environment or ecosystem including different systems and tools, stakeholders and users, and legal and security issues. This global approach must constantly address the problem of the security constraint that must adapt to the dynamics of the environment, of changing needs, risks, and technologies. The finality of such an approach will make it possible to sustain the CRIS platform via a number of key parameters while ensuring operational flexibility.
Some aspects:  CRIS researchers and administrators will manage only the application layer of the research platform through the extraction and exploitation of data and results, while the cloud architecture provider will ensure the operation and maintenance of the research platform services.  The CRIS database can be segmented and deployed on multiple servers, each server responding to a specific need.  The use of cloud access procedures will not only contribute to a better management of human workload but will undeniably provide better traceability and consolidate the security protocol specific to these types of systems.
This allows him to access virtualized resources such as hard disk capacity (storage), telephone systems, network functions, etc. Furthermore, virtual servers with different operating systems or simply structured databases can be set up by the user in the lowest level. Examples of the IaaS model are Amazon Elastic Compute Cloud (Amazon EC2), Microsoft Windows Azure Platform and HP Cloud Enabling Computing. 2. Platform as a Service (PaaS): PaaS is one level higher than IaaS and provides software platforms for application developers but also, simply, to run applications. Developers can access pre-configured programming environments for specific programming languages while executing the developed software in dedicated test environments. Examples of the PaaS model are Microsoft Azure Service, Google App Engine or Force.
INTRODUCTION A primary concern in visualization is how to efectively encode data values as visual variables. Beginning with Cleveland and McGill's seminal work  [11] , researchers have studied this question of graphical perception by conducting human subjects experiments. And increasingly, researchers are seeking to operationalize the guidelines such studies produce using handcrafted rule-based systems  [49, 73]  or learned models  [18, 27, 41]. To increase the scale and diversity of the subject pool, modern studies have eschewed traditional laboratory setups in favor of crowdsourcing platforms  [24]. But a constraining factor for true ecological validity remains. Collecting, curating, and cleaning data is a laborious and expensive process and, thus, researchers have relied on running studies with ad hoc datasets.
Such datasets, sometimes synthetically generated, do not display the same characteristics as data found in the wild. Moreover, as one-of exemplars, their use makes it difcult to compare approaches against a common baseline. Large-scale databases (such as WordNet  [47]  and Ima-geNet  [17] ) have proven instrumental in pushing the state-ofthe-art forward as they provide the data needed to train and test machine learning models, as well as a common baseline for evaluation, experimentation, and benchmarking. Their success has led researchers to call for a similar approach to advance data visualization  [3, 20]. However, insufcient attention has been paid to design and engineer a centralized and large-scale repository for evaluating the efectiveness of visual designs. In response, we introduce VizNet: a corpus of over 31 million datasets (657GB of data) compiled from the web, open data repositories, and online visualization platforms.
In characterizing these datasets, we fnd that they typically consist of 17 records describing 3 dimensions of data. 51% of the dimensions in the corpus record categorical data, 44% quantitative, and only 5% measure temporal information. Such high-level properties, and additional measures such as best statistical ft and entropy, contribute a taxonomy of real-world datasets that can inform assessments of ecological validity of prior studies. We demonstrate VizNet's viability as a platform for conducting online crowdsourced experiments at scale by replicating the  Kim and Heer (2018)  study assessing the efect of task and data distribution on the efectiveness of visual encodings  [29] , and extend it with an additional task: outlier detection. While largely in line with the original fndings, our results do exhibit several statistically signifcant diferences as a result of our more diverse backing datasets.
These diferences inform our discussion on how crowdsourced graphical perception studies must adapt to and account for the variation found in organic datasets. VizNet along with data collection and analysis scripts is publicly available at https://viznet. media. mit. edu. Data visualization is an inherently combinatorial design problem: a single dataset can be visualized in a multitude of ways, and a single visualization can be suitable for a range of analytic tasks. As the VizNet corpus grows, assessing the efectiveness of these (data, visualization, task) triplets, even using crowdsourcing, will quickly become time-and cost-prohibitive. To contend with this scale, we conclude by formulating efectiveness prediction as a machine learning task over these triplets. We demonstrate a proof-of-concept model that predicts the efectiveness of unseen triplets with non-random performance.
Previous studies have evaluated how user task, data types and distributions infuence the efectiveness of charts  [56]  and visual encoding variables  [29]. Graphical perception experiments in current practice are typically conducted on single datasets with small size and variety, lacking the characteristics of real-world data. Studies based on ad hoc datasets may provide useful results but are inherently partial and difcult to generalize, reproduce and compare against. VizNet provides a corpus of real-world tables from diverse domains to make it easier for researchers to run visualization design evaluation studies at scale. VizNet is sufciently rich both in size and variety to satisfy the data needs of a substantial number of experimental designs, facilitating the comparison of and reasoning about results from diferent experiments on a common baseline.
Data Collection for Visualization Research Although researchers recognize the need for data collection and generation to facilitate evaluation across a broad range of real datasets  [59, 61] , little efort has been made to create centralized corpora for data visualization research. Beagle  [4]  has been used to scrape over 41,000 visualizations from the web. Similarly, the MassVis  [5]  database was compiled by scraping over 5,000 visualizations from the web and partially annotating them. Lee et al. [62]  recently extracted and classifed 4. 8 million fgures from articles on PubMed Central. However, these datasets do not include the raw data represented by the visualizations, limiting their utility for generalized and reproducible visualization research.
Automated Visualization using Machine Learning Data-driven models based on responses elicited through human subjects experiments are common in the psychophysics and data visualization literature. For example, low-level perceptual models such as the Weber-Fechner Law, Stevens' Power Law, the CIELAB color space, and perceptual kernels  [15]  all ft various models to empirical user data, informing low-level visual encoding design. Earlier researchers propose using such models to generate and evaluate visualizations (e. g. ,  [15, 16, 60] ). In a natural extension to these earlier ideas, researchers have recently introduced machine learning-based systems for automated visualization design. Data2Vis  [18]  uses a neural machine translation approach to create a sequenceto-sequence model that maps JSON-encoded datasets to Vegalite visualization specifcations.
Draco-Learn  [49]  learns tradeofs between constraints in Draco. DeepEye  [41]  combines rule-based visualization generation with models trained to classify a visualization as "good" or "bad" and rank lists of visualizations. VizML  [27]  uses neural networks to predict visualization design choices from a corpus of one million dataset-visualization pairs harvested from a popular online visualization tool. Results from this recent work are promising but also point at the need for large-scale real-world training data with sufcient diversity  [57]. VizNet addresses this research gap and provides 31 million real-world datasets from everyday domains and can be used for training machine learning models to drive visualization systems.
Machine Learning Corpora Recent developments of large-scale data repositories have been instrumental in fostering machine learning research. Access to rich, voluminous data is crucial for developing successful machine learning models and comparing diferent approaches on a common baseline. To this end, researchers have created centralized data repositories for training, testing, and benchmarking models across many tasks. Publicly available repositories such as ImageNet  [17] , SUN  [74] , COCO  [39] , etc. are one of the main drivers behind the rapid advances in deep learning. VizNet is informed and inspired by the digital experimentation capabilities of large-scale data repositories in machine learning research.
Corpora The frst category of corpora includes data tables harvested from the web. In particular, we use horizontal relational tables from the WebTables 2015 corpus  [6] , which extracts structured tables from the Common Crawl. In these tables, entities are represented in rows and attributes in columns. The second type of corpus includes tabular data uploaded by users of two popular online data visualization and analysis systems. Plotly  [53]  is a software company that develops visualization tools and libraries. Once created, Plotly charts can be posted to the Plotly Community Feed  [54]. Using the Plotly API, we collected approximately 2. 5 years of public visualizations from the feed, starting from 2015-07-17 and ending at 2018-01-06. The second system, ManyEyes  [71]  allowed users to create and publish visualizations through a web interface.
It was available from 2007-2015, and was used by tens of thousands of users  [50]. The third type of corpus includes public data from the Open Data Portal Watch  [48, 51] , which catalogs and monitors 262 open data portals such as data. noaa. gov from CKAN, finances. worldbank. org from Socrata, and opendata. brussels. be from OpenDataSoft. The majority of these portals are hosted by governments, and collect civic and social data. VizNet aggregates these corpora into a centralized repository. However, the majority of datasets are from WebTables. Therefore, in the following sections, we describe each corpus individually with 250K randomly sampled datasets, to avoid oversampling the WebTable corpus. We combine these datasets into a balanced sample of one million datasets, which we refer to as the VizNet 1M corpus.