dataesr/scientific-paragraphs-categorization huggingface dataset:
Update of Node-Software The common Python software on which most data-nodes are built relies on a small suite of third-party libraries. We wish to keep this infrastructure baseline constant as long as possible to enhance stability and reduce maintenance for the node operators. However, it is now necessary to upgrade to newer versions of the libraries. Notably, Django 58 v2, which is no longer supported, must be replaced with v3. This may force some changes in the customisation of each node since Django upgrades are rarely compatible in all APIs. Provided that old and new versions of the node-software support the same version of VAMDC standards, we do not require all the data-nodes to be upgraded at the same time.
Applications and User Case Studies Any database lives from the use of its data, the more the better. EPN-TAP  [167]  VO service from IVOA registries) and identifies atomic and molecular species through its link to databases such as JPL  [15]  and CDMS  [74]  via a SQLite database. Species identification may also proceed via a direct access to VAMDC for any available spectroscopic database using TAP (the Table Access Protocol), which is based on the IVOA format. Modelling is performed in Kelvin units, and a conversion tool for Jansky units is available. LTE (Local Thermodynamic Equilibrium) and non-LTE modelling tools are provided. CASSIS gives ∼ 80 collision files constructed from the LAMDA  [168]  and BASECOL  [29, 30]  databases, as well as from local files, such that their quantum numbers, Einstein coefficients, upper energy levels and rest frequencies match the request made within CASSIS.
XCLASS -eXtended CASA Line Analysis Software Suite XCLASS 62  [170]  is a full message passing interface (MPI) parallelised toolbox for the Common Astronomy Software Applications package (CASA), aimed at fitting spectral line data from astronomical sources observed both with interferometers or single dish telescopes. XCLASS models a synthetic spectrum that is automatically compared to the data with the aim of providing a measurement of physical quantities, such as the temperature, molecular abundance and velocity. Molecular data required by XCLASS are taken from an embedded SQLite3 database containing entries from CDMS  [74]  and JPL  [15]  that is populated and updated via the Python library VAMDCLIB, which queries the data directly from the VAMDC nodes. XCLASS offers the possibility of describing molecules and radio recombination lines (RRLs) in LTE and non-LTE conditions using the RADEX 63 formalism.
Use of Stark-B Data From an analysis of applications of Stark broadening parameters  [173] , calculated with a semi-classical perturbation method  [68] , it was shown that the main users of Stark-B data are astronomers, who use them for investigation of A and B type stars, white dwarfs and hot stars in late evolution stages, as for example PG1159 type stars 64   [174]. The data most often used for this purpose primarily concern He I and Si II. Additional usage of these data is for plasma modelling in physics and technology, primarily for laser produced plasma with some emphasis on Stark broadening parameters of Zn I. An analysis of the citations to the Stark-B database shows that its data are also used for different analyses of regularities and systematic trends, as a source of a large number of homogeneous data obtained in the same way  [175].
Examples of Concrete User Issues Through the federation of resources and the adoption of common standards, the key features of the VAMDC e-infrastructure are its ability to query all resources with the same type of queries and to retrieve data identified with the same metadata. Theoretically this is a very big step towards speeding the retrieval of data in a safe and secure way. As an example, through VAMDC it is impossible to make a mistake when querying a given atomic species with its given ionisation stage: all databases will return data concerning that specific query. On the other hand, it is easy to query the wrong ionisation stage by going directly to the different databases. For example, a question from a user was: "I searched Ca + in the wavelength range from 4800 Å to 4950 Å directly from NIST, from VALD, and from VAMDC/NIST and VAMDC/VALD.
None of the transition wavelengths seem to be identical, what shall I take?" It turned out that the user had in fact queried Ca + in NIST and Ca in VALD. The VAMDC query was obviously the same both for VAMDC/NIST and VAMDC/VALD, but the values seemed to differ between NIST and VAMDC/NIST. It turned out that VAMDC transfers by default wavelengths in vacuum while a direct request to NIST provides wavelengths in air and direct request to VALD provides wavelengths in vacuum. This simple example shows the difficulty that users might encounter. In spite of major advances developed by the VAMDC federation, we have identified several user issues of such an e-infrastructure, which in turn might prevent its wider adoption by several communities. Users have their habits with queries and output formats; to a lesser extent, they question the data directly obtained from the long-established databases.
Databases sometimes offer science services that facilitate the users' work; for example, helping the user to choose the right set of the database's data for their application. Other databases may include recommendations of their own data whenever, for identical processes, the database contains different values. For example, the BASECOL database keeps historical collisional sets even when new data sets are included for the same process, and data sets are labelled as recommended or not. This local feature is not currently transferred through VAMDC. Some knowledgeable users would like that the VAMDC portal provide the functionality to choose the databases prior to specific queries on species and processes, thus avoiding unnecessary waiting time. Users deploying applications such as CASSIS (Section 4.
1) would appreciate the option to make total or partial copies of the databases locally in order to speed up their application. Such copies would be coupled to a rapid search for the availability of updates. Some users would appreciate a lighter format than the XSAMS format. We can therefore identify the following non-exhaustive and general issues to address in the future: (i) trust in a new numerical environment; (ii) the ability to quickly understand and compare data; (iii) the ability to choose the right data for its own usage and improvement/development of tools. Some of the future lines of work are outlined in Section 6. Closer collaborations between the VAMDC community and different user communities will be addressed through a VAMDC users committee whose representative will be part of our boards.
Node Data Standardisation and Quality Control An interesting impact that VAMDC has on participating databases is the pressure to introduce accurate and complete descriptions of data elements stored on individual nodes. Such descriptions are necessary for uniform representation of A&M data in XSAMS. In practice the construction of the translation dictionary between the local node and XSAMS requires unambiguous interpretation of data elements. While this sounds trivial for physical parameters like energy and frequency, the task becomes increasingly more difficult when dealing with energy level classification, quantum numbers, mixed states, etc. For the nodes that store data compilations from different sources and authors (e. g. VALD) the connection to VAMDC provides a good motivation for setting internal standards.
GrafOnto Collection of Scientific Plots The construction of databases, extracted from tabular resources in spectroscopy, required VAMDC to pay attention to the spectral functions presented in the form of plots and figures. The basis for creating a database of spectral functions was a digital library of more than a thousand articles with plots and figures, containing spectral functions for the problem of continuum absorption  [178] , the properties of weakly bounded complexes, including atmospheric molecules  [179]  and the absorption cross sections in the near and far ultraviolet ranges. In these subject domains the amount of graphical information is much larger than the amount of tabular information. We identified typical examples of plots and figures under study and created ontologies  [105]  of graphical resources, tools for integrating plots and means for citation quality assessment  [107].
New On-Line Databases and Community Portal The VAMDC project has directly stimulated the creation of new online databases such as TFSiCaSDa and UHeCaSDa  [19] , and the deployment of the theoretical PAH database within VAMDC  [3, 36]  while motivating the development of an experimental databases for PAHs. The VAMDC infrastructure deployment also encouraged a similar strategy for experimental databases for solid-state data in SSHADE (Section 2. 3. 2). Collaborations with VAMDC directly led to a renewed effort to compile local data in India (see Section 2. 3. 5). The RADAM collaboration adopted the VAMDC standards and designed a RADAM portal 67 that offers access to the VAMDC connected databases in addition to some other databases to their community. The Belgian repository of fundamental atomic data and stellar spectra (BRASS)  [180] -which aims to provide the largest systematic and homogeneous quality assessment of atomic data to date in terms of wavelength, atomic and stellar parameter coverage-retrieved atomic data from various repositories and cross-matched these data.
Common standards such as the DOI are important means to reach a reasonable level of documentation and accessibility. Databases connected to VAMDC offer a reliable and established way of supplying even further processed data, and therefore, are one of the most advanced answers to the concern of sustainability. The VAMDC consortium has agreed common standards and descriptions, which serve as a one-shop place for a plethora of atomic and molecular data. By defining common schemes for the data stored in the various databases, VAMDC has reached an invaluable level of interoperability and, therefore, a unique level of sophistication. VAMDC is an open platform that welcomes databases from all over the world and, as such, provides a focal point of atomic and molecular data. As a result, VAMDC serves as a role model to structure research data on international, national and local levels.
We can cite, for instance, the DAT@OSU portal that gathers metadata for all research databases in the Bourgogne VAMDC took a leading role in the RDA-Federated Identity management Interest Group: the RDA-recommendations produced by this interest Group  [182]  incorporate ideas and needs coming from the Authentication, Authorisation and Accounting strategy we developed for the VAMDC Consortium  [183]. • Since 2016, we have worked with the IVOA on converging VAMDC and the IVOA atomic and molecular standards 68. The Group (eIRG) agreed to take an observation role. Through our participation in GEDE-RDA, the VAMDC experience and vision directly contribute to the definition of international standards  [184]  and to the definition of the European Open Science agenda  [185]. Since November 2019, the VAMDC Portal, the Species Database and the Query Store (see, respectively, Sections 3.
User Interface Our attempts to be very precise in describing atomic and molecular configurations with XML schema have had an unexpected adverse effect on the user interface implemented in the portal. At first sight the portal seems to have an amazing flexibility: one can start from specific energy levels or processes, but the flexibility is massively reduced by the rigidity of the XSAMS requirements and "restrictables"  69  supported by each node. The only field that has some rudimentary attempt in translating user input into alternative representations is the field "species" that allows auto-completion. This is made possible through the species database described in Section 3. 2. 1. In the future VAMDC will explore the possibility of creating/adopting an engine for interpretation of much more relaxed formulations of the user's request, using iterations gradually narrowing the requested data set to the user's needs.
Large Data Sets As discussed above, the current VAMDC portal is not designed to easily handle databases with many millions or even billions of transitions. At the smaller end of these, it is possible to include the database but limit the amount of data that can be downloaded. At present really large databases are simply excluded. A possible solution could be to allow asynchronous queries followed by an e-mail sent to the user with a link to the result. More concise data sets may require sacrificing the completeness of XSAMS. Formats that represent projections of XSAMS structure onto a single or multiple tables, independent tables such as FITS are not able to carry complex inter-relations of the data model. Indeed, FITS is an open standard mainly designed for image transport (as its name indicates).
It may still be used in specific cases when requested data can be arranged in a table, for example a line list with data for computing opacity tables. In this case the FITS headers could provide the metadata explaining the meaning of numerical elements. In more general cases the complexity of relations in atomic and molecular physics calls for tree-based formats like XSAMS, that is a graph oriented data model. Relational sets of tables could be the right compromise. The most concise but generally useful format may be an SQLite file containing the query results in the form of an independent, relational database. Supplying extracted data as miniature databases also provides an easy way to query within the data subset stored on the users' own computers. SQLite is a well-established tool with widespread support in scientific computing.
Semantic Search of Qualitative Tabular and Graphic Resources in Quantitative Spectroscopy A semantic search for information resources is possible for systematic data, provided there is an explicit description of their properties in the form of ontologies (logical theories)  [105]. In the Semantic Web approach  [186]  the rules for constructing such ontologies are defined in the languages of their specification  [187]. At VAMDC the quality of spectral resources is determined by the data standardisation. This includes their analysis, validity control and trust assessment (see Section 5. 1). A semantic search in molecular spectroscopy was implemented in 2007-2019 to find qualitative information resources in quantitative spectroscopy of atmospheric molecules  [188] , qualitative energy levels and molecular transitions 71  [189, 190] , as well as spectral functions  [191] , used to solve the problems of continuum absorption and to describe the properties of weakly bounded complexes.
Visualisation and Data Access from Python Good data visualisation is important, particularly in data-set exploration. For larger data sets, dynamic display becomes essential, wherein the user scrolls through the data and zooms into areas of interest. Currently VAMDC provides some static displays of data (generated on the servers as web pages) and some dynamic displays (generated by a program on the user's computer using data downloaded from VAMDC). Neither of these approaches works well with large data sets; static pages are not sufficient and downloading large and rich data sets to plot only a small fraction of the data is frustratingly slow. 70 https://ui. adsabs. harvard. edu 71 http://wadis. saga. iao. ru/saga2/ontology-3-atomic-molecules/ Plotting large data sets is a common problem in scientific computing, and in recent years some solutions have been developed.
The Python libraries Bokeh 72 , Holoviews 73 and Datashader 74 can be combined to build plots that can be generated on a server, viewed in a web-browser and that are usefully interactive even with large data. The graphics packages also work well in Jupyter notebooks 75. To exploit this, we should provide code that allows such a notebook to extract data from VAMDC, either via a downloaded XSAMS-file or directly from data nodes.
Link Data to Evaluation Ratings We foresee handling the issue of linking data to evaluation ratings performed by review panels. The intention is to facilitate user choices among data sets provided by the various databases. This will involve a scientific organisation within VAMDC, collaborations with evaluation bodies and panels and, finally, technical developments in order to use the reviewed information. In the context of Open Science, this approach may be adopted by any other discipline. The VAMDC consortium is evaluating the opportunity to put these ideas into effect as part of a future European project proposal.
Quality Status and Future It should be stressed that the VAMDC infrastructure does not collectively address the issue of the data scientific quality. The scientific quality of the data taken from the e-infrastructure is that of the individual databases. The VAMDC infrastructure is currently a technical platform that ensures that the exchanged data are compliant with the standards, in particular with the data-format standards. As described in Section 5. 1 the standardisation imposed quality/coherence checks at the individual nodes, and those checks were beneficial to the overall quality of the outputs. The data nodes must be compliant with an internal quality chart voted by the board 76. Some specific quality improvements specify that references must be attached to the data, that the transferred data are timestamped and that the various queries are stored in the "Query Store" to ensure the tracing of the data.
When uncertainties are available in the databases, this information is transferred through VAMDC. In addition, as described above, it is planned to attach external rating reviews to data: rating reviews might concern comparison of several sets of data for a single range of applications or might concern a single set of data to provide its range of applicability. Obviously we should be aware that reviews are only a guide, they depend upon the group of people who performed the review, they can rapidly become out of date and should be updated. VAMDC offers an open infrastructure that allows third parties to establish some data assessment and to provide such services. VAMDC collects data and facilitates the comparison of data from different databases that might contain identical quantities: since all VAMDC nodes produce XSAMS formatted output data, it is easy for a user to identify overlaps between databases, e.
g. by identifying when the same data units have different values. 72  We would like to stress that VAMDC provides support to the work of scientists, but that users should be pro-active with VAMDC to enrich the scientific information. The system and service can be steadily improved and augmented by collecting and providing more central statistics about the content of the attached databases. Currently the species database allows the display of the species contained in the individual databases, but it could additionally list the processes and their range of applicability. The bibliographic service will display the papers attached to the data from the different databases, thus providing another type of central information system. Additional useful central information would be to display what fraction of data content in each database is accessible through VAMDC (sometimes it is 100% of data, sometimes it is not).
Finally some users would certainly like to have a "one-stop shop" for their application and such an effort can be made by users since VAMDC offers the technical possibility to easily build secondary data sets (see the discussion of impact on the BRASS database in Section 5. 3. 2). The VAMDC infrastructure could also provide the means to display such "user oriented secondary data collections" if the users wish to make them publicly available. The VAMDC consortium could also be the place where such demands are made and then collaborative projects could provide such "user oriented data sets". The communities should be aware that the VAMDC effort has already been huge, that the current people maintain efficiently the databases and the infrastructure with little manpower and financial resources, and that communities must get involved/engaged with VAMDC if they want to ensure the service they want.
Conclusions VAMDC is a sophisticated infrastructure that makes large sets of data publicly available in a common format. A fundamental feature of VAMDC is its ability to connect more databases from atomic and molecular domains already described in VAMDC, but also from other domains. Therefore, it serves as an ideal platform for sustainability of scientific data. A crucial key to the evolution of VAMDC is its usage in application software and the involvement of users in the orientation of the VAMDC e-infrastructure platform and services. Here, we have presented a few user case studies to illustrate this. A striking point in the evolution of VAMDC is its involvement in data management at the highest international level. This places the VAMDC e-infrastructure as an international prototype and example, where general concepts are developed and applied, thus providing to the international and national communities diverse high-level expertise in data management and handling.
In this paper we have demonstrated once again the strength and coherence of the VAMDC concept and the quality and the innovation in the development of the standards and software solutions that the VAMDC consortium has delivered. It summarises the advancement of a ten-year effort of a large group of leading scientists. The paper has also shown how the VAMDC continues to question the current status and to identify current weaknesses, while outlining how it will upgrade its services (Sections 4. 4 and 6) and make future improvements (see  Section 6). Finally, we provide help to newcomers and to users through detailed documentation and tutorials 77. A 'helpdesk' 78 led by experts in the VAMDC portal and tools answers questions and links the user with appropriate organisers of relevant databases as well as supports the inclusion of new nodes into VAMDC.
with sometimes different spectroscopic parameters, hindering the request for the modelling of line profiles. In the near future CASSIS will use the SPECTCOL tool [109, 110] , developed by the VAMDC consortium, to associate spectroscopic data provided by spectroscopic databases with collisional data provided by collisional databases to obtain the most up-to-date rate coefficients. Analysis modules for line and baseline fitting, resampling, rotational diagram analysis and χ 2 minimisation with Jython61     scripts are available to find best-fit model parameters such as column density, T k (or T ex ), n(H 2 ) or source size.
Comté region in France [181]. Sustainability issues are also related to human training and capabilities. In that respect, VAMDC has trained engineers and post-docs who are still involved in the fields of software developments, database management, development of standards for the benefit of the public sector in the VAMDC collaboration and in other domains, as well as in the private sector. Impact on Open Science Initiatives and International Data Alliances VAMDC delegates are active and have a high reputation in international data alliances, where they bring the VAMDC requirements and expertise about open-data sharing: • VAMDC has been active in the Research Data Alliance since 2014 (Section 3. 2. 2), when VAMDC became an early pilot for the data-citation recommendation. Starting from 2016.