dataesr/scientific-paragraphs-categorization huggingface dataset:
Introduction This article describes the data and software that have been added, modified, or enhanced in the HITRAN (High Resolution Transmission) compilation since the previous edition of 2012  [1]  (hereafter called HITRAN2012 in the text). The line-by-line portion and the absorption cross-sections of the new edition, hereafter called HITRAN2016, have now been cast into an underlying relational database structure as opposed to the long-standing fixedlength ASCII record format. A user-friendly powerful internet tool, HITRANonline (accessible through www. hitran. org), is provided to the user to query, filter, and plot sections of the data and to retrieve outputs in a host of convenient formats (see Hill et al. [2] ). The HITRAN compilation is composed of several components that include (1) line-by-line spectroscopic parameters for high resolution molecular absorption and radiance calculations (from the microwave through the ultraviolet region of the spectrum);  (2)  infrared absorption cross-sections (generally representing absorption by molecules that have very dense spectra or many lowlying vibrational modes); (3) collision-induced absorption datasets; (4) tables of aerosol refractive indices; and  (5)  global data that apply in a general manner to the archive.
There are also citations to more detailed studies of the modifications. With the recent advances in both laboratory spectrometers and the power of theoretical treatments such as ab initio calculations, the accuracy and the access to weak, yet important, transitions have had a big impact on this edition of HITRAN. Table  3  provides an overview of changes and additions to the database with respect to the HITRAN2012 edition. The line-by-line parameters (as well as the portion of HITRAN devoted to cross-sections) have now been cast into a relational database structure, described in Section 6. 1. This underlying structure has many advantages that were not available in the previous fixed-length format of previous HITRAN editions. The database is now able to add many more parameters (becoming necessary not only for planetary atmosphere applications, but also to satisfy sophisticated remote-sensing requirements), is not limited to constraints of field length, and can accommodate functional forms for parameters.
New line-shape representations Shapes of individual transitions in molecular spectra at atmospheric conditions are mainly determined by collisions and the Doppler effect. In the simplest case these two effects can be handled with the Voigt profile, where the collisions are described just as a single-parameter exponential relaxation. In the previous editions of HITRAN, the line shapes, with a very few exceptions, were represented with the Voigt profile. It turns out, however, that this model is too simple to reproduce atmospheric spectra with accuracy at the sub-percent level (see for instance Refs. [9, 10] ). For some light molecules, in particular for molecular hydrogen (important for research on dense atmospheres of gas giants), the non-Voigt effects are much more pronounced  [6]. At this point the database provides Voigt (for every transition), and speed-dependent Voigt, Galatry  [11]  and Hartmann-Tran  [7, 8]  profiles parameters where they had been measured and validated.
Ab initio line-shape models, originating from first principles, are far too complex to be simply implemented in a line-by-line spectroscopic database. Therefore, a compromise between simplicity of the model and the fidelity of experimental spectra representation had to be found. In recent decades, a number of phenomenological models that take into account both speed-dependent effects and velocity-changing collisions had been advanced. Recently, Tran et al. developed a computationally efficient algorithm  [7, 8]  for calculating one of them, the partially-correlated Complete representation of the line in the HITRAN2004 160-character format Parameters for broadening by new perturbing species, X c , and line-mixing parameters γ X cm À 1 atm À 1 gamma_X Lorentzian HWHM coefficient (for Voigt lineshape) for broadening by perturber X n X -n_X Temperature-dependence exponent for γ X n self -n_self Temperature-dependence exponent for γ self δ self cm À 1 atm À 1 delta_self Self-induced pressure line shift δ′ air cm À 1 atm À 1 K À 1 deltap_air Linear temperature dependence coefficient for δ air δ′ self cm À 1 atm À 1 K À 1 deltap_self Linear temperature dependence coefficient for δ self δ ′ H 2 cm À 1 atm À 1 K À 1 deltap_H2 Linear temperature dependence coefficient for δ H 2 δ X cm À 1 atm À 1 delta_X Pressure-induced line shift due to perturber X Y air cm À 1 atm À 1 y_air First-order (Rosenkranz) line coupling coefficient within Voigt profile; air-broadened environment Y self cm À 1 atm À 1 y_self First-order (Rosenkranz) line coupling coefficient within Voigt profile; air-broadened environment a The third column header, identifier, denotes the names of the variables used in the new relational database structure.
b These parameters are given in Table  1  in the way they appear in the ". par" file. However, one should note that they are stored in a flexible and substantially more detailed way in the database. c The names containing X correspond to the foreign broadeners, where X is the name of the perturbing species (CO 2 , H 2 , He). Unless explicitly given, the parameters imply transition between lower and upper states; double primes (″) and primes (′) are for lower and upper energy states, respectively. speed-dependent hard-collision profile  [12]  with quadratic speed dependence  [9]. An IUPAC (International Union of Pure and Applied Chemistry) task group recommended the use of this profile for high-resolution spectra representation  [13]  and call it the Hartmann-Tran (HT) profile. Ngo et al. [14]  demonstrated that the HT profile well reconstructs the line shapes of the most important atmospheric molecules.
H 2 O (molecule 1) The HITRAN2016 edition has undergone a substantial revision and expansion of the database of water vapor. In HITRAN2012  [1]  a very large expansion of the dynamic range of the line intensities for non-deuterated isotopologues of water vapor was implemented, thanks to the ab initio calculations from the BT2 line list  [15]  and, in selected spectral intervals, Lodi et al. [16]  for the principal isotopologue and from Lodi and Tennyson for H 2 18 O and H 2.
17 O  [17]. The ab initio results were replaced with high-quality experimental or-semi-empirical data wherever possible. In HITRAN2016, we have done a similar expansion of the dynamic (and spectral) range of the singly-deuterated isotopologues. In addition, the D 2  16  O isotopologue makes its debut in the database. D 2 O has a very low natural abundance on Earth but is measurable in the atmospheres of other planets, notably Venus which has an enhanced deuterium content  [18]  raising the importance of D 2 O. Also, for fitting D-enriched HDO laboratory spectra, a good D 2 O line list is essential. For this reason, D  2  O transitions have been included using a lower intensity cutoff of 10 À 32 cm À 1 /(molecule Á cm À 2 ) once isotopic abundance is accounted for. A dedicated paper detailing the update of the water-vapor dataset in HITRAN2016 along with atmospheric and laboratory validations is planned.
Line positions and intensities This update is informed by the work of an International Union of Pure and Applied Chemistry (IUPAC) task group which produced systematic sets of empirical energy levels (and hence transition frequencies) for all the stable isotopologues of water  [19] [20] [21] [22]. In HITRAN2012 non-deuterated isotopologues already benefited from the availability of the IUPAC dataset of empirically-derived energy levels (and transition wavenumbers derived from them), and we extend this to the deuterated species here. However, some important caveats of that compilation have to be noted. a) The accuracy of some of the transitions generated from the IUPAC set may be inferior to individual high-accuracy experiments. Therefore, just like in HITRAN2012, we have given SDV 0 cm À 1 atm À 1 delta_SDV_0_X _296 Air-(Self-) induced pressure line shift in 296 K (for SDV profile) δ ′SDV (X; 296) cm À 1 atm À 1 K À 1 deltap_SDV_X_296 Linear temperature dependence coefficient for δ ( ) X; 296 SDV 0 n SDV (X; 296) -n_SDV_X_296 Temperature dependence exponent around 296 K for γ ( ) X; 296 SDV 0 Y X SDV (296) cm À 1 atm À 1 Y_SDV_X_296 First-order (Rosenkranz) line coupling coefficient within SDV b profile; air-(self-) broadened case γ X G cm À 1 atm À 1 gamma_g_X Air-(Self-) broadened Lorentzian HWHM coefficient in 296 K (for Galatry profile) cm À 1 atm À 1 Y_HT_X_296 First-order (Rosenkranz) line coupling coefficient within HT profile; air-(self-) broadened case a The third column header, identifier, denotes the names of the variables used in the new relational database structure.
Molecule Isotopologue a HITRAN2012 Spectral Coverage (cm -1 ) preference to the experimental line position data from previous HITRAN editions if the uncertainty code was 5 (0. 00001-0. 0001 cm À 1 ) or higher. Comparisons with atmospheric spectra have shown that this was a correct choice. b) The datasets constructed in Refs. [19] [20] [21] [22]  do not include experimental data from the papers published after them. Quite a few new levels have become available over the years and some reassignments were in order (see for instance Mikhailenko et al. [23]  and Liu et al. [24] ). We partially updated the IUPAC datasets here. In particular, for the HD 17 O and HD 18 O species, levels from Kyuberis et al. [25]  were used. For H 2 18 O and H 2 17 O, and to a lesser extent for H 2 16 O, levels from Mikhailenko et al.
These line lists are presented and analyzed in articles published in this issue  [25]. Fig. 2  shows an example where atmospheric retrieval from a Total Carbon Column Observing Network (TCCON)  [32]  spectrum benefits from inclusion of HDO lines in the NIR. It is also important to note a significant increase in coverage for HD 17 O and HD 18 O which were poorly represented in previous HITRAN releases. The variational (calculated) intensities described above were then replaced with intensities from available high-quality experiments, many of which were already in the HITRAN2012 database. The details will be given in a dedicated paper, but some of the experiments that represent a substantial bulk of new experimental intensity data are described below. 2. 1. 1. 1. DLR experiments. An extensive work in the spectral ranges 1850-2280 cm À 1 and 2390-4000 cm À 1 was carried out by Loos et al.
[31]. These measurements were dedicated to water vapor parameters to be used in TROPOMI/S5-P  [36]  retrievals. The analysis was also based on a multispectrum fit using the HT profile. Line positions, intensities, self-and air-broadened line-shape parameters including speed-dependence and Dicke narrowing parameters as well as their temperature dependence LED refers to the experimental data from Ref. [26]. See text for details. were retrieved in the analysis. 2. 1. 1. 2. MiKaMo16 experimental set. Recently, Mikhailenko et al. [23]  have compiled a database of measurements in 5850-8340 cm À 1 region from the Grenoble-Tomsk collaboration reported in Refs. [37] [38] [39] [40] [41]  supplemented with measurements from Sironneau and Hodges  [42]  and for strong lines from Toth  [43]  and for very weak lines from ab initio work of Partridge and Schwenke  [44, 45].
This database is here referred to as MiKaMo16. Unfortunately, due to some differences in assignments with the variational Lodi-Tennyson-like line lists described above, it was hard to take full advantage of this line list. However, tests against TCCON spectra have shown that this line list is superior to any other line list in the region of 7000-8340 cm À 1. It was therefore used in HITRAN2016 in that region as is almost everywhere in this spectral region. This, however, creates some consistency issues with assignments of energy levels throughout the database, but substantial improvement of atmospheric retrievals justifies this approach. As examples of the complexity of the update, Figs. 3  and 4  show the flow diagrams of the construction of position-intensity line lists of the principal isotopologue (only up to 8340 cm À 1 for simplicity) and for H 2 18 O/H 2 17 O respectively.
17 O line lists The air-broadened half widths (in Voigt profile formalism) for the first three isotopologues of water in HITRAN2008 and, with some modifications, in HITRAN2012 were derived using the procedure "Diet" described by Gordon et al. [46]. This procedure continues to be used in this edition with experimental outliers identified and removed. One notable example is the removal of a large portion of the half widths measured in Jenouvrier et al. [47]  from the experimental data that feeds the "Diet". Indeed, some of the recent evaluations of atmospheric spectra showed that many data in Jenovrier et al. are questionable (see for instance Armante et al. [48] ). However, although the removal of this extensive dataset from the updated Gordon et al. [46]  procedure, as was done in the GEISA database  [49]  and the MiKaMo16 linelist, have improved the residuals in some of the lines, some of the residuals became worse than in HITRAN2012.
The main problem here is due to the fact that, while there are definite outliers in the values reported in Jenouvrier et al. [47] , it is the most extensive study of the air-broadened line widths in the 4200-6600 cm À 1 region. In fact, for many transitions no other measurements exist. Removal of these measurements from the experimental dataset has invoked the use of the approximations of different levels (some are very crude) from the work of Jacquemart et al. [50]  for many of the transitions. The only solution in this case was to filter the Jenouvrier et al. dataset. This was done through direct comparisons with the TCCON spectra for some of the transitions and crosscomparison with similar rotational transitions but from other bands, accounting for vibrational dependence from Eq. (1) of Jacquemart et al.
[50]. After this filtering procedure, about 1400 measurements (out of almost 5000) from Jenouvrier et al. [47]  were retained. It is also important to emphasize that measurements by Birk and Wagner  [51]  in the ν 2 band region were given a priority and were written into the database directly as was done in HITRAN2012. Since the modeling of water vapor absorption lines using a Voigt model is no longer sufficient in many applications and the HT profile provides considerably higher accuracy, experimental HT profile parameters of  Birk et al. and Loos et al. [31, 33, 34]  in the spectral ranges 1850-2280 cm À 1 , 2390-4000 cm À 1 and 4190-4340 cm À 1 were accommodated wherever possible. The experiments and analysis procedures applied are described briefly in Section 2. 1. 1.
Line-shape parameters for the deuterated isotopologues The addition of the line-shape parameters for D 2 O follows a similar procedure as the other isotopologues; however, the available data for air-broadening of D 2 O are considerably less complete than for the other isotopologues. There are some measurements for air-broadening of D 2 O  [52] [53] [54] [55]  as well as Complex Robert-Bonamy calculations  [56]. These data amount to several hundred lines from the measured data and just over 550 lines from the theoretical calculations. Using these data, average values as a function of J ″ were determined and extrapolation to J ″ ¼ 50 was done using a third-order polynomial fit. Using these data, the half widths and some line shifts have been added to the D 2 O lines in the database using a priority scheme.
First, the measured values were added; if the measurement did not exist for the transition in question, the theoretical value was used. If neither exists, the J-averaged value was used. This algorithm allowed half widths to be added to all D 2 O lines in the HITRAN database. For self-broadening parameters of D 2 O, the work of Gamache et al. [57]  was used. These data are an extensive set of calculations for D  2  O rotational band transitions extended to 6536 lines using the partner transition rule  [58]. Using these data, a set of J-averaged half widths was determined for J ¼ 0-50  [57]. These data have been added to the algorithm that adds line-shape data to the water lines in the HITRAN database.
Therefore fits of the effective Hamiltonian or the effective dipole moment  [78] , were based on only four major isotopologues 12 C 16    Note: ISO is the AFGL shorthand notation for the isotopologue, abundance is the terrestrial value assumed by HITRAN, and Q(  296 ) is the partition sum at 296 K. Fig. 5. Overview of the line lists of stable symmetric isotopologues of carbon dioxide in HITRAN2012 and HITRAN2016. 16 O 12 C 17 O, for which measured spectroscopic parameters were available. As a result, several spectral gaps were present in HI-TRAN2012 (see for instance discussion in Refs. [79, 80] ) which represent regions where experimental data were unavailable. For similar reasons, no entries were included in the database for the 18 O 13 C 17 O, 17 O 13 C 17 O, and 16 O 14 C 16 O isotopologues (837, 737, and 646 in old AFGL notation).
The most accurate entries in HI-TRAN2012 were taken from NASA JPL measurements by Toth et al. [88] [89] [90]  and covered the 1. 6-mm and 2. 06-mm spectral regions, which are used in remote-sensing measurements. The stated 1-5% accuracy of these experimental line intensities (HITRAN uncertainty index equal to 7 and 6), has been confirmed by a number of comparisons  [82, 83, 91] ; nonetheless the rigorous requirements for part-per-million resolution in measurements of CO 2 atmospheric concentration were not achieved. Since the 2012 release of the HITRAN database, a large number of experimental and theoretical studies have been devoted to improve the knowledge of line positions, line intensities and line shapes of CO 2 isotopologues. For a comprehensive review of measurements and theoretical models see Ref  [92].
Line positions were updated with respect to the previous version of the database. The majority of lines come from the effective Hamiltonian calculations included in UCL-IAO line lists, which are based on the fits to the observed line positions collected from the literature and published in the latest, 2015 release of the CDSD-296 database  [92]. These fits were completed and updated with recent, accurate measurements performed on isotopically-enriched samples of CO 2. Uncertainties in the fitted line positions depend on the quality of the experimental data and vary from 0. 001 cm À 1 to 10 À 9 cm À 1. For asymmetric isotopologues, a number of bands are affected by strong interpolyad anharmonic resonance interactions. The effective Hamiltonian model does not include this type of interaction for the asymmetric isotopologues.
Hence in such cases, line positions were taken directly from measurements  [80, [95] [96] [97] [98] 110, 111]. The uncertainty codes for the line positions were transferred from CDSD-296 to HITRAN2016. The uncertainty code 3 (0. 001-0. 01 cm À 1 ) was given for the line positions in the 9800-10,500 cm À 1 and 11,600-12,400 cm À 1 wavenumber regions. Partition functions in the current release of the database are based on the direct summations taken from the variational calculations of Huang et al. [79]. On average, the new partition functions agree excellently with those of CDSD-296  [83] ; however they do not agree perfectly with those in HITRAN2012 (from TIPS  [112] ) and three previous editions of the database and differ at 296 K by about À 0. 3%. Although this difference is marginal, it could have an effect in the applications where sub-percent accuracy is required.
As stated above, the radioactive isotopologue 14 CO 2 , 646, has been added to the database. This is the first edition of HITRAN where radioactive species have been incorporated (also for CO, see Section 2. 5). All lines of the 646 isotopologue were taken from the UCL line lists given in Ref. [83]. Due to issues with what constitutes a so-called natural terrestrial abundance of radioactive species (which is part of the traditional definition of intensities in HITRAN, see Eq. (1) of the Definitions and Units documentation in HI-TRANonline), line intensities for these type of species are given for unit abundance; a 10 À 27 cm À 1 /(molecule Á cm À 2 ) cut-off value for the intensity has been applied. This cut-off produced 41,610 lines in the J range 0 to 114. Vibrational assignments for the 646 isotopologue were based on isotopic shifts of energy levels and respective assignments for the 626 and 636 isotopologues, and hence should be regarded as provisional.
6-mm and 2. 06-mm bands respectively, due to lingering data and/or model biases (the use of partition function HITRAN 2012/TIPS is up to 0. 3% of this factor). After these studies, an update of the multi spectrum fitting code with CDSD partition functions was done. Additional methods to adjust the intensity distribution closer to the UCL list by scaling experimental conditions within the uncertainties are under evaluation. The intensities of the band at 2. 06 mm are already within 0. 7% of Zak et al. [82] , indicating that the additional 0. 7% scaling of ABSCO cross-sections may be unrelated to intensities. These issues will be considered for future editions of the database. For wavenumbers greater than 8000 cm À 1 , two sources of the line intensities are used: CDSD-296  [92]  and the newly-generated line list in the 9800-10,500 cm À 1 and 11,600-12,400 cm À 1 regions based on the new measurements  [105] [106] [107] [108].
The uncertainty codes of the CDSD-296 line intensities were transferred to HITRAN2016. Based on the uncertainties of the line intensity measurements in the 10,700-10,860 cm À 1 region  [106] , we use uncertainty code 5 (5% À 10%) for the line intensities of the 3003i-00001 (i ¼ 1,2,3,4) series of bands and based on the uncertainties of the line intensity measurements in the 10,000-10,300 cm À 1 and 11,600-12,400 cm À 1  [107]  wavenumber regions we use uncertainty code 3 (> 20%) for the line intensities of the 4003i-00001 (i ¼ 1,2,3,4,5) and of the 6001i-00001 (i ¼ 1,2,3,4,5,6,7) series of bands. The Voigt line-shape parameters throughout the entire database were calculated using the predictive routine of Gamache and Lamouroux explained in Refs. [114] [115] [116]. For the line mixing, we now provide a code from Lamouroux et al.
O 3 (molecule 3) The spectroscopic database for ozone is important for atmospheric and environmental applications due to its critical roles in UV shielding, radiative transfer and health. It is also considered a possible bio-marker for future observations of exoplanets  [118]. In order to understand the impacts of ozone on the atmosphere and to monitor ozone a precise, knowledge of its line parameters at all wavelengths is required. The ozone parameters for this edition of HITRAN have been updated in two steps. First, the parameters of spectroscopic models, line positions, intensities, and lower-state energy levels resulting from analyses of experimental spectra have been introduced into the S&MPO (Spectroscopy and Molecular Properties of Ozone) information system  [119]. The S&MPO offers various tools for spectral simulations and for comparisons with experimental records.
Newly included lines are colored green, updated lines blue and unchanged HITRAN2012 lines grey (intensities are given in cm -1 /molecule Á cm -2 at 296 K). 2ν 3 bands, the difference in the integrated intensities, S V , exceeded 6% between S&MPO and HITRAN2012. The analysis  [128]  covering the 1856-2273 cm À 1 region also included the 3ν 2 band, which is coupled with ν 1 þ ν 3 via an accidental Coriolis interaction. Another resonance accounted for in this list explains the anomalously strong ΔK a ¼ 5 lines that have been detected in atmospheric observations by Goldman et al. [129]. Janssen et al. [125]  reported a comparative study of the sensitivity of FTIR ground-based measurements of the atmospheric ozone concentration with respect to line parameters included in the HITRAN2012, GEISA2015 and S&MPO databases in 10-and 5mm regions.