dataesr/scientific-paragraphs-categorization huggingface dataset:
More than 1. 3 million station days of upper-air data and more than 1. 5 million station days of surface data have been catalogued. Of these, more than 700 000 station days of each upper-air and surface data have been digitised to date. The total number of digitised/inventoried records is 13/16 for atmospheric transmission data, 80/214 for surface data, 61/101 for moving upper-air data (i. e. data from ships etc. ), and 735/1783 for fixed station upper-air data. The largest single sources of moving upper-air data and upper-air data that have been inventoried, sorted according to the estimated number of station days, are listed in Tables  2  and 3. The metadatabase is organised according to the different types of observations, and contains a unique, project internal ID for each entry, followed by detailed information on the contact person/institution and data owner, station and/or vehicle information (the latter in the case of "moving upper-air records", e.
g. identifiers or vehicle name, measurement network, type of measurement platform, location or region, and brief reference to the source of data), the period of station operation and of the record, the frequency of observations and the estimated record volume (station days), the vertical coordinate (e. g. pressure or geometrical altitude for upperair data) or the number of resolved wavelengths (for transmission data), the available parameters and possibly instruments used, and the current status of the record (levels 0-5, correspondingly, imaged, digitised in native format, converted to common format/units, integrated to database, quality checked, and homogenised). Measurement platforms in   (1931, 1932, 1933, 1934)  ship the case of upper-air data are aircraft, pilot balloons, kites, registering balloons, captive balloons and radiosondes (see also Table  9  for the type specification in the data files).
Imaging and digitisation A web interface has been developed at UBERN to facilitate data handling and job distribution to assistants working on data digitisation. In practice, this tool proved to be very useful to keep track of the work flow and to supervise the occasionally more than 30 digitisers at this institution. It was also used by FFCUL and by the Universidad del Pacífico (a further work package partner dealing with surface data only). Seewarte (1904 Seewarte ( , 1905 Seewarte ( , 1906 Seewarte ( , 1907 Seewarte ( , 1908 Seewarte ( , 1909 Seewarte ( , 1910 Seewarte ( , 1911 Seewarte ( , 1912 Seewarte ( , 1913 Seewarte ( , 1914 Seewarte ( , 1915 Seewarte ( , 1919 Seewarte ( , 1920 Seewarte ( , 1921 Seewarte ( , 1922 Seewarte ( , 1923 Seewarte ( , 1924 Seewarte ( , 1925 Seewarte ( , 1926 Seewarte ( , 1927 Seewarte ( , 1928 Seewarte ( , 1929 Seewarte ( , 1930 Seewarte ( , 1931 Seewarte ( , 1932 Seewarte ( , 1933 Seewarte ( , 1934   logical Department (1931 logical Department ( , 1934ab, 1935ab, 1936, 1937ab) , 1937ab)  A/P/RB India and surrounding region 1/1928 12/1936 113 779.
P/R The digital images of all upper-air sources as well as the digitised data produced by all groups participating in the project (raw and corrected) were collected and stored centrally at UBERN. The images will be made available as stitched PDF files, sorted according to the different sources, via a web link from each record inside the metadatabase (http://www. oeschger-data. unibe. ch/metads). In the following subsections, we elaborate in more technical detail the imaging and digitisation process at the different institutions.
The images were then cut to a standard size using the Ir-fanView (http://www. irfanview. com) Batch Conversion function. This function was also used to quickly rotate large numbers of images. To rename the final image files to a standard notation, the program Bulk Rename Utility was used (http://www. bulkrenameutility. co. uk). The files were saved in JPEG (Joint Photographic Experts Group) format and later on merged to multi-page PDF files for distribution to the digitisers. For a relatively small part of the self-imaged, printed sources, it was possible to use optical character recognition (OCR) software for digitisation (the German Atlantic Meteor Expedition, the Bulletin of the Mount Weather Observatory, see Sect. 2, and for the supplements of the Monthly Weather Review mentioned above).
In most cases, the formats of the many relatively small sources changed too often (from source to source and even from year to year inside sources such as weather reports), or special characters appeared in the tables which made it too complicated to apply OCR. In the case of the NOAA Central Library downloads, the image quality was generally too low (resolution and contrast not optimal) for OCR to be useful and many sources were hand-written. In our case, with mainly tables and numbers to be digitised, AB-BYY FineReader 10 Professional (ABBYY, Russia) proved to be the optimal software product, with a clearly lower error rate compared to OmniPage 16 (Nuance, USA). However, for most sources, digitisation had to be done manually by keying. For this purpose, Excel templates were created for each different type of source containing an info tab with an image of a typical source page, describing exactly which data on the pages had to be digitised, and a template tab containing the data entry mask.
The pages containing the upper-air data were scanned with two A3 scanners (MFC-6490 CW printer, Brother, USA and Must Page Express A3 USB 1200 Pro, Mustek, USA) with a resolution of at least 300 dpi. Most of the images produced were in TIFF format and were used as input in an OCR program (ABBYY FineReader 8. 0 Professional Edition) to obtain Excel tables (digitisation process). The remaining files were subjected to typing in order to recover the data. Previously prepared tables in Excel format were used to transfer the OCR Excel files. The pilot balloon data were mostly subjected to this procedure, whereas the radiosonde data were all typed into previously prepared Excel tables. The initial format of the recovered tables was respected due to the fact that they were already typed with days in rows and levels in columns.
Every table corresponded to a month of data. A first level of quality control was applied when the OCR and typing processes were performed, signalling potentially wrong values. The pilot balloon data were relatively easy to deal with due to the fact that they only had wind direction and speed, but the radiosonde data contained surface pressure and temperatures, wind and geopotential values for each pressure level, as well as tropopause values. The geopotential heights were coded in the original documents, sometimes by omitting the first and last number, and needed to be restored to their real values. This process was performed by saving the monthly table values into tab-delimited ASCII (American Standard Code for Information Interchange) files, and introducing them into a Fortran program that calculated the required values and that also transformed the wind speeds published in knots into metres per second (for conversion constants used, see Table  4 ).
The Fortran program also verified that the wind direction "Calm" corresponded to 0 m s -1. The wind directions were multiplied by 10 to obtain the values in degrees (initially published in tens of degrees). The radiosondes for Maputo  (1952) (1953) (1954)  contained significant level values that we chose to digitise. The later year radiosondes did not present these levels. It was especially necessary to be careful with the vertical heights at which values of pilot balloon data were taken, as these changed from year to year, as did the time of pilot balloon ascents. All values were supplied to UBERN in tab-delimited ASCII files where one line represented an ascent, containing the year, month, day and hour (in UTC, coordinated universal time), as well as all the values measured by the device.
RIHMI At RIHMI-WDC, the collection contained book publications, which provided the most data, as well as observation booklets from different observatories and stations, and expedition reports. The fraction of data contained in typographically printed publications was about 80 % of that contained in all materials, while handwritten materials provided the remaining 20 %. Nevertheless, the paper in most of the typographical publications was aged, and the quality of the printed text was low. This led to numerous errors in the OCR processing, which could however efficiently be detected during the QC (quality check) process, as will be detailed in Sect. 4. The printed and handwritten materials were first scanned. The ELAR PlanScan C2X and ELAR PlanScan A2M highproduction units were used for scanning.
The page images of each publication were combined into electronic books of PDF format by scanning and image managing software. The OCR was the most essential part of the digitisation process at RIHMI. Like some of the other participants of the project (FFCUL, UBERN) we used ABBYY FineReader 10 Software (ABBYY, Russia). This software at RIHMI demonstrated acceptable quality of recognition and a smaller number of errors compared to other similar software products. In addition, it has a Russian language interface, thus allowing for its use by personnel that had no foreign language skills. The decisions on how to arrange the OCR process were taken based on the characteristics of the different table formats in each publication, case-by-case. Only few image sources were acceptable for recognition of whole pages, whereas most sources contained structures of tabular data that enabled recognition of separate parts (tabular blocks) of the pages.
METFR METFR has launched a national action creating an inventory of upper-air data sources and holdings in mainland France and overseas territories. Meanwhile, METFR has also started inventorying sources in the French National Archives, and in the departmental archives of the French West Indies. In total, 400 records for fixed upper-air stations have been inventoried and catalogued from the holdings of 37 French archives: the Météo-France centres and French institutional archives in mainland France, and Météo-France archives in the French overseas departments and territories. The three largest holdings of early mainland France upperair data sources are located in mainland France (library of Météo-France in Saint-Mandé, Directorate of Climatology in Toulouse and French National Archives in Fontainebleau), while the largest part of the overseas data sources is scattered all over the French overseas departments and territories (French West Indies, French Guyana, French Polynesia, Réunion Island and New Caledonia).
The largest upper-air sources encompass the Météo-France climate database and four long collections of daily reports and bulletins in hardcopy (see Table  3  Part of the collection of these original, handwritten daily reports, in hard copy for France and as microfiches for northern Africa, has been imaged in the framework of the project ERA-CLIM. Imaging was performed at several locations by private companies in mainland France, Guadeloupe, and Nouméa, with different professional flatbed scanners (300 dpi), and by Météo-France agents using cameras: eight long-term series in mainland France  (1923) (1924) (1925) (1926) (1927) (1928) (1929) (1930) (1931) (1932) (1933) (1934) (1935) (1936) , three stations in the overseas departments and territories, and three stations in the Antarctic territories.
Compte Rendu Aérologique and Compte Rendu Vent  (1948) (1949) (1950) (1951) (1952) (1953) (1954) (1955) (1956) (1957)  These are the largest sources for French (mainland France, overseas, Antarctic, ex-colonies) radiosonde data in hard copy. The pre-printed and handwritten daily aerological reports are stored on site at each upper-air station archive, with one exception: for the Antarctic territories it is archived in Toulouse. A microfiche listing, dated from 1985, with pressure, temperature, humidity and wind has been found at METFR for the period 1948-1957. A total of 43 979 microfiches have been imaged in order to recover the wind data that is still not integrated into CHUAN and the Météo-France climate database. (1929-1930, 1933-1916, 1951-1957)  This collection of daily bulletins published by the French meteorological service contains upper-air data from northern Africa.
). All these errors were detected, re-checked in the original publications or page images, and corrected in the Excel sheets. For the data manually digitised at RIHMI, pre-checking of possible values (min/max values, lists of acceptable values, etc. ) was implemented based on Microsoft Access options. Even though manual digitisation is a simpler, but slower process, it was maximally optimised. The further operations with data at RIHMI included import of Excel or Access data into SAS software: numerous transformations, data management operations, such as sorting, transposing, recalculations of variables, etc. , based on the SAS programming language. After that, SAS was used to A special, unexpected challenge for RIHMI occurred in connection with observation times. When the first version of the RIHMI data set was created, it still contained observation dates and times as "book values" (and, for the general goal of creating digital archives of the originals, it is useful to preserve these values).
The correction that was done in SAS then included several steps. The first step was to use the "book values" of year, month, day, hour, and minutes, to apply special transformation functions and to compose a BOOK DATETIME variable in the SAS format. The second step was to use the special shifting functions for date and time variables, and to produce a new UTC DATETIME variable. The third step was to apply special transformation functions to the UTC DATETIME variable in order to decompose it and to create modified UTC values separately of year, month, day, hour, and minutes. After all of the described operations, the data were output in column TXT (text file) format, so that each value had a complete set of spatial coordinates (horizontal and vertical) and temporal coordinate (unified UTC).
The new data are being integrated into CHUAN V2. 0, applying quality control and homogenisation methods introduced in Wartenburger et al. (  2014 ), and will presumably be assimilated into upcoming, full reanalyses at ECMWF and at similar centres. Furthermore, the University of Vienna is undertaking an effort to homogenise a combined upper-air data set merged from CHUAN including the new ERA-CLIM upper-air data plus IGRA  (Durre et al. , 2006)  and the ECMWF upper-air holdings, using well established homogenisation methods  (Ramella Pralungo et al. , 2013; Haimberger, 2007; Haimberger et al. , 2012). This data set will encompass pilot balloon, kite and radiosonde data. The ERA-CLIM data are available on the PANGAEA data repository (doi:10. 1594/PANGAEA. 821222). The historical upper-air data rescue efforts will continue in the follow-up project ERA-CLIM 2.
On the one hand, the related ERA-CLIM 2 activities will focus on the remaining data that have been inventoried, but not yet digitised during ERA-CLIM (see the ERA-CLIM metadatabase, including the whole record inventory on http://www. oeschger-data. unibe. ch/metads). On the other hand, additional data sources have already been identified in different archives and libraries (such as the MetOffice library, the Karlsruhe Institute of Technology central library, or at the Federal Maritime and Hydrographic Agency of Germany), or even imaged (e. g. at the Swedish Meteorological and Hydrological Institute).
Acknowledgements. All authors except R. Wartenburger received funding from the  EU   FP7  project  ERA-CLIM  (Grant  265229 ). R. Wartenburger was funded by the  SNF  project  EVAL-UATE. Upper-air data recovered by FFCUL was kindly provided by the  Instituto Português do Mar e da Atmosfera through their Anuários Climatológicos de Portugal (IV Parte -Territórios Ultramarinos -Observações de Altitude ). Edited by:  G. König - Langlo National Research Institute of Meteorology: Bulletin of the Upper Air Current Observations, Vol. I, Academia Sinica, Pei-Chi-Ko, Nanking ,  1930. National Research Institute of Meteorology: Bulletin of the Upper Air Current Observations, Vol. II, Academia Sinica, Pei-Chi-Ko, Nanking ,  1931. National Research Institute of Meteorology: Bulletin of the Upper Air Current Observations, Vol.
Generic integration process There are many kinds of application-specific integration technologies, but also for the more generic technologies there is a large freedom in integration concepts, and different foundries will have different integration technologies, even if they are functionally equivalent. In this section we will explain the basics of a generic photonic integration process using the COBRA process as a representative example; it was the first generic process that offered an MPW service and most of the process and building block information is already in the public domain. The integration process can roughly be subdivided into four process modules, which are illustrated in figure 8 and will be described below in broad outline. We will also indicate some new developments that are presently underway or planned for the continued evolution in generic processes at the JePPIX foundries Oclaro, Fraunhofer HHI and SMART Photonics.
Waveguide etching The second process module is the waveguide etching. In the COBRA process four etch steps are used for creating four different etch levels, as shown in figure  8(b ). The deepest level stretches into the substrate and is used in the deep etched waveguides, in deep-etched phase modulators and detector sections (reverse biased SOA sections), and for providing high index contrast. The deep etched waveguides are used where high contrast is essential, e. g. in small-radius bends and in compact MMI-couplers, MMI-reflectors and AWG-(de) multiplexers. The shallow etch level is used for defining shallow etched waveguides and amplifier sections, where etching through the active regions would cause increased surface recombination. Shallow waveguides show lower propagation losses due to reduced sidewall scattering; they are used for low loss interconnections.
Planarization and passivation After the etching process the sidewalls of deeply etched phase modulators and detectors have to be passivated and protected as shown in figure  8(c ). In the COBRA process this is done by deposition of a polyimide layer which also reduces the height difference on the chip (planarization). The passivation layer is necessary for obtaining low dark currents in reverse biased phase modulator and detector sections. Planarization is necessary in order to facilitate subsequent lithography steps, where the large difference in height between the different mesas complicates process steps like opening of the contact regions and definition of metal contacts. The planarization layer is also used as a substrate layer for metal interconnect between electrodes and bond or probe pads.
Thus the PICWave active models include a lot of relevant physics such as carrier diffusion, spatial hole burning and current spreading. Design kits using PICWave (figure  24 ) can provide the designer detailed models of their active building blocks that behave correctly over a wide range of operating conditions. This allows even the inexperienced circuit designer to include complex components like SOAs into a design without having to be an expert in optoelectronics modelling. 7. 1. 4. ADS (Agilent). Agilent's Advanced Design System is an electronic design automation software tool for RF, microwave, and high speed digital applications. By adding custom modules it becomes possible to use ADS also for simulating complex Photonic ICs. COBRA has added many components ranging from single and multi-mode waveguides to MIRs and AWGs.
The models employ an effective index based mode solver, and the simulation results are, therefore, firmly based on the physical properties of the circuit. With this approach, it is possible to see how the circuit responds to variations in etch depths or waveguide widths. Figure  25  shows a symbolic view of a PIC in ADS. The circuit consists of an AWG with one output channel connected to a FP cavity created by cascading two MIRs. The inset shows the simulation results, and the AWG channels plus the FP response can be clearly recognized. 7. 1. 5. Mask layout tools. After a circuit has been designed and optimized we want to validate conformity to the design rules of the foundry and generate the mask layout. Within the JePPIX consortium we are working with Mask Engineer from PhoeniX Software.
Once a satisfactory design has been created with a specific foundry and package in mind, it can be transferred via the photonic design automation (PDA) framework to Mask Engineer. With Mask Engineer the mask layout can be further completed and optimized (which is necessary because the circuit simulators cannot generate the full circuit mask layout) after which the PIC design is translated into mask files. During this process, automatic data processing takes place that obeys design rules set by the foundry. For example, a waveguide may have to be wider on the mask than in the original design in order to compensate for a known amount of under etch. Or the definition of a waveguide may involve a local mask inversion. In addition, the software will perform foundry specific design rule checks (DRC) at the logical and mask levels.
The foundry may also add process control features, alignment and cleaving marks, for example in the final stage of mask assembly. 7. 1. 6. PDAFlow API. In order to allow the efficient and errorfree transfer of designs between different software packages, a standard interface, the PDAFlow API  20  , has been developed and is now available as an open standard for use by all photonics software vendors via www. pdaflow. org [91]. An API is a particular set of rules ('code') and specifications that software programs can follow to communicate with each other. It serves as an interface between different software programs and facilitates their interaction, similar to the way a (graphical) user interface facilitates interaction between humans and computers. The PDAFlow API has been developed to allow the different software tools as mentioned above to exchange data, to simplify the development of PKDs by foundries and to allow IP-block  21 (IPB) development by design houses and academic groups.
Design manual The Design Manual is a document that contains the information needed by a user to design an ASPIC on a platform. It can be provided directly by the foundry, but usually it is distributed through a brokering organization (JePPIX for the InP platforms). All relevant technical information about the building blocks that are available within the developed generic technologies at the foundries is consolidated into the Design Manual which describes the complete generic platform offering including information on the available chip sizes, packages and the use of the software tools.
For such protected or 'private' building blocks the detailed information is inserted by software owned by the foundry before the masks are fabricated. In this way, IP of the owner can be effectively protected. If the library provides a good model description of the building block the designer is not hindered by lack of knowledge of exactly what's inside the box. So far mask layout can be generated either directly in the layout software or through the PDAFlow API from the circuit simulator. Optimization of the layout for minimal area usage and/or RF constraints is still required in the layout software. Longer term, auto-layout of a whole circuit starting from a symbolic circuit description may become feasible. But at the moment the interconnect problem (routing) has too many degrees of freedom and is too complex to be solved automatically.