dataesr/scientific-paragraphs-categorization huggingface dataset:
In the current default configuration of MUST, all processes are started together as MPI processes with a common MPI_COMM_WORLD. Using the MPI interception layer PMPI, MUST then makes sure, that only the processes intended to execute the application code will continue execution. The tool processes remain in a run loop which waits for incoming events to process. Whenever application code uses MPI_COMM_WORLD in an MPI function call, this communicator is replaced by a sub-communicator representing the application processes. This is transparent for the application, which will never see the additional analysis processes. For performance reasons, the tree layoutincluding the number of layers and placement of analysis functions-is hard-coded and compiled into a specific instance of the tool. This means that with the current tool infrastructure a dynamic reconfiguration of the tree is not possible at runtime.
XcalableMP XcalableMP (XMP) is a directive-based language extension for Fortran and C languages. Like in OpenMP, parallelism is introduced by the use of directives. If all the directives are ignored by the compiler, a serial program with the same semantics and results should remain. XMP targets parallel programming for distributed memory systems, in contrast to OpenMP targeting shared-memory parallelism. The implementation of XMP in the OmniCompiler is a source to source transformation, which translates the directives into additional code and calls into the XMP runtime library. The XMP runtime library communication is mainly performed using MPI. Therefore, it is in general also possible to use MPI communication in XMP programs or link a library written with XMP into an MPI application. Listing 1 shows an example of an XMP distributed parallel source code.
The functionality described so far is called as global-view programming in XMP. In global-view programming, the application programmer does not need to care where data is located. The array is transparently distributed and accessed. In the suggested workflow, the work is performed where the data is located. Furthermore, XMP extends Coarray Fortran and makes this functionality also available in C. In XMP this is called local-view programming. To access memory on a different process in local-view programming means to explicitly specify the target process, that holds the image of interest. The code example in Listing 2 demonstrates how XMP allows using the concept of Coarray in C code. The array a is declared as a Coarray of size 10 with an image on each process. The image selector is separated with a colon in the declaration.
Programming Errors in XMP Programs For global-view programming we identified a range of possible programming errors that violate restrictions provided for the specific XMP construct. As an example, the barrier construct has the following restriction: • The nodeset specified by the on clause must be a subset of the executing nodeset. The code example in Listing 3 violates this restriction, because the task construct limits the executing nodeset to process p(0), while the nodeset specified by the on clause is the complete nodeset p. The code presents also the MPI idiom with the same semantic. Since only the process with the rank number 0 reaches the barrier, this will finally result in a deadlock for the MPI code. Besides violations against restrictions imposed by the XMP specification, we also identified possible data races for asynchronous communication.
_XMP_Barrier is the XMP runtime implementation for any explicit or implicit barrier. The graph also shows that this XMP barrier is called from two different locations-lines 15 and 23-in the source code, although the original source code only has 15 lines. This example emphasizes, that correctness analysis for XMP applications can be done at the MPI level, but is not too useful for the application developer. In other previous work  [1, 11]  we have seen that we can achieve better results concerning precision and recall if we base the analysis on the semantics of the high-level parallel programming paradigm. Furthermore, the analysis at a higher abstraction level can help to provide more meaningful error reports. In the following, we will see how this applies to XMP.
Analysis in Global-View Programming For the errors, where XMP code might violate restrictions imposed by the XMP specification, we distinguish between static Restrictions on dynamic properties depend on the specific value a variable has at runtime. We distinguish between those that involve only the local process and those that involve multiple or all processes. For global-view programming the latter is only the case for collective operations, which require consistent clauses and values among all contributing processes. As for MPI collective communication functions, we analyze this as a reduction analysis, where each node in the TBON compares all incoming events and passes one representative event to the parent node. All other restrictions on dynamic properties can be analyzed locally. Various XMP constructs have the same restriction as mentioned in Sect.
Analysis in Local-View Programming For local-view programming, our main focus is on data race detection in remote memory access. A data race is commonly understood as concurrent access of multiple execution entities to the same data in memory while at least one access is writing to memory. Concurrent access implies that there is no synchronization between the two memory accesses. We can observe two ways of access to a coarray in XMP, both can be found in the code example in Listing 2. The remote image access denoted by a[:]:[target] has write semantics on the target memory for the put operation and read semantics for the get operation. The local image access denoted by a[:] or by a  [1]  has the memory access semantics as suggested by the base language. In general, an application might access and modify the local image through a pointer to the local image.
YvetteML YvetteML (YML) is a workflow description language for technical or scientific calculation that describes dependencies among tasks. YML interprets low source code and dependencies between tasks to generate the indicated DAG and execute the task according to the DAG. YML of the original casing is, P2P tasks that are written sequentially in the language it was assumed to run in an environment or a small cluster, of tasks written in a parallel language. By using YML it became possible to run the application on a large scale system. We also developed middleware for porting. The middleware used to implement the mSPMD programming model is OmniRPC-MPI  [13]. It provides Remote Procedure Call (RPC) based on MPI and is an extension of the library OmniRPC. Our OmniRPC-MPI middleware is a workflow scheduler to control remote programs which are created for task execution by use of MPI_Comm_spawn on request.
FP2C FP2C (Framework for Post-Petascale Computing) is a development and execution environment which supports multi-program methodologies across multiple architectural levels as suggested by Dufaud et al. [3]. FP2C integrates XMP to describe tasks into the workflow environment YML. Therefore FP2C is an implementation of YML to be executed on classical HPC clusters. FP2C is composed of three layers: 1. workflow programming, 2. parallel and distributed programming, and 3. shared-memory parallel programming/accelerator. The tasks are expected to be executed on sub-clusters or groups of nodes which are tightly connected. These tasks would be hybrid programs with distributed and shared programming models. The workflow scheduler among the sub-clusters or groups invokes and manages the tasks. The YML backend implementation used for this configuration is OmniRPC-MPI to allow dynamic creation and control of MPI processes needed to executed the YML tasks on an HPC cluster.
By the use of MPI point-to-point communication, the remote program is informed about the next task to execute and also communicates back about the completion of a task. If some YML tasks need a different number of parallel processes than the previously finished task, FP2C will terminate the remote program to spawn new remote programs as depicted for task1, which is replaced by smaller remote programs to execute task5 and task4. • The language Yvette, that expresses the graph semantics within YML, is quite similar to the hardware description language Esterel and suffers from similar correctness issues. In this section, we will especially cover potential deadlocks and data races as well as some semantic issues that come with this language. • The runtime system implementation of YML could also be subject of correctness analysis.
Challenges of Analyzing the YML Runtime System Analyzing the runtime system of YML has two mayor challenges for a runtime analysis tool like MUST, which is developed to support the analysis of common HPC applications. The first challenge is to understand the difference between code that represents the YML runtime system and code that belongs to the application code, in this context the YML task code. The bigger challenge is the dynamic MPI characteristic of the YML runtime system, which dynamically creates processes using MPI_Comm_spawn, that are then integrated into the execution and should also be supervised by the analysis tool. The analysis tool would also need to understand the resulting new MPI communicators as well as the communication patterns with those spawned processes. Supporting an application that exposes such dynamic behavior is currently not supported by MUST and the underlying TBON communication layer.
For the execution of a YML task with applied MUST analysis, the remote program controlled by FP2C then needs to select the appropriate prepared configuration of MUST, which is typically done by exporting some environmental variables. Since we specifically want to analyze the YML task, but not the YML infrastructure, the MPI functions called to implement the FP2C command and control workflow should be ignored by the analysis tool. Some of those MPI functions are used to communicate with the YML scheduler, which is outside of the process controlled by MUST. Such communication to the outside would confuse some analysis performed by MUST. We can avoid analysis of such functions by directly calling into the PMPI interface for functions that implement FP2C functionality. Circumventing MUST analysis for all FP2C-owned MPI communication can result in a deadlock: As Fig.
Conclusion In this paper, we discussed how we can apply runtime correctness checking to emerging multi-level parallel programming languages which try to encounter the challenges of multi-level concurrency of exascale systems. Specifically, we looked into possible correctness issues in XMP applications, which represent the field of PGAS languages. We described how we integrated runtime correctness analysis for XMP applications into the runtime correctness checking tool MUST and therefore specified the new tools interface XMPT for XcalableMP. The workflow description language YML allows to introduce another level of high-level concurrency and therefore better exploit the massive available concurrency of exascale systems. As an example application for such a high-level concurrency workflow, we introduced the unite and conquer method UCGLE.
The University of Valladolid (UVa, Spain) manages since 2006 a calibration center of the AErosol RObotic NETwork (AERONET). The CAELIS software tool, developed by UVa, was created to manage the data generated by the AERONET photometers, for calibration, quality control and data processing purposes. This paper exploits the potential of this tool in order to obtain products like the aerosol optical depth (AOD) and Ångström exponent (AE), which are of high interest for atmospheric and climate studies, as well as to enhance the quality control of the instruments and data managed by CAELIS. The AOD and cloud screening algorithms implemented in CAELIS, both based on AERONET version 3, are described in detail. The obtained products are compared with the AERONET database. In general, the differences in daytime AOD between CAELIS and AERONET are far below the expected uncertainty of the instrument, ranging the mean differences between -1.
3 × 10 -4 at 870 nm and 6. 2 × 10 -4 at 380 nm. The standard deviations of the differences range from 2. 8 × 10 -4 at 675 nm to 8. 1 × 10 -4 at 340 nm. The AOD and AE at night-time calculated by CAELIS from Moon observations are also presented, showing good continuity between day and night-time for different locations, aerosol loads and moon phase angles. Regarding cloud screening, around 99. 9% of the observations classified as cloud-free by CAELIS are also assumed cloud-free by AERONET; this percentage is similar for the cases considered as cloud-contaminated by both databases. The obtained results point out the capability of CAELIS as processing system. The AOD algorithm provides the opportunity to use this tool with other instrument types and to retrieve other aerosol products in the future.
The calibration needed for AOD evaluation is the extraterrestrial signal of the instrument, which is normally derived using the Langley plot technique  (Shaw, 1983)  for reference instruments, or side-by-side comparison for field instruments. The reference instruments are calibrated in high altitude stations like Mauna Loa and Izaña  (Toledano et al. , 2018). Field instruments are calibrated at inter-calibration sites. In the AERONET network, calibration facilities at GSFC/NASA(Greenbelt, USA); PHOTONS/LOA (Lille, France) and GOA/UVa (Valladolid, Spain) are used for this activity. In them, the instruments are routinely calibrated and maintained to ensure data quality. The facilities at Lille and Valladolid are also part of the Aerosol, Clouds and Trace Gases Research Infrastructure (ACTRIS, www.
actris. eu), a pan-European initiative to provide open and high-quality observations of those atmospheric constituents. There is a need to evaluate the photometer data in real time and control large amounts of data generated by the network. Hence, in order to help in the management of the AERONET/ACTRIS calibration facility at Valladolid, a software tool called CAELIS was developed  (Fuertes et al. , 2017). It provides tools for monitoring the instruments, processing the data in real time and offering the scientific community a new tool to work with the data. For this purpose, CAELIS contains a database and a web interface to visualize raw data and meta-data, provides processing of sky radiances and supports the monitoring of the instrument performance. This tool is capable to detect several technical problems with the network instruments by an automatic warning system based on the CAELIS's meta-data and products, which allows a quick response to detect and solve operation problems.
In this framework, the calculation of the AOD is important because several checks can be applied to the data to ensure the reliability of the measurements. Moreover, CAELIS also intends to be a framework to facilitate research activities, being the AOD a key product to the present and future investigations. Therefore, the main objective of this paper is to develop and describe the implementation of the aerosol optical depth and cloud screening algorithms in CAELIS. The AOD product must be robust and operational: it must work for any site, instrument configuration, etc. , and even with incomplete or damaged raw data files, which it should adequately flag if it is the case. CAELIS is focused on AERONET and the Cimel photometer simply because that is the framework of our calibration activity.
And this is actually the best reference point that we have in order to validate the AOD algorithm. Therefore we will compare the results with those provided by the AERONET version 3 AOD algorithm  (Giles et al. , 2019) , including the cloud screening, which is necessary because AOD can only be derived when the Sun or Moon are not obstructed by clouds. We present the general framework for the AOD calculation (section 2) and then the daytime (solar) and night-time (lunar) algorithms are described in detail (sections 3 and 4). The latter includes a novel correction  (Román et al. , 2020)  that considerably improves the quality of the lunar retrievals. The cloud screening is described in section 5 and finally the algorithm results are compared with the AERONET database (section 6).
General framework for AOD calculation The calculation of the aerosol optical depth in CAELIS is intended to provide this parameter for a number of instruments, i. e. the photometers within the AERONET network that are calibrated at the University of Valladolid and routinely provide measurements to CAELIS system. They constitute an operational network with about 40 active sites that deliver data in near real time. Therefore the algorithm needs to be robust and work in a large variety of circumstances: any site location, different instrument types, incomplete ancillary information, defective input data, etc. The algorithm to calculate the aerosol optical depth is composed by two main parts. In the first part the algorithm searches into the database all the meta information about the photometer in the specific date-time: calibration coefficients, location, filters, etc.
In the second part, the raw measurement data and the meta information are used to calculate the AOD. Figure  1  shows the process followed to generate the AOD, and as can be observed, it needs a lot of ancillary information. All that information is being stored in the CAELIS database. Each photometer on any particular date-time is linked in one deployment ('installation') with all that ancillary information. In this installation the information related with the beginning and ending dates when the photometer was deployed in one site, as well as the coordinates of the site, are stored. Once the photometer location is known, the next step is to know the specific instrument configuration. In this step all the information related with the instrument type 1 , which could change from one deployment to another, is needed in addition to the information about the interference filters of the photometer: e.
g. central and nominal wavelength of the spectral channels and the specific gaseous and water vapor absorption coefficients for those wavelengths. Then, the algorithm reads the photometer calibration (extraterrestrial signal at mean Earth-Sun distance) and the temperature correction coefficients. This information is provided by the Valladolid calibration facility to the AERONET database and is therefore identical to that used in the AERONET version 3 products. The calibration is then adjusted to the Earth-Sun distance for each observation date-time. The ancillary information needed for the processing is the local pressure and the column of absorbing gaseous species taken into account: ozone, nitrogen dioxide, carbon dioxide and methane. The detailed description on how this information is obtained by CAELIS for the specific date-time and location is provided in section 2.
The workflow of the computation is shown in Figure  2. As already mentioned, CAELIS stores all the data generated by the photometers that are calibrated at our facility. Therefore, the AOD algorithm only needs to get the raw data from the correct table of the database, and use them to calculate the AOD for each measurement. This procedure must be repeated a number of times, that depend on the instrument type. In digital and triple Cimel photometers, equipped with 10 spectral channels, a total number of 30 measurements are collected in each AOD observation. The 3 measurements per channel acquired over 1 minute (triplets) constitute the basic AOD measurement for each wavelength. Temperature correction is applied to these raw data according to the internal temperature recorded at the sensor head (see section 2.
CAELIS database structure for AOD CAELIS is composed of a relational database, a processing module and a web interface  (Fuertes et al. , 2017). As indicated above, in this database we can find all the information required to compute the AOD. Thanks to the deployment records ('installations') that exist in the database, we can link, for a specific date, all the physical and logical information about each particular instrument and how it is (or it was) configured. That means we can access to the calibration coefficients of each spectral channel, the raw data, the filter specifications, etc. All this information is stored in different tables of the database. Similarly, after running the AOD algorithm, all the information generated will be stored in different tables of the database. Specifically two tables are designed to store all the AOD information.
One table stores the information that is common for all the spectral channels: date and time, site, solar zenith angle, Earth-Sun distance, pressure, algorithm version, etc. All the information stored in the common table is used to calculate the AOD for each channel. It also stores the derived Ångström exponent and PWV values. The second table stores the specific information of each spectral channel. That means we can find in this table the exact central wavelength of the filter, the calibration coefficient and the temperature correction of the channel, the specific absorption coefficient for gases, and the calculated values for the various components (total, Rayleigh, gaseous absorption and aerosol optical depth). According to the CAELIS database structure  (Fuertes et al. , 2017) , the AOD is a level 1 product ('direct product'), therefore some redundant information is included in these tables in order to facilitate the data extraction by users.
Computing Thanks to the processing chain of CAELIS, the near real time provision of AOD can be achieved. Every raw data file that is received in CAELIS activates a set of triggers. First of all, the AOD algorithm runs between the first and last measurement included in the data file. Once a first version has been calculated, the system checks whether the AOD has been generated using a pressure value obtained by meteorological analysis (section 2. 3. 1) or only standard pressure was available. If pressure from meteorological analysis was not yet into the database, the system creates a new task (12 hours later), to reprocess the data until analysis pressure data are available. Each file received by the system activates the task to calculate the cloud screening. This task runs the cloud screening algorithm for the full day, between 00:00 and 23:59 local time, even if the file does not cover the entire day.
Global Data Assimilation System NOAA's Air Resource Laboratory runs a series of meteorological analyses and reanalyses; one of this is de Global Data Assimilation System (GDAS, see https://www. ncdc. noaa. gov/data-access/model-data/model-datasets/global-data-assimilation-system-https://doi. org/10. 5194/gi-2020-19 Preprint. Discussion started: 22 July 2020 c Author(s) 2020. CC BY 4. 0 License. gdas). The GDAS is run 4 times per day, at 00:00, 06:00, 12:00 and 18:00 UTC. Model output is a grid with 1 degree resolution (360-181 latitude-longitude). This grid contains several meteorological fields at a set of pressure levels. GDAS data are stored in CAELIS every 6 hours for the purpose of calculating the local pressure for every site of the network. The pressure at the site elevation is calculated from a set of standard geopotential heights and interpolated in time.