dataesr/scientific-paragraphs-categorization huggingface dataset:
g. , MPI using blocking communication or sophisticated alternatives like non-blocking communication. When using non-blocking communication, a further optimization is to decouple the computation of the inner region that can be calculated without needing updated halo data and outer regions that require data from another process to be computed. The translation tool extracts neighborhood information from the GGDML extensions. Such extracted information is analyzed by the tool to decide which halo regions should be exchanged between which nodes. Decisions and information from configuration files allow to generate the necessary code that handles the communication and the synchronization. This guarantees that the necessary data are consistent on the node where the computation takes place. The parallelization is a flexible technique.
No single library is used to handle the parallelization, rather, the communication library is provided through configuration files. Thus, different libraries or library versions can be used for this purpose. We have examined the use of MPI and GASPI as libraries for parallelization. Listing 1. 8 shows the resulting communication code of halo regions between multiple processes -in this case without decoupling of inner and outer area, code with decoupled areas is longer. In this listing, dirty flags are generated to communicate only necessary data. Flags are set global to all processes, and can be checked by each process such that processes that need to do communication can make use of them. This way we guarantee to handle communication properly.
Estimating DSL Impact on Code Quality and Development Costs To estimate the impact of using the DSL on the quality of the code and the costs of model development, we took two relevant kernels from each of the three icosahedral models, and ana-   [33]  lyzed the achieved code reduction in terms of lines of code (LOC)  [33]. We rewrote the kernels (originally written in Fortran) using GGDML + Fortran. Results are shown in Figure  4. I C O N 1 I C O N 2 N I C A M 1 N I C A M 2 D Y N. 1 D Y N. The average reduction in terms of LOC is 70%, i. e. LOC in GGDML+Fortran in comparison to original Fortran code is 30%. More reduction is noticed in some stencils (NICAM example No. 2, reduced to 12%). Influence on readability and maintainability: Using COCOMO  [9]  as a model to estimate complexity of development effort and costs, we estimated in Table  1  the benefits as a result of the code reductions when applying GGDML to develop a model comparable to the ICON model.
Evaluating Performance of our DSL To illustrate the performance benefits of using the DSL for modeling, we present some performance measurements that were measured for example codes written with the DSL and translated considering different optimization aspects (configurations). Two different testcodes were used to evaluate the DSL's support for different types of grids: One application uses an unstructured triangular grid, and the other uses a structured rectangular grid that could be applied for code in cubic sphere. Both were written using GGDML (our DSL) in addition to C as the host modeling language.
Evaluating Blocking To explore the benefit of organizing memory accesses efficiently across architectures, experiments using the shallow water equation solver code were conducted on Piz Daint. First, we generated code versions with and without blocking for the Broadwell processor and the P100 GPU. An excerpt of results presented for different size of grids is shown in  [30]  and in Figure  5a. The In the paper, we investigated the influence of the blocking factor on the application kernel further revealing that modest block sizes are leading to best performance (256 to 10k for CPU and 2-10k for GPU). On both architectures, wider grids run less efficiently as a results of an inefficient use of caches. The GPU implies additional overhead for the blocked version, requiring to run with a sufficient large grid to benefit from it.
Evaluating Inter-Kernel Optimization To explore the benefit of kernel merging, experiments were done using the shallow water equation solver code on the NEC testsystem (for vector engines) and Piz Daint (for GPUs and CPUs)  [30]. The performance of regular and merged kernels are shown in Table  3. The code achieves near optimal use of the memory bandwidth already before the merge and actually declines slightly after the merge. Exploiting the inter-kernel cache reuse allowed to reduce the data access in memory and increased the total performance of the application by 30-50%.
Scaling with Multiple-Node Runs To demonstrate the ability of the DSL to support global icosahedral models, we carried out experiments using the two applications. Scalability experiments of unstructured grid code were run on Mistral. Shallow water equation solver code experiments were run on Mistral for CPU tests, and on Piz Daint for GPU tests. In the experiment using the unstructured grid, we use the global grid of the application and apply a three-dimensional Laplacian stencil. We varied the number of nodes that we use to run the code up to 48 nodes. The minimum number of the grid points we used is 1,048,576. We used this number of points for the strong-scale analysis. Weak scalability experiments were based on this number of points for each node. Figure  6a  shows the results. We could do further numbers of nodes, however, we found that the code was scaling with the tested cases and further experiments needed resources and time to get jobs to run on the test machine.
For the measured cases, the weak scalability of the code is close to optimal. Thus, increasing the resolution of the grids and running the code on more nodes is achieved efficiently. This is an important point as higher resolution grids are essential for recent and future global simulations. We also carried out experiments to scale the shallow water equation solver on both Broadwell multi-core processors and on the P100 GPUs at Piz Daint. We generated code for Broadwell experiments with OpenMP as well as MPI, and for the GPUs with OpenACC as well as MPI. Figure  6b  shows the measured results of scaling the code. On the Broadwell processor, we used 36 OpenMP threads on each node. (a) Icosahedral code  [31]  (b) Shallow water solver  [32]  Fig. 6 : Scalability of the two different models (measured on Mistral; P100 on Piz Daint) While the performance of the GPU code scaled well, it loses quite some efficiency when running on two processes as the halo communication involves the host code.
Exploring the Use of Alternative DSLs: GridTools The GridTools framework is a set of libraries and utilities to develop performance-portable weather and climate applications. It is developed at The Swiss National Supercomputing Center  [2]. To achieve the goal of performance portability, the user-code is written in a generic form which is then optimized for a given architecture at compile-time. The core of GridTools is the stencil composition module which implements a DSL embedded in C++ for stencils and stencillike patterns. Further, GridTools provides modules for halo exchanges, boundary conditions, data management and bindings to C and Fortran. GridTools is successfully used to accelerate the dynamical core of the COSMO model with improved performance on CUDA-GPUs compared to the current official version, demonstrating production quality and feature-completeness of the library for models on lat-lon grids  [49].
Although GridTools was developed for weather and climate applications it might be applicable for other domains with a focus on stencil-like computations. In the context of AIMES project, we evaluated the viability of using GridTools for the dynamical core of NICAM: namely NICAM-DC. Since NICAM-DC is written in Fortan, we first had to port the code to C++, which includes also changing the build systems. Figure  1. 9 and Figure  1. 10 show simple examples codes extracted from NICAM-DC and ported to GridTools, respectively. We ported the dynamical core using the following incremental approach. First, each operator was ported individually to GridTools, i. e. re-written from Fortran to C++. Next, we used a verification tool to assure the same input to the C++ and Fortran version give the same output.
Next we move on to the following operator. Table  4  shows results from benchmarks extracted from NICAM-DC. It provides good speedup on GPU, and speed on CPU (in openMP) comparable to the hand-written version. Figure  7  shows results for running all operators of NICAM-DC on 10 nodes. It is worth mentioning that the most time consuming operator is more than 7x faster on GPU versus CPU. GridTools demonstrates good GPU performance and acceptable CPU performance. The functionalities and features included in GridTools were enough to support the regular mesh code of NICAM-DC without friction (i. e. no custom features were required in GridTools to support the requirements of NICAM-DC). In addition, GridTools, are transparent in the sense that no information about the platform is exposed to the end-user.
On the other hand, following is a list of issues that requires one's consideration when using GridTools: First, the requirement of rewriting the entire dynamical core in C++ is not a trivial task, specially that C++ templates make the code more convoluted, in comparison to Fortran. Second, while GridTools as a stencil framework does a good job for the dynamical core, separate solutions are required for the physics Table  4 : Execution time (seconds) of different benchmarks extracted from NICAM-DC. This includes the regular regions only, using 1 region, a single MPI rank for a 130x130x42 grid. Fig. 7 : NICAM-DC operators. Running on 10 nodes with one MPI rank per node. P100 is running GridTools generated kernels and Xeon uses the original Fortran code. Total grid is 32x32x10 using 40 vertical layers modules, the communicator module, and the non-regular compute modules (e.
Compression in HDF5 and NetCDF We tested the compression library SCIL using the icosahedral grid code. The code can use NetCDF to output the generated data periodically to output the fields. In this experiment, a high-resolution grid with 268 million grid cells (single precision floating point) in the horizontal times 64 vertical levels was used and executed on the supercomputer Mistral. The code was run on 128 nodes, with one MPI process per node. It wrote one field to the output file in one timestep. The field values range between -10 to +55 which is important for understanding the impact of the compression. The experiments varied the basic compression algorithms and parameters provided by SCIL. Compression is done with the algorithms memcopy: does not do any real compression, but allows to measure the overhead of the usage of enabling HDF5 compression and SCIL.
Standardized Icosahedral Benchmarks Our research on DSL and I/O is more practical. We started with real-world applications, namely three global atmospheric models developed by the participating countries. The global atmospheric model with the icosahedral grid system is one of the new generation global climate/weather models. The grid-point calculations, which are less computational intense than the spherical harmonics transformation, are used in the model. On the other hand, patterns of the data access in differential operators are more complicated than the traditional limited-area atmospheric model with the Cartesian grid system. There are different implementations of the dynamical core on the icosahedral grid: direct vs. indirect memory access, staggered vs. co-located data distribution, and so on.
The objective of standardization of benchmarks is to provide a variety of computational patterns of the icosahedral atmospheric models. The kernels are useful for evaluating the performance of new machines and new programming models. Not only for our studies but also for the existing/future DSL studies, this benchmark set provides various samples of the source code. The icosahedral grid system is an unstructured grid coordinate, and there are a lot of challenging issues about data decomposition, data layout, loop structures, cache blocking, threading, offloading the accelerators, and so on. By applying DSLs or frameworks to the kernels, the developers can try the detailed, practical evaluation of their software. The benchmarks were used in the final evaluation of SCIL and they steered the DSL development by providing the relevant patterns.
IcoAtmosBenchmark v1: Kernels from Icosahedral Atmospheric Models IcoAtmosBenchmark v1 is the package of kernels extracted from three Icosahedral Global Atmospheric models, NICAM, ICON, DYNAMICO. As shown in Figure  11 , we prepared input data and reference output data for easy validation of results. We also arranged documentation about the kernels. The package is available online  5. Documentation An excerpt to the NICAM kernel serves as an example. The icosahedral grid on the sphere of NICAM is the unstructured grid system. In NICAM code, the complex topology of the grid system is separated into the structured and unstructured part. The grids are decomposed into tiles, and one or more tiles are allocated to each MPI process. The horizontal grids in the tile are kept in a 2-dimensional structure.
On the other hand, a topology of the network of the tiles is complex. We selected and extracted 6+1 computational kernels from the dynamical core of NICAM, as samples of the stencil calculation on the structured grid system. We also extracted a communication kernel, as a sample of halo exchange in the complex node topology. The features of each kernel are documented on the GitHub page. All kernels are single subroutines and almost the same as the source codes in the original model, except to the ICON kernels. They are put into the wrapper for the kernel program. Values of input variables in the argument list of the kernel subroutine are stored as a data file, just before the call in the execution of the original model. They are read and given to the subroutine in the kernel program. Similarly, the values of output variables in the argument list are stored, just after the call in execution.
Summary and Conclusion The numerical simulation of climate and weather is demanding for computational and I/O resources. Within the AIMES project, we addressed those challenges and researched approaches that foster the separation of concerns. This idea unites our approaches for the DSL and the compression library. While a higher-level of abstraction can improve the productivity for scientists, most importantly the decoupling of requirements from the implementation allows scientific programmers to develop and improve architecture-specific optimizations. Promising candidates for DSLs have been explored and with GGDML an alternative has been developed that covers the most relevant formulations of the three icosahedral models: DYNAM-ICO, ICON, and NICAM. The DSL and toolchain we developed integrates into existing code bases and suits for incremental reformulation of the code.
We estimated the benefit for code reduction and demonstrated several optimizations for CPU, GPU, and vector architectures. Our DSL allows to reduce code to around 30% of the LOC in comparison to code written with GPL code. With the semantics of GGDML, we could achieve near optimal use of memory hierarchies and memory throughput which is critical for the family of computations in hand. Our experiments show running codes with around 80% of achievable memory throughput on differnt archiectures. Furthermore, we could scale models to multiple nodes, which is essential for exascale computing, using the same code that is used for a single node. The separation of concerns in our approach allowed us to keep models code separate of underlying hardware changes. The single GGDML source code is used to generate code for the different architectures and on single vs.
multiple nodes. To address the I/O challenge, we developed the library SCIL, a metacompressor supporting various lossy and lossless algorithms. It allows users to specify various quantities for the tolerable error and expected performance, and allows the library to chose a suitable algorithm. SCIL is a stand-alone library but also integrates into NetCDF and HDF5 allowing users to explore the benefits of using alternative compression algorithms with their existing middleware. We evaluated the performance and compression ratio for various scientific datasets and on moderate scale. The results show that the choice of the best algorithm depends on the data and performance expectation which, in turn, motivates the need for the decoupling of quantities from the selection of the algorithm. A blocker for applying parallel large-scale compression in existing NetCDF workflows is the performance limitation of the current HDF5 stack.
Finally, benchmarks and mini-applications were created that represent the key features of the icosahedral applications. Beside the achieved research in the AIMES project, the work done opens the door for further research in the software engineering of climate and weather prediction models. The performance portability, where we used the same code to run on different architectures and machines, including single and multiple nodes, shows that techniques to be viable to continue the research in this direction. The code reduction offered by DSLs promise to save million in development cost that can be used to contribute to the DSL development. During the runtime of the project, it became apparent that the concurrently developed solutions GridTools and PSYclone that also provide such features are preferred by most scientists as they are developed by a bigger community and supported by national centers.
Exploiting low and high-level information, the 2SEES can act as an interface between objects and their individual functionalities. But more can be done: the 2SEES could exploit the SOs information and holistically combine them to provide high-level knowledge to the VIP. The general querying process is the following: based on their wish, the VIP expresses a query in natural language, the system converts it into a formal high-level query, which is broken down into several sub-queries. Reasoning is then performed on queried data to answer the high-level query (Figure  5 ). For instance, to answer the ''Why is it cold?'' question, the 2SEES system needs to: • Identify objects that have a relationship with the local temperature: it could be a thermometer, the heating system, or a window; • Identify their relationship with the temperature: the thermometer measures it, the heating system measures and can influence it, and the window can influence it by being open or closed; • Query these objects to retrieve data about that relationship: the temperature can be obtained from the thermometer, the state of the heating system (i.
e. on/off, setpoint temperature), and the state of the window (i. e. open or closed); • For instance, the system can check the underlying judgment of the question using the temperature value, and then check possible reasons: the window may be open and/or the heating system may be turned off. Since the 2SEES acts as a gateway between the user and the surrounding SOs, a lot of information can be transmitted to the VIP. However, because of the limitations of transmission channels (see section I), it is not possible to exchange large quantities of data between the VIP and SOs. Thus, reducing the amount of exchanged data is a central issue of the design of the 2SEES-user interface. In order to provide only meaningful, concise and intelligible information, available data have to be filtered. If a low-level querying model were to be adopted, users would have to verbally navigate and access SOs data, and most of their effort would be spent telling the system which SOs and attributes it must query, which is a rather tedious communication process.
All these SOs sent the data they generated to a back-end platform acting as the cloud via a multi-support gateway (IEEE802. 15. 4 and IEEE802. 3y) (Fig. 10, 11 ). The gateway also acts as the local communication link between the SO environment and the SO2SEES system: it advertises the IRIs of the SOs to the SO2SEES system. When the VIP enters the room, they open and close the door. The gateway detects an entrance in the room by monitoring the status of the door: it triggers the local, wireless transmission of the list of the SOs' IRIs to the local platform of the 2SEES (LMSC). The LMSC transmits this list to the smartphone, which in turn transmits it to the 2SEES backend, which was part of the same LAN as the other back-end and the gateway. The IRIs point to files containing WoT-TD descriptions of the objects, hosted on the back-end of the SOs.
Therefore, the query execution program consists in retrieving the status of every object in the room related to the temperature -and transmitting it to the user: ''The thermometer measures 14 • C; the window is closed; the radiator is turned off. '' This process induces context relevance constraints: the only properties to be checked must (i) be related to temperature and (ii) belong to objects located inside the room. In our implementation, the first constraint was solved by the local transmission mechanism: the gateway knew where SOs were and which room had been entered. Embedding objects location in their descriptive metadata could also solve this constraint. The temperature relevance has to be embedded in the knowledge base, either vendor's side or 2SEES side. To illustrate the full functionality of the SO2SEES, a specific ontology for VIPs still needs to be developed, but in this work we only focused on the feasibility of using SOs to improve the perception of VIPs' environment.
B. INTEROPERABILITY ISSUES The above corrections of navigation and obstacle detection raise interoperability issues. Indeed, data produced by SOs need to be encoded in a format that the 2SEES can understand at the semantic level, and the possible interactions with the SOs should be described in an equally coherent semantic format. In that aim, the SO2SEES functionality relies on several assumptions: • All SOs have an associated IRI; • The 2SEES can access a formal description of SOs from their IRI; • This IRI can lead to the data of the SO, either directly, if it represents an accessible location, or indirectly, (e. g. the resource name) and a mean to access the SO from its name; • The SOs periodically broadcast locally their IRI. Alternatively, any type of discovery protocol can be employed, as long as it is standardized across devices (e.
On a physical level, different interfaces and protocols are adapted to different applications, and the 2SEES is not likely to have hardware and software for all of them. Furthermore, wired networks, which are commonly used in building automation systems, for instance, do not allow direct connection. Indirect access is more flexible: Internet access is to be expected in areas with a rich SO environment and access to SO data through cloud back-ends is also very likely. Another important choice is the access method, of which a first aspect is the discovery of objects. We have chosen to use local broadcast from the edge router, a middle ground between discovery broadcast/answering from objects and a mechanism purely independent from local communication where objects identifiers would be taken from a geographical database.
The latter possibility requires an accessible and up to date set of databases (each vendor probably having its own) and accurate localization. The second aspect is the addressing and the access itself. In the OSI or TCP/IP network model, Layer 3/network addressing could be possible, thanks to the large address space of IPv6, but some of the most constrained SOs do not have enough resources to be directly connected. Indirect or higher level addressing solves this issue: the location pointed by the address can lead to an intermediate service representing the node (i. e. a virtual node). We have chosen to use dereferencable IRIs as identifiers and access methods for objects. IRIs are standard  [35]  and widely used, in particular, they are the chosen method to identify objects in the W3C standard RDF  [36].