pinecone/core-2020-05-10-deduplication huggingface dataset:
A domain-specific high-level programming model

International audienceNowadays, computing hardware continues to move toward more parallelism and more heterogeneity, to obtain more computing power. From personal computers to supercomputers, we can find several levels of parallelism expressed by the interconnections of multi-core and many-core accelerators. On the other hand, computing software needs to adapt to this trend, and programmers can use parallel programming models (PPM) to fulfil this difficult task. There are different PPMs available that are based on tasks, directives, or low level languages or library. These offer higher or lower abstraction levels from the architecture by handling their own syntax. However, to offer an efficient PPM with a greater (additional) high-levelabstraction level while saving on performance, one idea is to restrict this to a specific domain and to adapt it to a family of applications. In the present study, we propose a high-level PPM specific to digital signal processing applications. It is based on data-flow graph models of computation, and a dynamic runtime model of execution (StarPU). We show how the user can easily express this digital signal processing application, and can take advantage of task, data and graph parallelism in the implementation, to enhance the performances of targeted heterogeneous clusters composed of CPUs and different accelerators (e.g., GPU, Xeon Phi
Monitoring and improving performance in human-computer interaction

Monitoring an individual's performance in a task, especially in the workplace context, is becoming an increasingly interesting and controversial topic in a time in which workers are expected to produce more, better and faster. The tension caused by this competitiveness, together with the pressure of monitoring, may not work in favour of the organization's objectives. In this paper, we present an innovative approach on the problem of performance management. We build on the fact that computers are nowadays used as major work tools in many workplaces to devise a non-invasive method for distributed performance monitoring based on the observation of the worker's interaction with the computer. We then look at musical selection both as a pleasant and as an effective method for improving performance in the workplace. The proposed approach will allow team coordinators to assess and manage their co-workers' performance continuously and in real-time, using a distributed service-based architecture. Copyright (c) 2015 John Wiley & Sons, Ltd.This work is part-funded by European Regional Development Fund (ERDF) through the COMPETE Programme (operational programme for competitiveness) and by the national funds through the Fundacao para a Ciencia e a Tecnologia (FCT; Portuguese Foundation for Science and Technology) within projects FCOMP-01-0124-FEDER-028980 (PTDC/EEI-SII/1386/2012) and project PEst-OE/EEI/UI0752/2014.info:eu-repo/semantics/publishedVersio
Towards an autonomous decentralised orchestration system

Orchestrating workflows needed for modern scientific data analysis presents a significant research challenge: they are typically executed in a centralised manner such that all data pass through a single compute server known as the engine, which causes unnecessary network traffic that leads to a performance bottleneck. This paper presents a scalable decentralised orchestration system that relies on a functional, high-level data coordination language for executing workflows. This system consists of distributed execution engines, each of which is responsible for executing part of the overall workflow. It exploits parallelism in the workflow by partitioning it into smaller sub workflows, and determines the most appropriate engines to execute them using network resource monitoring and placement analysis. This permits the computation logic of the workflow to be moved towards the services providing the data, which improves the overall execution time. The system supports data-driven execution that allows each sub workflow to be executed as soon as the data needed for its execution becomes available from other sources. Therefore, a scheduling mechanism is not required to manage the order in which the sub workflows are orchestrated. This paper provides an evaluation of the proposed system, which demonstrate that decentralised orchestration provides scalability over centralised orchestration.PostprintPeer reviewe
Performance evaluation of a distributed storage service in community network clouds

Community networks are self-organized and decentralized communication networks built and operated by citizens, for citizens. The consolidation of today's cloud technologies offers now, for community networks, the possibility to collectively develop community clouds, building upon user-provided networks and extending toward cloud services. Cloud storage, and in particular secure and reliable cloud storage, could become a key community cloud service to enable end-user applications. In this paper, we evaluate in a real deployment the performance of Tahoe least-authority file system (Tahoe-LAFS), a decentralized storage system with provider-independent security that guarantees privacy to the users. We evaluate how the Tahoe-LAFS storage system performs when it is deployed over distributed community cloud nodes in a real community network such as Guifi.net. Furthermore, we evaluate Tahoe-LAFS in the Microsoft Azure commercial cloud platform, to compare and understand the impact of homogeneous network and hardware resources on the performance of the Tahoe-LAFS. We observed that the write operation of Tahoe-LAFS resulted in similar performance when using either the community network cloud or the commercial cloud. However, the read operation achieved better performance in the Azure cloud, where the reading from multiple nodes of Tahoe-LAFS benefited from the homogeneity of the network and nodes. Our results suggest that Tahoe-LAFS can run on community network clouds with suitable performance for the needed end-user experience.Peer ReviewedPreprin
User Applications Driven by the Community Contribution Framework
  MPContribs in the Materials Project

This work discusses how the MPContribs framework in the Materials Project
(MP) allows user-contributed data to be shown and analyzed alongside the core
MP database. The Materials Project is a searchable database of electronic
structure properties of over 65,000 bulk solid materials that is accessible
through a web-based science-gateway. We describe the motivation for enabling
user contributions to the materials data and present the framework's features
and challenges in the context of two real applications. These use-cases
illustrate how scientific collaborations can build applications with their own
"user-contributed" data using MPContribs. The Nanoporous Materials Explorer
application provides a unique search interface to a novel dataset of hundreds
of thousands of materials, each with tables of user-contributed values related
to material adsorption and density at varying temperature and pressure. The
Unified Theoretical and Experimental x-ray Spectroscopy application discusses a
full workflow for the association, dissemination and combined analyses of
experimental data from the Advanced Light Source with MP's theoretical core
data, using MPContribs tools for data formatting, management and exploration.
The capabilities being developed for these collaborations are serving as the
model for how new materials data can be incorporated into the Materials Project
website with minimal staff overhead while giving powerful tools for data search
and display to the user community.Comment: 12 pages, 5 figures, Proceedings of 10th Gateway Computing
  Environments Workshop (2015), to be published in "Concurrency in Computation:
  Practice and Experience
Quality measures for ETL processes: from goals to implementation

Extraction transformation loading (ETL) processes play an increasingly important role for the support of modern business operations. These business processes are centred around artifacts with high variability and diverse lifecycles, which correspond to key business entities. The apparent complexity of these activities has been examined through the prism of business process management, mainly focusing on functional requirements and performance optimization. However, the quality dimension has not yet been thoroughly investigated, and there is a need for a more human-centric approach to bring them closer to business-users requirements. In this paper, we take a first step towards this direction by defining a sound model for ETL process quality characteristics and quantitative measures for each characteristic, based on existing literature. Our model shows dependencies among quality characteristics and can provide the basis for subsequent analysis using goal modeling techniques. We showcase the use of goal modeling for ETL process design through a use case, where we employ the use of a goal model that includes quantitative components (i.e., indicators) for evaluation and analysis of alternative design decisions.Peer ReviewedPostprint (author's final draft
Dynamic web worker pool management for highly parallel javascript web applications

JavaScript web applications are improving performance mainly thanks to the inclusion of new standards by HTML5. Among others, web workers API allows multithreaded JavaScript web apps to exploit parallel processors. However, developers have difficulties to determine the minimum number of web workers that provide the highest performance. But even if developers found out this optimal number, it is a static value configured at the beginning of the execution. Because users tend to execute other applications in background, the estimated number of web workers could be non-optimal, because it may overload or underutilize the system. In this paper, we propose a solution for highly parallel web apps to dynamically adapt the number of running web workers to the actual available resources, avoiding the hassle to estimate a static optimal number of threads. The solution consists in the inclusion of a web worker pool and a simple management algorithm in the web app. Even though there are co-running applications, the results show our approach dynamically enables a number of web workers close to the optimal. Our proposal, which is independent of the web browser, overcomes the lack of knowledge of the underlying processor architecture as well as dynamic resources availability changes.Peer ReviewedPostprint (author's final draft
HPC-GAP : engineering a 21st-century High-Performance Computer algebra system

Symbolic computation has underpinned a number of key advances in Mathematics and Computer Science. Applications are typically large and potentially highly parallel, making them good candidates for parallel execution at a variety of scales from multi-core to high-performance computing systems. However, much existing work on parallel computing is based around numeric rather than symbolic computations. In particular, symbolic computing presents particular problems in terms of varying granularity and irregular task sizes that do not match conventional approaches to parallelisation. It also presents problems in terms of the structure of the algorithms and data. This paper describes a new implementation of the free open-source GAP computational algebra system that places parallelism at the heart of the design, dealing with the key scalability and cross-platform portability problems. We provide three system layers that deal with the three most important classes of hardware: individual shared memory multi-core nodes, mid-scale distributed clusters of (multi-core) nodes, and full- blown HPC systems, comprising large-scale tightly-connected networks of multi-core nodes. This requires us to develop new cross-layer programming abstractions in the form of new domain-specific skeletons that allow us to seamlessly target different hardware levels. Our results show that, using our approach, we can achieve good scalability and speedups for two realistic exemplars, on high-performance systems comprising up to 32,000 cores, as well as on ubiquitous multi-core systems and distributed clusters. The work reported here paves the way towards full scale exploitation of symbolic computation by high-performance computing systems, and we demonstrate the potential with two major case studies.Publisher PDFPeer reviewe
RTS: road topology-based scheme for traffic condition estimation via vehicular crowdsensing

Urban traffic condition usually serves as basic information for some intelligent urban applications, for example, intelligent transportation system. The traditional acquisition of such information is often costly because of the dependencies on infrastructures, such as cameras and loop detectors. Crowdsensing, as a new economic paradigm, can be utilized together with vehicular networks to efficiently gather vehicle-sensed data for estimating the traffic condition. However, it has the problem of being lack of data uploading efficiency and data usage effectiveness. In this paper, we take into account the topology of the road net to deal with these problems. Specifically, we divide the road net into road sections and junction areas. Based on this division, we introduce a two-phased data collection and processing scheme named road topology-based scheme. It leverages the correlations among adjacent roads. In a junction area, data collected by vehicles are first processed and integrated by a sponsor vehicle to locally calculate traffic condition. Both the selection of the sponsor and the calculation of road condition utilize the road correlation. The sponsor then uploads the local data to a server. By employing the inherent relations among roads, the server processes data and estimates traffic condition for the road sections without vehicular data in a global vision. We conduct experiments based on real vehicle trace data. The results indicate that our design can commendably handle the problems of efficiency and effectiveness in traffic condition evaluation using the vehicular crowdsensing data
HTC-Sim : a trace-driven simulation framework for energy consumption in high-throughput computing systems.

High-throughput computing (HTC) is a powerful paradigm allowing vast quantities of independent work to be performed simultaneously across many loosely coupled computers. These systems often exploit the idle time available on computers provisioned for other purposes – volunteer computing. However, until recently, little evaluation has been performed on the energy impact of HTC. Many organisations now seek to minimise energy consumption across their IT infrastructure. However, it is unclear how this will affect the usability of HTC systems especially when exploiting volunteer computers. We present here HTC-Sim, a simulation system that allows the evaluation of different energy reduction policies across an HTC system. We model systems that comprised both collections of computational resources dedicated to HTC work and resources provided through volunteer computing – a desktop grid. We demonstrate that our simulation software scales linearly with increasing HTC workload. We go further to evaluate a number of resource selection policies in terms of the overheads/slowdown incurred and the energy impact on the HTC system
Performance and Portability of Accelerated Lattice Boltzmann
  Applications with OpenACC

An increasingly large number of HPC systems rely on heterogeneous
architectures combining traditional multi-core CPUs with power efficient
accelerators. Designing efficient applications for these systems has been
troublesome in the past as accelerators could usually be programmed using
specific programming languages threatening maintainability, portability and
correctness. Several new programming environments try to tackle this problem.
Among them, OpenACC offers a high-level approach based on compiler directive
clauses to mark regions of existing C, C++ or Fortran codes to run on
accelerators. This approach directly addresses code portability, leaving to
compilers the support of each different accelerator, but one has to carefully
assess the relative costs of portable approaches versus computing efficiency.
In this paper we address precisely this issue, using as a test-bench a
massively parallel Lattice Boltzmann algorithm. We first describe our
multi-node implementation and optimization of the algorithm, using OpenACC and
MPI. We then benchmark the code on a variety of processors, including
traditional CPUs and GPUs, and make accurate performance comparisons with other
GPU implementations of the same algorithm using CUDA and OpenCL. We also asses
the performance impact associated to portable programming, and the actual
portability and performance-portability of OpenACC-based applications across
several state-of-the- art architectures
A performance analysis of a mimetic finite difference scheme for acoustic wave propagation on GPU platforms

Realistic applications of numerical modeling of acoustic wave dynamics usually demand high-performance computing because of the large size of study domains and demanding accuracy requirements on simulation results. Forward modeling of seismic motion on a given subsurface geological structure is by itself a good example of such applications, and when used as a component of seismic inversion tools or as a guide for the design of seismic surveys, its computational cost increases enormously. In the case of finite difference methods (or any other volumen-discretization scheme), memory and computing demands rise with grid refinement, which may be necessary to reduce errors on numerical wave patterns and better capture target physical devices. In this work, we present several implementations of a mimetic finite difference method for the simulation of acoustic wave propagation on highly dense staggered grids. These implementations evolve as different optimization strategies are employed starting from appropriate setting of compilation flags, code vectorization by using streaming SIMD extensions Advanced Vector Extensions (AVX), CPU parallelization by exploiting the Open Multi-Processing framework to the final code parallelization on graphics processing unit platforms. We present and discuss the increasing processing speed up of this mimetic scheme achieved by the gradual implementation and testing of all these performance optimizations. In terms of simulation times, the performance of our graphics processing unit parallel implementations is consistently better than the best CPU version.Authors from Universidad Central de Venezuela (UCV) were partially supported by: Consejo de Desarrollo Científico y Humanístico de la UCV, Vice-rectorado Académico de la UCV, Coordinación de Investigación de la Facultad de Ciencias UCV, Banco Central de Venezuela (BCV) and Generalitat Valenciana under project PROMETEOII/2015/015
A performance analysis of a mimetic finite difference scheme for acoustic wave propagation on GPU platforms

Realistic applications of numerical modeling of acoustic wave dynamics usually demand high-performance computing because of the large size of study domains and demanding accuracy requirements on simulation results. Forward modeling of seismic motion on a given subsurface geological structure is by itself a good example of such applications, and when used as a component of seismic inversion tools or as a guide for the design of seismic surveys, its computational cost increases enormously. In the case of finite difference methods (or any other volumen-discretization scheme), memory and computing demands rise with grid refinement, which may be necessary to reduce errors on numerical wave patterns and better capture target physical devices. In this work, we present several implementations of a mimetic finite difference method for the simulation of acoustic wave propagation on highly dense staggered grids. These implementations evolve as different optimization strategies are employed starting from appropriate setting of compilation flags, code vectorization by using streaming SIMD extensions Advanced Vector Extensions (AVX), CPU parallelization by exploiting the Open Multi-Processing framework to the final code parallelization on graphics processing unit platforms. We present and discuss the increasing processing speed up of this mimetic scheme achieved by the gradual implementation and testing of all these performance optimizations. In terms of simulation times, the performance of our graphics processing unit parallel implementations is consistently better than the best CPU version. Copyright © 2016 John Wiley & Sons, Ltd.Peer ReviewedPostprint (author's final draft
CAP Bench: a benchmark suite for performance and energy evaluation of low-power many-core processors

International audienceSUMMARY The constant need for faster and more energy-efficient processors has been stimulating the development of new architectures, such as low-power many-core architectures. Researchers aiming to study these architectures are challenged by peculiar characteristics of some components such as Networks-on-Chip and lack of specific tools to evaluate their performance. In this context, the goal of this paper is to present a benchmark suite to evaluate state-of-the-art low-power many-core architectures such as the Kalray MPPA-256 low-power processor, which features 256 compute cores in a single chip. The benchmark was designed and used to highlight important aspects and details that need to be considered when developing parallel applications for emerging low-power many-core architectures. As a result, this paper demonstrates that the benchmark offers a diverse suite of programs with regard to parallel patterns, job types, communication intensity and task load strategies, suitable for a broad understanding of performance and energy consumption of MPPA-256 and upcoming many-core architectures
Performance analysis of the Kahan-enhanced scalar product on current
  multi- and manycore processors

We investigate the performance characteristics of a numerically enhanced
scalar product (dot) kernel loop that uses the Kahan algorithm to compensate
for numerical errors, and describe efficient SIMD-vectorized implementations on
recent multi- and manycore processors. Using low-level instruction analysis and
the execution-cache-memory (ECM) performance model we pinpoint the relevant
performance bottlenecks for single-core and thread-parallel execution, and
predict performance and saturation behavior. We show that the Kahan-enhanced
scalar product comes at almost no additional cost compared to the naive
(non-Kahan) scalar product if appropriate low-level optimizations, notably SIMD
vectorization and unrolling, are applied. The ECM model is extended
appropriately to accommodate not only modern Intel multicore chips but also the
Intel Xeon Phi "Knights Corner" coprocessor and an IBM POWER8 CPU. This allows
us to discuss the impact of processor features on the performance across four
modern architectures that are relevant for high performance computing.Comment: 15 pages, 10 figure