OpenRaiser/Intern-Atlas huggingface dataset:
Batch 1: Definition of several Weather & Climate Dwarfs
Venue: arXiv cs.DC 2019
Year: 2019

This document is one of the deliverable reports created for the ESCAPE project. ESCAPE stands for Energy-efficient Scalable Algorithms for Weather Prediction at Exascale. The project develops world-class, extreme-scale computing capabilities for European operational numerical weather prediction and future climate models. This is done by identifying weather & climate dwarfs which are key patterns in terms of computation and communication (in the spirit of the Berkeley dwarfs). These dwarfs are then optimised for different hardware architectures (single and multi-node) and alternative algorithms are explored. Performance portability is addressed through the use of domain specific languages. This deliverable contains the description of the characteristics of the weather & climate dwarfs that form key functional components of prediction models in terms of the science that they encapsulate and in terms of computational cost they impose on the forecast production. The ESCAPE work flow between work packages centres on these dwarfs and hence their selection, their performance assessment, code adaptation and optimization is crucial for the success of the project. At this stage of ESCAPE, a selection of established and new dwarfs has been made, their documentation been compiled and the software been made available on the software exchange platform. The selection of dwarfs will be extended throughout the course of the project (see Deliverable D1.2). The current selection includes the spectral transforms, the cloud microphysics scheme, two and three-dimensional elliptic solvers, a bi-Fourier spectral transform, an interpolation needed for the semi-Lagrangian advection scheme and a first version of the semi-Lagrangian advection scheme itself. This deliverable includes their scientific description and the guidance for installation, execution and testing.
Development of Atlas, a flexible data structure framework
Venue: arXiv cs.DC 2019
Year: 2019

This document is one of the deliverable reports created for the ESCAPE project. ESCAPE stands for Energy-efficient Scalable Algorithms for Weather Prediction at Exascale. The project develops world-class, extreme-scale computing capabilities for European operational numerical weather prediction and future climate models. This is done by identifying Weather & Climate dwarfs which are key patterns in terms of computation and communication (in the spirit of the Berkeley dwarfs). These dwarfs are then optimised for different hardware architectures (single and multi-node) and alternative algorithms are explored. Performance portability is addressed through the use of domain specific languages. In this deliverable report, we present Atlas, a new software library that is currently being developed at the European Centre for Medium-Range Weather Forecasts (ECMWF), with the scope of handling data structures required for NWP applications in a flexible and massively parallel way. Atlas provides a versatile framework for the future development of efficient NWP and climate applications on emerging HPC architectures. The applications range from full Earth system models, to specific tools required for post-processing weather forecast products. Atlas provides data structures for building various numerical strategies to solve equations on the sphere or limited area's on the sphere. These data structures may contain a distribution of points (grid) and, possibly, a composition of elements (mesh), required to implement the numerical operations required. Atlas can also represent a given field within a specific spatial projection. Atlas is capable of mapping fields between different grids as part of pre- and post-processing stages or as part of coupling processes whose respective fields are discretised on different grids or meshes.
Performance report and optimized implementation of Weather & Climate Dwarfs on GPU, MIC and Optalysys Optical Processor
Venue: arXiv cs.DC 2019
Year: 2019

This document is one of the deliverable reports created for the ESCAPE project. ESCAPE stands for Energy-efficient Scalable Algorithms for Weather Prediction at Exascale. The project develops world-class, extreme-scale computing capabilities for European operational numerical weather prediction and future climate models. This is done by identifying Weather & Climate dwarfs which are key patterns in terms of computation and communication (in the spirit of the Berkeley dwarfs). These dwarfs are then optimised for different hardware architectures (single and multi-node) and alternative algorithms are explored. Performance portability is addressed through the use of domain specific languages. Here we summarize the work performed on optimizations of the dwarfs on CPUs, Xeon Phi, GPUs and on the Optalysys optical processor. We limit ourselves to a subset of the dwarf configurations and to problem sizes small enough to execute on a single node. Also, we use time-to-solution as the main performance metric. Multi-node optimizations of the dwarfs and energy-specific optimizations are beyond the scope of this report and will be described in Deliverable D3.4. To cover the important algorithmic motifs we picked dwarfs related to the dynamical core as well as column physics. Specifically, we focused on the formulation relevant to spectral codes like ECMWF's IFS code. The main findings of this report are: (a) Acceleration of 1.1x - 2.5x of the dwarfs on CPU based systems using compiler directives, (b) order of magnitude acceleration of the dwarfs on GPUs (23x for spectral transform, 9x for MPDATA) using data locality optimizations and (c) demonstrated feasibility of a spectral transform in a purely optical fashion.
Performance report and optimized implementations of Weather & Climate dwarfs on multi-node systems
Venue: arXiv cs.DC 2019
Year: 2019

This document is one of the deliverable reports created for the ESCAPE project. ESCAPE stands for Energy-efficient Scalable Algorithms for Weather Prediction at Exascale. The project develops world-class, extreme-scale computing capabilities for European operational numerical weather prediction and future climate models. This is done by identifying Weather & Climate dwarfs which are key patterns in terms of computation and communication (in the spirit of the Berkeley dwarfs). These dwarfs are then optimised for different hardware architectures (single and multi-node) and alternative algorithms are explored. Performance portability is addressed through the use of domain specific languages. Here we summarize the work performed on optimizations of the dwarfs focusing on CPU multi-nodes and multi-GPUs. We limit ourselves to a subset of the dwarf configurations chosen by the consortium. Intra-node optimizations of the dwarfs and energy-specific optimizations have been described in Deliverable D3.3. To cover the important algorithmic motifs we picked dwarfs related to the dynamical core as well as column physics. Specifically, we focused on the formulation relevant to spectral codes like ECMWF's IFS code. The main findings of this report are: (a) Up-to 30% performance gain with CPU based multi-node systems compared to optimized version of dwarfs from task 3.3 (see D3.3), (b) up to 10X performance gain on multiple GPUs from optimizations to keep data resident on the GPU and enable fast inter-GPU communication mechanisms, and (c) multi-GPU systems which feature a high-bandwidth all-to-all interconnect topology with NVLink/NVSwitch hardware are particularly well suited to the algorithms.
Simulation and Learning for Urban Mobility: City-scale Traffic Reconstruction and Autonomous Driving
Venue: arXiv cs.OH 2019
Year: 2019

Traffic congestion has become one of the most critical issues worldwide. The costs due to traffic gridlock and jams are approximately $160 billion in the United States, more than £13 billion in the United Kingdom, and over one trillion dollars across the globe annually. As more metropolitan areas will experience increasingly severe traffic conditions, the ability to analyze, understand, and improve traffic dynamics becomes critical. This dissertation is an effort towards achieving such an ability. I propose various techniques combining simulation and machine learning to tackle the problem of traffic from two perspectives: city-scale traffic reconstruction and autonomous driving.
The Transference Architecture for Automatic Post-Editing
Venue: arXiv cs.CL 2019
Year: 2019

In automatic post-editing (APE) it makes sense to condition post-editing (pe) decisions on both the source (src) and the machine translated text (mt) as input. This has led to multi-source encoder based APE approaches. A research challenge now is the search for architectures that best support the capture, preparation and provision of src and mt information and its integration with pe decisions. In this paper we present a new multi-source APE model, called transference. Unlike previous approaches, it (i) uses a transformer encoder block for src, (ii) followed by a decoder block, but without masking for self-attention on mt, which effectively acts as second encoder combining src -> mt, and (iii) feeds this representation into a final decoder block generating pe. Our model outperforms the state-of-the-art by 1 BLEU point on the WMT 2016, 2017, and 2018 English--German APE shared tasks (PBSMT and NMT). We further investigate the importance of our newly introduced second encoder and find that a too small amount of layers does hurt the performance, while reducing the number of layers of the decoder does not matter much.
Extending editing capabilities of subdivision schemes by refinement of point-normal pairs
Venue: arXiv cs.GR 2019
Year: 2019

In this paper we extend the 2D circle average of [11] to a 3D binary average of point-normal pairs, and study its properties. We modify classical surface-generating linear subdivision schemes with this average obtaining surface-generating schemes refining point-normal pairs. The modified schemes give the possibility to generate more geometries by editing the initial normals. For the case of input data consisting of a mesh only, we present a method for computing "naive" initial normals from the initial mesh. The performance of several modified schemes is compared to their linear variants, when operating on the same initial mesh, and examples of the editing capabilities of the modified schemes are given. In addition we provide a link to our repository, where we store the initial and refined mesh files, and the implementation code. Several videos, demonstrating the editing capabilities of the initial normals are provided in our Youtube channel.
Distributional Negative Sampling for Knowledge Base Completion
Venue: arXiv cs.LG 2019
Year: 2019

State-of-the-art approaches for Knowledge Base Completion (KBC) exploit deep neural networks trained with both false and true assertions: positive assertions are explicitly taken from the knowledge base, whereas negative ones are generated by random sampling of entities. In this paper, we argue that random sampling is not a good training strategy since it is highly likely to generate a huge number of nonsensical assertions during training, which does not provide relevant training signal to the system. Hence, it slows down the learning process and decreases accuracy. To address this issue, we propose an alternative approach called Distributional Negative Sampling that generates meaningful negative examples which are highly likely to be false. Our approach achieves a significant improvement in Mean Reciprocal Rank values amongst two different KBC algorithms in three standard academic benchmarks.
Conv2Warp: An unsupervised deformable image registration with continuous convolution and warping
Venue: arXiv cs.CV 2019
Year: 2019

Recent successes in deep learning based deformable image registration (DIR) methods have demonstrated that complex deformation can be learnt directly from data while reducing computation time when compared to traditional methods. However, the reliance on fully linear convolutional layers imposes a uniform sampling of pixel/voxel locations which ultimately limits their performance. To address this problem, we propose a novel approach of learning a continuous warp of the source image. Here, the required deformation vector fields are obtained from a concatenated linear and non-linear convolution layers and a learnable bicubic Catmull-Rom spline resampler. This allows to compute smooth deformation field and more accurate alignment compared to using only linear convolutions and linear resampling. In addition, the continuous warping technique penalizes disagreements that are due to topological changes. Our experiments demonstrate that this approach manages to capture large non-linear deformations and minimizes the propagation of interpolation errors. While improving accuracy the method is computationally efficient. We present comparative results on a range of public 4D CT lung (POPI) and brain datasets (CUMC12, MGH10).
Hard but Robust, Easy but Sensitive: How Encoder and Decoder Perform in Neural Machine Translation
Venue: arXiv cs.CL 2019
Year: 2019

Neural machine translation (NMT) typically adopts the encoder-decoder framework. A good understanding of the characteristics and functionalities of the encoder and decoder can help to explain the pros and cons of the framework, and design better models for NMT. In this work, we conduct an empirical study on the encoder and the decoder in NMT, taking Transformer as an example. We find that 1) the decoder handles an easier task than the encoder in NMT, 2) the decoder is more sensitive to the input noise than the encoder, and 3) the preceding words/tokens in the decoder provide strong conditional information, which accounts for the two observations above. We hope those observations can shed light on the characteristics of the encoder and decoder and inspire future research on NMT.
EmotionX-IDEA: Emotion BERT -- an Affectional Model for Conversation
Venue: arXiv cs.CL 2019
Year: 2019

In this paper, we investigate the emotion recognition ability of the pre-training language model, namely BERT. By the nature of the framework of BERT, a two-sentence structure, we adapt BERT to continues dialogue emotion prediction tasks, which rely heavily on the sentence-level context-aware understanding. The experiments show that by mapping the continues dialogue into a causal utterance pair, which is constructed by the utterance and the reply utterance, models can better capture the emotions of the reply utterance. The present method has achieved 0.815 and 0.885 micro F1 score in the testing dataset of Friends and EmotionPush, respectively.
Towards an Integrated Graph Algebra for Graph Pattern Matching with Gremlin (Extended Version)
Venue: arXiv cs.DB 2019
Year: 2019

Graph data management (also called NoSQL) has revealed beneficial characteristics in terms of flexibility and scalability by differently balancing between query expressivity and schema flexibility. This peculiar advantage has resulted into an unforeseen race of developing new task-specific graph systems, query languages and data models, such as property graphs, key-value, wide column, resource description framework (RDF), etc. Present-day graph query languages are focused towards flexible graph pattern matching (aka sub-graph matching), whereas graph computing frameworks aim towards providing fast parallel (distributed) execution of instructions. The consequence of this rapid growth in the variety of graph-based data management systems has resulted in a lack of standardization. Gremlin, a graph traversal language, and machine provides a common platform for supporting any graph computing system (such as an OLTP graph database or OLAP graph processors). We present a formalization of graph pattern matching for Gremlin queries. We also study, discuss and consolidate various existing graph algebra operators into an integrated graph algebra.
A Sharp Threshold Phenomenon for the Distributed Complexity of the Lovász Local Lemma
Venue: arXiv cs.DS 2019
Year: 2019

The Lovász Local Lemma (LLL) says that, given a set of bad events that depend on the values of some random variables and where each event happens with probability at most $p$ and depends on at most $d$ other events, there is an assignment of the variables that avoids all bad events if the LLL criterion $ep(d+1)<1$ is satisfied. In this paper, we study the dependency of the distributed complexity of the LLL problem on the chosen LLL criterion. We show that for the fundamental case of each random variable of the considered LLL instance being associated with an edge of the input graph, that is, each random variable influences at most two events, a sharp threshold phenomenon occurs at $p = 2^{-d}$: we provide a simple deterministic (!) algorithm that matches a known $Ω(\log^* n)$ lower bound in bounded degree graphs, if $p < 2^{-d}$, whereas for $p \geq 2^{-d}$, a known $Ω(\log \log n)$ randomized and a known $Ω(\log n)$ deterministic lower bounds hold. In many applications variables affect more than two events; our main contribution is to extend our algorithm to the case where random variables influence at most three different bad events. We show that, surprisingly, the sharp threshold occurs at the exact same spot, providing evidence for our conjecture that this phenomenon always occurs at $p = 2^{-d}$, independent of the number $r$ of events that are affected by a variable. Almost all steps of the proof framework we provide for the case $r=3$ extend directly to the case of arbitrary $r$; consequently, our approach serves as a step towards characterizing the complexity of the LLL under different exponential criteria.
Scheduling Policies for Federated Learning in Wireless Networks
Venue: arXiv cs.IT 2019
Year: 2019

Motivated by the increasing computational capacity of wireless user equipments (UEs), e.g., smart phones, tablets, or vehicles, as well as the increasing concerns about sharing private data, a new machine learning model has emerged, namely federated learning (FL), that allows a decoupling of data acquisition and computation at the central unit. Unlike centralized learning taking place in a data center, FL usually operates in a wireless edge network where the communication medium is resource-constrained and unreliable. Due to limited bandwidth, only a portion of UEs can be scheduled for updates at each iteration. Due to the shared nature of the wireless medium, transmissions are subjected to interference and are not guaranteed. The performance of FL system in such a setting is not well understood. In this paper, an analytical model is developed to characterize the performance of FL in wireless networks. Particularly, tractable expressions are derived for the convergence rate of FL in a wireless setting, accounting for effects from both scheduling schemes and inter-cell interference. Using the developed analysis, the effectiveness of three different scheduling policies, i.e., random scheduling (RS), round robin (RR), and proportional fair (PF), are compared in terms of FL convergence rate. It is shown that running FL with PF outperforms RS and RR if the network is operating under a high signal-to-interference-plus-noise ratio (SINR) threshold, while RR is more preferable when the SINR threshold is low. Moreover, the FL convergence rate decreases rapidly as the SINR threshold increases, thus confirming the importance of compression and quantization of the update parameters. The analysis also reveals a trade-off between the number of scheduled UEs and subchannel bandwidth under a fixed amount of available spectrum.
Occlusion Robust Face Recognition Based on Mask Learning with PairwiseDifferential Siamese Network
Venue: arXiv cs.CV 2019
Year: 2019

Deep Convolutional Neural Networks (CNNs) have been pushing the frontier of the face recognition research in the past years. However, existing general CNN face models generalize poorly to the scenario of occlusions on variable facial areas. Inspired by the fact that a human visual system explicitly ignores occlusions and only focuses on non-occluded facial areas, we propose a mask learning strategy to find and discard the corrupted feature elements for face recognition. A mask dictionary is firstly established by exploiting the differences between the top convoluted features of occluded and occlusion-free face pairs using an innovatively designed Pairwise Differential Siamese Network (PDSN). Each item of this dictionary captures the correspondence between occluded facial areas and corrupted feature elements, which is named Feature Discarding Mask (FDM). When dealing with a face image with random partial occlusions, we generate its FDM by combining relevant dictionary items and then multiply it with the original features to eliminate those corrupted feature elements. Comprehensive experiments on both synthesized and realistic occluded face datasets show that the proposed approach significantly outperforms the state-of-the-arts.
The stable set problem in graphs with bounded genus and bounded odd cycle packing number
Venue: arXiv cs.DM 2019
Year: 2019

Consider the family of graphs without $ k $ node-disjoint odd cycles, where $ k $ is a constant. Determining the complexity of the stable set problem for such graphs $ G $ is a long-standing problem. We give a polynomial-time algorithm for the case that $ G $ can be further embedded in a (possibly non-orientable) surface of bounded genus. Moreover, we obtain polynomial-size extended formulations for the respective stable set polytopes. To this end, we show that $2$-sided odd cycles satisfy the Erdős-Pósa property in graphs embedded in a fixed surface. This extends the fact that odd cycles satisfy the Erdős-Pósa property in graphs embedded in a fixed orientable surface (Kawarabayashi & Nakamoto, 2007). Eventually, our findings allow us to reduce the original problem to the problem of finding a minimum-cost non-negative integer circulation of a certain homology class, which turns out to be efficiently solvable in our case.
A Conceptual Framework for Successful E-commerce Smartphone Applications: The Context of GCC
Venue: arXiv cs.CY 2019
Year: 2019

Rapid expansion of online business has engulfed the GCC region. Such expansion causes competition among business entities, causing the need to identify the factors that the customers use to choose a suitable mobile business application. Instead of just focusing on the visitors/users of the application, a shift in focus towards transforming casual customers to loyal customers is needed. The IS Success Model, whose main constructs are Information Quality, Quality Systems, Service Quality, User Satisfaction, Intention to Use and Net Benefits, includes diversified indicators along with their measures. This research considers User Satisfaction, Intention to Use and Net Benefits constructs as it is, but modified System Quality, Information Quality and Service Quality constructs based on previous state-of-the-art literature. The developed theoretical model was further tested surveying 803 GCC participants. Responses were analyzed using exploratory and confirmatory factor analysis. Study reveals the significance of Service Quality (consisting of M-loyalty Building, Customer Chat and feedback, Help and technical support, and Credibility and Reliability Build) over Information Quality and System Quality, impacting the importance of User Satisfaction over the other constructs.