OpenRaiser/Intern-Atlas huggingface dataset:
Deploying Jupyter Notebooks at scale on XSEDE resources for Science Gateways and workshops
Venue: arXiv cs.DC 2018
Year: 2018

Jupyter Notebooks have become a mainstream tool for interactive computing in every field of science. Jupyter Notebooks are suitable as companion applications for Science Gateways, providing more flexibility and post-processing capability to the users. Moreover they are often used in training events and workshops to provide immediate access to a pre-configured interactive computing environment. The Jupyter team released the JupyterHub web application to provide a platform where multiple users can login and access a Jupyter Notebook environment. When the number of users and memory requirements are low, it is easy to setup JupyterHub on a single server. However, setup becomes more complicated when we need to serve Jupyter Notebooks at scale to tens or hundreds of users. In this paper we will present three strategies for deploying JupyterHub at scale on XSEDE resources. All options share the deployment of JupyterHub on a Virtual Machine on XSEDE Jetstream. In the first scenario, JupyterHub connects to a supercomputer and launches a single node job on behalf of each user and proxies back the Notebook from the computing node back to the user's browser. In the second scenario, implemented in the context of a XSEDE consultation for the IRIS consortium for Seismology, we deploy Docker in Swarm mode to coordinate many XSEDE Jetstream virtual machines to provide Notebooks with persistent storage and quota. In the last scenario we install the Kubernetes containers orchestration framework on Jetstream to provide a fault-tolerant JupyterHub deployment with a distributed filesystem and capability to scale to thousands of users. In the conclusion section we provide a link to step-by-step tutorials complete with all the necessary commands and configuration files to replicate these deployments.
Video Processing on the Edge for Multimedia IoT Systems
Venue: arXiv cs.MM 2018
Year: 2018

In this article, we first survey the current situation of video processing on the edge for multimedia Internet-of-Things (M-IoT) systems in three typical scenarios, i.e., smart cities, satellite networks, and Internet-of-Vehicles. By summarizing a general model of the edge video processing, the importance of developing an edge computing platform is highlighted. Then, we give a method of implementing cooperative video processing on an edge computing platform based on light-weighted virtualization technologies. Performance evaluation is conducted and some insightful observations can be obtained. Moreover, we summarize challenges and opportunities of realizing effective edge video processing for M-IoT systems.
An Almost Tight RMR Lower Bound for Abortable Test-And-Set
Venue: arXiv cs.DC 2018
Year: 2018

We prove a lower bound of Omega(log n/loglog n) for the remote memory reference (RMR) complexity of abortable test-and-set (leader election) in the cache-coherent (CC) and the distributed shared memory (DSM) model. This separates the complexities of abortable and non-abortable test-and-set, as the latter has constant RMR complexity (Golab, Hendler, Woelfel, SIAM Journal of Computing Vol. 39, 2010). Golab, Hendler, Hadzilacos and Woelfel (Distributed Computing Vol. 25, 2012) showed that compare-and-swap can be implemented from registers and TAS objects with constant RMR complexity. We observe that a small modification to that implementation is abortable, provided that the used TAS objects are abortable.
SPAIDS and OAMS Models in Wireless Ad Hoc Networks
Venue: arXiv cs.DC 2018
Year: 2018

In this paper, we present two randomized distributed algorithms in wireless ad hoc networks. We consider that the network is structured into pairs of nodes (sender, receiver) in a decay space. We take into account the following: Each node has its own power assignment and the distance between them does not follow the symmetry property. Then, we consider a non-uniform network or a realistic wireless network, which is beyond the geometry. Our model is based on the Signal to Interference plus Noise Ratio (SINR) model. In this work, the main problem is to solve the scheduling task aiming the successful transmission of messages in a realistic environment. Therefore, we propose the first randomized scheduling and power selection algorithm in a decay space and is called as SPAIDS. In order to solve the problem in this non-uniform network, we introduce a new way to study the affectance (the interference) among the links, which is defined as Weighted Average Affectance (WAFF). Moreover, we study the online broadcast problem in a metric space, in which the nodes are activated in case that they receive packets. We propose an online algorithm in a metric space which is denoted as OAMS. Our aim is to obtain the maximum subset of nodes that receive the message from a sender node with enough energy supplies. Finally, we compare the performance of OAMS to the optimal.
UnibucKernel Reloaded: First Place in Arabic Dialect Identification for the Second Year in a Row
Venue: arXiv cs.CL 2018
Year: 2018

We present a machine learning approach that ranked on the first place in the Arabic Dialect Identification (ADI) Closed Shared Tasks of the 2018 VarDial Evaluation Campaign. The proposed approach combines several kernels using multiple kernel learning. While most of our kernels are based on character p-grams (also known as n-grams) extracted from speech or phonetic transcripts, we also use a kernel based on dialectal embeddings generated from audio recordings by the organizers. In the learning stage, we independently employ Kernel Discriminant Analysis (KDA) and Kernel Ridge Regression (KRR). Preliminary experiments indicate that KRR provides better classification results. Our approach is shallow and simple, but the empirical results obtained in the 2018 ADI Closed Shared Task prove that it achieves the best performance. Furthermore, our top macro-F1 score (58.92%) is significantly better than the second best score (57.59%) in the 2018 ADI Shared Task, according to the statistical significance test performed by the organizers. Nevertheless, we obtain even better post-competition results (a macro-F1 score of 62.28%) using the audio embeddings released by the organizers after the competition. With a very similar approach (that did not include phonetic features), we also ranked first in the ADI Closed Shared Tasks of the 2017 VarDial Evaluation Campaign, surpassing the second best method by 4.62%. We therefore conclude that our multiple kernel learning method is the best approach to date for Arabic dialect identification.
Building Near-Real-Time Processing Pipelines with the Spark-MPI Platform
Venue: arXiv cs.DC 2018
Year: 2018

Advances in detectors and computational technologies provide new opportunities for applied research and the fundamental sciences. Concurrently, dramatic increases in the three Vs (Volume, Velocity, and Variety) of experimental data and the scale of computational tasks produced the demand for new real-time processing systems at experimental facilities. Recently, this demand was addressed by the Spark-MPI approach connecting the Spark data-intensive platform with the MPI high-performance framework. In contrast with existing data management and analytics systems, Spark introduced a new middleware based on resilient distributed datasets (RDDs), which decoupled various data sources from high-level processing algorithms. The RDD middleware significantly advanced the scope of data-intensive applications, spreading from SQL queries to machine learning to graph processing. Spark-MPI further extended the Spark ecosystem with the MPI applications using the Process Management Interface. The paper explores this integrated platform within the context of online ptychographic and tomographic reconstruction pipelines.
Emergence and Evolution of Hierarchical Structure in Complex Systems
Venue: arXiv cs.NE 2018
Year: 2018

It is well known that many complex systems, both in technology and nature, exhibit hierarchical modularity: smaller modules, each of them providing a certain function, are used within larger modules that perform more complex functions. What is not well understood however is how this hierarchical structure (which is fundamentally a network property) emerges, and how it evolves over time. We propose a modeling framework, referred to as Evo-Lexis, that provides insight to some fundamental questions about evolving hierarchical systems. Evo-Lexis models the most elementary modules of the system as symbols ("sources") and the modules at the highest level of the hierarchy as sequences of those symbols ("targets"). Evo-Lexis computes the optimized adjustment of a given hierarchy when the set of targets changes over time by additions and removals (a process referred to as "incremental design"). In this paper we use computation modeling to show that: - Low-cost and deep hierarchies emerge when the population of target sequences evolves through tinkering and mutation. - Strong selection on the cost of new candidate targets results in reuse of more complex (longer) nodes in an optimized hierarchy. - The bias towards reuse of complex nodes results in an "hourglass architecture" (i.e., few intermediate nodes that cover almost all source-target paths). - With such bias, the core nodes are conserved for relatively long time periods although still being vulnerable to major transitions and punctuated equilibria. - Finally, we analyze the differences in terms of cost and structure between incrementally designed hierarchies and the corresponding "clean-slate" hierarchies which result when the system is designed from scratch after a change.
Doing the impossible: Why neural networks can be trained at all
Venue: arXiv cs.LG 2018
Year: 2018

As deep neural networks grow in size, from thousands to millions to billions of weights, the performance of those networks becomes limited by our ability to accurately train them. A common naive question arises: if we have a system with billions of degrees of freedom, don't we also need billions of samples to train it? Of course, the success of deep learning indicates that reliable models can be learned with reasonable amounts of data. Similar questions arise in protein folding, spin glasses and biological neural networks. With effectively infinite potential folding/spin/wiring configurations, how does the system find the precise arrangement that leads to useful and robust results? Simple sampling of the possible configurations until an optimal one is reached is not a viable option even if one waited for the age of the universe. On the contrary, there appears to be a mechanism in the above phenomena that forces them to achieve configurations that live on a low-dimensional manifold, avoiding the curse of dimensionality. In the current work we use the concept of mutual information between successive layers of a deep neural network to elucidate this mechanism and suggest possible ways of exploiting it to accelerate training. We show that adding structure to the neural network that enforces higher mutual information between layers speeds training and leads to more accurate results. High mutual information between layers implies that the effective number of free parameters is exponentially smaller than the raw number of tunable weights.
Dyna: A Method of Momentum for Stochastic Optimization
Venue: arXiv cs.LG 2018
Year: 2018

An algorithm is presented for momentum gradient descent optimization based on the first-order differential equation of the Newtonian dynamics. The fictitious mass is introduced to the dynamics of momentum for regularizing the adaptive stepsize of each individual parameter. The dynamic relaxation is adapted for stochastic optimization of nonlinear objective functions through an explicit time integration with varying damping ratio. The adaptive stepsize is optimized for each individual neural network layer based on the number of inputs. The adaptive stepsize for every parameter over the entire neural network is uniformly optimized with one upper bound, independent of sparsity, for better overall convergence rate. The numerical implementation of the algorithm is similar to the Adam Optimizer, possessing computational efficiency, similar memory requirements, etc. There are three hyper-parameters in the algorithm with clear physical interpretation. Preliminary trials show promise in performance and convergence.
Learning Temporal Strategic Relationships using Generative Adversarial Imitation Learning
Venue: arXiv cs.CV 2018
Year: 2018

This paper presents a novel framework for automatic learning of complex strategies in human decision making. The task that we are interested in is to better facilitate long term planning for complex, multi-step events. We observe temporal relationships at the subtask level of expert demonstrations, and determine the different strategies employed in order to successfully complete a task. To capture the relationship between the subtasks and the overall goal, we utilise two external memory modules, one for capturing dependencies within a single expert demonstration, such as the sequential relationship among different sub tasks, and a global memory module for modelling task level characteristics such as best practice employed by different humans based on their domain expertise. Furthermore, we demonstrate how the hidden state representation of the memory can be used as a reward signal to smooth the state transitions, eradicating subtle changes. We evaluate the effectiveness of the proposed model for an autonomous highway driving application, where we demonstrate its capability to learn different expert policies and outperform state-of-the-art methods. The scope in industrial applications extends to any robotics and automation application which requires learning from complex demonstrations containing series of subtasks.
Multiple Antenna Aided NOMA in UAV Networks: A Stochastic Geometry Approach
Venue: arXiv cs.IT 2018
Year: 2018

This article investigates the multiple-input multiple-output (MIMO) non-orthogonal multiple access (NOMA) assisted unmanned aerial vehicles (UAVs) networks. By utilizing a stochastic geometry model, a new 3-Dimension UAV framework for providing wireless service to randomly roaming NOMA users has been proposed. In an effort to evaluate the performance of the proposed framework, we derive analytical expressions for the outage probability and the ergodic rate of MIMO-NOMA enhanced UAV networks. We examine tractable upper bounds for the whole proposed framework, with deriving asymptotic results for scenarios that transmit power of interference sources being proportional or being fixed to the UAV. For obtaining more insights for the proposed framework, we investigate the diversity order and high signal-to-noise (SNR) slope of MIMO-NOMA assisted UAV networks. Our results confirm that: i) The outage probability of NOMA enhanced UAV networks is affected to a large extent by the targeted transmission rates and power allocation factors of NOMA users; and ii) For the case that the interference power is proportional to the UAV power, there are error floors for the outage probabilities.
Double-Spending Risk Quantification in Private, Consortium and Public Ethereum Blockchains
Venue: arXiv cs.CR 2018
Year: 2018

Recently, several works conjectured the vulnerabilities of mainstream blockchains under several network attacks. All these attacks translate into showing that the assumptions of these blockchains can be violated in theory or under simulation at best. Unfortunately, previous results typically omit both the nature of the network under which the blockchain code runs and whether blockchains are private, consortium or public. In this paper, we study the public Ethereum blockchain as well as a consortium and private blockchains and quantify the feasibility of man-in-the-middle and double spending attacks against them. To this end, we list important properties of the Ethereum public blockchain topology, we deploy VMs with constrained CPU quantum to mimic the top-10 mining pools of Ethereum and we develop full-fledged attacks, that first partition the network through BGP hijacking or ARP spoofing before issuing a Balance Attack to steal coins. Our results demonstrate that attacking Ethereum is remarkably devastating in a consortium or private context as the adversary can multiply her digital assets by 200, 000x in 10 hours through BGP hijacking whereas it would be almost impossible in a public context.
A Deep Learning Approach with an Attention Mechanism for Automatic Sleep Stage Classification
Venue: arXiv cs.LG 2018
Year: 2018

Automatic sleep staging is a challenging problem and state-of-the-art algorithms have not yet reached satisfactory performance to be used instead of manual scoring by a sleep technician. Much research has been done to find good feature representations that extract the useful information to correctly classify each epoch into the correct sleep stage. While many useful features have been discovered, the amount of features have grown to an extent that a feature reduction step is necessary in order to avoid the curse of dimensionality. One reason for the need of such a large feature set is that many features are good for discriminating only one of the sleep stages and are less informative during other stages. This paper explores how a second feature representation over a large set of pre-defined features can be learned using an auto-encoder with a selective attention for the current sleep stage in the training batch. This selective attention allows the model to learn feature representations that focuses on the more relevant inputs without having to perform any dimensionality reduction of the input data. The performance of the proposed algorithm is evaluated on a large data set of polysomnography (PSG) night recordings of patients with sleep-disordered breathing. The performance of the auto-encoder with selective attention is compared with a regular auto-encoder and previous works using a deep belief network (DBN).
Improved Reconciliation With Polar Codes In Quantum Key Distribution
Venue: arXiv cs.IT 2018
Year: 2018

Quantum key distribution (QKD) is a cryptographic system that generates an information-theoretically secure key shared by two legitimate parties. QKD consists of two parts: quantum and classical. The latter is referred to as classical post-processing (CPP). Information reconciliation is a part of CPP in which parties are given correlated variables and attempt to eliminate the discrepancies between them while disclosing a minimum amount of information. The elegant reconciliation protocol known as \emph{Cascade} was developed specifically for QKD in 1992 and has become the de-facto standard for all QKD implementations. However, the protocol is highly interactive. Thus, other protocols based on linear block codes such as Hamming codes, low-density parity-check (LDPC) codes, and polar codes have been researched. In particular, reconciliation using LDPC codes has been mainly studied because of its outstanding performance. Nevertheless, with small block size, the bit error rate performance of polar codes under successive-cancellation list (SCL) decoding with a cyclic redundancy check (CRC) is comparable to state-of-the-art turbo and LDPC codes. In this study, we demonstrate the use of polar codes to improve the performance of information reconciliation in a QKD system with small block size. The best decoder for polar codes, a CRC-aided SCL decoder, requires CRC-precoded messages. However, messages that are sifted keys in QKD are obtained arbitrarily as a result of a characteristic of the QKD protocol and cannot be CRC-precoded. We propose a method that allows arbitrarily obtained sifted keys to be CRC precoded by introducing a virtual string. Thus the best decoder can be used for reconciliation using polar codes and improves the efficiency of the protocol.
Functional Baby Talk: Analysis of Code Fragments from Novice Haskell Programmers
Venue: arXiv cs.PL 2018
Year: 2018

What kinds of mistakes are made by novice Haskell developers, as they learn about functional programming? Is it possible to analyze these errors in order to improve the pedagogy of Haskell? In 2016, we delivered a massive open online course which featured an interactive code evaluation environment. We captured and analyzed 161K interactions from learners. We report typical novice developer behavior; for instance, the mean time spent on an interactive tutorial is around eight minutes. Although our environment was restricted, we gain some understanding of Haskell novice errors. Parenthesis mismatches, lexical scoping errors and do block misunderstandings are common. Finally, we make recommendations about how such beginner code evaluation environments might be enhanced.
BioPhysical Modeling, Characterization and Optimization of Electro-Quasistatic Human Body Communication
Venue: arXiv cs.ET 2018
Year: 2018

Human Body Communication (HBC) has emerged as an alternative to radio wave communication for connecting low power, miniaturized wearable and implantable devices in, on and around the human body which uses the human body as the communication channel. Previous studies characterizing the human body channel has reported widely varying channel response much of which has been attributed to the variation in measurement setup. This calls for the development of a unifying bio physical model of HBC supported by in depth analysis and an understanding of the effect of excitation, termination modality on HBC measurements. This paper characterizes the human body channel up to 1MHz frequency to evaluate it as a medium for broadband communication. A lumped bio physical model of HBC is developed, supported by experimental validations that provides insight into some of the key discrepancies found in previous studies. Voltage loss measurements are carried out both with an oscilloscope and a miniaturized wearable prototype to capture the effects of non common ground. Results show that the channel loss is strongly dependent on the termination impedance at the receiver end, with up to 4dB variation in average loss for different termination in an oscilloscope and an additional 9 dB channel loss with wearable prototype compared to an oscilloscope measurement. The measured channel response with capacitive termination reduces low frequency loss and allows flat band transfer function down to 13 KHz, establishing the human body as a broadband communication channel. Analysis of the measured results and the simulation model shows that (1) high impedance (2) capacitive termination should be used at the receiver end for accurate voltage mode loss measurements of the HBC channel at low frequencies.
Breaking the Scalability Barrier of Causal Broadcast for Large and Dynamic Systems
Venue: arXiv cs.DC 2018
Year: 2018

Many distributed protocols and applications rely on causal broadcast to ensure consistency criteria. However, none of causality tracking state-of-the-art approaches scale in large and dynamic systems. This paper presents a new non-blocking causal broadcast protocol suited for dynamic systems. The proposed protocol outperforms state-of-the-art in size of messages, execution time complexity, and local space complexity. Most importantly, messages piggyback control information the size of which is constant. We prove that for both static and dynamic systems. Consequently, large and dynamic systems can finally afford causal broadcast.