OpenRaiser/Intern-Atlas huggingface dataset:
Henge: Intent-driven Multi-Tenant Stream Processing
Venue: arXiv cs.DC 2018
Year: 2018

We present Henge, a system to support intent-based multi-tenancy in modern stream processing applications. Henge supports multi-tenancy as a first-class citizen: everyone inside an organization can now submit their stream processing jobs to a single, shared, consolidated cluster. Additionally, Henge allows each tenant (job) to specify its own intents (i.e., requirements) as a Service Level Objective (SLO) that captures latency and/or throughput. In a multi-tenant cluster, the Henge scheduler adapts continually to meet jobs' SLOs in spite of limited cluster resources, and under dynamic input workloads. SLOs are soft and are based on utility functions. Henge continually tracks SLO satisfaction, and when jobs miss their SLOs, it wisely navigates the state space to perform resource allocations in real time, maximizing total system utility achieved by all jobs in the system. Henge is integrated in Apache Storm and we present experimental results using both production topologies and real datasets.
The Great Division
Venue: arXiv cs.SI 2018
Year: 2018

When information flow fails, when Democrats and Republicans do not talk to each other, when Israelis and Palestinians do not talk to each other, and when North Koreans and South Koreans do not talk to each other, mis-perceptions, biases and fake news arise. In this paper we present an in-depth study of political polarization and social division using Twitter data and Monte Carlo simulations. First, we study at the aggregate level people's inclination to retweet within their own ideological circle. Introducing the concept of cocoon ratio, we show that Donald Trump's followers are 2.56 more likely to retweet a fellow Trump follower than to retweet a Hillary Clinton follower. Second, going down to the individual level, we show that the tendency of retweeting exclusively within one's ideological circle is stronger for women than for men and that such tendency weakens as one's social capital grows. Third, we use a one-dimensional Ising model to simulate how a society with high cocoon ratios could end up becoming completely divided. We conclude with a discussion of our findings with respect to fake news.
Full Image Recover for Block-Based Compressive Sensing
Venue: arXiv cs.CV 2018
Year: 2018

Recent years, compressive sensing (CS) has improved greatly for the application of deep learning technology. For convenience, the input image is usually measured and reconstructed block by block. This usually causes block effect in reconstructed images. In this paper, we present a novel CNN-based network to solve this problem. In measurement part, the input image is adaptively measured block by block to acquire a group of measurements. While in reconstruction part, all the measurements from one image are used to reconstruct the full image at the same time. Different from previous method recovering block by block, the structure information destroyed in measurement part is recovered in our framework. Block effect is removed accordingly. We train the proposed framework by mean square error (MSE) loss function. Experiments show that there is no block effect at all in the proposed method. And our results outperform 1.8 dB compared with existing methods.
Interference Mitigation Methods for Unmanned Aerial Vehicles Served by Cellular Networks
Venue: arXiv cs.IT 2018
Year: 2018

A main challenge in providing connectivity to the low altitude unmanned aerial vehicles (UAVs) through existing cellular network arises due to the increased interference in the network. The increased altitude and favourable propagation condition cause UAVs to generate more interference to the neighbouring cells, and at the same time experience more interference from the downlink transmissions of the neighbouring base stations. The uplink interference problem may result in terrestrial UEs having degraded performance, whereas the downlink interference problem may make it challenging for a UAV to maintain connection with the network. In this paper, we propose several uplink and downlink interference mitigation techniques to address these issues. The results indicate that the proposed solutions can reduce the uplink throughput degradation of terrestrial UEs and ensure UAVs to remain in LTE coverage under the worst case scenarios when all the base stations transmit at full power.
On the state space and dynamics selection in linear stochastic models: a spectral factorization approach
Venue: arXiv cs.SY 2018
Year: 2018

Matrix spectral factorization is traditionally described as finding spectral factors having a fixed analytic pole configuration. The classification of spectral factors then involves studying the solutions of a certain algebraic Riccati equation which parametrizes their zero structure. The pole structure of the spectral factors can be also parametrized in terms of solutions of another Riccati equation. We study the relation between the solution sets of these two Riccati equations and describe the construction of general spectral factors which involve both zero- and pole-flipping on an arbitrary reference spectral factor.
Anomaly Detection in Log Data using Graph Databases and Machine Learning to Defend Advanced Persistent Threats
Venue: arXiv cs.DB 2018
Year: 2018

Advanced Persistent Threats (APTs) are a main impendence in cyber security of computer networks. In 2015, a successful breach remains undetected 146 days on average, reported by [Fi16].With our work we demonstrate a feasible and fast way to analyse real world log data to detect breaches or breach attempts. By adapting well-known kill chain mechanisms and a combine of a time series database and an abstracted graph approach, it is possible to create flexible attack profiles. Using this approach, it can be demonstrated that the graph analysis successfully detects simulated attacks by analysing the log data of a simulated computer network. Considering another source for log data, the framework is capable to deliver sufficient performance for analysing real-world data in short time. By using the computing power of the graph database it is possible to identify the attacker and furthermore it is feasible to detect other affected system components. We believe to significantly reduce the detection time of breaches with this approach and react fast to new attack vectors.
Robust Sequential Detection in Distributed Sensor Networks
Venue: arXiv cs.IT 2018
Year: 2018

We consider the problem of sequential binary hypothesis testing with a distributed sensor network in a non-Gaussian noise environment. To this end, we present a general formulation of the Consensus + Innovations Sequential Probability Ratio Test (CISPRT). Furthermore, we introduce two different concepts for robustifying the CISPRT and propose four different algorithms, namely, the Least-Favorable-Density-CISPRT, the Median-CISPRT, the M-CISPRT, and the Myriad-CISPRT. Subsequently, we analyze their suitability for different binary hypothesis tests before verifying and evaluating their performance in a shift-in-mean and a shift-in-variance scenario.
Emerging Language Spaces Learned From Massively Multilingual Corpora
Venue: arXiv cs.CL 2018
Year: 2018

Translations capture important information about languages that can be used as implicit supervision in learning linguistic properties and semantic representations. In an information-centric view, translated texts may be considered as semantic mirrors of the original text and the significant variations that we can observe across various languages can be used to disambiguate a given expression using the linguistic signal that is grounded in translation. Parallel corpora consisting of massive amounts of human translations with a large linguistic variation can be applied to increase abstractions and we propose the use of highly multilingual machine translation models to find language-independent meaning representations. Our initial experiments show that neural machine translation models can indeed learn in such a setup and we can show that the learning algorithm picks up information about the relation between languages in order to optimize transfer leaning with shared parameters. The model creates a continuous language space that represents relationships in terms of geometric distances, which we can visualize to illustrate how languages cluster according to language families and groups. Does this open the door for new ideas of data-driven language typology with promising models and techniques in empirical cross-linguistic research?
Dominance analysis of linear complementarity systems
Venue: arXiv cs.SY 2018
Year: 2018

The paper extends the concepts of dominance and p-dissipativity to the non-smooth family of linear complementarity systems. Dominance generalizes incremental stability whereas p-dissipativity generalizes incremental passivity. The generalization aims at an interconnection theory for the design and analysis of switching and oscillatory systems. The approach is illustrated by a detailed study of classical electrical circuits that switch and oscillate.
Correlation and Prediction of Evaluation Metrics in Information Retrieval
Venue: arXiv cs.IR 2018
Year: 2018

Because researchers typically do not have the time or space to present more than a few evaluation metrics in any published study, it can be difficult to assess relative effectiveness of prior methods for unreported metrics when baselining a new method or conducting a systematic meta-review. While sharing of study data would help alleviate this, recent attempts to encourage consistent sharing have been largely unsuccessful. Instead, we propose to enable relative comparisons with prior work across arbitrary metrics by predicting unreported metrics given one or more reported metrics. In addition, we further investigate prediction of high-cost evaluation measures using low-cost measures as a potential strategy for reducing evaluation cost. We begin by assessing the correlation between 23 IR metrics using 8 TREC test collections. Measuring prediction error wrt. R-square and Kendall's tau, we show that accurate prediction of MAP, P@10, and RBP can be achieved using only 2-3 other metrics. With regard to lowering evaluation cost, we show that RBP(p=0.95) can be predicted with high accuracy using measures with only evaluation depth of 30. Taken together, our findings provide a valuable proof-of-concept which we expect to spur follow-on work by others in proposing more sophisticated models for metric prediction.
A Dynamic Game Approach for Demand-Side Management: Scheduling Energy Storage with Forecasting Errors
Venue: arXiv cs.GT 2018
Year: 2018

Smart metering infrastructure allows for two-way communication and power transfer. Based on this promising technology, we propose a demand-side management (DSM) scheme for a residential neighbourhood of prosumers. Its core is a discrete time dynamic game to schedule individually owned home energy storage. The system model includes an advanced battery model, local generation of renewable energy, and forecasting errors for demand and generation. We derive a closed-form solution for the best-response problem of a player and construct an iterative algorithm to solve the game. Empirical analysis shows exponential convergence towards the Nash equilibrium. A comparison to a DSM scheme with a static game, reveals the advantages of the dynamic game approach. We provide an extensive analysis on the influence of the forecasting error on the outcome of the game. A key result demonstrates that our approach is robust even in the worst-case scenario. This grants considerable gains for the utility company organising the DSM scheme and its participants.
The Helping Hand: An Assistive Manipulation Framework Using Augmented Reality and a Tongue-Drive Interfaces
Venue: arXiv cs.RO 2018
Year: 2018

A human-in-the-loop system is proposed to enable collaborative manipulation tasks for person with physical disabilities. Studies show that the cognitive burden of subject reduces with increased autonomy of assistive system. Our framework obtains high-level intent from the user to specify manipulation tasks. The system processes sensor input to interpret the user's environment. Augmented reality glasses provide ego-centric visual feedback of the interpretation and summarize robot affordances on a menu. A tongue drive system serves as the input modality for triggering a robotic arm to execute the tasks. Assistance experiments compare the system to Cartesian control and to state-of-the-art approaches. Our system achieves competitive results with faster completion time by simplifying manipulation tasks.
Approximating power by weights
Venue: arXiv cs.GT 2018
Year: 2018

Determining the power distribution of the members of a shareholder meeting or a legislative committee is a well-known problem for many applications. In some cases it turns out that power is nearly proportional to relative voting weights, which is very beneficial for both theoretical considerations and practical computations with many members. We present quantitative approximation results with precise error bounds for several power indices as well as impossibility results for such approximations between power and weights.
The effects of anger on automated long-term-spectra based speaker-identification
Venue: arXiv cs.HC 2018
Year: 2018

Forensic speaker identification has traditionally considered approaches based on long term spectra analysis as especially robust, given that they work well for short recordings, are not sensitive to changes in the intensity of the sample, and continue to function in the presence of noise and limited passband. We find, however, that anger induces a significant distortion of the acoustic signal for long term spectra analysis purposes. Even moderate anger offsets speaker identification results by 33% in the direction of a different speaker altogether. Thus, caution should be exercised when applying this tool.
Multipath Communication with Finite Sliding Window Network Coding for Ultra-Reliability and Low Latency
Venue: arXiv cs.NI 2018
Year: 2018

We use random linear network coding (RLNC) based scheme for multipath communication in the presence of lossy links with different delay characteristics to obtain ultra-reliability and low latency. A sliding window version of RLNC is proposed where the coded packets are generated using packets in a window size and are inserted among systematic packets in different paths. The packets are scheduled in the paths in a round robin fashion proportional to the data rates. We use finite encoding and decoding window size and do not rely on feedback for closing the sliding window, unlike the previous work. Our implementation of two paths with LTE and WiFi characteristics shows that the proposed sliding window scheme achieves better latency compared to the block RLNC code. It is also shown that the proposed scheme achieves low latency communication through multiple paths compared to the individual paths for bursty traffic by translating the throughput on both the paths into latency gain.
Decentralized Control of Stochastically Switched Linear System with Unreliable Communication
Venue: arXiv cs.SY 2018
Year: 2018

We consider a networked control system (NCS) consisting of two plants, a global plant and a local plant, and two controllers, a global controller and a local controller. The global (resp. local) plant follows discrete-time stochastically switched linear dynamics with a continuous global (resp. local) state and a discrete global (resp. local) mode. We assume that the state and mode of the global plant are observed by both controllers while the state and mode of the local plant are only observed by the local controller. The local controller can inform the global controller of the local plant's state and mode through an unreliable TCP-like communication channel where successful transmissions are acknowledged. The objective of the controllers is to cooperatively minimize a modes-dependent quadratic cost over a finite time horizon. Following the method developed in [1] and [2], we construct a dynamic program based on common information and a decomposition of strategies, and use it to obtain explicit optimal strategies for the controllers. In the optimal strategies, both controllers compute a common estimate of the local plant's state. The global controller's action is linear in the state of the global plant and the common estimated state, and the local controller's action is linear in the actual states of both plants and the common estimated state. Furthermore, the gain matrices for the global controller depend on the global mode and its observation about the local mode, while the gain matrices for the local controller depend on the actual modes of both plants and the global controller's observation about the local mode.
Detecting Zones and Threat on 3D Body for Security in Airports using Deep Machine Learning
Venue: arXiv cs.CV 2018
Year: 2018

In this research, it was used a segmentation and classification method to identify threat recognition in human scanner images of airport security. The Department of Homeland Security's (DHS) in USA has a higher false alarm, produced from theirs algorithms using today's scanners at the airports. To repair this problem they started a new competition at Kaggle site asking the science community to improve their detection with new algorithms. The dataset used in this research comes from DHS at https://www.kaggle.com/c/passenger-screening-algorithm-challenge/data According to DHS: "This dataset contains a large number of body scans acquired by a new generation of millimeter wave scanner called the High Definition-Advanced Imaging Technology (HD-AIT) system. They are comprised of volunteers wearing different clothing types (from light summer clothes to heavy winter clothes), different body mass indices, different genders, different numbers of threats, and different types of threats". Using Python as a principal language, the preprocessed of the dataset images extracted features from 200 bodies using: intensity, intensity differences and local neighbourhood to detect, to produce segmentation regions and label those regions to be used as a truth in a training and test dataset. The regions are subsequently give to a CNN deep learning classifier to predict 17 classes (that represents the body zones): zone1, zone2, ... zone17 and zones with threat in a total of 34 zones. The analysis showed the results of the classifier an accuracy of 98.2863% and a loss of 0.091319, as well as an average of 100% for recall and precision.
Ensuring Data Integrity in Electronic Health Records: A Quality Health Care Implication
Venue: arXiv cs.CY 2018
Year: 2018

An Electronic Health Record (EHR) system must enable efficient availability of meaningful, accurate and complete data to assist improved clinical administration through the development, implementation and optimisation of clinical pathways. Therefore data integrity is the driving force in EHR systems and is an essential aspect of service delivery at all levels. However, preserving data integrity in EHR systems has become a major problem because of its consequences in promoting high standards of patient care. In this paper, we review and address the impact of data integrity of the use of EHR system and its associated issues. We determine and analyse three phases of data integrity of an EHR system. Finally, we also present an appropriate method to preserve the integrity in EHR systems. To analyse and evaluate the data integrity, one of the major clinical systems in Australia is considered. This will demonstrate the impact on quality and safety of patient care.
Explainable Software Analytics
Venue: arXiv cs.SE 2018
Year: 2018

Software analytics has been the subject of considerable recent attention but is yet to receive significant industry traction. One of the key reasons is that software practitioners are reluctant to trust predictions produced by the analytics machinery without understanding the rationale for those predictions. While complex models such as deep learning and ensemble methods improve predictive performance, they have limited explainability. In this paper, we argue that making software analytics models explainable to software practitioners is as \emph{important} as achieving accurate predictions. Explainability should therefore be a key measure for evaluating software analytics models. We envision that explainability will be a key driver for developing software analytics models that are useful in practice. We outline a research roadmap for this space, building on social science, explainable artificial intelligence and software engineering.
Visual Interpretability for Deep Learning: a Survey
Venue: arXiv cs.CV 2018
Year: 2018

This paper reviews recent studies in understanding neural-network representations and learning neural networks with interpretable/disentangled middle-layer representations. Although deep neural networks have exhibited superior performance in various tasks, the interpretability is always the Achilles' heel of deep neural networks. At present, deep neural networks obtain high discrimination power at the cost of low interpretability of their black-box representations. We believe that high model interpretability may help people to break several bottlenecks of deep learning, e.g., learning from very few annotations, learning via human-computer communications at the semantic level, and semantically debugging network representations. We focus on convolutional neural networks (CNNs), and we revisit the visualization of CNN representations, methods of diagnosing representations of pre-trained CNNs, approaches for disentangling pre-trained CNN representations, learning of CNNs with disentangled representations, and middle-to-end learning based on model interpretability. Finally, we discuss prospective trends in explainable artificial intelligence.
Advanced Symbolic Time Series Analysis in Cyber Physical Systems
Venue: arXiv cs.CE 2018
Year: 2018

This paper presents advanced symbolic time series analysis (ASTSA) for large data sets emanating from cyber physical systems (CPS). The definition of CPS most pertinent to this paper is: A CPS is a system with a coupling of the cyber aspects of computing and communications with the physical aspects of dynamics and engineering that must abide by the laws of physics. This includes sensor networks, real-time and hybrid systems. To ensure that the computation results conform to the laws of physics a linear differential operator (LDO) is embedded in the processing channel for each sensor. In this manner the dynamics of the system can be incorporated prior to performing symbolic analysis. A non-linear quantization is used for the intervals corresponding to the symbols. The intervals are based on observed modes of the system, which can be determined either during an exploratory phase or online during operation of the system. A complete processing channel is called a single channel lexical analyser; one is made available for each sensor on the machine being observed. The implementation of LDO in the system is particularly important since it enables the establishment of a causal link between the observations of the dynamic system and their cause. Without causality there can be no semantics and without semantics no knowledge acquisition based on the physical background of the system being observed. Correlation alone is not a guarantee for causality. This work was originally motivated from the observation of large bulk mate- rial handling systems. Typically, there are $n = 150\dots250$ sensors per machine, and data is collected in a multi rate manner; whereby general sensors are sampled with $f_s = 1Hz$ and vibration data being sampled in the kilo-hertz range.