OpenRaiser/Intern-Atlas huggingface dataset:
Precision of the ENDGame: Mixed-precision arithmetic in the iterative solver of the Unified Model
Venue: arXiv cs.NA 2018
Year: 2018

The Met Office's weather and climate simulation code the Unified Model is used for both operational Numerical Weather Prediction and Climate modelling. The computational performance of the model running on parallel supercomputers is a key consideration. A Krylov sub-space solver is employed to solve the equations of the dynamical core of the model, known as ENDGame. These describe the evolution of the Earth's atmosphere. Typically, 64-bit precision is used throughout weather and climate applications. This work presents a mixed-precision implementation of the solver, the beneficial effect on run-time and the impact on solver convergence. The complex interplay of errors arising from accumulated round-off in floating-point arithmetic and other numerical effects is discussed. A careful analysis is required, however, the mixed-precision solver is now employed in the operational forecast to satisfy run-time constraints without compromising the accuracy of the solution.
Sample-Efficient Policy Learning based on Completely Behavior Cloning
Venue: arXiv cs.LG 2018
Year: 2018

Direct policy search is one of the most important algorithm of reinforcement learning. However, learning from scratch needs a large amount of experience data and can be easily prone to poor local optima. In addition to that, a partially trained policy tends to perform dangerous action to agent and environment. In order to overcome these challenges, this paper proposed a policy initialization algorithm called Policy Learning based on Completely Behavior Cloning (PLCBC). PLCBC first transforms the Model Predictive Control (MPC) controller into a piecewise affine (PWA) function using multi-parametric programming, and uses a neural network to express this function. By this way, PLCBC can completely clone the MPC controller without any performance loss, and is totally training-free. The experiments show that this initialization strategy can help agent learn at the high reward state region, and converge faster and better.
Learning Semantic Representations for Novel Words: Leveraging Both Form and Context
Venue: arXiv cs.CL 2018
Year: 2018

Word embeddings are a key component of high-performing natural language processing (NLP) systems, but it remains a challenge to learn good representations for novel words on the fly, i.e., for words that did not occur in the training data. The general problem setting is that word embeddings are induced on an unlabeled training corpus and then a model is trained that embeds novel words into this induced embedding space. Currently, two approaches for learning embeddings of novel words exist: (i) learning an embedding from the novel word's surface-form (e.g., subword n-grams) and (ii) learning an embedding from the context in which it occurs. In this paper, we propose an architecture that leverages both sources of information - surface-form and context - and show that it results in large increases in embedding quality. Our architecture obtains state-of-the-art results on the Definitional Nonce and Contextual Rare Words datasets. As input, we only require an embedding set and an unlabeled corpus for training our architecture to produce embeddings appropriate for the induced embedding space. Thus, our model can easily be integrated into any existing NLP system and enhance its capability to handle novel words.
Suggesting Cooking Recipes Through Simulation and Bayesian Optimization
Venue: arXiv cs.AI 2018
Year: 2018

Cooking typically involves a plethora of decisions about ingredients and tools that need to be chosen in order to write a good cooking recipe. Cooking can be modelled in an optimization framework, as it involves a search space of ingredients, kitchen tools, cooking times or temperatures. If we model as an objective function the quality of the recipe, several problems arise. No analytical expression can model all the recipes, so no gradients are available. The objective function is subjective, in other words, it contains noise. Moreover, evaluations are expensive both in time and human resources. Bayesian Optimization (BO) emerges as an ideal methodology to tackle problems with these characteristics. In this paper, we propose a methodology to suggest recipe recommendations based on a Machine Learning (ML) model that fits real and simulated data and BO. We provide empirical evidence with two experiments that support the adequacy of the methodology.
RadIoT: Radio Communications Intrusion Detection for IoT - A Protocol Independent Approach
Venue: arXiv cs.CR 2018
Year: 2018

Internet-of-Things (IoT) devices are nowadays massively integrated in daily life: homes, factories, or public places. This technology offers attractive services to improve the quality of life as well as new economic markets through the exploitation of the collected data. However, these connected objects have also become attractive targets for attackers because their current security design is often weak or flawed, as illustrated by several vulnerabilities such as Mirai, Blueborne, etc. This paper presents a novel approach for detecting intrusions in smart spaces such as smarthomes, or smartfactories, that is based on the monitoring and profiling of radio communications at the physical layer using machine learning techniques. The approach is designed to be independent of the large and heterogeneous set of wireless communication protocols typically implemented by connected objects such as WiFi, Bluetooth, Zigbee, Bluetooth-Low-Energy (BLE) or proprietary communication protocols. The main concepts of the proposed approach are presented together with an experimental case study illustrating its feasibility based on data collected during the deployment of the intrusion detection approach in a smart home under real-life conditions.
Modeling Rape Reporting Delays Using Spatial, Temporal and Social Features
Venue: arXiv cs.CY 2018
Year: 2018

We present a novel approach to estimate the delay observed between the occurrence and reporting of rape crimes. We explore spatial, temporal and social effects in sparse aggregated (area-level) and high-dimensional disaggregated (event-level) data for New York and Los Angeles. Focusing on inference, we apply Gradient Boosting and Random Forests to assess predictor importance, as well as Gaussian Processes to model spatial disparities in reporting times. Our results highlight differences and similarities between the two cities. We identify at-risk populations and communities which may be targeted with focused policies and interventions to support rape victims, apprehend perpetrators, and prevent future crimes.
Deep Compression of Sum-Product Networks on Tensor Networks
Venue: arXiv cs.LG 2018
Year: 2018

Sum-product networks (SPNs) represent an emerging class of neural networks with clear probabilistic semantics and superior inference speed over graphical models. This work reveals a strikingly intimate connection between SPNs and tensor networks, thus leading to a highly efficient representation that we call tensor SPNs (tSPNs). For the first time, through mapping an SPN onto a tSPN and employing novel optimization techniques, we demonstrate remarkable parameter compression with negligible loss in accuracy.
Science Hackathons for Cyberphysical System Security Research: Putting CPS testbed platforms to good use
Venue: arXiv cs.CR 2018
Year: 2018

A challenge is to develop cyber-physical system scenarios that reflect the diversity and complexity of real-life cyber-physical systems in the research questions that they address. Time-bounded collaborative events, such as hackathons, jams and sprints, are increasingly used as a means of bringing groups of individuals together, in order to explore challenges and develop solutions. This paper describes our experiences, using a science hackathon to bring individual researchers together, in order to develop a common use-case implemented on a shared CPS testbed platform that embodies the diversity in their own security research questions. A qualitative study of the event was conducted, in order to evaluate the success of the process, with a view to improving future similar events.
Representation-Oblivious Error Correction by Natural Redundancy
Venue: arXiv cs.IT 2018
Year: 2018

Storage systems have a strong need for substantially improving their error correction capabilities, especially for long-term storage where the accumulating errors can exceed the decoding threshold of error-correcting codes (ECCs). In this work, a new scheme is presented that uses deep learning to perform soft decoding for noisy files based on their natural redundancy. The soft decoding result is then combined with ECCs for substantially better error correction performance. The scheme is representation-oblivious: it requires no prior knowledge on how data are represented (e.g., mapped from symbols to bits, compressed, and combined with meta data) in different types of files, which makes the solution more convenient to use for storage systems. Experimental results confirm that the scheme can substantially improve the ability to recover data for different types of files even when the bit error rates in the files have significantly exceeded the decoding threshold of the ECC.
Polynomial-time Approximation Scheme for Minimum k-cut in Planar and Minor-free Graphs
Venue: arXiv cs.DS 2018
Year: 2018

The $k$-cut problem asks, given a connected graph $G$ and a positive integer $k$, to find a minimum-weight set of edges whose removal splits $G$ into $k$ connected components. We give the first polynomial-time algorithm with approximation factor $2-ε$ (with constant $ε> 0$) for the $k$-cut problem in planar and minor-free graphs. Applying more complex techniques, we further improve our method and give a polynomial-time approximation scheme for the $k$-cut problem in both planar and minor-free graphs. Despite persistent effort, to the best of our knowledge, this is the first improvement for the $k$-cut problem over standard approximation factor of $2$ in any major class of graphs.
ADNet: A Deep Network for Detecting Adverts
Venue: arXiv cs.MM 2018
Year: 2018

Online video advertising gives content providers the ability to deliver compelling content, reach a growing audience, and generate additional revenue from online media. Recently, advertising strategies are designed to look for original advert(s) in a video frame, and replacing them with new adverts. These strategies, popularly known as product placement or embedded marketing, greatly help the marketing agencies to reach out to a wider audience. However, in the existing literature, such detection of candidate frames in a video sequence for the purpose of advert integration, is done manually. In this paper, we propose a deep-learning architecture called ADNet, that automatically detects the presence of advertisements in video frames. Our approach is the first of its kind that automatically detects the presence of adverts in a video frame, and achieves state-of-the-art results on a public dataset.
Zero-shot Neural Transfer for Cross-lingual Entity Linking
Venue: arXiv cs.CL 2018
Year: 2018

Cross-lingual entity linking maps an entity mention in a source language to its corresponding entry in a structured knowledge base that is in a different (target) language. While previous work relies heavily on bilingual lexical resources to bridge the gap between the source and the target languages, these resources are scarce or unavailable for many low-resource languages. To address this problem, we investigate zero-shot cross-lingual entity linking, in which we assume no bilingual lexical resources are available in the source low-resource language. Specifically, we propose pivot-based entity linking, which leverages information from a high-resource "pivot" language to train character-level neural entity linking models that are transferred to the source low-resource language in a zero-shot manner. With experiments on 9 low-resource languages and transfer through a total of 54 languages, we show that our proposed pivot-based framework improves entity linking accuracy 17% (absolute) on average over the baseline systems, for the zero-shot scenario. Further, we also investigate the use of language-universal phonological representations which improves average accuracy (absolute) by 36% when transferring between languages that use different scripts.
Dual Latent Variable Model for Low-Resource Natural Language Generation in Dialogue Systems
Venue: arXiv cs.CL 2018
Year: 2018

Recent deep learning models have shown improving results to natural language generation (NLG) irrespective of providing sufficient annotated data. However, a modest training data may harm such models performance. Thus, how to build a generator that can utilize as much of knowledge from a low-resource setting data is a crucial issue in NLG. This paper presents a variational neural-based generation model to tackle the NLG problem of having limited labeled dataset, in which we integrate a variational inference into an encoder-decoder generator and introduce a novel auxiliary autoencoding with an effective training procedure. Experiments showed that the proposed methods not only outperform the previous models when having sufficient training dataset but also show strong ability to work acceptably well when the training data is scarce.
New Movement and Transformation Principle of Fuzzy Reasoning and Its Application to Fuzzy Neural Network
Venue: arXiv cs.AI 2018
Year: 2018

In this paper, we propose a new fuzzy reasoning principle, so called Movement and Transformation Principle(MTP). This Principle is to obtain a new fuzzy reasoning result by Movement and Transformation the consequent fuzzy set in response to the Movement, Transformation, and Movement-Transformation operations between the antecedent fuzzy set and fuzzificated observation information. And then we presented fuzzy modus ponens and fuzzy modus tollens based on MTP. We compare proposed method with Mamdani fuzzy system, Sugeno fuzzy system, Wang distance type fuzzy reasoning method and Hellendoorn functional type method. And then we applied to the learning experiments of the fuzzy neural network based on MTP and compared it with the Sugeno method. Through prediction experiments of fuzzy neural network on the precipitation data and security situation data, learning accuracy and time performance are clearly improved. Consequently we show that our method based on MTP is computationally simple and does not involve nonlinear operations, so it is easy to handle mathematically.
An efficient branch-and-bound algorithm for submodular function maximization
Venue: arXiv cs.DS 2018
Year: 2018

The submodular function maximization is an attractive optimization model that appears in many real applications. Although a variety of greedy algorithms quickly find good feasible solutions for many instances while guaranteeing (1-1/e)-approximation ratio, we still encounter many real applications that ask optimal or better feasible solutions within reasonable computation time. In this paper, we present an efficient branch-and-bound algorithm for the non-decreasing submodular function maximization problem based on its binary integer programming (BIP) formulation with a huge number of constraints. Nemhauser and Wolsey developed an exact algorithm called the constraint generation algorithm that starts from a reduced BIP problem with a small subset of constraints taken from the constraints and repeats solving a reduced BIP problem while adding a new constraint at each iteration. However, their algorithm is still computationally expensive due to many reduced BIP problems to be solved. To overcome this, we propose an improved constraint generation algorithm to add a promising set of constraints at each iteration. We incorporate it into a branch-and-bound algorithm to attain good upper bounds while solving a smaller number of reduced BIP problems. According to computational results for well-known benchmark instances, our algorithm achieved better performance than the state-of-the-art exact algorithms.
Adversarially-Trained Normalized Noisy-Feature Auto-Encoder for Text Generation
Venue: arXiv cs.CL 2018
Year: 2018

This article proposes Adversarially-Trained Normalized Noisy-Feature Auto-Encoder (ATNNFAE) for byte-level text generation. An ATNNFAE consists of an auto-encoder where the internal code is normalized on the unit sphere and corrupted by additive noise. Simultaneously, a replica of the decoder (sharing the same parameters as the AE decoder) is used as the generator and fed with random latent vectors. An adversarial discriminator is trained to distinguish training samples reconstructed from the AE from samples produced through the random-input generator, making the entire generator-discriminator path differentiable for discrete data like text. The combined effect of noise injection in the code and shared weights between the decoder and the generator can prevent the mode collapsing phenomenon commonly observed in GANs. Since perplexity cannot be applied to non-sequential text generation, we propose a new evaluation method using the total variance distance between frequencies of hash-coded byte-level n-grams (NGTVD). NGTVD is a single benchmark that can characterize both the quality and the diversity of the generated texts. Experiments are offered in 6 large-scale datasets in Arabic, Chinese and English, with comparisons against n-gram baselines and recurrent neural networks (RNNs). Ablation study on both the noise level and the discriminator is performed. We find that RNNs have trouble competing with the n-gram baselines, and the ATNNFAE results are generally competitive.
A Note on Local Mode-in-State Participation Factors for Nonlinear Systems
Venue: arXiv cs.SY 2018
Year: 2018

The paper studies an extension to nonlinear systems of a recently proposed approach to the concept of modal participation factors. First, a definition is given for local mode-in-state participation factors for smooth nonlinear autonomous systems. The definition is general, and, unlike in the more traditional approach, the resulting participation measures depend on the assumed uncertainty law governing the system initial condition. The work follows Hashlamoun, Hassouneh and Abed (2009) in taking a mathematical expectation (or set-theoretic average) of a modal contribution measure with respect to an assumed uncertain initial state. As in the linear case, it is found that a symmetry assumption on the distribution of the initial state results in a tractable calculation and an explicit and simple formula for mode-in-state participation factors.
Deep Learning Approach for Building Detection in Satellite Multispectral Imagery
Venue: arXiv cs.CV 2018
Year: 2018

Building detection from satellite multispectral imagery data is being a fundamental but a challenging problem mainly because it requires correct recovery of building footprints from high-resolution images. In this work, we propose a deep learning approach for building detection by applying numerous enhancements throughout the process. Initial dataset is preprocessed by 2-sigma percentile normalization. Then data preparation includes ensemble modelling where 3 models were created while incorporating OpenStreetMap data. Binary Distance Transformation (BDT) is used for improving data labeling process and the U-Net (Convolutional Networks for Biomedical Image Segmentation) is modified by adding batch normalization wrappers. Afterwards, it is explained how each component of our approach is correlated with the final detection accuracy. Finally, we compare our results with winning solutions of SpaceNet 2 competition for real satellite multispectral images of Vegas, Paris, Shanghai and Khartoum, demonstrating the importance of our solution for achieving higher building detection accuracy.
IP Geolocation through Reverse DNS
Venue: arXiv cs.NI 2018
Year: 2018

IP Geolocation databases are widely used in online services to map end user IP addresses to their geographical locations. However, they use proprietary geolocation methods and in some cases they have poor accuracy. We propose a systematic approach to use publicly accessible reverse DNS hostnames for geolocating IP addresses. Our method is designed to be combined with other geolocation data sources. We cast the task as a machine learning problem where for a given hostname, we generate and rank a list of potential location candidates. We evaluate our approach against three state of the art academic baselines and two state of the art commercial IP geolocation databases. We show that our work significantly outperforms the academic baselines, and is complementary and competitive with commercial databases. To aid reproducibility, we open source our entire approach.
Coronary Calcium Detection using 3D Attention Identical Dual Deep Network Based on Weakly Supervised Learning
Venue: arXiv cs.CV 2018
Year: 2018

Coronary artery calcium (CAC) is biomarker of advanced subclinical coronary artery disease and predicts myocardial infarction and death prior to age 60 years. The slice-wise manual delineation has been regarded as the gold standard of coronary calcium detection. However, manual efforts are time and resource consuming and even impracticable to be applied on large-scale cohorts. In this paper, we propose the attention identical dual network (AID-Net) to perform CAC detection using scan-rescan longitudinal non-contrast CT scans with weakly supervised attention by only using per scan level labels. To leverage the performance, 3D attention mechanisms were integrated into the AID-Net to provide complementary information for classification tasks. Moreover, the 3D Gradient-weighted Class Activation Mapping (Grad-CAM) was also proposed at the testing stage to interpret the behaviors of the deep neural network. 5075 non-contrast chest CT scans were used as training, validation and testing datasets. Baseline performance was assessed on the same cohort. From the results, the proposed AID-Net achieved the superior performance on classification accuracy (0.9272) and AUC (0.9627).
Micro-Operator driven Local 5G Network Architecture for Industrial Internet
Venue: arXiv cs.NI 2018
Year: 2018

In addition to the high degree of flexibility and customization required by different vertical sectors, 5G calls for a network architecture that ensures ultra-responsive and ultra-reliable communication links. The novel concept called micro-operator (uO) enables a versatile set of stakeholders to operate local 5G networks within their premises with a guaranteed quality and reliability to complement mobile network operators' (MNOs) offerings. In this paper, we propose a descriptive architecture for emerging 5G uOs which provides user specific and location specific services in a spatially confined environment. The architecture is discussed in terms of network functions and the operational units which entail the core and radio access networks in a smart factory environment which supports industry 4.0 standards. Moreover, in order to realize the conceptual design, we provide simulation results for the latency measurements of the proposed uO architecture with respect to an augmented reality use case in industrial internet. Thereby we discuss the benefits of having uO driven local 5G networks for specialized user requirements, rather than continuing with the conventional approach where only MNOs can deploy cellular networks.