OpenRaiser/Intern-Atlas huggingface dataset:
Impact of Batch Size on Stopping Active Learning for Text Classification
Venue: arXiv cs.LG 2018
Year: 2018

When using active learning, smaller batch sizes are typically more efficient from a learning efficiency perspective. However, in practice due to speed and human annotator considerations, the use of larger batch sizes is necessary. While past work has shown that larger batch sizes decrease learning efficiency from a learning curve perspective, it remains an open question how batch size impacts methods for stopping active learning. We find that large batch sizes degrade the performance of a leading stopping method over and above the degradation that results from reduced learning efficiency. We analyze this degradation and find that it can be mitigated by changing the window size parameter of how many past iterations of learning are taken into account when making the stopping decision. We find that when using larger batch sizes, stopping methods are more effective when smaller window sizes are used.
TritanDB: Time-series Rapid Internet of Things Analytics
Venue: arXiv cs.DB 2018
Year: 2018

The efficient management of data is an important prerequisite for realising the potential of the Internet of Things (IoT). Two issues given the large volume of structured time-series IoT data are, addressing the difficulties of data integration between heterogeneous Things and improving ingestion and query performance across databases on both resource-constrained Things and in the cloud. In this paper, we examine the structure of public IoT data and discover that the majority exhibit unique flat, wide and numerical characteristics with a mix of evenly and unevenly-spaced time-series. We investigate the advances in time-series databases for telemetry data and combine these findings with microbenchmarks to determine the best compression techniques and storage data structures to inform the design of a novel solution optimised for IoT data. A query translation method with low overhead even on resource-constrained Things allows us to utilise rich data models like the Resource Description Framework (RDF) for interoperability and data integration on top of the optimised storage. Our solution, TritanDB, shows an order of magnitude performance improvement across both Things and cloud hardware on many state-of-the-art databases within IoT scenarios. Finally, we describe how TritanDB supports various analyses of IoT time-series data like forecasting.
An LSTM Network for Highway Trajectory Prediction
Venue: arXiv cs.RO 2018
Year: 2018

In order to drive safely and efficiently on public roads, autonomous vehicles will have to understand the intentions of surrounding vehicles, and adapt their own behavior accordingly. If experienced human drivers are generally good at inferring other vehicles' motion up to a few seconds in the future, most current Advanced Driving Assistance Systems (ADAS) are unable to perform such medium-term forecasts, and are usually limited to high-likelihood situations such as emergency braking. In this article, we present a first step towards consistent trajectory prediction by introducing a long short-term memory (LSTM) neural network, which is capable of accurately predicting future longitudinal and lateral trajectories for vehicles on highway. Unlike previous work focusing on a low number of trajectories collected from a few drivers, our network was trained and validated on the NGSIM US-101 dataset, which contains a total of 800 hours of recorded trajectories in various traffic densities, representing more than 6000 individual drivers.
A Collective Knowledge workflow for collaborative research into multi-objective autotuning and machine learning techniques
Venue: arXiv cs.HC 2018
Year: 2018

Developing efficient software and hardware has never been harder whether it is for a tiny IoT device or an Exascale supercomputer. Apart from the ever growing design and optimization complexity, there exist even more fundamental problems such as lack of interdisciplinary knowledge required for effective software/hardware co-design, and a growing technology transfer gap between academia and industry. We introduce our new educational initiative to tackle these problems by developing Collective Knowledge (CK), a unified experimental framework for computer systems research and development. We use CK to teach the community how to make their research artifacts and experimental workflows portable, reproducible, customizable and reusable while enabling sustainable R&D and facilitating technology transfer. We also demonstrate how to redesign multi-objective autotuning and machine learning as a portable and extensible CK workflow. Such workflows enable researchers to experiment with different applications, data sets and tools; crowdsource experimentation across diverse platforms; share experimental results, models, visualizations; gradually expose more design and optimization choices using a simple JSON API; and ultimately build upon each other's findings. As the first practical step, we have implemented customizable compiler autotuning, crowdsourced optimization of diverse workloads across Raspberry Pi 3 devices, reduced the execution time and code size by up to 40%, and applied machine learning to predict optimizations. We hope such approach will help teach students how to build upon each others' work to enable efficient and self-optimizing software/hardware/model stack for emerging workloads.
On Scale-out Deep Learning Training for Cloud and HPC
Venue: arXiv cs.DC 2018
Year: 2018

The exponential growth in use of large deep neural networks has accelerated the need for training these deep neural networks in hours or even minutes. This can only be achieved through scalable and efficient distributed training, since a single node/card cannot satisfy the compute, memory, and I/O requirements of today's state-of-the-art deep neural networks. However, scaling synchronous Stochastic Gradient Descent (SGD) is still a challenging problem and requires continued research/development. This entails innovations spanning algorithms, frameworks, communication libraries, and system design. In this paper, we describe the philosophy, design, and implementation of Intel Machine Learning Scalability Library (MLSL) and present proof-points demonstrating scaling DL training on 100s to 1000s of nodes across Cloud and HPC systems.
Intel nGraph: An Intermediate Representation, Compiler, and Executor for Deep Learning
Venue: arXiv cs.DC 2018
Year: 2018

The Deep Learning (DL) community sees many novel topologies published each year. Achieving high performance on each new topology remains challenging, as each requires some level of manual effort. This issue is compounded by the proliferation of frameworks and hardware platforms. The current approach, which we call "direct optimization", requires deep changes within each framework to improve the training performance for each hardware backend (CPUs, GPUs, FPGAs, ASICs) and requires $\mathcal{O}(fp)$ effort; where $f$ is the number of frameworks and $p$ is the number of platforms. While optimized kernels for deep-learning primitives are provided via libraries like Intel Math Kernel Library for Deep Neural Networks (MKL-DNN), there are several compiler-inspired ways in which performance can be further optimized. Building on our experience creating neon (a fast deep learning library on GPUs), we developed Intel nGraph, a soon to be open-sourced C++ library to simplify the realization of optimized deep learning performance across frameworks and hardware platforms. Initially-supported frameworks include TensorFlow, MXNet, and Intel neon framework. Initial backends are Intel Architecture CPUs (CPU), the Intel(R) Nervana Neural Network Processor(R) (NNP), and NVIDIA GPUs. Currently supported compiler optimizations include efficient memory management and data layout abstraction. In this paper, we describe our overall architecture and its core components. In the future, we envision extending nGraph API support to a wider range of frameworks, hardware (including FPGAs and ASICs), and compiler optimizations (training versus inference optimizations, multi-node and multi-device scaling via efficient sub-graph partitioning, and HW-specific compounding of operations).
The challenge of simultaneous object detection and pose estimation: a comparative study
Venue: arXiv cs.CV 2018
Year: 2018

Detecting objects and estimating their pose remains as one of the major challenges of the computer vision research community. There exists a compromise between localizing the objects and estimating their viewpoints. The detector ideally needs to be view-invariant, while the pose estimation process should be able to generalize towards the category-level. This work is an exploration of using deep learning models for solving both problems simultaneously. For doing so, we propose three novel deep learning architectures, which are able to perform a joint detection and pose estimation, where we gradually decouple the two tasks. We also investigate whether the pose estimation problem should be solved as a classification or regression problem, being this still an open question in the computer vision community. We detail a comparative analysis of all our solutions and the methods that currently define the state of the art for this problem. We use PASCAL3D+ and ObjectNet3D datasets to present the thorough experimental evaluation and main results. With the proposed models we achieve the state-of-the-art performance in both datasets.
Outage Performance of A Unified Non-Orthogonal Multiple Access Framework
Venue: arXiv cs.IT 2018
Year: 2018

In this paper, a unified framework of non-orthogonal multiple access (NOMA) networks is proposed, which can be applied to code-domain NOMA (CD-NOMA) and power-domain NOMA (PD-NOMA). Since the detection of NOMA users mainly depend on efficient successive interference cancellation (SIC) schemes, both imperfect SIC (ipSIC) and perfect SIC (pSIC) are taken into considered. To characterize the performance of this unified framework, the exact and asymptotic expressions of outage probabilities as well as delay-limited throughput for CD/PD-NOMA with ipSIC/pSIC are derived. Based on the asymptotic analysis, the diversity orders of CD/PD-NOMA are provided. It is confirmed that due to the impact of residual interference (RI), the outage probability of the n-th user with ipSIC for CD/PD-NOMA converges to an error floor in the high signal-to-noise ratio (SNR) region. Numerical simulations demonstrate that the outage behavior of CD-NOMA is superior to that of PD-NOMA.
User Association and Resource Allocation in Unified NOMA Enabled Heterogeneous Ultra Dense Networks
Venue: arXiv cs.IT 2018
Year: 2018

Heterogeneous ultra dense networks (HUDNs) and non-orthogonal multiple access (NOMA) have been identified as two proposing techniques for the fifth generation (5G) mobile communication systems due to their great capabilities to enhance spectrum efficiency. This article investigates the application of NOMA techniques in HUDNs to support massive connectivity in 5G systems. Particularly, a unified NOMA framework is proposed, including power-domain NOMA and code-domain NOMA, which can be configured flexibly to serve different applications scenarios. As a further advance, the unified NOMA framework enabled HUDNs is further investigated, with particular focuses on the user association and resource allocation. Two case studies are provided for demonstrating the effectiveness of the unified NOMA enabled HUDNs. Finally, some main challenges and promising research directions in NOMA enabled HUDNs are identified. Index Terms Heterogeneous ultra dense networks, non-orthogonal multiple access, massive connectivity, user association, and resource allocation
Sparse Representation for Wireless Communications: A Compressive Sensing Approach
Venue: arXiv cs.IT 2018
Year: 2018

Sparse representation can efficiently model signals in different applications to facilitate processing. In this article, we will discuss various applications of sparse representation in wireless communications, with focus on the most recent compressive sensing (CS) enabled approaches. With the help of the sparsity property, CS is able to enhance the spectrum efficiency and energy efficiency for the fifth generation (5G) networks and Internet of Things (IoT) networks. This article starts from a comprehensive overview of CS principles and different sparse domains potentially used in 5G and IoT networks. Then recent research progress on applying CS to address the major opportunities and challenges in 5G and IoT networks is introduced, including wideband spectrum sensing in cognitive radio networks, data collection in IoT networks, and channel estimation and feedback in massive MIMO systems. Moreover, other potential applications and research challenges on sparse representation for 5G and IoT networks are identified. This article will provide readers a clear picture of how to exploit the sparsity properties to process wireless signals in different applications.
A Tutorial on Modeling and Inference in Undirected Graphical Models for Hyperspectral Image Analysis
Venue: arXiv cs.CV 2018
Year: 2018

Undirected graphical models have been successfully used to jointly model the spatial and the spectral dependencies in earth observing hyperspectral images. They produce less noisy, smooth, and spatially coherent land cover maps and give top accuracies on many datasets. Moreover, they can easily be combined with other state-of-the-art approaches, such as deep learning. This has made them an essential tool for remote sensing researchers and practitioners. However, graphical models have not been easily accessible to the larger remote sensing community as they are not discussed in standard remote sensing textbooks and not included in the popular remote sensing software and toolboxes. In this tutorial, we provide a theoretical introduction to Markov random fields and conditional random fields based spatial-spectral classification for land cover mapping along with a detailed step-by-step practical guide on applying these methods using freely available software. Furthermore, the discussed methods are benchmarked on four public hyperspectral datasets for a fair comparison among themselves and easy comparison with the vast number of methods in literature which use the same datasets. The source code necessary to reproduce all the results in the paper is published on-line to make it easier for the readers to apply these techniques to different remote sensing problems.
Discovering Markov Blanket from Multiple interventional Datasets
Venue: arXiv cs.AI 2018
Year: 2018

In this paper, we study the problem of discovering the Markov blanket (MB) of a target variable from multiple interventional datasets. Datasets attained from interventional experiments contain richer causal information than passively observed data (observational data) for MB discovery. However, almost all existing MB discovery methods are designed for finding MBs from a single observational dataset. To identify MBs from multiple interventional datasets, we face two challenges: (1) unknown intervention variables; (2) nonidentical data distributions. To tackle the challenges, we theoretically analyze (a) under what conditions we can find the correct MB of a target variable, and (b) under what conditions we can identify the causes of the target variable via discovering its MB. Based on the theoretical analysis, we propose a new algorithm for discovering MBs from multiple interventional datasets, and present the conditions/assumptions which assure the correctness of the algorithm. To our knowledge, this work is the first to present the theoretical analyses about the conditions for MB discovery in multiple interventional datasets and the algorithm to find the MBs in relation to the conditions. Using benchmark Bayesian networks and real-world datasets, the experiments have validated the effectiveness and efficiency of the proposed algorithm in the paper.
Algebraic Geometric codes on minimal Hirzebruch surfaces
Venue: arXiv cs.IT 2018
Year: 2018

We define a linear code $C_η(δ_T,δ_X)$ by evaluating polynomials of bidegree $(δ_T,δ_X)$ in the Cox ring on $\mathbb{F}_q$-rational points of the Hirzebruch surface of parameter $η$ on the finite field $\mathbb{F}_q$. We give explicit parameters of the code, notably using Gröbner bases. The minimum distance provides an upper bound of the number of $\mathbb{F}_q$-rational points of a non-filling curve on a Hirzebruch surface.
Analyzing Similarity in Mathematical Content To Enhance the Detection of Academic Plagiarism
Venue: arXiv cs.IR 2018
Year: 2018

Despite the effort put into the detection of academic plagiarism, it continues to be a ubiquitous problem spanning all disciplines. Various tools have been developed to assist human inspectors by automatically identifying suspicious documents. However, to our knowledge currently none of these tools use mathematical content for their analysis. This is problematic, because mathematical content potentially represents a significant amount of the scientific contribution in academic documents. Hence, ignoring mathematical content limits the detection of plagiarism considerably, especially in disciplines with frequent use of mathematics. This paper aims to help close this gap by providing an overview of existing approaches in mathematical information retrieval and an analysis of their applicability for different possible cases of mathematical plagiarism. I find that whereas syntax-based approaches perform particularly well in detecting undisguised plagiarism, structure-based and hybrid approaches promise to also detect forms of disguised mathematical plagiarism, such as plagiarism with renamed identifiers. However, more research in this area is needed to enable the detection of more complex mathematical plagiarism: the scope of current approaches is restricted to the formula-level, an extension to the section-level is needed. Additionally, the general detection of equivalence transformations is currently not feasible. Despite these remaining problems, I conclude that the presented approaches could already be used for a basic automated detection system targeting mathematical plagiarism and therefore enhance current plagiarism detection systems.
Hardware implementation of auto-mutual information function for condition monitoring
Venue: arXiv cs.OH 2018
Year: 2018

This study is aimed at showing applicability of mutual information, namely auto-mutual information function for condition monitoring in electrical motors, through age detection in accelerated motor aging. Vibration data collected in artificial induction motor experiment is used for verification of both the original auto-mutual information function algorithm and its hardware implementation in Verilog, produced from an initial version made with Matlab HDL (Hardware Description Language) Coder. A conceptual model for industry and education based on a field programmable logic array development board is developed and demonstrated on the auto-mutual information function example, while suggesting other applications as well. It has also been shown that attractor reconstruction for the vibration data cannot be straightforward.
Convolutional Invasion and Expansion Networks for Tumor Growth Prediction
Venue: arXiv cs.CV 2018
Year: 2018

Tumor growth is associated with cell invasion and mass-effect, which are traditionally formulated by mathematical models, namely reaction-diffusion equations and biomechanics. Such models can be personalized based on clinical measurements to build the predictive models for tumor growth. In this paper, we investigate the possibility of using deep convolutional neural networks (ConvNets) to directly represent and learn the cell invasion and mass-effect, and to predict the subsequent involvement regions of a tumor. The invasion network learns the cell invasion from information related to metabolic rate, cell density and tumor boundary derived from multimodal imaging data. The expansion network models the mass-effect from the growing motion of tumor mass. We also study different architectures that fuse the invasion and expansion networks, in order to exploit the inherent correlations among them. Our network can easily be trained on population data and personalized to a target patient, unlike most previous mathematical modeling methods that fail to incorporate population data. Quantitative experiments on a pancreatic tumor data set show that the proposed method substantially outperforms a state-of-the-art mathematical model-based approach in both accuracy and efficiency, and that the information captured by each of the two subnetworks are complementary.
Sine Cosine Crow Search Algorithm: A powerful hybrid meta heuristic for global optimization
Venue: arXiv cs.NE 2018
Year: 2018

This paper presents a novel hybrid algorithm named Since Cosine Crow Search Algorithm. To propose the SCCSA, two novel algorithms are considered including Crow Search Algorithm (CSA) and Since Cosine Algorithm (SCA). The advantages of the two algorithms are considered and utilize to design an efficient hybrid algorithm which can perform significantly better in various benchmark functions. The combination of concept and operators of the two algorithms enable the SCCSA to make an appropriate trade-off between exploration and exploitation abilities of the algorithm. To evaluate the performance of the proposed SCCSA, seven well-known benchmark functions are utilized. The results indicated that the proposed hybrid algorithm is able to provide very competitive solution comparing to other state-of-the-art meta heuristics.
Etymo: A New Discovery Engine for AI Research
Venue: arXiv cs.IR 2018
Year: 2018

We present Etymo (https://etymo.io), a discovery engine to facilitate artificial intelligence (AI) research and development. It aims to help readers navigate a large number of AI-related papers published every week by using a novel form of search that finds relevant papers and displays related papers in a graphical interface. Etymo constructs and maintains an adaptive similarity-based network of research papers as an all-purpose knowledge graph for ranking, recommendation, and visualisation. The network is constantly evolving and can learn from user feedback to adjust itself.
A New Algorithm for Double Scalar Multiplication over Koblitz Curves
Venue: arXiv cs.CR 2018
Year: 2018

Koblitz curves are a special set of elliptic curves and have improved performance in computing scalar multiplication in elliptic curve cryptography due to the Frobenius endomorphism. Double-base number system approach for Frobenius expansion has improved the performance in single scalar multiplication. In this paper, we present a new algorithm to generate a sparse and joint $τ$-adic representation for a pair of scalars and its application in double scalar multiplication. The new algorithm is inspired from double-base number system. We achieve 12% improvement in speed against state-of-the-art $τ$-adic joint sparse form.