OpenRaiser/Intern-Atlas huggingface dataset:
Improved cross-validation for classifiers that make algorithmic choices to minimise runtime without compromising output correctness
Venue: arXiv cs.SC 2019
Year: 2019

Our topic is the use of machine learning to improve software by making choices which do not compromise the correctness of the output, but do affect the time taken to produce such output. We are particularly concerned with computer algebra systems (CASs), and in particular, our experiments are for selecting the variable ordering to use when performing a cylindrical algebraic decomposition of $n$-dimensional real space with respect to the signs of a set of polynomials. In our prior work we explored the different ML models that could be used, and how to identify suitable features of the input polynomials. In the present paper we both repeat our prior experiments on problems which have more variables (and thus exponentially more possible orderings), and examine the metric which our ML classifiers targets. The natural metric is computational runtime, with classifiers trained to pick the ordering which minimises this. However, this leads to the situation were models do not distinguish between any of the non-optimal orderings, whose runtimes may still vary dramatically. In this paper we investigate a modification to the cross-validation algorithms of the classifiers so that they do distinguish these cases, leading to improved results.
Joint User Association and Resource Allocation in the Uplink of Heterogeneous Networks
Venue: arXiv cs.NI 2019
Year: 2019

This letter considers the problem of joint user association (UA), sub-channel assignment, antenna selection (AS), and power control in the uplink (UL) of a heterogeneous network such that the data rate of small cell users can be maximized while the macro-cell users are protected by imposing a threshold on the cross-tier interference. The considered problem is a non-convex mixed integer non-linear programming (MINLP). To tackle the problem, we decompose the original problem into two sub-problems: (i) joint UA, sub-channel assignment, and AS, and (ii) power control. Then, we iteratively solve the subproblems by applying the tools from majorization-minimization (MM) theory and augmented Lagrange method, respectively, and obtain locally optimal solutions for each sub-problem. Simulation results illustrate that our proposed scheme outperforms existing schemes. Complexity analysis of the proposed algorithm is also presented.
Data-Driven Compression of Convolutional Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

Deploying trained convolutional neural networks (CNNs) to mobile devices is a challenging task because of the simultaneous requirements of the deployed model to be fast, lightweight and accurate. Designing and training a CNN architecture that does well on all three metrics is highly non-trivial and can be very time-consuming if done by hand. One way to solve this problem is to compress the trained CNN models before deploying to mobile devices. This work asks and answers three questions on compressing CNN models automatically: a) How to control the trade-off between speed, memory and accuracy during model compression? b) In practice, a deployed model may not see all classes and/or may not need to produce all class labels. Can this fact be used to improve the trade-off? c) How to scale the compression algorithm to execute within a reasonable amount of time for many deployments? The paper demonstrates that a model compression algorithm utilizing reinforcement learning with architecture search and knowledge distillation can answer these questions in the affirmative. Experimental results are provided for current state-of-the-art CNN model families for image feature extraction like VGG and ResNet with CIFAR datasets.
Motion Equivariance OF Event-based Camera Data with the Temporal Normalization Transform
Venue: arXiv cs.CV 2019
Year: 2019

In this work, we focus on using convolution neural networks (CNN) to perform object recognition on the event data. In object recognition, it is important for a neural network to be robust to the variations of the data during testing. For traditional cameras, translations are well handled because CNNs are naturally equivariant to translations. However, because event cameras record the change of light intensity of an image, the geometric shape of event volumes will not only depend on the objects but also on their relative motions with respect to the camera. The deformation of the events caused by motions causes the CNN to be less robust to unseen motions during inference. To address this problem, we would like to explore the equivariance property of CNNs, a well-studied area that demonstrates to produce predictable deformation of features under certain transformations of the input image.
RIoTS: Risk Analysis of IoT Supply Chain Threats
Venue: arXiv cs.CR 2019
Year: 2019

Securing the supply chain of information and communications technology (ICT) has recently emerged as a critical concern for national security and integrity. With the proliferation of Internet of Things (IoT) devices and their increasing role in controlling real world infrastructure, there is a need to analyze risks in networked systems beyond established security analyses. Existing methods in literature typically leverage attack and fault trees to analyze malicious activity and its impact. In this paper, we develop RIoTS, a security risk assessment framework borrowing from system reliability theory to incorporate the supply chain. We also analyze the impact of grouping within suppliers that may pose hidden risks to the systems from malicious supply chain actors. The results show that the proposed analysis is able to reveal hidden threats posed to the IoT ecosystem from potential supplier collusion.
A PHY Layer Security Analysis of Uplink Cooperative Jamming-Based Underlay CRNs with Multi-Eavesdroppers
Venue: arXiv cs.IT 2019
Year: 2019

In this paper, the physical layer security of a dual-hop underlay uplink cognitive radio network is investigated over Nakagami-m fading channels. Specifically, multiple secondary sources are taking turns in accessing the licensed spectrum of the primary users and communicating with a multiantenna secondary base station D through the aid of a multiantenna relay R in the presence of M eavesdroppers that are also equipped with multiple antennas. Among the remaining nodes, one jammer is randomly selected to transmit an artificial noise to disrupt all the eavesdroppers that are attempting to intercept the communication of the legitimate links i.e., S-R and R-D. The received signals at each node are combined using maximal-ratio combining. Secrecy analysis is provided by deriving closed-form and asymptotic expressions for the secrecy outage probability. The impact of several key parameters on the system's secrecy e.g., transmit power of the sources, number of eavesdroppers, maximum tolerated interference power, and the number of diversity branches is investigated. Importantly, by considering two scenarios, namely (i) absence and (ii) presence of a friendly jammer, new insights are obtained for the considered communication system. Especially, we tend to answer to the following question: Can better secrecy be achieved without jamming by considering a single antenna at eavesdroppers and multiple-ones at the legitimate users (i.e., relay and end-user) rather than sending permanently an artificial noise and considering that both the relay and the destination are equipped with a single antenna, while multiple antennas are used by the eavesdroppers? The obtained results are corroborated through Monte Carlo simulation and show that the system's security can be enhanced by adjusting the aforementioned parameters.
Tropical Polynomial Division and Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

In this work, we examine the process of Tropical Polynomial Division, a geometric method which seeks to emulate the division of regular polynomials, when applied to those of the max-plus semiring. This is done via the approximation of the Newton Polytope of the dividend polynomial by that of the divisor. This process is afterwards generalized and applied in the context of neural networks with ReLU activations. In particular, we make use of the intuition it provides, in order to minimize a two-layer fully connected network, trained for a binary classification problem. This method is later evaluated on a variety of experiments, demonstrating its capability to approximate a network, with minimal loss in performance.
Incremental Clustering Techniques for Multi-Party Privacy-Preserving Record Linkage
Venue: arXiv cs.DB 2019
Year: 2019

Privacy-Preserving Record Linkage (PPRL) supports the integration of sensitive information from multiple datasets, in particular the privacy-preserving matching of records referring to the same entity. PPRL has gained much attention in many application areas, with the most prominent ones in the healthcare domain. PPRL techniques tackle this problem by conducting linkage on masked (encoded) values. Employing PPRL on records from multiple (more than two) parties/sources (multi-party PPRL, MP-PPRL) is an increasingly important but challenging problem that so far has not been sufficiently solved. Existing MP-PPRL approaches are limited to finding only those entities that are present in all parties thereby missing entities that match only in a subset of parties. Furthermore, previous MP-PPRL approaches face substantial scalability limitations due to the need of a large number of comparisons between masked records. We thus propose and evaluate new MP-PPRL approaches that find matches in any subset of parties and still scale to many parties. Our approaches maintain all matches within clusters, where these clusters are incrementally extended or refined by considering records from one party after the other. An empirical evaluation using multiple real datasets ranging from 3 to 26 parties each containing up to $5$ million records validates that our protocols are efficient, and significantly outperform existing MP-PPRL approaches in terms of linkage quality and scalability.
Correlation-aware Adversarial Domain Adaptation and Generalization
Venue: arXiv cs.CV 2019
Year: 2019

Domain adaptation (DA) and domain generalization (DG) have emerged as a solution to the domain shift problem where the distribution of the source and target data is different. The task of DG is more challenging than DA as the target data is totally unseen during the training phase in DG scenarios. The current state-of-the-art employs adversarial techniques, however, these are rarely considered for the DG problem. Furthermore, these approaches do not consider correlation alignment which has been proven highly beneficial for minimizing domain discrepancy. In this paper, we propose a correlation-aware adversarial DA and DG framework where the features of the source and target data are minimized using correlation alignment along with adversarial learning. Incorporating the correlation alignment module along with adversarial learning helps to achieve a more domain agnostic model due to the improved ability to reduce domain discrepancy with unlabeled target data more effectively. Experiments on benchmark datasets serve as evidence that our proposed method yields improved state-of-the-art performance.
Sentiment Analysis of German Twitter
Venue: arXiv cs.CL 2019
Year: 2019

This thesis explores the ways by how people express their opinions on German Twitter, examines current approaches to automatic mining of these feelings, and proposes novel methods, which outperform state-of-the-art techniques. For this purpose, I introduce a new corpus of German tweets that have been manually annotated with sentiments, their targets and holders, as well as polar terms and their contextual modifiers. Using these data, I explore four major areas of sentiment research: (i) generation of sentiment lexicons, (ii) fine-grained opinion mining, (iii) message-level polarity classification, and (iv) discourse-aware sentiment analysis. In the first task, I compare three popular groups of lexicon generation methods: dictionary-, corpus-, and word-embedding-based ones, finding that dictionary-based systems generally yield better lexicons than the last two groups. Apart from this, I propose a linear projection algorithm, whose results surpass many existing automatic lexicons. Afterwords, in the second task, I examine two common approaches to automatic prediction of sentiments, sources, and targets: conditional random fields and recurrent neural networks, obtaining higher scores with the former model and improving these results even further by redefining the structure of CRF graphs. When dealing with message-level polarity classification, I juxtapose three major sentiment paradigms: lexicon-, machine-learning-, and deep-learning-based systems, and try to unite the first and last of these groups by introducing a bidirectional neural network with lexicon-based attention. Finally, in order to make the new classifier aware of discourse structure, I let it separately analyze the elementary discourse units of each microblog and infer the overall polarity of a message from the scores of its EDUs with the help of two new approaches: latent-marginalized CRFs and Recursive Dirichlet Process.
Kurdish (Sorani) Speech to Text: Presenting an Experimental Dataset
Venue: arXiv cs.CL 2019
Year: 2019

We present an experimental dataset, Basic Dataset for Sorani Kurdish Automatic Speech Recognition (BD-4SK-ASR), which we used in the first attempt in developing an automatic speech recognition for Sorani Kurdish. The objective of the project was to develop a system that automatically could recognize simple sentences based on the vocabulary which is used in grades one to three of the primary schools in the Kurdistan Region of Iraq. We used CMUSphinx as our experimental environment. We developed a dataset to train the system. The dataset is publicly available for non-commercial use under the CC BY-NC-SA 4.0 license.
A spiking neural algorithm for the Network Flow problem
Venue: arXiv cs.NE 2019
Year: 2019

It is currently not clear what the potential is of neuromorphic hardware beyond machine learning and neuroscience. In this project, a problem is investigated that is inherently difficult to fully implement in neuromorphic hardware by introducing a new machine model in which a conventional Turing machine and neuromorphic oracle work together to solve such types of problems. We show that the P-complete Max Network Flow problem is intractable in models where the oracle may be consulted only once (`create-and-run' model) but becomes tractable using an interactive (`neuromorphic co-processor') model of computation. More in specific we show that a logspace-constrained Turing machine with access to an interactive neuromorphic oracle with linear space, time, and energy constraints can solve Max Network Flow. A modified variant of this algorithm is implemented on the Intel Loihi chip; a neuromorphic manycore processor developed by Intel Labs. We show that by off-loading the search for augmenting paths to the neuromorphic processor we can get energy efficiency gains, while not sacrificing runtime resources. This result demonstrates how P-complete problems can be mapped on neuromorphic architectures in a theoretically and potentially practically efficient manner.
Learning Domain-Independent Planning Heuristics with Hypergraph Networks
Venue: arXiv cs.AI 2019
Year: 2019

We present the first approach capable of learning domain-independent planning heuristics entirely from scratch. The heuristics we learn map the hypergraph representation of the delete-relaxation of the planning problem at hand, to a cost estimate that approximates that of the least-cost path from the current state to the goal through the hypergraph. We generalise Graph Networks to obtain a new framework for learning over hypergraphs, which we specialise to learn planning heuristics by training over state/value pairs obtained from optimal cost plans. Our experiments show that the resulting architecture, STRIPS-HGNs, is capable of learning heuristics that are competitive with existing delete-relaxation heuristics including LM-cut. We show that the heuristics we learn are able to generalise across different problems and domains, including to domains that were not seen during training.
Tight Bounds for Planar Strongly Connected Steiner Subgraph with Fixed Number of Terminals (and Extensions)
Venue: arXiv cs.DS 2019
Year: 2019

(see paper for full abstract) Given a vertex-weighted directed graph $G=(V,E)$ and a set $T=\{t_1, t_2, \ldots t_k\}$ of $k$ terminals, the objective of the SCSS problem is to find a vertex set $H\subseteq V$ of minimum weight such that $G[H]$ contains a $t_{i}\rightarrow t_j$ path for each $i\neq j$. The problem is NP-hard, but Feldman and Ruhl [FOCS '99; SICOMP '06] gave a novel $n^{O(k)}$ algorithm for the SCSS problem, where $n$ is the number of vertices in the graph and $k$ is the number of terminals. We explore how much easier the problem becomes on planar directed graphs: - Our main algorithmic result is a $2^{O(k)}\cdot n^{O(\sqrt{k})}$ algorithm for planar SCSS, which is an improvement of a factor of $O(\sqrt{k})$ in the exponent over the algorithm of Feldman and Ruhl. - Our main hardness result is a matching lower bound for our algorithm: we show that planar SCSS does not have an $f(k)\cdot n^{o(\sqrt{k})}$ algorithm for any computable function $f$, unless the Exponential Time Hypothesis (ETH) fails. The following additional results put our upper and lower bounds in context: - In general graphs, we cannot hope for such a dramatic improvement over the $n^{O(k)}$ algorithm of Feldman and Ruhl: assuming ETH, SCSS in general graphs does not have an $f(k)\cdot n^{o(k/\log k)}$ algorithm for any computable function $f$. - Feldman and Ruhl generalized their $n^{O(k)}$ algorithm to the more general Directed Steiner Network (DSN) problem; here the task is to find a subgraph of minimum weight such that for every source $s_i$ there is a path to the corresponding terminal $t_i$. We show that, assuming ETH, there is no $f(k)\cdot n^{o(k)}$ time algorithm for DSN on acyclic planar graphs.
Deep autofocus with cone-beam CT consistency constraint
Venue: arXiv cs.LG 2019
Year: 2019

High quality reconstruction with interventional C-arm cone-beam computed tomography (CBCT) requires exact geometry information. If the geometry information is corrupted, e. g., by unexpected patient or system movement, the measured signal is misplaced in the backprojection operation. With prolonged acquisition times of interventional C-arm CBCT the likelihood of rigid patient motion increases. To adapt the backprojection operation accordingly, a motion estimation strategy is necessary. Recently, a novel learning-based approach was proposed, capable of compensating motions within the acquisition plane. We extend this method by a CBCT consistency constraint, which was proven to be efficient for motions perpendicular to the acquisition plane. By the synergistic combination of these two measures, in and out-plane motion is well detectable, achieving an average artifact suppression of 93 [percent]. This outperforms the entropy-based state-of-the-art autofocus measure which achieves on average an artifact suppression of 54 [percent].
Optimal checkpointing for heterogeneous chains: how to train deep neural networks with limited memory
Venue: arXiv cs.LG 2019
Year: 2019

This paper introduces a new activation checkpointing method which allows to significantly decrease memory usage when training Deep Neural Networks with the back-propagation algorithm. Similarly to checkpoint-ing techniques coming from the literature on Automatic Differentiation, it consists in dynamically selecting the forward activations that are saved during the training phase, and then automatically recomputing missing activations from those previously recorded. We propose an original computation model that combines two types of activation savings: either only storing the layer inputs, or recording the complete history of operations that produced the outputs (this uses more memory, but requires fewer recomputations in the backward phase), and we provide an algorithm to compute the optimal computation sequence for this model. This paper also describes a PyTorch implementation that processes the entire chain, dealing with any sequential DNN whose internal layers may be arbitrarily complex and automatically executing it according to the optimal checkpointing strategy computed given a memory limit. Through extensive experiments, we show that our implementation consistently outperforms existing checkpoint-ing approaches for a large class of networks, image sizes and batch sizes.
AuthorGAN: Improving GAN Reproducibility using a Modular GAN Framework
Venue: arXiv cs.LG 2019
Year: 2019

Generative models are becoming increasingly popular in the literature, with Generative Adversarial Networks (GAN) being the most successful variant, yet. With this increasing demand and popularity, it is becoming equally difficult and challenging to implement and consume GAN models. A qualitative user survey conducted across 47 practitioners show that expert level skill is required to use GAN model for a given task, despite the presence of various open source libraries. In this research, we propose a novel system called AuthorGAN, aiming to achieve true democratization of GAN authoring. A highly modularized library agnostic representation of GAN model is defined to enable interoperability of GAN architecture across different libraries such as Keras, Tensorflow, and PyTorch. An intuitive drag-and-drop based visual designer is built using node-red platform to enable custom architecture designing without the need for writing any code. Five different GAN models are implemented as a part of this framework and the performance of the different GAN models are shown using the benchmark MNIST dataset.
Non-Intrusive Electrical Appliances Monitoring and Classification using K-Nearest Neighbors
Venue: arXiv cs.LG 2019
Year: 2019

Non-Intrusive Load Monitoring (NILM) is the method of detecting an individual device's energy signal from an aggregated energy consumption signature [1]. As existing energy meters provide very little to no information regarding the energy consumption of individual appliances apart from the aggregated power rating, the spotting of individual appliances' energy usages by NILM will not only provide consumers the feedback of appliance-specific energy usage but also lead to the changes of their consumption behavior which facilitate energy conservation. B Neenan et al. [2] have demonstrated that direct individual appliance-specific energy usage signals lead to consumers' behavioral changes which improves energy efficiency by as much as 15%. Upon disaggregation of an energy signal, the signal needs to be classified according to the appropriate appliance. Hence, the goal of this paper is to disaggregate total energy consumption data to individual appliance signature and then classify appliance-specific energy loads using a prominent supervised classification method known as K-Nearest Neighbors (KNN). To perform this operation we have used a publicly accessible dataset of power signals from several houses known as the REDD dataset. Before applying KNN, data is preprocessed for each device. Then KNN is applied to check whether their energy consumption signature is separable or not. KNN is applied with K=5.