OpenRaiser/Intern-Atlas huggingface dataset:
Efficient Sampling for Predictor-Based Neural Architecture Search
Venue: arXiv cs.LG 2020
Year: 2020

Recently, predictor-based algorithms emerged as a promising approach for neural architecture search (NAS). For NAS, we typically have to calculate the validation accuracy of a large number of Deep Neural Networks (DNNs), what is computationally complex. Predictor-based NAS algorithms address this problem. They train a proxy model that can infer the validation accuracy of DNNs directly from their network structure. During optimization, the proxy can be used to narrow down the number of architectures for which the true validation accuracy must be computed, what makes predictor-based algorithms sample efficient. Usually, we compute the proxy for all DNNs in the network search space and pick those that maximize the proxy as candidates for optimization. However, that is intractable in practice, because the search spaces are often very large and contain billions of network architectures. The contributions of this paper are threefold: 1) We define a sample efficiency gain to compare different predictor-based NAS algorithms. 2) We conduct experiments on the NASBench-101 dataset and show that the sample efficiency of predictor-based algorithms decreases dramatically if the proxy is only computed for a subset of the search space. 3) We show that if we choose the subset of the search space on which the proxy is evaluated in a smart way, the sample efficiency of the original predictor-based algorithm that has access to the full search space can be regained. This is an important step to make predictor-based NAS algorithms useful, in practice.
Minimal Virtual Machines on IoT Microcontrollers: The Case of Berkeley Packet Filters with rBPF
Venue: arXiv cs.NI 2020
Year: 2020

Virtual machines (VM) are widely used to host and isolate software modules. However, extremely small memory and low-energy budgets have so far prevented wide use of VMs on typical microcontroller-based IoT devices. In this paper, we explore the potential of two minimal VM approaches on such low-power hardware. We design rBPF, a register-based VM based on extended Berkeley Packet Filters (eBPF). We compare it with a stack-based VM based on WebAssembly (Wasm) adapted for embedded systems. We implement prototypes of each VM, hosted in the IoT operating system RIOT. We perform measurements on commercial off-the-shelf IoT hardware. Unsurprisingly, we observe that both Wasm and rBPF virtual machines yield execution time and memory overhead, compared to not using a VM. We show however that this execution time overhead is tolerable for low-throughput, low-energy IoT devices. We further show that, while using a VM based on Wasm entails doubling the memory budget for a simple networked IoT application using a 6LoWPAN/CoAP stack, using a VM based on rBPF requires only negligible memory overhead (less than 10% more memory). rBPF is thus a promising approach to host small software modules, isolated from OS software, and updatable on-demand, over low-power networks.
SEA: Sentence Encoder Assembly for Video Retrieval by Textual Queries
Venue: arXiv cs.CV 2020
Year: 2020

Retrieving unlabeled videos by textual queries, known as Ad-hoc Video Search (AVS), is a core theme in multimedia data management and retrieval. The success of AVS counts on cross-modal representation learning that encodes both query sentences and videos into common spaces for semantic similarity computation. Inspired by the initial success of previously few works in combining multiple sentence encoders, this paper takes a step forward by developing a new and general method for effectively exploiting diverse sentence encoders. The novelty of the proposed method, which we term Sentence Encoder Assembly (SEA), is two-fold. First, different from prior art that use only a single common space, SEA supports text-video matching in multiple encoder-specific common spaces. Such a property prevents the matching from being dominated by a specific encoder that produces an encoding vector much longer than other encoders. Second, in order to explore complementarities among the individual common spaces, we propose multi-space multi-loss learning. As extensive experiments on four benchmarks (MSR-VTT, TRECVID AVS 2016-2019, TGIF and MSVD) show, SEA surpasses the state-of-the-art. In addition, SEA is extremely ease to implement. All this makes SEA an appealing solution for AVS and promising for continuously advancing the task by harvesting new sentence encoders.
CCIC-WSN: An Architecture for Single Channel Cluster-based Information-Centric Wireless Sensor Networks
Venue: arXiv cs.NI 2020
Year: 2020

The promising vision of Information-Centric Networking (ICN) and of its realization, Named Data Networking (NDN), has attracted extensive attention in recent years in the context of the Internet of Things (IoT) and Wireless Sensor Networks (WSNs). However, a comprehensive NDN/ICN-based architectural design for WSNs, including specially tailored naming schemes and forwarding mechanisms, has yet to be explored. In this paper, we present single-Channel Cluster-based Information-Centric WSN (CCIC-WSN), an NDN/ICN-based framework to fulfill the requirements of cluster-based WSNs, such as communication between child nodes and cluster heads, association of new child nodes with cluster heads, discovery of the namespace of newly associated nodes, and child node mobility. Through an extensive simulation study, we demonstrate that CCIC-WSN achieves 71-90% lower energy consumption and 74-96% lower data retrieval delays than recently proposed frameworks for NDN/ICN-based WSNs under various evaluation settings.
Lipophilicity Prediction with Multitask Learning and Molecular Substructures Representation
Venue: arXiv cs.LG 2020
Year: 2020

Lipophilicity is one of the factors determining the permeability of the cell membrane to a drug molecule. Hence, accurate lipophilicity prediction is an essential step in the development of new drugs. In this paper, we introduce a novel approach to encoding additional graph information by extracting molecular substructures. By adding a set of generalized atomic features of these substructures to an established Direct Message Passing Neural Network (D-MPNN) we were able to achieve a new state-of-the-art result at the task of prediction of two main lipophilicity coefficients, namely logP and logD descriptors. We further improve our approach by employing a multitask approach to predict logP and logD values simultaneously. Additionally, we present a study of the model performance on symmetric and asymmetric molecules, that may yield insight for further research.
Tight Integrated End-to-End Training for Cascaded Speech Translation
Venue: arXiv cs.CL 2020
Year: 2020

A cascaded speech translation model relies on discrete and non-differentiable transcription, which provides a supervision signal from the source side and helps the transformation between source speech and target text. Such modeling suffers from error propagation between ASR and MT models. Direct speech translation is an alternative method to avoid error propagation; however, its performance is often behind the cascade system. To use an intermediate representation and preserve the end-to-end trainability, previous studies have proposed using two-stage models by passing the hidden vectors of the recognizer into the decoder of the MT model and ignoring the MT encoder. This work explores the feasibility of collapsing the entire cascade components into a single end-to-end trainable model by optimizing all parameters of ASR and MT models jointly without ignoring any learned parameters. It is a tightly integrated method that passes renormalized source word posterior distributions as a soft decision instead of one-hot vectors and enables backpropagation. Therefore, it provides both transcriptions and translations and achieves strong consistency between them. Our experiments on four tasks with different data scenarios show that the model outperforms cascade models up to 1.8% in BLEU and 2.0% in TER and is superior compared to direct models.
Generating Intelligible Plumitifs Descriptions: Use Case Application with Ethical Considerations
Venue: arXiv cs.CL 2020
Year: 2020

Plumitifs (dockets) were initially a tool for law clerks. Nowadays, they are used as summaries presenting all the steps of a judicial case. Information concerning parties' identity, jurisdiction in charge of administering the case, and some information relating to the nature and the course of the preceding are available through plumitifs. They are publicly accessible but barely understandable; they are written using abbreviations and referring to provisions from the Criminal Code of Canada, which makes them hard to reason about. In this paper, we propose a simple yet efficient multi-source language generation architecture that leverages both the plumitif and the Criminal Code's content to generate intelligible plumitifs descriptions. It goes without saying that ethical considerations rise with these sensitive documents made readable and available at scale, legitimate concerns that we address in this paper.
Generative Adversarial Stacked Autoencoders
Venue: arXiv cs.LG 2020
Year: 2020

Generative Adversarial Networks (GANs) have become predominant in image generation tasks. Their success is attributed to the training regime which employs two models: a generator G and discriminator D that compete in a minimax zero sum game. Nonetheless, GANs are difficult to train due to their sensitivity to hyperparameter and parameter initialisation, which often leads to vanishing gradients, non-convergence, or mode collapse, where the generator is unable to create samples with different variations. In this work, we propose a novel Generative Adversarial Stacked Convolutional Autoencoder(GASCA) model and a generative adversarial gradual greedy layer-wise learning algorithm de-signed to train Adversarial Autoencoders in an efficient and incremental manner. Our training approach produces images with significantly lower reconstruction error than vanilla joint training.
Low Complexity Precoding and Detection in Multi-user Massive MIMO OTFS Downlink
Venue: arXiv cs.IT 2020
Year: 2020

We consider the problem of degradation in performance of multi-carrier multi-user massive MIMO systems when channel induced Doppler spread is high. Recently, Orthogonal Time Frequency Space (OTFS) modulation has been shown to be robust to channel induced Doppler spread. In OTFS based systems, information symbols are embedded in the delay-Doppler (DD) domain where they are jointly modulated to generate the time-domain transmit signal. Due to the multi-path delay and Doppler shifts, the DD domain information symbols need to be jointly demodulated at the receiver. For multi-carrier based communication (e.g., Orthogonal Frequency Division Multiplexing (OFDM)), massive MIMO systems have been shown to achieve high spectral and energy efficiency with low complexity multi-user precoding in the downlink. Extending the same to OTFS based downlink multi-user massive MIMO systems is challenging due to the requirement for joint demodulation of all information symbols at the user terminal (UT). In this paper, we solve this problem by proposing a novel OTFS based multi-user precoder at the base station (BS) and a corresponding low complexity detector (LCD) at the user terminals (UTs), which allows for separate demodulation of each DD domain information symbol at the UT. The complexity of the proposed precoder increases only linearly with increasing number BS antennas Q and the number of UTs. We show, through analysis, that as Q increases (with total transmitted power decreased linearly with Q), the proposed low complexity detector achieves a sum SE close to that achieved with optimal joint demodulation at each UT. Numerical simulations confirm our analysis and also show that the SE and error rate performance of the proposed OTFS based massive MIMO precoder and LCD detector is significantly more robust to channel induced Doppler spread when compared to OFDM based multi-user massive MIMO systems.
PeleNet: A Reservoir Computing Framework for Loihi
Venue: arXiv cs.NE 2020
Year: 2020

High-level frameworks for spiking neural networks are a key factor for fast prototyping and efficient development of complex algorithms. Such frameworks have emerged in the last years for traditional computers, but programming neuromorphic hardware is still a challenge. Often low level programming with knowledge about the hardware of the neuromorphic chip is required. The PeleNet framework aims to simplify reservoir computing for the neuromorphic hardware Loihi. It is build on top of the NxSDK from Intel and is written in Python. The framework manages weight matrices, parameters and probes. In particular, it provides an automatic and efficient distribution of networks over several cores and chips. With this, the user is not confronted with technical details and can concentrate on experiments.
Improving Clinical Outcome Predictions Using Convolution over Medical Entities with Multimodal Learning
Venue: arXiv cs.LG 2020
Year: 2020

Early prediction of mortality and length of stay(LOS) of a patient is vital for saving a patient's life and management of hospital resources. Availability of electronic health records(EHR) makes a huge impact on the healthcare domain and there has seen several works on predicting clinical problems. However, many studies did not benefit from the clinical notes because of the sparse, and high dimensional nature. In this work, we extract medical entities from clinical notes and use them as additional features besides time-series features to improve our predictions. We propose a convolution based multimodal architecture, which not only learns effectively combining medical entities and time-series ICU signals of patients, but also allows us to compare the effect of different embedding techniques such as Word2vec, FastText on medical entities. In the experiments, our proposed method robustly outperforms all other baseline models including different multimodal architectures for all clinical tasks. The code for the proposed method is available at https://github.com/tanlab/ConvolutionMedicalNer.
Lethean Attack: An Online Data Poisoning Technique
Venue: arXiv cs.CR 2020
Year: 2020

Data poisoning is an adversarial scenario where an attacker feeds a specially crafted sequence of samples to an online model in order to subvert learning. We introduce Lethean Attack, a novel data poisoning technique that induces catastrophic forgetting on an online model. We apply the attack in the context of Test-Time Training, a modern online learning framework aimed for generalization under distribution shifts. We present the theoretical rationale and empirically compare it against other sample sequences that naturally induce forgetting. Our results demonstrate that using lethean attacks, an adversary could revert a test-time training model back to coin-flip accuracy performance using a short sample sequence.
Play Fair: Frame Attributions in Video Models
Venue: arXiv cs.CV 2020
Year: 2020

In this paper, we introduce an attribution method for explaining action recognition models. Such models fuse information from multiple frames within a video, through score aggregation or relational reasoning. We break down a model's class score into the sum of contributions from each frame, fairly. Our method adapts an axiomatic solution to fair reward distribution in cooperative games, known as the Shapley value, for elements in a variable-length sequence, which we call the Element Shapley Value (ESV). Critically, we propose a tractable approximation of ESV that scales linearly with the number of frames in the sequence. We employ ESV to explain two action recognition models (TRN and TSN) on the fine-grained dataset Something-Something. We offer detailed analysis of supporting/distracting frames, and the relationships of ESVs to the frame's position, class prediction, and sequence length. We compare ESV to naive baselines and two commonly used feature attribution methods: Grad-CAM and Integrated-Gradients.
Experiments in Autonomous Driving Through Imitation Learning
Venue: arXiv cs.RO 2020
Year: 2020

This report demonstrates several methods used to make a self-driving vehicle using a supervised learning algorithm and a forward-facing RGBD camera. The project originally involved research in creating an adversarial attack on the vehicle's model, but due to difficulties with the initial training of the car, the plans were discarded in favor of completing the imitation learning portion of the project. Many approaches were explored, but due to challenges introduced by an unbalanced data set, the approaches had limited effectiveness.
Dynamic Hybrid Precoding Relying on Twin-Resolution Phase Shifters in Millimeter-Wave Communication Systems
Venue: arXiv cs.IT 2020
Year: 2020

Hybrid analog/digital precoding in millimeter-wave (mmWave) multi-input multi-ouput (MIMO) systems is capable of achieving the near-optimal full-digital performance at reduced hardware cost and power consumption compared to its full-RF digital counterpart. However, having numerous phase shifters is still costly, especially when the phase shifters are of high resolution. In this paper, we propose a novel twin-resolution phase-shifter network for mmWave MIMO systems, which reduces the power consumption of an entirely high-resolution network, whilst mitigating the severe array gain reduction of an entirely low-resolution network. The connections between the twin phase shifters having different resolutions and the antennas are either fixed or dynamically configured. In the latter, we jointly design the phase-shifter network and the hybrid precoding matrix, where the phase of each entry in the analog precoding matrix can be dynamically designed according to the required resolution. This method is slightly modified for the fixed network's hybrid precoding matrix. Furthermore, we extend the proposed method to multi-user MIMO systems and provide its performance analysis. Our simulation results show that the proposed dynamic hybrid precoding method strikes an attractive performance vs. power consumption trade-off.
CRACT: Cascaded Regression-Align-Classification for Robust Visual Tracking
Venue: arXiv cs.CV 2020
Year: 2020

High quality object proposals are crucial in visual tracking algorithms that utilize region proposal network (RPN). Refinement of these proposals, typically by box regression and classification in parallel, has been popularly adopted to boost tracking performance. However, it still meets problems when dealing with complex and dynamic background. Thus motivated, in this paper we introduce an improved proposal refinement module, Cascaded Regression-Align-Classification (CRAC), which yields new state-of-the-art performances on many benchmarks. First, having observed that the offsets from box regression can serve as guidance for proposal feature refinement, we design CRAC as a cascade of box regression, feature alignment and box classification. The key is to bridge box regression and classification via an alignment step, which leads to more accurate features for proposal classification with improved robustness. To address the variation in object appearance, we introduce an identification-discrimination component for box classification, which leverages offline reliable fine-grained template and online rich background information to distinguish the target from background. Moreover, we present pyramid RoIAlign that benefits CRAC by exploiting both the local and global cues of proposals. During inference, tracking proceeds by ranking all refined proposals and selecting the best one. In experiments on seven benchmarks including OTB-2015, UAV123, NfS, VOT-2018, TrackingNet, GOT-10k and LaSOT, our CRACT exhibits very promising results in comparison with state-of-the-art competitors and runs in real-time.
Solving the r-pseudoforest Deletion Problem in Time Independent of r
Venue: arXiv cs.DS 2020
Year: 2020

The feedback vertex set problem is one of the most studied parameterized problems. Several generalizations of the problem have been studied where one is to delete vertices to obtain graphs close to acyclic. In this paper, we give an FPT algorithm for the problem of deleting at most $k$ vertices to get an $r$-pseudoforest. A graph is an $r$-pseudoforest if we can delete at most $r$ edges from each component to get a forest. Philip et al. introduced this problem and gave an $O^*(c_{r}^{k})$ algorithm for it, where $c_r$ depends on $r$ double exponentially. In comparison, our algorithm runs in time $O^*((10k)^{k})$, independent of $r$.
Match Them Up: Visually Explainable Few-shot Image Classification
Venue: arXiv cs.CV 2020
Year: 2020

Few-shot learning (FSL) approaches are usually based on an assumption that the pre-trained knowledge can be obtained from base (seen) categories and can be well transferred to novel (unseen) categories. However, there is no guarantee, especially for the latter part. This issue leads to the unknown nature of the inference process in most FSL methods, which hampers its application in some risk-sensitive areas. In this paper, we reveal a new way to perform FSL for image classification, using visual representations from the backbone model and weights generated by a newly-emerged explainable classifier. The weighted representations only include a minimum number of distinguishable features and the visualized weights can serve as an informative hint for the FSL process. Finally, a discriminator will compare the representations of each pair of the images in the support set and the query set. Pairs with the highest scores will decide the classification results. Experimental results prove that the proposed method can achieve both good accuracy and satisfactory explainability on three mainstream datasets.
Developing a Security Testbed for Industrial Internet of Things
Venue: arXiv cs.CR 2020
Year: 2020

While achieving security for Industrial Internet of Things (IIoT) is a critical and non-trivial task, more attention is required for brownfield IIoT systems. This is a consequence of long life cycles of their legacy devices which were initially designed without considering security and IoT connectivity, but they are now becoming more connected and integrated with emerging IoT technologies and messaging communication protocols. Deploying today's methodologies and solutions in brownfield IIoT systems is not viable, as security solutions must co-exist and fit these systems requirements. This necessitates a realistic standardized IIoT testbed that can be used as an optimal format to measure the credibility of security solutions of IIoT networks, analyze IIoT attack landscapes and extract threat intelligence. Developing a testbed for brownfield IIoT systems is considered a significant challenge as these systems are comprised of legacy, heterogeneous devices, communication layers and applications that need to be implemented holistically to achieve high fidelity. In this paper, we propose a new generic end-to-end IIoT security testbed, with a particular focus on the brownfield system and provide details of the testbed's architectural design and the implementation process. The proposed testbed can be easily reproduced and reconfigured to support the testing activities of new processes and various security scenarios. The proposed testbed operation is demonstrated on different connected devices, communication protocols and applications. The experiments demonstrate that this testbed is effective in terms of its operation and security testing. A comparison with existing testbeds, including a table of features is provided.