OpenRaiser/Intern-Atlas huggingface dataset:
Aging-Aware Request Scheduling for Non-Volatile Main Memory
Venue: arXiv cs.AR 2020
Year: 2020

Modern computing systems are embracing non-volatile memory (NVM) to implement high-capacity and low-cost main memory. Elevated operating voltages of NVM accelerate the aging of CMOS transistors in the peripheral circuitry of each memory bank. Aggressive device scaling increases power density and temperature, which further accelerates aging, challenging the reliable operation of NVM-based main memory. We propose HEBE, an architectural technique to mitigate the circuit aging-related problems of NVM-based main memory. HEBE is built on three contributions. First, we propose a new analytical model that can dynamically track the aging in the peripheral circuitry of each memory bank based on the bank's utilization. Second, we develop an intelligent memory request scheduler that exploits this aging model at run time to de-stress the peripheral circuitry of a memory bank only when its aging exceeds a critical threshold. Third, we introduce an isolation transistor to decouple parts of a peripheral circuit operating at different voltages, allowing the decoupled logic blocks to undergo long-latency de-stress operations independently and off the critical path of memory read and write accesses, improving performance. We evaluate HEBE with workloads from the SPEC CPU2017 Benchmark suite. Our results show that HEBE significantly improves both performance and lifetime of NVM-based main memory.
On Error Exponents of Almost-Fixed-Length Channel Codes and Hypothesis Tests
Venue: arXiv cs.IT 2020
Year: 2020

We examine a new class of channel coding strategies, and hypothesis tests referred to as almost-fixed-length strategies that have little flexibility in the stopping time over fixed-length strategies. The stopping time of these strategies is allowed to be slightly large only on a rare set of sample paths with an exponentially small probability. We show that almost-fixed-length channel coding strategies can achieve Burnashev's optimal error exponent. Similarly, almost-fixed length hypothesis tests are shown to bridge the gap between hypothesis testing with fixed sample size and sequential hypothesis testing and improve the trade-off between type-I and type-II error exponents.
Using Data Analytics to predict students score
Venue: arXiv cs.CY 2020
Year: 2020

Education is very important to Singapore, and the government has continued to invest heavily in our education system to become one of the world-class systems today. A strong foundation of Science, Technology, Engineering, and Mathematics (STEM) was what underpinned Singapore's development over the past 50 years. PISA is a triennial international survey that evaluates education systems worldwide by testing the skills and knowledge of 15-year-old students who are nearing the end of compulsory education. In this paper, the authors used the PISA data from 2012 and 2015 and developed machine learning techniques to predictive the students' scores and understand the inter-relationships among social, economic, and education factors. The insights gained would be useful to have fresh perspectives on education, useful for policy formulation.
Dynamic Image for 3D MRI Image Alzheimer's Disease Classification
Venue: arXiv cs.CV 2020
Year: 2020

We propose to apply a 2D CNN architecture to 3D MRI image Alzheimer's disease classification. Training a 3D convolutional neural network (CNN) is time-consuming and computationally expensive. We make use of approximate rank pooling to transform the 3D MRI image volume into a 2D image to use as input to a 2D CNN. We show our proposed CNN model achieves $9.5\%$ better Alzheimer's disease classification accuracy than the baseline 3D models. We also show that our method allows for efficient training, requiring only 20% of the training time compared to 3D CNN models. The code is available online: https://github.com/UkyVision/alzheimer-project.
Utilizing UNet for the future traffic map prediction task Traffic4cast challenge 2020
Venue: arXiv cs.CV 2020
Year: 2020

This paper describes our UNet based experiments on the Traffic4cast challenge 2020. Similar to the Traffic4cast challenge 2019, the task is to predict traffic flow volume, direction and speed on a high resolution map of three large cities worldwide. We mainly experimented with UNet based deep convolutional networks with various compositions of densely connected convolution layers, average pooling layers and max pooling layers. Three base UNet model types are tried and predictions are combined by averaging prediction scores or taking median value. Our method achieved best performance in this years newly built challenge dataset.
Improving accuracy of rare words for RNN-Transducer through unigram shallow fusion
Venue: arXiv cs.CL 2020
Year: 2020

End-to-end automatic speech recognition (ASR) systems, such as recurrent neural network transducer (RNN-T), have become popular, but rare word remains a challenge. In this paper, we propose a simple, yet effective method called unigram shallow fusion (USF) to improve rare words for RNN-T. In USF, we extract rare words from RNN-T training data based on unigram count, and apply a fixed reward when the word is encountered during decoding. We show that this simple method can improve performance on rare words by 3.7% WER relative without degradation on general test set, and the improvement from USF is additive to any additional language model based rescoring. Then, we show that the same USF does not work on conventional hybrid system. Finally, we reason that USF works by fixing errors in probability estimates of words due to Viterbi search used during decoding with subword-based RNN-T.
Solvable Model for Inheriting the Regularization through Knowledge Distillation
Venue: arXiv cs.LG 2020
Year: 2020

In recent years the empirical success of transfer learning with neural networks has stimulated an increasing interest in obtaining a theoretical understanding of its core properties. Knowledge distillation where a smaller neural network is trained using the outputs of a larger neural network is a particularly interesting case of transfer learning. In the present work, we introduce a statistical physics framework that allows an analytic characterization of the properties of knowledge distillation (KD) in shallow neural networks. Focusing the analysis on a solvable model that exhibits a non-trivial generalization gap, we investigate the effectiveness of KD. We are able to show that, through KD, the regularization properties of the larger teacher model can be inherited by the smaller student and that the yielded generalization performance is closely linked to and limited by the optimality of the teacher. Finally, we analyze the double descent phenomenology that can arise in the considered KD setting.
High Quality Real-Time Structured Debate Generation
Venue: arXiv cs.CL 2020
Year: 2020

Automatically generating debates is a challenging task that requires an understanding of arguments and how to negate or support them. In this work we define debate trees and paths for generating debates while enforcing a high level structure and grammar. We leverage a large corpus of tree-structured debates that have metadata associated with each argument. We develop a framework for generating plausible debates which is agnostic to the sentence embedding model. Our results demonstrate the ability to generate debates in real-time on complex topics at a quality that is close to humans, as evaluated by the style, content, and strategy metrics used for judging competitive human debates. In the spirit of reproducible research we make our data, models, and code publicly available.
Audience and Streamer Participation at Scale on Twitch
Venue: arXiv cs.SI 2020
Year: 2020

Large-scale streaming platforms such as Twitch are becoming increasingly popular, but detailed audience-streamer interaction dynamics remain unexplored at scale. In this paper, we perform a mixed-methods study on a dataset with over 12 million audience chat messages and 45 hours of streaming video to understand audience participation and streamer performance on Twitch. We uncover five types of streams based on size and audience participation styles: Clique Streams, small streams with close streamer-audience interactions; Rising Streamers, mid-range streams using custom technology and moderators to formalize their communities; Chatter-boxes, mid-range streams with established conversational dynamics; Spotlight Streamers, large streams that engage large numbers of viewers while still retaining a sense of community; and Professionals, massive streams with the stadium-style audiences. We discuss challenges and opportunities emerging for streamers and audiences from each style and conclude by providing data-backed design implications that empower streamers, audiences, live streaming platforms, and game designers
Uncertainty-Constrained Differential Dynamic Programming in Belief Space for Vision Based Robots
Venue: arXiv cs.RO 2020
Year: 2020

Most mobile robots follow a modular sense-planact system architecture that can lead to poor performance or even catastrophic failure for visual inertial navigation systems due to trajectories devoid of feature matches. Planning in belief space provides a unified approach to tightly couple the perception, planning and control modules, leading to trajectories that are robust to noisy measurements and disturbances. However, existing methods handle uncertainties as costs that require manual tuning for varying environments and hardware. We therefore propose a novel trajectory optimization formulation that incorporates inequality constraints on uncertainty and a novel Augmented Lagrangian based stochastic differential dynamic programming method in belief space. Furthermore, we develop a probabilistic visibility model that accounts for discontinuities due to feature visibility limits. Our simulation tests demonstrate that our method can handle inequality constraints in different environments, for holonomic and nonholonomic motion models with no manual tuning of uncertainty costs involved. We also show the improved optimization performance in belief space due to our visibility model.
Strike on Stage: a percussion and media performance
Venue: arXiv cs.SD 2020
Year: 2020

This paper describes Strike on Stage, an interface and corresponding audio-visual performance work developed and performed in 2010 by percussionists and media artists Chi-Hsia Lai and Charles Martin. The concept of Strike on Stage is to integrate computer visuals and sound into an improvised percussion performance. A large projection surface is positioned directly behind the performers, while a computer vision system tracks their movements. The setup allows computer visualisation and sonification to be directly responsive and unified with the performers' gestures.
CycleGAN without checkerboard artifacts for counter-forensics of fake-image detection
Venue: arXiv cs.CV 2020
Year: 2020

In this paper, we propose a novel CycleGAN without checkerboard artifacts for counter-forensics of fake-image detection. Recent rapid advances in image manipulation tools and deep image synthesis techniques, such as Generative Adversarial Networks (GANs) have easily generated fake images, so detecting manipulated images has become an urgent issue. Most state-of-the-art forgery detection methods assume that images include checkerboard artifacts which are generated by using DNNs. Accordingly, we propose a novel CycleGAN without any checkerboard artifacts for counter-forensics of fake-mage detection methods for the first time, as an example of GANs without checkerboard artifacts.
Unsupervised Part Discovery via Feature Alignment
Venue: arXiv cs.CV 2020
Year: 2020

Understanding objects in terms of their individual parts is important, because it enables a precise understanding of the objects' geometrical structure, and enhances object recognition when the object is seen in a novel pose or under partial occlusion. However, the manual annotation of parts in large scale datasets is time consuming and expensive. In this paper, we aim at discovering object parts in an unsupervised manner, i.e., without ground-truth part or keypoint annotations. Our approach builds on the intuition that objects of the same class in a similar pose should have their parts aligned at similar spatial locations. We exploit the property that neural network features are largely invariant to nuisance variables and the main remaining source of variations between images of the same object category is the object pose. Specifically, given a training image, we find a set of similar images that show instances of the same object category in the same pose, through an affine alignment of their corresponding feature maps. The average of the aligned feature maps serves as pseudo ground-truth annotation for a supervised training of the deep network backbone. During inference, part detection is simple and fast, without any extra modules or overheads other than a feed-forward neural network. Our experiments on several datasets from different domains verify the effectiveness of the proposed method. For example, we achieve 37.8 mAP on VehiclePart, which is at least 4.2 better than previous methods.
Python Workflows on HPC Systems
Venue: arXiv cs.LG 2020
Year: 2020

The recent successes and wide spread application of compute intensive machine learning and data analytics methods have been boosting the usage of the Python programming language on HPC systems. While Python provides many advantages for the users, it has not been designed with a focus on multi-user environments or parallel programming - making it quite challenging to maintain stable and secure Python workflows on a HPC system. In this paper, we analyze the key problems induced by the usage of Python on HPC clusters and sketch appropriate workarounds for efficiently maintaining multi-user Python software environments, securing and restricting resources of Python jobs and containing Python processes, while focusing on Deep Learning applications running on GPU clusters.
Deep Ad-hoc Beamforming Based on Speaker Extraction for Target-Dependent Speech Separation
Venue: arXiv cs.SD 2020
Year: 2020

Recently, the research on ad-hoc microphone arrays with deep learning has drawn much attention, especially in speech enhancement and separation. Because an ad-hoc microphone array may cover such a large area that multiple speakers may locate far apart and talk independently, target-dependent speech separation, which aims to extract a target speaker from a mixed speech, is important for extracting and tracing a specific speaker in the ad-hoc array. However, this technique has not been explored yet. In this paper, we propose deep ad-hoc beamforming based on speaker extraction, which is to our knowledge the first work for target-dependent speech separation based on ad-hoc microphone arrays and deep learning. The algorithm contains three components. First, we propose a supervised channel selection framework based on speaker extraction, where the estimated utterance-level SNRs of the target speech are used as the basis for the channel selection. Second, we apply the selected channels to a deep learning based MVDR algorithm, where a single-channel speaker extraction algorithm is applied to each selected channel for estimating the mask of the target speech. We conducted an extensive experiment on a WSJ0-adhoc corpus. Experimental results demonstrate the effectiveness of the proposed method.
A Generative Model to Synthesize EEG Data for Epileptic Seizure Prediction
Venue: arXiv cs.LG 2020
Year: 2020

Prediction of seizure before they occur is vital for bringing normalcy to the lives of patients. Researchers employed machine learning methods using hand-crafted features for seizure prediction. However, ML methods are too complicated to select the best ML model or best features. Deep Learning methods are beneficial in the sense of automatic feature extraction. One of the roadblocks for accurate seizure prediction is scarcity of epileptic seizure data. This paper addresses this problem by proposing a deep convolutional generative adversarial network to generate synthetic EEG samples. We use two methods to validate synthesized data namely, one-class SVM and a new proposal which we refer to as convolutional epileptic seizure predictor (CESP). Another objective of our study is to evaluate performance of well-known deep learning models (e.g., VGG16, VGG19, ResNet50, and Inceptionv3) by training models on augmented data using transfer learning with average time of 10 min between true prediction and seizure onset. Our results show that CESP model achieves sensitivity of 78.11% and 88.21%, and FPR of 0.27/h and 0.14/h for training on synthesized and testing on real Epilepsyecosystem and CHB-MIT datasets, respectively. Effective results of CESP trained on synthesized data shows that synthetic data acquired the correlation between features and labels very well. We also show that employment of idea of transfer learning and data augmentation in patient-specific manner provides highest accuracy with sensitivity of 90.03% and 0.03 FPR/h which was achieved using Inceptionv3, and that augmenting data with samples generated from DCGAN increased prediction results of our CESP model and Inceptionv3 by 4-5% as compared to state-of-the-art traditional augmentation techniques. Finally, we note that prediction results of CESP achieved by using augmented data are better than chance level for both datasets.
Towards a Universal Features Set for IoT Botnet Attacks Detection
Venue: arXiv cs.CR 2020
Year: 2020

The security pitfalls of IoT devices make it easy for the attackers to exploit the IoT devices and make them a part of a botnet. Once hundreds of thousands of IoT devices are compromised and become the part of a botnet, the attackers use this botnet to launch the large and complex distributed denial of service (DDoS) attacks which take down the target websites or services and make them unable to respond the legitimate users. So far, many botnet detection techniques have been proposed but their performance is limited to a specific dataset on which they are trained. This is because the features used to train a machine learning model on one botnet dataset, do not perform well on other datasets due to the diversity of attack patterns. Therefore, in this paper, we propose a universal features set to better detect the botnet attacks regardless of the underlying dataset. The proposed features set manifest preeminent results for detecting the botnet attacks when tested the trained machine learning models over three different botnet attack datasets.
Problems of representation of electrocardiograms in convolutional neural networks
Venue: arXiv cs.LG 2020
Year: 2020

Using electrocardiograms as an example, we demonstrate the characteristic problems that arise when modeling one-dimensional signals containing inaccurate repeating pattern by means of standard convolutional networks. We show that these problems are systemic in nature. They are due to how convolutional networks work with composite objects, parts of which are not fixed rigidly, but have significant mobility. We also demonstrate some counterintuitive effects related to generalization in deep networks.
Power Control for a URLLC-enabled UAV system incorporated with DNN-Based Channel Estimation
Venue: arXiv cs.IT 2020
Year: 2020

This letter is concerned with power control for a ultra-reliable and low-latency communications (URLLC) enabled unmanned aerial vehicle (UAV) system incorporated with deep neural network (DNN) based channel estimation. Particularly, we formulate the power control problem for the UAV system as an optimization problem to accommodate the URLLC requirement of uplink control and non-payload signal delivery while ensuring the downlink high-speed payload transmission. This problem is challenging to be solved due to the requirement of analytically tractable channel models and the non-convex characteristic as well. To address the challenges, we propose a novel power control algorithm, which constructs analytically tractable channel models based on DNN estimation results and explores a semidefinite relaxation (SDR) scheme to tackle the non-convexity. Simulation results demonstrate the accuracy of the DNN estimation and verify the effectiveness of the proposed algorithm.
Neural language models for text classification in evidence-based medicine
Venue: arXiv cs.CL 2020
Year: 2020

The COVID-19 has brought about a significant challenge to the whole of humanity, but with a special burden upon the medical community. Clinicians must keep updated continuously about symptoms, diagnoses, and effectiveness of emergent treatments under a never-ending flood of scientific literature. In this context, the role of evidence-based medicine (EBM) for curating the most substantial evidence to support public health and clinical practice turns essential but is being challenged as never before due to the high volume of research articles published and pre-prints posted daily. Artificial Intelligence can have a crucial role in this situation. In this article, we report the results of an applied research project to classify scientific articles to support Epistemonikos, one of the most active foundations worldwide conducting EBM. We test several methods, and the best one, based on the XLNet neural language model, improves the current approach by 93\% on average F1-score, saving valuable time from physicians who volunteer to curate COVID-19 research articles manually.
Meta-Embeddings for Natural Language Inference and Semantic Similarity tasks
Venue: arXiv cs.CL 2020
Year: 2020

Word Representations form the core component for almost all advanced Natural Language Processing (NLP) applications such as text mining, question-answering, and text summarization, etc. Over the last two decades, immense research is conducted to come up with one single model to solve all major NLP tasks. The major problem currently is that there are a plethora of choices for different NLP tasks. Thus for NLP practitioners, the task of choosing the right model to be used itself becomes a challenge. Thus combining multiple pre-trained word embeddings and forming meta embeddings has become a viable approach to improve tackle NLP tasks. Meta embedding learning is a process of producing a single word embedding from a given set of pre-trained input word embeddings. In this paper, we propose to use Meta Embedding derived from few State-of-the-Art (SOTA) models to efficiently tackle mainstream NLP tasks like classification, semantic relatedness, and text similarity. We have compared both ensemble and dynamic variants to identify an efficient approach. The results obtained show that even the best State-of-the-Art models can be bettered. Thus showing us that meta-embeddings can be used for several NLP tasks by harnessing the power of several individual representations.
Simulating Surface Wave Dynamics with Convolutional Networks
Venue: arXiv cs.LG 2020
Year: 2020

We investigate the performance of fully convolutional networks to simulate the motion and interaction of surface waves in open and closed complex geometries. We focus on a U-Net architecture and analyse how well it generalises to geometric configurations not seen during training. We demonstrate that a modified U-Net architecture is capable of accurately predicting the height distribution of waves on a liquid surface within curved and multi-faceted open and closed geometries, when only simple box and right-angled corner geometries were seen during training. We also consider a separate and independent 3D CNN for performing time-interpolation on the predictions produced by our U-Net. This allows generating simulations with a smaller time-step size than the one the U-Net has been trained for.