OpenRaiser/Intern-Atlas huggingface dataset:
Fooling Adversarial Training with Inducing Noise
Venue: arXiv cs.LG 2021
Year: 2021

Adversarial training is widely believed to be a reliable approach to improve model robustness against adversarial attack. However, in this paper, we show that when trained on one type of poisoned data, adversarial training can also be fooled to have catastrophic behavior, e.g., $<1\%$ robust test accuracy with $>90\%$ robust training accuracy on CIFAR-10 dataset. Previously, there are other types of noise poisoned in the training data that have successfully fooled standard training ($15.8\%$ standard test accuracy with $99.9\%$ standard training accuracy on CIFAR-10 dataset), but their poisonings can be easily removed when adopting adversarial training. Therefore, we aim to design a new type of inducing noise, named ADVIN, which is an irremovable poisoning of training data. ADVIN can not only degrade the robustness of adversarial training by a large margin, for example, from $51.7\%$ to $0.57\%$ on CIFAR-10 dataset, but also be effective for fooling standard training ($13.1\%$ standard test accuracy with $100\%$ standard training accuracy). Additionally, ADVIN can be applied to preventing personal data (like selfies) from being exploited without authorization under whether standard or adversarial training.
Grounded Situation Recognition with Transformers
Venue: arXiv cs.CV 2021
Year: 2021

Grounded Situation Recognition (GSR) is the task that not only classifies a salient action (verb), but also predicts entities (nouns) associated with semantic roles and their locations in the given image. Inspired by the remarkable success of Transformers in vision tasks, we propose a GSR model based on a Transformer encoder-decoder architecture. The attention mechanism of our model enables accurate verb classification by capturing high-level semantic feature of an image effectively, and allows the model to flexibly deal with the complicated and image-dependent relations between entities for improved noun classification and localization. Our model is the first Transformer architecture for GSR, and achieves the state of the art in every evaluation metric on the SWiG benchmark. Our code is available at https://github.com/jhcho99/gsrtr .
Improved Prosodic Clustering for Multispeaker and Speaker-independent Phoneme-level Prosody Control
Venue: arXiv cs.SD 2021
Year: 2021

This paper presents a method for phoneme-level prosody control of F0 and duration on a multispeaker text-to-speech setup, which is based on prosodic clustering. An autoregressive attention-based model is used, incorporating multispeaker architecture modules in parallel to a prosody encoder. Several improvements over the basic single-speaker method are proposed that increase the prosodic control range and coverage. More specifically we employ data augmentation, F0 normalization, balanced clustering for duration, and speaker-independent prosodic clustering. These modifications enable fine-grained phoneme-level prosody control for all speakers contained in the training set, while maintaining the speaker identity. The model is also fine-tuned to unseen speakers with limited amounts of data and it is shown to maintain its prosody control capabilities, verifying that the speaker-independent prosodic clustering is effective. Experimental results verify that the model maintains high output speech quality and that the proposed method allows efficient prosody control within each speaker's range despite the variability that a multispeaker setting introduces.
Word-Level Style Control for Expressive, Non-attentive Speech Synthesis
Venue: arXiv cs.SD 2021
Year: 2021

This paper presents an expressive speech synthesis architecture for modeling and controlling the speaking style at a word level. It attempts to learn word-level stylistic and prosodic representations of the speech data, with the aid of two encoders. The first one models style by finding a combination of style tokens for each word given the acoustic features, and the second outputs a word-level sequence conditioned only on the phonetic information in order to disentangle it from the style information. The two encoder outputs are aligned and concatenated with the phoneme encoder outputs and then decoded with a Non-Attentive Tacotron model. An extra prior encoder is used to predict the style tokens autoregressively, in order for the model to be able to run without a reference utterance. We find that the resulting model gives both word-level and global control over the style, as well as prosody transfer capabilities.
Streaming Deletion Problems Parameterized by Vertex Cover
Venue: arXiv cs.DS 2021
Year: 2021

Streaming is a model where an input graph is provided one edge at a time, instead of being able to inspect it at will. In this work, we take a parameterized approach by assuming a vertex cover of the graph is given, building on work of Bishnu et al. [COCOON 2020]. We show the further potency of combining this parameter with the Adjacency List streaming model to obtain results for vertex deletion problems. This includes kernels, parameterized algorithms, and lower bounds for the problems of Pi-free Deletion, H-free Deletion, and the more specific forms of Cluster Vertex Deletion and Odd Cycle Transversal. We focus on the complexity in terms of the number of passes over the input stream, and the memory used. This leads to a pass/memory trade-off, where a different algorithm might be favourable depending on the context and instance. We also discuss implications for parameterized complexity in the non-streaming setting.
Optimisation of job scheduling for supercomputers with burst buffers
Venue: arXiv cs.PF 2021
Year: 2021

The ever-increasing gap between compute and I/O performance in HPC platforms, together with the development of novel NVMe storage devices (NVRAM), led to the emergence of the burst buffer concept - an intermediate persistent storage layer logically positioned between random-access main memory and a parallel file system. Since the appearance of this technology, numerous supercomputers have been equipped with burst buffers exploring various architectures. Despite the development of real-world architectures as well as research concepts, Resource and Job Management Systems, such as Slurm, provide only marginal support for scheduling jobs with burst buffer requirements. This research is primarily motivated by the alerting observation that burst buffers are omitted from reservations in the procedure of backfilling in existing job schedulers. In this dissertation, we forge a detailed supercomputer simulator based on Batsim and SimGrid, which is capable of simulating I/O contention and I/O congestion effects. Due to the lack of publicly available workloads with burst buffer requests, we create a burst buffer request distribution model derived from Parallel Workload Archive logs. We investigate the impact of burst buffer reservations on the overall efficiency of online job scheduling for canonical algorithms: First-Come-First-Served (FCFS) and Shortest-Job-First (SJF) EASY-backfilling. Our results indicate that the lack of burst buffer reservations in backfilling may significantly deteriorate the performance of scheduling. [...] Furthermore, this lack of reservations may cause the starvation of medium-size and wide jobs. Finally, we propose a burst-buffer-aware plan-based scheduling algorithm with simulated annealing optimisation, which improves the mean waiting time by over 20% and mean bounded slowdown by 27% compared to the SJF EASY-backfilling.
Ubi-SleepNet: Advanced Multimodal Fusion Techniques for Three-stage Sleep Classification Using Ubiquitous Sensing
Venue: arXiv cs.LG 2021
Year: 2021

Sleep is a fundamental physiological process that is essential for sustaining a healthy body and mind. The gold standard for clinical sleep monitoring is polysomnography(PSG), based on which sleep can be categorized into five stages, including wake/rapid eye movement sleep (REM sleep)/Non-REM sleep 1 (N1)/Non-REM sleep 2 (N2)/Non-REM sleep 3 (N3). However, PSG is expensive, burdensome, and not suitable for daily use. For long-term sleep monitoring, ubiquitous sensing may be a solution. Most recently, cardiac and movement sensing has become popular in classifying three-stage sleep, since both modalities can be easily acquired from research-grade or consumer-grade devices (e.g., Apple Watch). However, how best to fuse the data for the greatest accuracy remains an open question. In this work, we comprehensively studied deep learning (DL)-based advanced fusion techniques consisting of three fusion strategies alongside three fusion methods for three-stage sleep classification based on two publicly available datasets. Experimental results demonstrate important evidence that three-stage sleep can be reliably classified by fusing cardiac/movement sensing modalities, which may potentially become a practical tool to conduct large-scale sleep stage assessment studies or long-term self-tracking on sleep. To accelerate the progression of sleep research in the ubiquitous/wearable computing community, we made this project open source, and the code can be found at: https://github.com/bzhai/Ubi-SleepNet.
A 3D 2D convolutional Neural Network Model for Hyperspectral Image Classification
Venue: arXiv cs.CV 2021
Year: 2021

In the proposed SEHybridSN model, a dense block was used to reuse shallow feature and aimed at better exploiting hierarchical spatial spectral feature. Subsequent depth separable convolutional layers were used to discriminate the spatial information. Further refinement of spatial spectral features was realized by the channel attention method, which were performed behind every 3D convolutional layer and every 2D convolutional layer. Experiment results indicate that our proposed model learn more discriminative spatial spectral features using very few training data. SEHybridSN using only 0.05 and 0.01 labeled data for training, a very satisfactory performance is obtained.
Unsupervised Visual Time-Series Representation Learning and Clustering
Venue: arXiv cs.LG 2021
Year: 2021

Time-series data is generated ubiquitously from Internet-of-Things (IoT) infrastructure, connected and wearable devices, remote sensing, autonomous driving research and, audio-video communications, in enormous volumes. This paper investigates the potential of unsupervised representation learning for these time-series. In this paper, we use a novel data transformation along with novel unsupervised learning regime to transfer the learning from other domains to time-series where the former have extensive models heavily trained on very large labelled datasets. We conduct extensive experiments to demonstrate the potential of the proposed approach through time-series clustering.
Max-algebraic hybrid automata: Modelling and equivalences
Venue: arXiv cs.FL 2021
Year: 2021

This article introduces the novel framework of max-algebraic hybrid automata as a hybrid modelling language in the max-plus algebra. We show that the modelling framework unifies and extends the switching max-plus linear systems framework and is analogous to the discrete hybrid automata framework in conventional algebra. In addition, we show that the framework serves as a bridge between automata-theoretic models in max-plus algebra and switching max-plus linear systems. In doing so, we formalise the relationship between max-plus automata and switching max-plus linear systems in a behavioural sense. This also serves as another step towards importing tools for analysis and optimal control from conventional time-driven hybrid systems to discrete-event systems in max-plus algebra.
Solving Visual Analogies Using Neural Algorithmic Reasoning
Venue: arXiv cs.LG 2021
Year: 2021

We consider a class of visual analogical reasoning problems that involve discovering the sequence of transformations by which pairs of input/output images are related, so as to analogously transform future inputs. This program synthesis task can be easily solved via symbolic search. Using a variation of the `neural analogical reasoning' approach of (Velickovic and Blundell 2021), we instead search for a sequence of elementary neural network transformations that manipulate distributed representations derived from a symbolic space, to which input images are directly encoded. We evaluate the extent to which our `neural reasoning' approach generalizes for images with unseen shapes and positions.
RacketStore: Measurements of ASO Deception in Google Play via Mobile and App Usage
Venue: arXiv cs.CR 2021
Year: 2021

Online app search optimization (ASO) platforms that provide bulk installs and fake reviews for paying app developers in order to fraudulently boost their search rank in app stores, were shown to employ diverse and complex strategies that successfully evade state-of-the-art detection methods. In this paper we introduce RacketStore, a platform to collect data from Android devices of participating ASO providers and regular users, on their interactions with apps which they install from the Google Play Store. We present measurements from a study of 943 installs of RacketStore on 803 unique devices controlled by ASO providers and regular users, that consists of 58,362,249 data snapshots collected from these devices, the 12,341 apps installed on them and their 110,511,637 Google Play reviews. We reveal significant differences between ASO providers and regular users in terms of the number and types of user accounts registered on their devices, the number of apps they review, and the intervals between the installation times of apps and their review times. We leverage these insights to introduce features that model the usage of apps and devices, and show that they can train supervised learning algorithms to detect paid app installs and fake reviews with an F1-measure of 99.72% (AUC above 0.99), and detect devices controlled by ASO providers with an F1-measure of 95.29% (AUC = 0.95). We discuss the costs associated with evading detection by our classifiers and also the potential for app stores to use our approach to detect ASO work with privacy.
Tweet Analysis for Enhancement of COVID-19 Epidemic Simulation: A Case Study in Japan
Venue: arXiv cs.SI 2021
Year: 2021

The COVID-19 pandemic, which began in December 2019, progressed in a complicated manner and thus caused problems worldwide. Seeking clues to the reasons for the complicated progression is necessary but challenging in the fight against the pandemic. We sought clues by investigating the relationship between reactions on social media and the COVID-19 epidemic in Japan. Twitter was selected as the social media platform for study because it has a large user base in Japan and because it quickly propagates short topic-focused messages ("tweets"). Analysis using Japanese Twitter data suggests that reactions on social media and the progression of the COVID-19 pandemic may have a close relationship. Experiments to evaluate the potential of using tweets to support the prediction of how an epidemic will progress demonstrated the value of using epidemic-related social media data. Our findings provide insights into the relationship between user reactions on social media, particularly Twitter, and epidemic progression, which can be used to fight pandemics.
Inter-Domain Fusion for Enhanced Intrusion Detection in Power Systems: An Evidence Theoretic and Meta-Heuristic Approach
Venue: arXiv cs.AI 2021
Year: 2021

False alerts due to misconfigured/ compromised IDS in ICS networks can lead to severe economic and operational damage. To solve this problem, research has focused on leveraging deep learning techniques that help reduce false alerts. However, a shortcoming is that these works often require or implicitly assume the physical and cyber sensors to be trustworthy. Implicit trust of data is a major problem with using artificial intelligence or machine learning for CPS security, because during critical attack detection time they are more at risk, with greater likelihood and impact, of also being compromised. To address this shortcoming, the problem is reframed on how to make good decisions given uncertainty. Then, the decision is detection, and the uncertainty includes whether the data used for ML-based IDS is compromised. Thus, this work presents an approach for reducing false alerts in CPS power systems by dealing uncertainty without the knowledge of prior distribution of alerts. Specifically, an evidence theoretic based approach leveraging Dempster Shafer combination rules are proposed for reducing false alerts. A multi-hypothesis mass function model is designed that leverages probability scores obtained from various supervised-learning classifiers. Using this model, a location-cum-domain based fusion framework is proposed and evaluated with different combination rules, that fuse multiple evidence from inter-domain and intra-domain sensors. The approach is demonstrated in a cyber-physical power system testbed with Man-In-The-Middle attack emulation in a large-scale synthetic electric grid. For evaluating the performance, plausibility, belief, pignistic, etc. metrics as decision functions are considered. To improve the performance, a multi-objective based genetic algorithm is proposed for feature selection considering the decision metrics as the fitness function.
Feature selection or extraction decision process for clustering using PCA and FRSD
Venue: arXiv cs.LG 2021
Year: 2021

This paper concerns the critical decision process of extracting or selecting the features before applying a clustering algorithm. It is not obvious to evaluate the importance of the features since the most popular methods to do it are usually made for a supervised learning technique process. A clustering algorithm is an unsupervised method. It means that there is no known output label to match the input data. This paper proposes a new method to choose the best dimensionality reduction method (selection or extraction) according to the data scientist's parameters, aiming to apply a clustering process at the end. It uses Feature Ranking Process Based on Silhouette Decomposition (FRSD) algorithm, a Principal Component Analysis (PCA) algorithm, and a K-Means algorithm along with its metric, the Silhouette Index (SI). This paper presents 5 use cases based on a smart city dataset. This research also aims to discuss the impacts, the advantages, and the disadvantages of each choice that can be made in this unsupervised learning process.
Combining Data-driven Supervision with Human-in-the-loop Feedback for Entity Resolution
Venue: arXiv cs.CL 2021
Year: 2021

The distribution gap between training datasets and data encountered in production is well acknowledged. Training datasets are often constructed over a fixed period of time and by carefully curating the data to be labeled. Thus, training datasets may not contain all possible variations of data that could be encountered in real-world production environments. Tasked with building an entity resolution system - a model that identifies and consolidates data points that represent the same person - our first model exhibited a clear training-production performance gap. In this case study, we discuss our human-in-the-loop enabled, data-centric solution to closing the training-production performance divergence. We conclude with takeaways that apply to data-centric learning at large.
An End-to-End Framework for Dynamic Crime Profiling of Places
Venue: arXiv cs.MM 2021
Year: 2021

Much effort is being made to ensure the safety of people. One of the main requirements of travellers and city administrators is to have knowledge of places that are more prone to criminal activities. To rate a place as a potential crime location, it needs the past crime history at that location. Such data is not easily available in the public domain, however, it floats around on the Internet in the form of newspaper and social media posts, in an unstructured manner though. Consequently, a large number of works are reported on extracting crime information from news articles, providing piecemeal solutions to the problem. This chapter complements these works by building an end-to-end framework for crime profiling of any given location/area. It customizes individual components of the framework and provides a Spatio-temporal integration of crime information. It develops an automated framework that crawls online news articles, analyzes them, and extracts relevant information to create a crime knowledge base that gets dynamically updated in real-time. The crime density can be easily visualized in the form of a heat map which is generated by the knowledge base. As a case study, it investigates 345448 news articles published by 6 daily English newspapers collected for approximately two years. Experimental results show that the crime profiling matches with the ratings calculated manually by various organizations.
Learning Non-Stationary Time-Series with Dynamic Pattern Extractions
Venue: arXiv cs.LG 2021
Year: 2021

The era of information explosion had prompted the accumulation of a tremendous amount of time-series data, including stationary and non-stationary time-series data. State-of-the-art algorithms have achieved a decent performance in dealing with stationary temporal data. However, traditional algorithms that tackle stationary time-series do not apply to non-stationary series like Forex trading. This paper investigates applicable models that can improve the accuracy of forecasting future trends of non-stationary time-series sequences. In particular, we focus on identifying potential models and investigate the effects of recognizing patterns from historical data. We propose a combination of \rebuttal{the} seq2seq model based on RNN, along with an attention mechanism and an enriched set features extracted via dynamic time warping and zigzag peak valley indicators. Customized loss functions and evaluating metrics have been designed to focus more on the predicting sequence's peaks and valley points. Our results show that our model can predict 4-hour future trends with high accuracy in the Forex dataset, which is crucial in realistic scenarios to assist foreign exchange trading decision making. We further provide evaluations of the effects of various loss functions, evaluation metrics, model variants, and components on model performance.
Deep Spoken Keyword Spotting: An Overview
Venue: arXiv cs.SD 2021
Year: 2021

Spoken keyword spotting (KWS) deals with the identification of keywords in audio streams and has become a fast-growing technology thanks to the paradigm shift introduced by deep learning a few years ago. This has allowed the rapid embedding of deep KWS in a myriad of small electronic devices with different purposes like the activation of voice assistants. Prospects suggest a sustained growth in terms of social use of this technology. Thus, it is not surprising that deep KWS has become a hot research topic among speech scientists, who constantly look for KWS performance improvement and computational complexity reduction. This context motivates this paper, in which we conduct a literature review into deep spoken KWS to assist practitioners and researchers who are interested in this technology. Specifically, this overview has a comprehensive nature by covering a thorough analysis of deep KWS systems (which includes speech features, acoustic modeling and posterior handling), robustness methods, applications, datasets, evaluation metrics, performance of deep KWS systems and audio-visual KWS. The analysis performed in this paper allows us to identify a number of directions for future research, including directions adopted from automatic speech recognition research and directions that are unique to the problem of spoken KWS.