OpenRaiser/Intern-Atlas huggingface dataset:
Combining Spatial Clustering with LSTM Speech Models for Multichannel Speech Enhancement
Venue: arXiv cs.SD 2020
Year: 2020

Recurrent neural networks using the LSTM architecture can achieve significant single-channel noise reduction. It is not obvious, however, how to apply them to multi-channel inputs in a way that can generalize to new microphone configurations. In contrast, spatial clustering techniques can achieve such generalization, but lack a strong signal model. This paper combines the two approaches to attain both the spatial separation performance and generality of multichannel spatial clustering and the signal modeling performance of multiple parallel single-channel LSTM speech enhancers. The system is compared to several baselines on the CHiME3 dataset in terms of speech quality predicted by the PESQ algorithm and word error rate of a recognizer trained on mis-matched conditions, in order to focus on generalization. Our experiments show that by combining the LSTM models with the spatial clustering, we reduce word error rate by 4.6\% absolute (17.2\% relative) on the development set and 11.2\% absolute (25.5\% relative) on test set compared with spatial clustering system, and reduce by 10.75\% (32.72\% relative) on development set and 6.12\% absolute (15.76\% relative) on test data compared with LSTM model.
Modeling Updates of Scholarly Webpages Using Archived Data
Venue: arXiv cs.DL 2020
Year: 2020

The vastness of the web imposes a prohibitive cost on building large-scale search engines with limited resources. Crawl frontiers thus need to be optimized to improve the coverage and freshness of crawled content. In this paper, we propose an approach for modeling the dynamics of change in the web using archived copies of webpages. To evaluate its utility, we conduct a preliminary study on the scholarly web using 19,977 seed URLs of authors' homepages obtained from their Google Scholar profiles. We first obtain archived copies of these webpages from the Internet Archive (IA), and estimate when their actual updates occurred. Next, we apply maximum likelihood to estimate their mean update frequency ($λ$) values. Our evaluation shows that $λ$ values derived from a short history of archived data provide a good estimate for the true update frequency in the short-term, and that our method provides better estimations of updates at a fraction of resources compared to the baseline models. Based on this, we demonstrate the utility of archived data to optimize the crawling strategy of web crawlers, and uncover important challenges that inspire future research directions.
Learning Graph Neural Networks with Approximate Gradient Descent
Venue: arXiv cs.LG 2020
Year: 2020

The first provably efficient algorithm for learning graph neural networks (GNNs) with one hidden layer for node information convolution is provided in this paper. Two types of GNNs are investigated, depending on whether labels are attached to nodes or graphs. A comprehensive framework for designing and analyzing convergence of GNN training algorithms is developed. The algorithm proposed is applicable to a wide range of activation functions including ReLU, Leaky ReLU, Sigmod, Softplus and Swish. It is shown that the proposed algorithm guarantees a linear convergence rate to the underlying true parameters of GNNs. For both types of GNNs, sample complexity in terms of the number of nodes or the number of graphs is characterized. The impact of feature dimension and GNN structure on the convergence rate is also theoretically characterized. Numerical experiments are further provided to validate our theoretical analysis.
Multi-Instrumentalist Net: Unsupervised Generation of Music from Body Movements
Venue: arXiv cs.SD 2020
Year: 2020

We propose a novel system that takes as an input body movements of a musician playing a musical instrument and generates music in an unsupervised setting. Learning to generate multi-instrumental music from videos without labeling the instruments is a challenging problem. To achieve the transformation, we built a pipeline named 'Multi-instrumentalistNet' (MI Net). At its base, the pipeline learns a discrete latent representation of various instruments music from log-spectrogram using a Vector Quantized Variational Autoencoder (VQ-VAE) with multi-band residual blocks. The pipeline is then trained along with an autoregressive prior conditioned on the musician's body keypoints movements encoded by a recurrent neural network. Joint training of the prior with the body movements encoder succeeds in the disentanglement of the music into latent features indicating the musical components and the instrumental features. The latent space results in distributions that are clustered into distinct instruments from which new music can be generated. Furthermore, the VQ-VAE architecture supports detailed music generation with additional conditioning. We show that a Midi can further condition the latent space such that the pipeline will generate the exact content of the music being played by the instrument in the video. We evaluate MI Net on two datasets containing videos of 13 instruments and obtain generated music of reasonable audio quality, easily associated with the corresponding instrument, and consistent with the music audio content.
Adaptive Local Bayesian Optimization Over Multiple Discrete Variables
Venue: arXiv cs.LG 2020
Year: 2020

In the machine learning algorithms, the choice of the hyperparameter is often an art more than a science, requiring labor-intensive search with expert experience. Therefore, automation on hyperparameter optimization to exclude human intervention is a great appeal, especially for the black-box functions. Recently, there have been increasing demands of solving such concealed tasks for better generalization, though the task-dependent issue is not easy to solve. The Black-Box Optimization challenge (NeurIPS 2020) required competitors to build a robust black-box optimizer across different domains of standard machine learning problems. This paper describes the approach of team KAIST OSI in a step-wise manner, which outperforms the baseline algorithms by up to +20.39%. We first strengthen the local Bayesian search under the concept of region reliability. Then, we design a combinatorial kernel for a Gaussian process kernel. In a similar vein, we combine the methodology of Bayesian and multi-armed bandit,(MAB) approach to select the values with the consideration of the variable types; the real and integer variables are with Bayesian, while the boolean and categorical variables are with MAB. Empirical evaluations demonstrate that our method outperforms the existing methods across different tasks.
STSIR: Spatial Temporal Pandemic Model with Mobility Data
Venue: arXiv cs.CE 2020
Year: 2020

With the outbreak of COVID-19, how to mitigate and suppress its spread is a big issue to the government. Department of public health need powerful models to model and predict the trend and scale of such pandemic. And models that could evaluate the effect of the public policy are also essential to the fight with the COVID-19. A main limitation of existing models is that they can only evaluate the policy by calculating $R_0$ after infection happens instead of giving observable index. To tackle this, based on the transmission character of the COVID-19, we preposed a novel framework Spatial-Temporal-Susceptible-Infected-Removed (STSIR) model. In particular, we merged both intra-city and inter-city mobility index with the traditional SIR dynamics and make it a dynamic system. And we proved that the STSIR system is a closed system which makes the system self-consistent. And finally we proposed a Multi-Stage Simulated Annealing (MSSA) algorithm to find optimal parameter of the system. In our experiments, based on Baidu Mobility dataset, and China pandemic dataset provided by Dingxiangyuan, our model can effectively predict the total scale of the pandemic and also gives clear policy analysis with observable index.
Estimation of Gas Turbine Shaft Torque and Fuel Flow of a CODLAG Propulsion System Using Genetic Programming Algorithm
Venue: arXiv cs.AI 2020
Year: 2020

In this paper, the publicly available dataset of condition based maintenance of combined diesel-electric and gas (CODLAG) propulsion system for ships has been utilized to obtain symbolic expressions which could estimate gas turbine shaft torque and fuel flow using genetic programming (GP) algorithm. The entire dataset consists of 11934 samples that was divided into training and testing portions of dataset in an 80:20 ratio. The training dataset used to train the GP algorithm to obtain symbolic expressions for gas turbine shaft torque and fuel flow estimation consisted of 9548 samples. The best symbolic expressions obtained for gas turbine shaft torque and fuel flow estimation were obtained based on their $R^2$ score generated as a result of the application of the testing portion of the dataset on the aforementioned symbolic expressions. The testing portion of the dataset consisted of 2386 samples. The three best symbolic expressions obtained for gas turbine shaft torque estimation generated $R^2$ scores of 0.999201, 0.999296, and 0.999374, respectively. The three best symbolic expressions obtained for fuel flow estimation generated $R^2$ scores of 0.995495, 0.996465, and 0.996487, respectively.
Describing the syntax of programming languages using conjunctive and Boolean grammars
Venue: arXiv cs.FL 2020
Year: 2020

A classical result by Floyd ("On the non-existence of a phrase structure grammar for ALGOL 60", 1962) states that the complete syntax of any sensible programming language cannot be described by the ordinary kind of formal grammars (Chomsky's ``context-free''). This paper uses grammars extended with conjunction and negation operators, known as conjunctive grammars and Boolean grammars, to describe the set of well-formed programs in a simple typeless procedural programming language. A complete Boolean grammar, which defines such concepts as declaration of variables and functions before their use, is constructed and explained. Using the Generalized LR parsing algorithm for Boolean grammars, a program can then be parsed in time $O(n^4)$ in its length, while another known algorithm allows subcubic-time parsing. Next, it is shown how to transform this grammar to an unambiguous conjunctive grammar, with square-time parsing. This becomes apparently the first specification of the syntax of a programming language entirely by a computationally feasible formal grammar.
PPKE: Knowledge Representation Learning by Path-based Pre-training
Venue: arXiv cs.CL 2020
Year: 2020

Entities may have complex interactions in a knowledge graph (KG), such as multi-step relationships, which can be viewed as graph contextual information of the entities. Traditional knowledge representation learning (KRL) methods usually treat a single triple as a training unit, and neglect most of the graph contextual information exists in the topological structure of KGs. In this study, we propose a Path-based Pre-training model to learn Knowledge Embeddings, called PPKE, which aims to integrate more graph contextual information between entities into the KRL model. Experiments demonstrate that our model achieves state-of-the-art results on several benchmark datasets for link prediction and relation prediction tasks, indicating that our model provides a feasible way to take advantage of graph contextual information in KGs.
Speech Imagery Classification using Length-Wise Training based on Deep Learning
Venue: arXiv cs.HC 2020
Year: 2020

Brain-computer interface uses brain signals to control external devices without actual control behavior. Recently, speech imagery has been studied for direct communication using language. Speech imagery uses brain signals generated when the user imagines speech. Unlike motor imagery, speech imagery still has unknown characteristics. Additionally, electroencephalography has intricate and non-stationary properties resulting in insufficient decoding performance. In addition, speech imagery is difficult to utilize spatial features. In this study, we designed length-wise training that allows the model to classify a series of a small number of words. In addition, we proposed hierarchical convolutional neural network structure and loss function to maximize the training strategy. The proposed method showed competitive performance in speech imagery classification. Hence, we demonstrated that the length of the word is a clue at improving classification performance.
Generalised Perceptron Learning
Venue: arXiv cs.LG 2020
Year: 2020

We present a generalisation of Rosenblatt's traditional perceptron learning algorithm to the class of proximal activation functions and demonstrate how this generalisation can be interpreted as an incremental gradient method applied to a novel energy function. This novel energy function is based on a generalised Bregman distance, for which the gradient with respect to the weights and biases does not require the differentiation of the activation function. The interpretation as an energy minimisation algorithm paves the way for many new algorithms, of which we explore a novel variant of the iterative soft-thresholding algorithm for the learning of sparse perceptrons.
Human vs. supervised machine learning: Who learns patterns faster?
Venue: arXiv cs.AI 2020
Year: 2020

The capabilities of supervised machine learning (SML), especially compared to human abilities, are being discussed in scientific research and in the usage of SML. This study provides an answer to how learning performance differs between humans and machines when there is limited training data. We have designed an experiment in which 44 humans and three different machine learning algorithms identify patterns in labeled training data and have to label instances according to the patterns they find. The results show a high dependency between performance and the underlying patterns of the task. Whereas humans perform relatively similarly across all patterns, machines show large performance differences for the various patterns in our experiment. After seeing 20 instances in the experiment, human performance does not improve anymore, which we relate to theories of cognitive overload. Machines learn slower but can reach the same level or may even outperform humans in 2 of the 4 of used patterns. However, machines need more instances compared to humans for the same results. The performance of machines is comparably lower for the other 2 patterns due to the difficulty of combining input features.
Non-Repudiation for VoIP Communication in UMTS and LTE Networks
Venue: arXiv cs.CR 2020
Year: 2020

This thesis work presents an architectural design of a system to bring non-repudiation concept into the IP based digital voice conversations (VoIP) in LTE and UMTS networks, using electronic signatures, by considering a centralized approach. Moreover, functionalities and technical methods to support such a system are researched. Last but not least, ways to introduce this system as a public and commercial service are discussed. Non-repudiation concept provided by electronic signatures and related cryptographic functions, as introduced in this study, allow using digital records of these voice conversations as legally binding statements or proofs likewise and even instead of traditional wet signatures. The system is designed as a subsystem to IMS based 3G and 4G networks and maximum compatibility with current configurations, components and interfaces of these networks is intended. On the other hand non-repudiation is achieved by special signature, storage and verification units located in the IMS core network. Voice data is proposed to be processed in MRF unit of the IMS. Additionally, a USSD/USSI based special solution to initiate these signed calls is developed. According to the proposed scheme; during a signed call, two unidirectional voice streams originating from two parties of the call, which are transferred in IP and UDP encapsulated RTP packages, are received by the signature unit and interweaved using their arrival times, so that they become a unified stream. Signature unit generates hashes of groups of received packages and signs them using PKI algorithms and applying hash/signature chaining to increase integrity protection and to empower non-repudiation. Then, it forwards packages and signature information to the storage unit. Storage unit keeps all the call records, signature data and metadata of these calls. Verification unit later gathers relevant data from the storage unit...
G-RCN: Optimizing the Gap between Classification and Localization Tasks for Object Detection
Venue: arXiv cs.CV 2020
Year: 2020

Multi-task learning is widely used in computer vision. Currently, object detection models utilize shared feature map to complete classification and localization tasks simultaneously. By comparing the performance between the original Faster R-CNN and that with partially separated feature maps, we show that: (1) Sharing high-level features for the classification and localization tasks is sub-optimal; (2) Large stride is beneficial for classification but harmful for localization; (3) Global context information could improve the performance of classification. Based on these findings, we proposed a paradigm called Gap-optimized region based convolutional network (G-RCN), which aims to separating these two tasks and optimizing the gap between them. The paradigm was firstly applied to correct the current ResNet protocol by simply reducing the stride and moving the Conv5 block from the head to the feature extraction network, which brings 3.6 improvement of AP70 on the PASCAL VOC dataset and 1.5 improvement of AP on the COCO dataset for ResNet50. Next, the new method is applied on the Faster R-CNN with backbone of VGG16,ResNet50 and ResNet101, which brings above 2.0 improvement of AP70 on the PASCAL VOC dataset and above 1.9 improvement of AP on the COCO dataset. Noticeably, the implementation of G-RCN only involves a few structural modifications, with no extra module added.
Exploring the Usage of Online Food Delivery Data for Intra-Urban Job and Housing Mobility Detection and Characterization
Venue: arXiv cs.CY 2020
Year: 2020

Human mobility plays a critical role in urban planning and policy-making. However, at certain spatial and temporal resolutions, it is very challenging to track, for example, job and housing mobility. In this study, we explore the usage of a new modality of dataset, online food delivery data, to detect job and housing mobility. By leveraging millions of meal orders from a popular online food ordering and delivery service in Beijing, China, we are able to detect job and housing moves at much higher spatial and temporal resolutions than using traditional data sources. Popular moving seasons and origins/destinations can be well identified. More importantly, we match the detected moves to both macro- and micro-level factors so as to characterize job and housing dynamics. Our findings suggest that commuting distance is a major factor for job and housing mobility. We also observe that: (1) For home movers, there is a trade-off between lower housing cost and shorter commuting distance given the urban spatial structure; (2) For job hoppers, those who frequently work overtime are more likely to reduce their working hours by switching jobs. While this new modality of dataset has its limitations, we believe that ensemble approaches would be promising, where a mash-up of multiple datasets with different characteristic limitations can provide a more comprehensive picture of job and housing dynamics. Our work demonstrates the effectiveness of utilizing food delivery data to detect and analyze job and housing mobility, and contributes to realizing the full potential of ensemble-based approaches.
Using Side Channel Information and Artificial Intelligence for Malware Detection
Venue: arXiv cs.CR 2020
Year: 2020

Cybersecurity continues to be a difficult issue for society especially as the number of networked systems grows. Techniques to protect these systems range from rules-based to artificial intelligence-based intrusion detection systems and anti-virus tools. These systems rely upon the information contained in the network packets and download executables to function. Side channel information leaked from hardware has been shown to reveal secret information in systems such as encryption keys. This work demonstrates that side channel information can be used to detect malware running on a computing platform without access to the code involved.
Inter-layer Information Similarity Assessment of Deep Neural Networks Via Topological Similarity and Persistence Analysis of Data Neighbour Dynamics
Venue: arXiv cs.LG 2020
Year: 2020

The quantitative analysis of information structure through a deep neural network (DNN) can unveil new insights into the theoretical performance of DNN architectures. Two very promising avenues of research towards quantitative information structure analysis are: 1) layer similarity (LS) strategies focused on the inter-layer feature similarity, and 2) intrinsic dimensionality (ID) strategies focused on layer-wise data dimensionality using pairwise information. Inspired by both LS and ID strategies for quantitative information structure analysis, we introduce two novel complimentary methods for inter-layer information similarity assessment premised on the interesting idea of studying a data sample's neighbourhood dynamics as it traverses through a DNN. More specifically, we introduce the concept of Nearest Neighbour Topological Similarity (NNTS) for quantifying the information topology similarity between layers of a DNN. Furthermore, we introduce the concept of Nearest Neighbour Topological Persistence (NNTP) for quantifying the inter-layer persistence of data neighbourhood relationships throughout a DNN. The proposed strategies facilitate the efficient inter-layer information similarity assessment by leveraging only local topological information, and we demonstrate their efficacy in this study by performing analysis on a deep convolutional neural network architecture on image data to study the insights that can be gained with respect to the theoretical performance of a DNN.
The Lab vs The Crowd: An Investigation into Data Quality for Neural Dialogue Models
Venue: arXiv cs.CL 2020
Year: 2020

Challenges around collecting and processing quality data have hampered progress in data-driven dialogue models. Previous approaches are moving away from costly, resource-intensive lab settings, where collection is slow but where the data is deemed of high quality. The advent of crowd-sourcing platforms, such as Amazon Mechanical Turk, has provided researchers with an alternative cost-effective and rapid way to collect data. However, the collection of fluid, natural spoken or textual interaction can be challenging, particularly between two crowd-sourced workers. In this study, we compare the performance of dialogue models for the same interaction task but collected in two different settings: in the lab vs. crowd-sourced. We find that fewer lab dialogues are needed to reach similar accuracy, less than half the amount of lab data as crowd-sourced data. We discuss the advantages and disadvantages of each data collection method.