OpenRaiser/Intern-Atlas huggingface dataset:
GANtron: Emotional Speech Synthesis with Generative Adversarial Networks
Venue: arXiv cs.SD 2021
Year: 2021

Speech synthesis is used in a wide variety of industries. Nonetheless, it always sounds flat or robotic. The state of the art methods that allow for prosody control are very cumbersome to use and do not allow easy tuning. To tackle some of these drawbacks, in this work we target the implementation of a text-to-speech model where the inferred speech can be tuned with the desired emotions. To do so, we use Generative Adversarial Networks (GANs) together with a sequence-to-sequence model using an attention mechanism. We evaluate four different configurations considering different inputs and training strategies, study them and prove how our best model can generate speech files that lie in the same distribution as the initial training dataset. Additionally, a new strategy to boost the training convergence by applying a guided attention loss is proposed.
SERAB: A multi-lingual benchmark for speech emotion recognition
Venue: arXiv cs.SD 2021
Year: 2021

Recent developments in speech emotion recognition (SER) often leverage deep neural networks (DNNs). Comparing and benchmarking different DNN models can often be tedious due to the use of different datasets and evaluation protocols. To facilitate the process, here, we present the Speech Emotion Recognition Adaptation Benchmark (SERAB), a framework for evaluating the performance and generalization capacity of different approaches for utterance-level SER. The benchmark is composed of nine datasets for SER in six languages. Since the datasets have different sizes and numbers of emotional classes, the proposed setup is particularly suitable for estimating the generalization capacity of pre-trained DNN-based feature extractors. We used the proposed framework to evaluate a selection of standard hand-crafted feature sets and state-of-the-art DNN representations. The results highlight that using only a subset of the data included in SERAB can result in biased evaluation, while compliance with the proposed protocol can circumvent this issue.
PWG-IDS: An Intrusion Detection Model for Solving Class Imbalance in IIoT Networks Using Generative Adversarial Networks
Venue: arXiv cs.CR 2021
Year: 2021

With the continuous development of industrial IoT (IIoT) technology, network security is becoming more and more important. And intrusion detection is an important part of its security. However, since the amount of attack traffic is very small compared to normal traffic, this imbalance makes intrusion detection in it very difficult. To address this imbalance, an intrusion detection system called pretraining Wasserstein generative adversarial network intrusion detection system (PWG-IDS) is proposed in this paper. This system is divided into two main modules: 1) In this module, we introduce the pretraining mechanism in the Wasserstein generative adversarial network with gradient penalty (WGAN-GP) for the first time, firstly using the normal network traffic to train the WGAN-GP, and then inputting the imbalance data into the pre-trained WGAN-GP to retrain and generate the final required data. 2) Intrusion detection module: We use LightGBM as the classification algorithm to detect attack traffic in IIoT networks. The experimental results show that our proposed PWG-IDS outperforms other models, with F1-scores of 99% and 89% on the 2 datasets, respectively. And the pretraining mechanism we proposed can also be widely used in other GANs, providing a new way of thinking for the training of GANs.
A novel approach for MBSFN Area Formation aided by D2D Communications for eMBB Service Delivery in 5G NR Systems
Venue: arXiv cs.NI 2021
Year: 2021

Forthcoming 5G New Radio (NR) systems will be asked to handle a huge number of devices accessing or delivering "resource-hungry" and high-quality services. In view of this, the new 5G Radio Access Technology (RAT) aims to support, in next releases, Multimedia Broadcast/Multicast Service Single Frequency Network (MBSFN) to enable the simultaneous delivery of the same content to a set of users covered by different cells. According to MBSFN, all cells belonging to the same MBSFN Area are synchronized in time and the MBSFN transmission occurs over the same radio resources. In such a way, the same content flow is delivered by several cells to all the receivers in the MBSFN Area. A further means to enhance the network coverage and provide high data rate and low latency in future 5G-enabled MBSFN networks is Device-to-Device (D2D) connectivity. Along these lines, in this paper we propose a D2D-aided MBSFN Area Formation (D2D-MAF) algorithm to dynamically create MBSFN Areas with the aim to improve the system aggregate data rate while satisfying all user requests. The proposed D2D-MAF foresees that users could receive the service through either MBSFN, or D2D, or unicast transmissions. Performance evaluation results, carried out under a wide range of conditions, testify to the high effectiveness of the proposed algorithm.
Using Contrastive Learning and Pseudolabels to learn representations for Retail Product Image Classification
Venue: arXiv cs.CV 2021
Year: 2021

Retail product Image classification problems are often few shot classification problems, given retail product classes cannot have the type of variations across images like a cat or dog or tree could have. Previous works have shown different methods to finetune Convolutional Neural Networks to achieve better classification accuracy on such datasets. In this work, we try to address the problem statement : Can we pretrain a Convolutional Neural Network backbone which yields good enough representations for retail product images, so that training a simple logistic regression on these representations gives us good classifiers ? We use contrastive learning and pseudolabel based noisy student training to learn representations that get accuracy in order of finetuning the entire Convnet backbone for retail product image classification.
Joint inference of multiple graphs with hidden variables from stationary graph signals
Venue: arXiv cs.SI 2021
Year: 2021

Learning graphs from sets of nodal observations represents a prominent problem formally known as graph topology inference. However, current approaches are limited by typically focusing on inferring single networks, and they assume that observations from all nodes are available. First, many contemporary setups involve multiple related networks, and second, it is often the case that only a subset of nodes is observed while the rest remain hidden. Motivated by these facts, we introduce a joint graph topology inference method that models the influence of the hidden variables. Under the assumptions that the observed signals are stationary on the sought graphs and the graphs are closely related, the joint estimation of multiple networks allows us to exploit such relationships to improve the quality of the learned graphs. Moreover, we confront the challenging problem of modeling the influence of the hidden nodes to minimize their detrimental effect. To obtain an amenable approach, we take advantage of the particular structure of the setup at hand and leverage the similarity between the different graphs, which affects both the observed and the hidden nodes. To test the proposed method, numerical simulations over synthetic and real-world graphs are provided.
Adversarial Attack by Limited Point Cloud Surface Modifications
Venue: arXiv cs.CV 2021
Year: 2021

Recent research has revealed that the security of deep neural networks that directly process 3D point clouds to classify objects can be threatened by adversarial samples. Although existing adversarial attack methods achieve high success rates, they do not restrict the point modifications enough to preserve the point cloud appearance. To overcome this shortcoming, two constraints are proposed. These include applying hard boundary constraints on the number of modified points and on the point perturbation norms. Due to the restrictive nature of the problem, the search space contains many local maxima. The proposed method addresses this issue by using a high step-size at the beginning of the algorithm to search the main surface of the point cloud fast and effectively. Then, in order to converge to the desired output, the step-size is gradually decreased. To evaluate the performance of the proposed method, it is run on the ModelNet40 and ScanObjectNN datasets by employing the state-of-the-art point cloud classification models; including PointNet, PointNet++, and DGCNN. The obtained results show that it can perform successful attacks and achieve state-of-the-art results by only a limited number of point modifications while preserving the appearance of the point cloud. Moreover, due to the effective search algorithm, it can perform successful attacks in just a few steps. Additionally, the proposed step-size scheduling algorithm shows an improvement of up to $14.5\%$ when adopted by other methods as well. The proposed method also performs effectively against popular defense methods.
Efficient large-scale image retrieval with deep feature orthogonality and Hybrid-Swin-Transformers
Venue: arXiv cs.CV 2021
Year: 2021

We present an efficient end-to-end pipeline for largescale landmark recognition and retrieval. We show how to combine and enhance concepts from recent research in image retrieval and introduce two architectures especially suited for large-scale landmark identification. A model with deep orthogonal fusion of local and global features (DOLG) using an EfficientNet backbone as well as a novel Hybrid-Swin-Transformer is discussed and details how to train both architectures efficiently using a step-wise approach and a sub-center arcface loss with dynamic margins are provided. Furthermore, we elaborate a novel discriminative re-ranking methodology for image retrieval. The superiority of our approach was demonstrated by winning the recognition and retrieval track of the Google Landmark Competition 2021.
Toward a Theory of Programming Language and Reasoning Assistant Design: Minimizing Cognitive Load
Venue: arXiv cs.PL 2021
Year: 2021

Current approaches to making programming languages and reasoning assistants more effective for people focus on leveraging feedback from users and on evaluating the success of particular techniques. These approaches, although helpful, may not result in systems that are as usable as possible, and may not lead to general design principles. This paper advocates for leveraging theories from cognitive science, focusing on cognitive load theory, to design more effective programming languages and reasoning assistants. Development of these theories may enable designers to create more effective programming languages and reasoning assistants at lower cost.
Dyn-Backdoor: Backdoor Attack on Dynamic Link Prediction
Venue: arXiv cs.AI 2021
Year: 2021

Dynamic link prediction (DLP) makes graph prediction based on historical information. Since most DLP methods are highly dependent on the training data to achieve satisfying prediction performance, the quality of the training data is crucial. Backdoor attacks induce the DLP methods to make wrong prediction by the malicious training data, i.e., generating a subgraph sequence as the trigger and embedding it to the training data. However, the vulnerability of DLP toward backdoor attacks has not been studied yet. To address the issue, we propose a novel backdoor attack framework on DLP, denoted as Dyn-Backdoor. Specifically, Dyn-Backdoor generates diverse initial-triggers by a generative adversarial network (GAN). Then partial links of the initial-triggers are selected to form a trigger set, according to the gradient information of the attack discriminator in the GAN, so as to reduce the size of triggers and improve the concealment of the attack. Experimental results show that Dyn-Backdoor launches successful backdoor attacks on the state-of-the-art DLP models with success rate more than 90%. Additionally, we conduct a possible defense against Dyn-Backdoor to testify its resistance in defensive settings, highlighting the needs of defenses for backdoor attacks on DLP.
Ranking Cost: Building An Efficient and Scalable Circuit Routing Planner with Evolution-Based Optimization
Venue: arXiv cs.AI 2021
Year: 2021

Circuit routing has been a historically challenging problem in designing electronic systems such as very large-scale integration (VLSI) and printed circuit boards (PCBs). The main challenge is that connecting a large number of electronic components under specific design rules involves a very large search space. Early solutions are typically designed with hard-coded heuristics, which suffer from problems of non-optimal solutions and lack of flexibility for new design needs. Although a few learning-based methods have been proposed recently, they are typically cumbersome and hard to extend to large-scale applications. In this work, we propose a new algorithm for circuit routing, named Ranking Cost, which innovatively combines search-based methods (i.e., A* algorithm) and learning-based methods (i.e., Evolution Strategies) to form an efficient and trainable router. In our method, we introduce a new set of variables called cost maps, which can help the A* router to find out proper paths to achieve the global objective. We also train a ranking parameter, which can produce the ranking order and further improve the performance of our method. Our algorithm is trained in an end-to-end manner and does not use any artificial data or human demonstration. In the experiments, we compare with the sequential A* algorithm and a canonical reinforcement learning approach, and results show that our method outperforms these baselines with higher connectivity rates and better scalability.
Anomaly Detection in Beehives: An Algorithm Comparison
Venue: arXiv cs.LG 2021
Year: 2021

Sensor-equipped beehives allow monitoring the living conditions of bees. Machine learning models can use the data of such hives to learn behavioral patterns and find anomalous events. One type of event that is of particular interest to apiarists for economical reasons is bee swarming. Other events of interest are behavioral anomalies from illness and technical anomalies, e.g. sensor failure. Beekeepers can be supported by suitable machine learning models which can detect these events. In this paper we compare multiple machine learning models for anomaly detection and evaluate them for their applicability in the context of beehives. Namely we employed Deep Recurrent Autoencoder, Elliptic Envelope, Isolation Forest, Local Outlier Factor and One-Class SVM. Through evaluation with real world datasets of different hives and with different sensor setups we find that the autoencoder is the best multi-purpose anomaly detector in comparison.
A Framework for Aspectual Requirements Validation: An Experimental Study
Venue: arXiv cs.SE 2021
Year: 2021

Requirements engineering is a discipline of software engineering that is concerned with the identification and handling of user and system requirements. Aspect-Oriented Requirements Engineering (AORE) extends the existing requirements engineering approaches to cope with the issue of tangling and scattering resulted from crosscutting concerns. Crosscutting concerns are considered as potential aspects and can lead to the phenomena tyranny of the dominant decomposition. Requirements-level aspects are responsible for producing scattered and tangled descriptions of requirements in the requirements document. Validation of requirements artefacts is an essential task in software development. This task ensures that requirements are correct and valid in terms of completeness and consistency, hence, reducing the development cost, maintenance and establish an approximately correct estimate of effort and completion time of the project. In this paper, we present a validation framework to validate the aspectual requirements and the crosscutting relationship of concerns that are resulted from the requirements engineering phase. The proposed framework comprises a high-level and low-level validation to implement on software requirements specification (SRS). The high-level validation validates the concerns with stakeholders, whereas the low-level validation validates the aspectual requirement by requirements engineers and analysts using a checklist. The approach has been evaluated using an experimental study on two AORE approaches. The approaches are viewpoint-based called AORE with ArCaDe and lexical analysis based on Theme/Doc approach. The results obtained from the study demonstrate that the proposed framework is an effective validation model for AORE artefacts.
Pathologies in priors and inference for Bayesian transformers
Venue: arXiv cs.LG 2021
Year: 2021

In recent years, the transformer has established itself as a workhorse in many applications ranging from natural language processing to reinforcement learning. Similarly, Bayesian deep learning has become the gold-standard for uncertainty estimation in safety-critical applications, where robustness and calibration are crucial. Surprisingly, no successful attempts to improve transformer models in terms of predictive uncertainty using Bayesian inference exist. In this work, we study this curiously underpopulated area of Bayesian transformers. We find that weight-space inference in transformers does not work well, regardless of the approximate posterior. We also find that the prior is at least partially at fault, but that it is very hard to find well-specified weight priors for these models. We hypothesize that these problems stem from the complexity of obtaining a meaningful mapping from weight-space to function-space distributions in the transformer. Therefore, moving closer to function-space, we propose a novel method based on the implicit reparameterization of the Dirichlet distribution to apply variational inference directly to the attention weights. We find that this proposed method performs competitively with our baselines.
Context-LGM: Leveraging Object-Context Relation for Context-Aware Object Recognition
Venue: arXiv cs.CV 2021
Year: 2021

Context, as referred to situational factors related to the object of interest, can help infer the object's states or properties in visual recognition. As such contextual features are too diverse (across instances) to be annotated, existing attempts simply exploit image labels as supervision to learn them, resulting in various contextual tricks, such as features pyramid, context attention, etc. However, without carefully modeling the context's properties, especially its relation to the object, their estimated context can suffer from large inaccuracy. To amend this problem, we propose a novel Contextual Latent Generative Model (Context-LGM), which considers the object-context relation and models it in a hierarchical manner. Specifically, we firstly introduce a latent generative model with a pair of correlated latent variables to respectively model the object and context, and embed their correlation via the generative process. Then, to infer contextual features, we reformulate the objective function of Variational Auto-Encoder (VAE), where contextual features are learned as a posterior distribution conditioned on the object. Finally, to implement this contextual posterior, we introduce a Transformer that takes the object's information as a reference and locates correlated contextual factors. The effectiveness of our method is verified by state-of-the-art performance on two context-aware object recognition tasks, i.e. lung cancer prediction and emotion recognition.
Test-time Batch Statistics Calibration for Covariate Shift
Venue: arXiv cs.CV 2021
Year: 2021

Deep neural networks have a clear degradation when applying to the unseen environment due to the covariate shift. Conventional approaches like domain adaptation requires the pre-collected target data for iterative training, which is impractical in real-world applications. In this paper, we propose to adapt the deep models to the novel environment during inference. An previous solution is test time normalization, which substitutes the source statistics in BN layers with the target batch statistics. However, we show that test time normalization may potentially deteriorate the discriminative structures due to the mismatch between target batch statistics and source parameters. To this end, we present a general formulation $α$-BN to calibrate the batch statistics by mixing up the source and target statistics for both alleviating the domain shift and preserving the discriminative structures. Based on $α$-BN, we further present a novel loss function to form a unified test time adaptation framework Core, which performs the pairwise class correlation online optimization. Extensive experiments show that our approaches achieve the state-of-the-art performance on total twelve datasets from three topics, including model robustness to corruptions, domain generalization on image classification and semantic segmentation. Particularly, our $α$-BN improves 28.4\% to 43.9\% on GTA5 $\rightarrow$ Cityscapes without any training, even outperforms the latest source-free domain adaptation method.
MToFNet: Object Anti-Spoofing with Mobile Time-of-Flight Data
Venue: arXiv cs.CV 2021
Year: 2021

In online markets, sellers can maliciously recapture others' images on display screens to utilize as spoof images, which can be challenging to distinguish in human eyes. To prevent such harm, we propose an anti-spoofing method using the paired rgb images and depth maps provided by the mobile camera with a Time-of-Fight sensor. When images are recaptured on display screens, various patterns differing by the screens as known as the moiré patterns can be also captured in spoof images. These patterns lead the anti-spoofing model to be overfitted and unable to detect spoof images recaptured on unseen media. To avoid the issue, we build a novel representation model composed of two embedding models, which can be trained without considering the recaptured images. Also, we newly introduce mToF dataset, the largest and most diverse object anti-spoofing dataset, and the first to utilize ToF data. Experimental results confirm that our model achieves robust generalization even across unseen domains.
Landslide Detection in Real-Time Social Media Image Streams
Venue: arXiv cs.CV 2021
Year: 2021

Lack of global data inventories obstructs scientific modeling of and response to landslide hazards which are oftentimes deadly and costly. To remedy this limitation, new approaches suggest solutions based on citizen science that requires active participation. However, as a non-traditional data source, social media has been increasingly used in many disaster response and management studies in recent years. Inspired by this trend, we propose to capitalize on social media data to mine landslide-related information automatically with the help of artificial intelligence (AI) techniques. Specifically, we develop a state-of-the-art computer vision model to detect landslides in social media image streams in real time. To that end, we create a large landslide image dataset labeled by experts and conduct extensive model training experiments. The experimental results indicate that the proposed model can be deployed in an online fashion to support global landslide susceptibility maps and emergency response.