OpenRaiser/Intern-Atlas huggingface dataset:
Parallel Algorithms for Densest Subgraph Discovery Using Shared Memory Model
Venue: arXiv cs.IR 2021
Year: 2021

The problem of finding dense components of a graph is a widely explored area in data analysis, with diverse applications in fields and branches of study including community mining, spam detection, computer security and bioinformatics. This research project explores previously available algorithms in order to study them and identify potential modifications that could result in an improved version with considerable performance and efficiency leap. Furthermore, efforts were also steered towards devising a novel algorithm for the problem of densest subgraph discovery. This paper presents an improved implementation of a widely used densest subgraph discovery algorithm and a novel parallel algorithm which produces better results than a 2-approximation.
Burst-Error Propagation Suppression for Decision-Feedback Equalizer in Field-Trial Submarine Fiber-Optic Communications
Venue: arXiv cs.IT 2021
Year: 2021

In this paper, we present a field-trial C-band 72Gbit/s optical on-off keying (OOK) system over 18.8km dispersion-uncompensated submarine optical cable in the South China Sea. Chromatic dispersion (CD) of 18.8km submarine optical cable causes four spectral nulls on the 36GHz bandwidth of 72Gbit/s OOK signal, which is the main obstacle for achieving an acceptable bit-error-rate (BER) performance. Decision feedback equalizer (DFE) is effective to compensate for the spectral nulls. However, DFE has a serious defect of burst-error propagation when the burst errors emerge due to the unstable submarine environment. Weighted DFE (WDFE) can be used to mitigate the burst-error propagation, but it cannot fully compensate for the spectral nulls because only a part of feedback symbols is directly decided. Fortunately, maximum likelihood sequence estimation (MLSE) can be added after the WDFE to simultaneously eliminate the resisting spectral distortions and implement optimal detection. Compared to the joint DFE and MLSE algorithm, the joint WDFE and MLSE algorithm can effectively suppress the burst-error propagation to obtain a maximum 2.9dB improvement of $\boldsymbol{Q}$ factor and eliminate the phenomenon of BER floor. In conclusion, the joint WDFE and MLSE algorithm can solve the burst-error propagation for the field-trial fiber-optic communications.
COVID-19 Tweets Analysis through Transformer Language Models
Venue: arXiv cs.CL 2021
Year: 2021

Understanding the public sentiment and perception in a healthcare crisis is essential for developing appropriate crisis management techniques. While some studies have used Twitter data for predictive modelling during COVID-19, fine-grained sentiment analysis of the opinion of people on social media during this pandemic has not yet been done. In this study, we perform an in-depth, fine-grained sentiment analysis of tweets in COVID-19. For this purpose, we perform supervised training of four transformer language models on the downstream task of multi-label classification of tweets into seven tone classes: [confident, anger, fear, joy, sadness, analytical, tentative]. We achieve a LRAP (Label Ranking Average Precision) score of 0.9267 through RoBERTa. This trained transformer model is able to correctly predict, with high accuracy, the tone of a tweet. We then leverage this model for predicting tones for 200,000 tweets on COVID-19. We then perform a country-wise analysis of the tone of tweets, and extract useful indicators of the psychological condition about the people in this pandemic.
Remote Sensing Image Change Detection with Transformers
Venue: arXiv cs.CV 2021
Year: 2021

Modern change detection (CD) has achieved remarkable success by the powerful discriminative ability of deep convolutions. However, high-resolution remote sensing CD remains challenging due to the complexity of objects in the scene. Objects with the same semantic concept may show distinct spectral characteristics at different times and spatial locations. Most recent CD pipelines using pure convolutions are still struggling to relate long-range concepts in space-time. Non-local self-attention approaches show promising performance via modeling dense relations among pixels, yet are computationally inefficient. Here, we propose a bitemporal image transformer (BIT) to efficiently and effectively model contexts within the spatial-temporal domain. Our intuition is that the high-level concepts of the change of interest can be represented by a few visual words, i.e., semantic tokens. To achieve this, we express the bitemporal image into a few tokens, and use a transformer encoder to model contexts in the compact token-based space-time. The learned context-rich tokens are then feedback to the pixel-space for refining the original features via a transformer decoder. We incorporate BIT in a deep feature differencing-based CD framework. Extensive experiments on three CD datasets demonstrate the effectiveness and efficiency of the proposed method. Notably, our BIT-based model significantly outperforms the purely convolutional baseline using only 3 times lower computational costs and model parameters. Based on a naive backbone (ResNet18) without sophisticated structures (e.g., FPN, UNet), our model surpasses several state-of-the-art CD methods, including better than four recent attention-based methods in terms of efficiency and accuracy. Our code is available at https://github.com/justchenhao/BIT\_CD.
Generative Chemical Transformer: Neural Machine Learning of Molecular Geometric Structures from Chemical Language via Attention
Venue: arXiv cs.LG 2021
Year: 2021

Discovering new materials better suited to specific purposes is an important issue in improving the quality of human life. Here, a neural network that creates molecules that meet some desired conditions based on a deep understanding of chemical language is proposed (Generative Chemical Transformer, GCT). The attention mechanism in GCT allows a deeper understanding of molecular structures beyond the limitations of chemical language itself which cause semantic discontinuity by paying attention to characters sparsely. It is investigated that the significance of language models for inverse molecular design problems by quantitatively evaluating the quality of the generated molecules. GCT generates highly realistic chemical strings that satisfy both chemical and linguistic grammar rules. Molecules parsed from generated strings simultaneously satisfy the multiple target properties and vary for a single condition set. These advances will contribute to improving the quality of human life by accelerating the process of desired material discovery.
ProbLP: A framework for low-precision probabilistic inference
Venue: arXiv cs.AR 2021
Year: 2021

Bayesian reasoning is a powerful mechanism for probabilistic inference in smart edge-devices. During such inferences, a low-precision arithmetic representation can enable improved energy efficiency. However, its impact on inference accuracy is not yet understood. Furthermore, general-purpose hardware does not natively support low-precision representation. To address this, we propose ProbLP, a framework that automates the analysis and design of low-precision probabilistic inference hardware. It automatically chooses an appropriate energy-efficient representation based on worst-case error-bounds and hardware energy-models. It generates custom hardware for the resulting inference network exploiting parallelism, pipelining and low-precision operation. The framework is validated on several embedded-sensing benchmarks.
RA-GCN: Graph Convolutional Network for Disease Prediction Problems with Imbalanced Data
Venue: arXiv cs.LG 2021
Year: 2021

Disease prediction is a well-known classification problem in medical applications. GCNs provide a powerful tool for analyzing the patients' features relative to each other. This can be achieved by modeling the problem as a graph node classification task, where each node is a patient. Due to the nature of such medical datasets, class imbalance is a prevalent issue in the field of disease prediction, where the distribution of classes is skewed. When the class imbalance is present in the data, the existing graph-based classifiers tend to be biased towards the major class(es) and neglect the samples in the minor class(es). On the other hand, the correct diagnosis of the rare positive cases among all the patients is vital in a healthcare system. In conventional methods, such imbalance is tackled by assigning appropriate weights to classes in the loss function which is still dependent on the relative values of weights, sensitive to outliers, and in some cases biased towards the minor class(es). In this paper, we propose a Re-weighted Adversarial Graph Convolutional Network (RA-GCN) to prevent the graph-based classifier from emphasizing the samples of any particular class. This is accomplished by associating a graph-based neural network to each class, which is responsible for weighting the class samples and changing the importance of each sample for the classifier. Therefore, the classifier adjusts itself and determines the boundary between classes with more attention to the important samples. The parameters of the classifier and weighting networks are trained by an adversarial approach. We show experiments on synthetic and three publicly available medical datasets. RA-GCN demonstrates the superiority compared to recent methods in identifying the patient's status on all three datasets. The detailed analysis is provided as quantitative and qualitative experiments on synthetic datasets.
Towards Intelligent RAN Slicing for B5G: Opportunities and Challenges
Venue: arXiv cs.NI 2021
Year: 2021

To meet the diverse demands for wireless communication, fifth-generation (5G) networks and beyond (B5G) embrace the concept of network slicing by forging virtual instances (slices) of its physical infrastructure. While network slicing constitutes dynamic allocation of core network and radio access network (RAN) resources, this article emphasizes RAN slicing (RAN-S) design. Forming on-demand RAN-S that can be flexibly (re)-configured while ensuring slice isolation is challenging. A variety of machine learning (ML) techniques have been recently introduced for traffic forecasting and classification, resource usage prediction, admission control, scheduling, and dynamic resource allocation in RAN-S. Albeit these approaches grant opportunities towards intelligent RAN-S design, they raise critical challenges that need to be examined. This article underlines the opportunities and the challenges of incorporating ML into RAN-S by reviewing the cutting-edge ML-based techniques for RAN-S. It also draws few directions for future research towards intelligent RAN-S (iRAN-S).
Color-Coded Symbology and New Computer Vision Tool to Predict the Historical Color Pallets of the Renaissance Oil Artworks
Venue: arXiv cs.CV 2021
Year: 2021

In this paper, we discuss possible color palletes, prediction and analysis of originality of the colors that Artists used on the Renaissance oil paintings. This framework goal is to help to use the color symbology and image enhancement tools, to predict the historical color palletes of the Renaissance oil artworks. This work is only the start of a development to explore the possibilities of prediction of color palletes of the Renaissance oil artworks. We believe that framework might be very useful in the prediction of color palletes of the Renaissance oil artworks and other artworks. The images in number 105 have been taken from the paintings of three well-known artists, Rafael, Leonardo Da Vinci, and Rembrandt that are available in the Olga's Gallery. Images are processed in the frequency domain to enhance a quality of images and ratios of primary colors are calculated and analyzed by using new measurements of color-ratios.
Effective Universal Unrestricted Adversarial Attacks using a MOE Approach
Venue: arXiv cs.LG 2021
Year: 2021

Recent studies have shown that Deep Leaning models are susceptible to adversarial examples, which are data, in general images, intentionally modified to fool a machine learning classifier. In this paper, we present a multi-objective nested evolutionary algorithm to generate universal unrestricted adversarial examples in a black-box scenario. The unrestricted attacks are performed through the application of well-known image filters that are available in several image processing libraries, modern cameras, and mobile applications. The multi-objective optimization takes into account not only the attack success rate but also the detection rate. Experimental results showed that this approach is able to create a sequence of filters capable of generating very effective and undetectable attacks.
A Brief Introduction to Generative Models
Venue: arXiv cs.LG 2021
Year: 2021

We introduce and motivate generative modeling as a central task for machine learning and provide a critical view of the algorithms which have been proposed for solving this task. We overview how generative modeling can be defined mathematically as trying to make an estimating distribution the same as an unknown ground truth distribution. This can then be quantified in terms of the value of a statistical divergence between the two distributions. We outline the maximum likelihood approach and how it can be interpreted as minimizing KL-divergence. We explore a number of approaches in the maximum likelihood family, while discussing their limitations. Finally, we explore the alternative adversarial approach which involves studying the differences between an estimating distribution and a real data distribution. We discuss how this approach can give rise to new divergences and methods that are necessary to make adversarial learning successful. We also discuss new evaluation metrics which are required by the adversarial approach.
BiconNet: An Edge-preserved Connectivity-based Approach for Salient Object Detection
Venue: arXiv cs.CV 2021
Year: 2021

Salient object detection (SOD) is viewed as a pixel-wise saliency modeling task by traditional deep learning-based methods. A limitation of current SOD models is insufficient utilization of inter-pixel information, which usually results in imperfect segmentation near edge regions and low spatial coherence. As we demonstrate, using a saliency mask as the only label is suboptimal. To address this limitation, we propose a connectivity-based approach called bilateral connectivity network (BiconNet), which uses connectivity masks together with saliency masks as labels for effective modeling of inter-pixel relationships and object saliency. Moreover, we propose a bilateral voting module to enhance the output connectivity map, and a novel edge feature enhancement method that efficiently utilizes edge-specific features. Through comprehensive experiments on five benchmark datasets, we demonstrate that our proposed method can be plugged into any existing state-of-the-art saliency-based SOD framework to improve its performance with negligible parameter increase.
Constrained Differentially Private Federated Learning for Low-bandwidth Devices
Venue: arXiv cs.CR 2021
Year: 2021

Federated learning becomes a prominent approach when different entities want to learn collaboratively a common model without sharing their training data. However, Federated learning has two main drawbacks. First, it is quite bandwidth inefficient as it involves a lot of message exchanges between the aggregating server and the participating entities. This bandwidth and corresponding processing costs could be prohibitive if the participating entities are, for example, mobile devices. Furthermore, although federated learning improves privacy by not sharing data, recent attacks have shown that it still leaks information about the training data. This paper presents a novel privacy-preserving federated learning scheme. The proposed scheme provides theoretical privacy guarantees, as it is based on Differential Privacy. Furthermore, it optimizes the model accuracy by constraining the model learning phase on few selected weights. Finally, as shown experimentally, it reduces the upstream and downstream bandwidth by up to 99.9% compared to standard federated learning, making it practical for mobile systems.
SUM: A Benchmark Dataset of Semantic Urban Meshes
Venue: arXiv cs.CV 2021
Year: 2021

Recent developments in data acquisition technology allow us to collect 3D texture meshes quickly. Those can help us understand and analyse the urban environment, and as a consequence are useful for several applications like spatial analysis and urban planning. Semantic segmentation of texture meshes through deep learning methods can enhance this understanding, but it requires a lot of labelled data. The contributions of this work are threefold: (1) a new benchmark dataset of semantic urban meshes, (2) a novel semi-automatic annotation framework, and (3) an annotation tool for 3D meshes. In particular, our dataset covers about 4 km2 in Helsinki (Finland), with six classes, and we estimate that we save about 600 hours of labelling work using our annotation framework, which includes initial segmentation and interactive refinement. We also compare the performance of several state-of-theart 3D semantic segmentation methods on the new benchmark dataset. Other researchers can use our results to train their networks: the dataset is publicly available, and the annotation tool is released as open-source.
They'll Know It When They See It: Analyzing Post-Release Feedback from the Android Community
Venue: arXiv cs.SE 2021
Year: 2021

It is known that user involvement and user-centered design enhance system acceptance, particularly when end-users' views are considered early in the process. However, the increasingly common method of system deployment, through frequent releases via an online application distribution platform, relies more on post-release feedback from a virtual community. Such feedback may be received from large and diverse communities of users, posing challenges to developers in terms of extracting and identifying the most pressing requests to address. In seeking to tackle these challenges we have used natural language processing techniques to study enhancement requests logged by the Android community. We observe that features associated with a specific subset of topics were most frequently requested for improvement, and that end-users expressed particular discontent with the Jellybean release. End-users also tended to request improvements to specific issues together, potentially posing a prioritization challenge to Google.
SpikeDyn: A Framework for Energy-Efficient Spiking Neural Networks with Continual and Unsupervised Learning Capabilities in Dynamic Environments
Venue: arXiv cs.NE 2021
Year: 2021

Spiking Neural Networks (SNNs) bear the potential of efficient unsupervised and continual learning capabilities because of their biological plausibility, but their complexity still poses a serious research challenge to enable their energy-efficient design for resource-constrained scenarios (like embedded systems, IoT-Edge, etc.). We propose SpikeDyn, a comprehensive framework for energy-efficient SNNs with continual and unsupervised learning capabilities in dynamic environments, for both the training and inference phases. It is achieved through the following multiple diverse mechanisms: 1) reduction of neuronal operations, by replacing the inhibitory neurons with direct lateral inhibitions; 2) a memory- and energy-constrained SNN model search algorithm that employs analytical models to estimate the memory footprint and energy consumption of different candidate SNN models and selects a Pareto-optimal SNN model; and 3) a lightweight continual and unsupervised learning algorithm that employs adaptive learning rates, adaptive membrane threshold potential, weight decay, and reduction of spurious updates. Our experimental results show that, for a network with 400 excitatory neurons, our SpikeDyn reduces the energy consumption on average by 51% for training and by 37% for inference, as compared to the state-of-the-art. Due to the improved learning algorithm, SpikeDyn provides on avg. 21% accuracy improvement over the state-of-the-art, for classifying the most recently learned task, and by 8% on average for the previously learned tasks.
EKMP: Generalized Imitation Learning with Adaptation, Nonlinear Hard Constraints and Obstacle Avoidance
Venue: arXiv cs.RO 2021
Year: 2021

As a user-friendly and straightforward solution for robot trajectory generation, imitation learning has been viewed as a vital direction in the context of robot skill learning. In contrast to unconstrained imitation learning which ignores possible internal and external constraints arising from environments and robot kinematics/dynamics, recent works on constrained imitation learning allow for transferring human skills to unstructured scenarios, further enlarging the application domain of imitation learning. While various constraints have been studied, e.g., joint limits, obstacle avoidance and plane constraints, the problem of nonlinear hard constraints has not been well-addressed. In this paper, we propose extended kernelized movement primitives (EKMP) to cope with most of the key problems in imitation learning, including nonlinear hard constraints. Specifically, EKMP is capable of learning the probabilistic features of multiple demonstrations, adapting the learned skills towards arbitrary desired points in terms of joint position and velocity, avoiding obstacles at the level of robot links, as well as satisfying arbitrary linear and nonlinear, equality and inequality hard constraints. Besides, the connections between EKMP and state-of-the-art motion planning approaches are discussed. Several evaluations including the planning of joint trajectories for a 7-DoF robotic arm are provided to verify the effectiveness of our framework.
Criminal Networks Analysis in Missing Data scenarios through Graph Distances
Venue: arXiv cs.SI 2021
Year: 2021

Data collected in criminal investigations may suffer from: (i) incompleteness, due to the covert nature of criminal organisations; (ii) incorrectness, caused by either unintentional data collection errors and intentional deception by criminals; (iii) inconsistency, when the same information is collected into law enforcement databases multiple times, or in different formats. In this paper we analyse nine real criminal networks of different nature (i.e., Mafia networks, criminal street gangs and terrorist organizations) in order to quantify the impact of incomplete data and to determine which network type is most affected by it. The networks are firstly pruned following two specific methods: (i) random edges removal, simulating the scenario in which the Law Enforcement Agencies (LEAs) fail to intercept some calls, or to spot sporadic meetings among suspects; (ii) nodes removal, that catches the hypothesis in which some suspects cannot be intercepted or investigated. Finally we compute spectral (i.e., Adjacency, Laplacian and Normalised Laplacian Spectral Distances) and matrix (i.e., Root Euclidean Distance) distances between the complete and pruned networks, which we compare using statistical analysis. Our investigation identified two main features: first, the overall understanding of the criminal networks remains high even with incomplete data on criminal interactions (i.e., 10% removed edges); second, removing even a small fraction of suspects not investigated (i.e., 2% removed nodes) may lead to significant misinterpretation of the overall network.