OpenRaiser/Intern-Atlas huggingface dataset:
On the Transferability of Adversarial Attacksagainst Neural Text Classifier
Venue: arXiv cs.LG 2021
Year: 2021

Deep neural networks are vulnerable to adversarial attacks, where a small perturbation to an input alters the model prediction. In many cases, malicious inputs intentionally crafted for one model can fool another model. In this paper, we present the first study to systematically investigate the transferability of adversarial examples for text classification models and explore how various factors, including network architecture, tokenization scheme, word embedding, and model capacity, affect the transferability of adversarial examples. Based on these studies, we propose a genetic algorithm to find an ensemble of models that can be used to induce adversarial examples to fool almost all existing models. Such adversarial examples reflect the defects of the learning process and the data bias in the training set. Finally, we derive word replacement rules that can be used for model diagnostics from these adversarial examples.
Control Strategies for Autonomous Vehicles
Venue: arXiv cs.RO 2021
Year: 2021

This chapter focuses on the self-driving technology from a control perspective and investigates the control strategies used in autonomous vehicles and advanced driver-assistance systems from both theoretical and practical viewpoints. First, we introduce the self-driving technology as a whole, including perception, planning and control techniques required for accomplishing the challenging task of autonomous driving. We then dwell upon each of these operations to explain their role in the autonomous system architecture, with a prime focus on control strategies. The core portion of this chapter commences with detailed mathematical modeling of autonomous vehicles followed by a comprehensive discussion on control strategies. The chapter covers longitudinal as well as lateral control strategies for autonomous vehicles with coupled and de-coupled control schemes. We as well discuss some of the machine learning techniques applied to autonomous vehicle control task. Finally, we briefly summarize some of the research works that our team has carried out at the Autonomous Systems Lab and conclude the chapter with a few thoughtful remarks.
Analyzing and Mitigating JPEG Compression Defects in Deep Learning
Venue: arXiv cs.CV 2021
Year: 2021

With the proliferation of deep learning methods, many computer vision problems which were considered academic are now viable in the consumer setting. One drawback of consumer applications is lossy compression, which is necessary from an engineering standpoint to efficiently and cheaply store and transmit user images. Despite this, there has been little study of the effect of compression on deep neural networks and benchmark datasets are often losslessly compressed or compressed at high quality. Here we present a unified study of the effects of JPEG compression on a range of common tasks and datasets. We show that there is a significant penalty on common performance metrics for high compression. We test several methods for mitigating this penalty, including a novel method based on artifact correction which requires no labels to train.
A Unified Seeding Framework
Venue: arXiv cs.SI 2021
Year: 2021

Online social networks have become a crucial medium to disseminate the latest political, commercial, and social information. Users with high visibility are often selected as seeds to spread information and affect their adoption in target groups. We study how gender differences and similarities can impact the information spreading process. Using a large-scale Instagram dataset and a small-scale Facebook dataset, we first conduct a multi-faceted analysis taking the interaction type, directionality and frequency into account. To this end, we explore a variety of existing and new single and multihop centrality measures. Our analysis unveils that males and females interact differently depending on the interaction types, e.g., likes or comments, and they feature different support and promotion patterns. We complement prior work showing that females do not reach top visibility (often referred to as the glass ceiling effect) jointly factoring in the connectivity and interaction intensity, both of which were previously mainly discussed independently. Inspired by these observations, we propose a novel seeding framework, called Disparity Seeding, which aims at maximizing spread while reaching a target user group, e.g., a certain percentage of females -- promoting the influence of under-represented groups. Disparity Seeding ranks influential users with two gender-aware measures, the Target HI-index and the Embedding index. Extensive simulations comparing Disparity Seeding with target-agnostic algorithms show that Disparity Seeding meets the target percentage while effectively maximizing the spread. Disparity Seeding can be generalized to counter different types of inequality, e.g., race, and proactively promote minorities in the society.
Visual Drift Detection for Sequence Data Analysis of Business Processes
Venue: arXiv cs.HC 2021
Year: 2021

Event sequence data is increasingly available in various application domains, such as business process management, software engineering, or medical pathways. Processes in these domains are typically represented as process diagrams or flow charts. So far, various techniques have been developed for automatically generating such diagrams from event sequence data. An open challenge is the visual analysis of drift phenomena when processes change over time. In this paper, we address this research gap. Our contribution is a system for fine-granular process drift detection and corresponding visualizations for event logs of executed business processes. We evaluated our system both on synthetic and real-world data. On synthetic logs, we achieved an average F-score of 0.96 and outperformed all the state-of-the-art methods. On real-world logs, we identified all types of process drifts in a comprehensive manner. Finally, we conducted a user study highlighting that our visualizations are easy to use and useful as perceived by process mining experts. In this way, our work contributes to research on process mining, event sequence analysis, and visualization of temporal data.
Quantum Broadcast Channels with Cooperating Decoders: An Information-Theoretic Perspective on Quantum Repeaters
Venue: arXiv cs.IT 2021
Year: 2021

Communication over a quantum broadcast channel with cooperation between the receivers is considered. The first form of cooperation addressed is classical conferencing, where Receiver 1 can send classical messages to Receiver 2. Another cooperation setting involves quantum conferencing, where Receiver 1 can teleport a quantum state to Receiver 2. When Receiver 1 is not required to recover information and its sole purpose is to help the transmission to Receiver 2, the model reduces to the quantum primitive relay channel. The quantum conferencing setting is intimately related to quantum repeaters, as the sender, Receiver 1, and Receiver 2 can be viewed as the transmitter, the repeater, and the destination receiver, respectively. We develop lower and upper bounds on the capacity region in each setting. In particular, the cutset upper bound and the decode-forward lower bound are derived for the primitive relay channel. Furthermore, we present an entanglement-formation lower bound, where a virtual channel is simulated through the conference link. At last, we show that as opposed to the multiple access channel with entangled encoders, entanglement between decoders does not increase the classical communication rates for the broadcast dual.
A Privacy-Preserving Healthcare Framework Using Hyperledger Fabric
Venue: arXiv cs.CR 2021
Year: 2021

Electronic health record (EHR) management systems require the adoption of effective technologies when health information is being exchanged. Current management approaches often face risks that may expose medical record storage solutions to common security attack vectors. However, healthcare-oriented blockchain solutions can provide a decentralized, anonymous and secure EHR handling approach. This paper presents PREHEALTH, a privacy-preserving EHR management solution that uses distributed ledger technology and an Identity Mixer (Idemix). The paper describes a proof-of-concept implementation that uses the Hyperledger Fabric's permissioned blockchain framework. The proposed solution is able to store patient records effectively whilst providing anonymity and unlinkability. Experimental performance evaluation results demonstrate the scheme's efficiency and feasibility for real-world scale deployment.
Analysis of Cryptocurrency Transactions from a Network Perspective: An Overview
Venue: arXiv cs.SI 2021
Year: 2021

As one of the most important and famous applications of blockchain technology, cryptocurrency has attracted extensive attention recently. Empowered by blockchain technology, all the transaction records of cryptocurrencies are irreversible and recorded in the blocks. These transaction records containing rich information and complete traces of financial activities are publicly accessible, thus providing researchers with unprecedented opportunities for data mining and knowledge discovery in this area. Networks are a general language for describing interacting systems in the real world, and a considerable part of existing work on cryptocurrency transactions is studied from a network perspective. This survey aims to analyze and summarize the existing literature on analyzing and understanding cryptocurrency transactions from a network perspective. Aiming to provide a systematic guideline for researchers and engineers, we present the background information of cryptocurrency transaction network analysis and review existing research in terms of three aspects, i.e., network modeling, network profiling, and network-based detection. For each aspect, we introduce the research issues, summarize the methods, and discuss the results and findings given in the literature. Furthermore, we present the main challenges and several future directions in this area.
A Knowledge Distillation Ensemble Framework for Predicting Short and Long-term Hospitalisation Outcomes from Electronic Health Records Data
Venue: arXiv cs.LG 2021
Year: 2021

The ability to perform accurate prognosis of patients is crucial for proactive clinical decision making, informed resource management and personalised care. Existing outcome prediction models suffer from a low recall of infrequent positive outcomes. We present a highly-scalable and robust machine learning framework to automatically predict adversity represented by mortality and ICU admission from time-series vital signs and laboratory results obtained within the first 24 hours of hospital admission. The stacked platform comprises two components: a) an unsupervised LSTM Autoencoder that learns an optimal representation of the time-series, using it to differentiate the less frequent patterns which conclude with an adverse event from the majority patterns that do not, and b) a gradient boosting model, which relies on the constructed representation to refine prediction, incorporating static features of demographics, admission details and clinical summaries. The model is used to assess a patient's risk of adversity over time and provides visual justifications of its prediction based on the patient's static features and dynamic signals. Results of three case studies for predicting mortality and ICU admission show that the model outperforms all existing outcome prediction models, achieving PR-AUC of 0.891 (95$%$ CI: 0.878 - 0.969) in predicting mortality in ICU and general ward settings and 0.908 (95$%$ CI: 0.870-0.935) in predicting ICU admission.
Causal Contextual Prediction for Learned Image Compression
Venue: arXiv cs.CV 2021
Year: 2021

Over the past several years, we have witnessed impressive progress in the field of learned image compression. Recent learned image codecs are commonly based on autoencoders, that first encode an image into low-dimensional latent representations and then decode them for reconstruction purposes. To capture spatial dependencies in the latent space, prior works exploit hyperprior and spatial context model to build an entropy model, which estimates the bit-rate for end-to-end rate-distortion optimization. However, such an entropy model is suboptimal from two aspects: (1) It fails to capture spatially global correlations among the latents. (2) Cross-channel relationships of the latents are still underexplored. In this paper, we propose the concept of separate entropy coding to leverage a serial decoding process for causal contextual entropy prediction in the latent space. A causal context model is proposed that separates the latents across channels and makes use of cross-channel relationships to generate highly informative contexts. Furthermore, we propose a causal global prediction model, which is able to find global reference points for accurate predictions of unknown points. Both these two models facilitate entropy estimation without the transmission of overhead. In addition, we further adopt a new separate attention module to build more powerful transform networks. Experimental results demonstrate that our full image compression model outperforms standard VVC/H.266 codec on Kodak dataset in terms of both PSNR and MS-SSIM, yielding the state-of-the-art rate-distortion performance.
An experiment on the mechanisms of racial bias in ML-based credit scoring in Brazil
Venue: arXiv cs.CY 2021
Year: 2021

We dissect an experimental credit scoring model developed with real data and demonstrate - without access to protected attributes - how the use of location information introduces racial bias. We analyze the tree gradient boosting model with the aid of a game-theoretic inspired machine learning explainability technique, counterfactual experiments and Brazilian census data. By exposing algorithmic racial bias explaining the trained machine learning model inner mechanisms, this experiment comprises an interesting artifact to aid the endeavor of theoretical understanding of the emergence of racial bias in machine learning systems. Without access to individuals' racial categories, we show how classification parity measures using geographically defined groups could carry information about model racial bias. The experiment testifies to the need for methods and language that do not presuppose access to protected attributes when auditing ML models, the importance of considering regional specifics when addressing racial issues, and the central role of census data in the AI research community. To the best of our knowledge, this is the first documented case of algorithmic racial bias in ML-based credit scoring in Brazil, the country with the second largest Black population in the world.
A Stable High-order Tuner for General Convex Functions
Venue: arXiv cs.LG 2021
Year: 2021

Iterative gradient-based algorithms have been increasingly applied for the training of a broad variety of machine learning models including large neural-nets. In particular, momentum-based methods, with accelerated learning guarantees, have received a lot of attention due to their provable guarantees of fast learning in certain classes of problems and multiple algorithms have been derived. However, properties for these methods hold only for constant regressors. When time-varying regressors occur, which is commonplace in dynamic systems, many of these momentum-based methods cannot guarantee stability. Recently, a new High-order Tuner (HT) was developed for linear regression problems and shown to have 1) stability and asymptotic convergence for time-varying regressors and 2) non-asymptotic accelerated learning guarantees for constant regressors. In this paper, we extend and discuss the results of this same HT for general convex loss functions. Through the exploitation of convexity and smoothness definitions, we establish similar stability and asymptotic convergence guarantees. Finally, we provide numerical simulations supporting the satisfactory behavior of the HT algorithm as well as an accelerated learning property.
The Fundamental Principles of Reproducibility
Venue: arXiv cs.LG 2021
Year: 2021

Reproducibility is a confused terminology. In this paper, I take a fundamental view on reproducibility rooted in the scientific method. The scientific method is analysed and characterised in order to develop the terminology required to define reproducibility. Further, the literature on reproducibility and replication is surveyed, and experiments are modeled as tasks and problem solving methods. Machine learning is used to exemplify the described approach. Based on the analysis, reproducibility is defined and three different types of reproducibility as well as four degrees of reproducibility are specified.
ConvTransformer: A Convolutional Transformer Network for Video Frame Synthesis
Venue: arXiv cs.CV 2021
Year: 2021

Deep Convolutional Neural Networks (CNNs) are powerful models that have achieved excellent performance on difficult computer vision tasks. Although CNNs perform well whenever large labeled training samples are available, they work badly on video frame synthesis due to objects deforming and moving, scene lighting changes, and cameras moving in video sequence. In this paper, we present a novel and general end-to-end architecture, called convolutional Transformer or ConvTransformer, for video frame sequence learning and video frame synthesis. The core ingredient of ConvTransformer is the proposed attention layer, i.e., multi-head convolutional self-attention layer, that learns the sequential dependence of video sequence. ConvTransformer uses an encoder, built upon multi-head convolutional self-attention layer, to encode the sequential dependence between the input frames, and then a decoder decodes the long-term dependence between the target synthesized frames and the input frames. Experiments on video future frame extrapolation task show ConvTransformer to be superior in quality while being more parallelizable to recent approaches built upon convolutional LSTM (ConvLSTM). To the best of our knowledge, this is the first time that ConvTransformer architecture is proposed and applied to video frame synthesis.
Deep reinforcement learning for feedback control in a collective flashing ratchet
Venue: arXiv cs.LG 2021
Year: 2021

A collective flashing ratchet transports Brownian particles using a spatially periodic, asymmetric, and time-dependent on-off switchable potential. The net current of the particles in this system can be substantially increased by feedback control based on the particle positions. Several feedback policies for maximizing the current have been proposed, but optimal policies have not been found for a moderate number of particles. Here, we use deep reinforcement learning (RL) to find optimal policies, with results showing that policies built with a suitable neural network architecture outperform the previous policies. Moreover, even in a time-delayed feedback situation where the on-off switching of the potential is delayed, we demonstrate that the policies provided by deep RL provide higher currents than the previous strategies.
Weighted automata are compact and actively learnable
Venue: arXiv cs.FL 2021
Year: 2021

We show that weighted automata over the field of two elements can be exponentially more compact than non-deterministic finite state automata. To show this, we combine ideas from automata theory and communication complexity. However, weighted automata are also efficiently learnable in Angluin's minimal adequate teacher model in a number of queries that is polynomial in the size of the minimal weighted automaton.. We include an algorithm for learning WAs over any field based on a linear algebraic generalization of the Angluin-Schapire algorithm. Together, this produces a surprising result: weighted automata over fields are structured enough that even though they can be very compact, they are still efficiently learnable.
Chitrakar: Robotic System for Drawing Jordan Curve of Facial Portrait
Venue: arXiv cs.RO 2021
Year: 2021

This paper presents a robotic system (\textit{Chitrakar}) which autonomously converts any image of a human face to a recognizable non-self-intersecting loop (Jordan Curve) and draws it on any planar surface. The image is processed using Mask R-CNN for instance segmentation, Laplacian of Gaussian (LoG) for feature enhancement and intensity-based probabilistic stippling for the image to points conversion. These points are treated as a destination for a travelling salesman and are connected with an optimal path which is calculated heuristically by minimizing the total distance to be travelled. This path is converted to a Jordan Curve in feasible time by removing intersections using a combination of image processing, 2-opt, and Bresenham's Algorithm. The robotic system generates $n$ instances of each image for human aesthetic judgement, out of which the most appealing instance is selected for the final drawing. The drawing is executed carefully by the robot's arm using trapezoidal velocity profiles for jerk-free and fast motion. The drawing, with a decent resolution, can be completed in less than 30 minutes which is impossible to do by hand. This work demonstrates the use of robotics to augment humans in executing difficult craft-work instead of replacing them altogether.
BARS: Joint Search of Cell Topology and Layout for Accurate and Efficient Binary ARchitectures
Venue: arXiv cs.AI 2021
Year: 2021

Binary Neural Networks (BNNs) have received significant attention due to their promising efficiency. Currently, most BNN studies directly adopt widely-used CNN architectures, which can be suboptimal for BNNs. This paper proposes a novel Binary ARchitecture Search (BARS) flow to discover superior binary architecture in a large design space. Specifically, we analyze the information bottlenecks that are related to both the topology and layout architecture design choices. And we propose to automatically search for the optimal information flow. To achieve that, we design a two-level (Macro & Micro) search space tailored for BNNs and apply a differentiable neural architecture search (NAS) to explore this search space efficiently. The macro-level search space includes width and depth decisions, which is required for better balancing the model performance and complexity. We also design the micro-level search space to strengthen the information flow for BNN. %A notable challenge of BNN architecture search lies in that binary operations exacerbate the "collapse" problem of differentiable NAS, for which we incorporate various search and derive strategies to stabilize the search process. On CIFAR-10, BARS achieves 1.5% higher accuracy with 2/3 binary operations and 1/10 floating-point operations comparing with existing BNN NAS studies. On ImageNet, with similar resource consumption, BARS-discovered architecture achieves a 6% accuracy gain than hand-crafted binary ResNet-18 architectures and outperforms other binary architectures while fully binarizing the architecture backbone.
Predictive maintenance on event logs: Application on an ATM fleet
Venue: arXiv cs.LG 2021
Year: 2021

Predictive maintenance is used in industrial applications to increase machine availability and optimize cost related to unplanned maintenance. In most cases, predictive maintenance applications use output from sensors, recording physical phenomenons such as temperature or vibration which can be directly linked to the degradation process of the machine. However, in some applications, outputs from sensors are not available, and event logs generated by the machine are used instead. We first study the approaches used in the literature to solve predictive maintenance problems and present a new public dataset containing the event logs from 156 machines. After this, we define an evaluation framework for predictive maintenance systems, which takes into account business constraints, and conduct experiments to explore suitable solutions, which can serve as guidelines for future works using this new dataset.