OpenRaiser/Intern-Atlas huggingface dataset:
Noise Modulation: Let Your Model Interpret Itself
Venue: arXiv cs.LG 2021
Year: 2021

Given the great success of Deep Neural Networks(DNNs) and the black-box nature of it,the interpretability of these models becomes an important issue.The majority of previous research works on the post-hoc interpretation of a trained model.But recently, adversarial training shows that it is possible for a model to have an interpretable input-gradient through training.However,adversarial training lacks efficiency for interpretability.To resolve this problem, we construct an approximation of the adversarial perturbations and discover a connection between adversarial training and amplitude modulation. Based on a digital analogy,we propose noise modulation as an efficient and model-agnostic alternative to train a model that interprets itself with input-gradients.Experiment results show that noise modulation can effectively increase the interpretability of input-gradients model-agnosticly.
Knowledge-Based Hierarchical POMDPs for Task Planning
Venue: arXiv cs.AI 2021
Year: 2021

The main goal in task planning is to build a sequence of actions that takes an agent from an initial state to a goal state. In robotics, this is particularly difficult because actions usually have several possible results, and sensors are prone to produce measurements with error. Partially observable Markov decision processes (POMDPs) are commonly employed, thanks to their capacity to model the uncertainty of actions that modify and monitor the state of a system. However, since solving a POMDP is computationally expensive, their usage becomes prohibitive for most robotic applications. In this paper, we propose a task planning architecture for service robotics. In the context of service robot design, we present a scheme to encode knowledge about the robot and its environment, that promotes the modularity and reuse of information. Also, we introduce a new recursive definition of a POMDP that enables our architecture to autonomously build a hierarchy of POMDPs, so that it can be used to generate and execute plans that solve the task at hand. Experimental results show that, in comparison to baseline methods, by following a recursive hierarchical approach the architecture is able to significantly reduce the planning time, while maintaining (or even improving) the robustness under several scenarios that vary in uncertainty and size.
API2Com: On the Improvement of Automatically Generated Code Comments Using API Documentations
Venue: arXiv cs.SE 2021
Year: 2021

Code comments can help in program comprehension and are considered as important artifacts to help developers in software maintenance. However, the comments are mostly missing or are outdated, specially in complex software projects. As a result, several automatic comment generation models are developed as a solution. The recent models explore the integration of external knowledge resources such as Unified Modeling Language class diagrams to improve the generated comments. In this paper, we propose API2Com, a model that leverages the Application Programming Interface Documentations (API Docs) as a knowledge resource for comment generation. The API Docs include the description of the methods in more details and therefore, can provide better context in the generated comments. The API Docs are used along with the code snippets and Abstract Syntax Trees in our model. We apply the model on a large Java dataset of over 130,000 methods and evaluate it using both Transformer and RNN-base architectures. Interestingly, when API Docs are used, the performance increase is negligible. We therefore run different experiments to reason about the results. For methods that only contain one API, adding API Docs improves the results by 4% BLEU score on average (BLEU score is an automatic evaluation metric used in machine translation). However, as the number of APIs that are used in a method increases, the performance of the model in generating comments decreases due to long documentations used in the input. Our results confirm that the API Docs can be useful in generating better comments, but, new techniques are required to identify the most informative ones in a method rather than using all documentations simultaneously.
Cost-effective Deployment of BERT Models in Serverless Environment
Venue: arXiv cs.CL 2021
Year: 2021

In this study we demonstrate the viability of deploying BERT-style models to serverless environments in a production setting. Since the freely available pre-trained models are too large to be deployed in this way, we utilize knowledge distillation and fine-tune the models on proprietary datasets for two real-world tasks: sentiment analysis and semantic textual similarity. As a result, we obtain models that are tuned for a specific domain and deployable in serverless environments. The subsequent performance analysis shows that this solution results in latency levels acceptable for production use and that it is also a cost-effective approach for small-to-medium size deployments of BERT models, all without any infrastructure overhead.
Selectively Delaying Instructions to Prevent Microarchitectural Replay Attacks
Venue: arXiv cs.CR 2021
Year: 2021

MicroScope, and microarchitectural replay attacks in general, take advantage of the characteristics of speculative execution to trap the execution of the victim application in an infinite loop, enabling the attacker to amplify a side-channel attack by executing it indefinitely. Due to the nature of the replay, it can be used to effectively attack security critical trusted execution environments (secure enclaves), even under conditions where a side-channel attack would not be possible. At the same time, unlike speculative side-channel attacks, MicroScope can be used to amplify the correct path of execution, rendering many existing speculative side-channel defences ineffective. In this work, we generalize microarchitectural replay attacks beyond MicroScope and present an efficient defence against them. We make the observation that such attacks rely on repeated squashes of so-called "replay handles" and that the instructions causing the side-channel must reside in the same reorder buffer window as the handles. We propose Delay-on-Squash, a technique for tracking squashed instructions and preventing them from being replayed by speculative replay handles. Our evaluation shows that it is possible to achieve full security against microarchitectural replay attacks with very modest hardware requirements, while still maintaining 97% of the insecure baseline performance.
Connecting Images through Time and Sources: Introducing Low-data, Heterogeneous Instance Retrieval
Venue: arXiv cs.CV 2021
Year: 2021

With impressive results in applications relying on feature learning, deep learning has also blurred the line between algorithm and data. Pick a training dataset, pick a backbone network for feature extraction, and voilà ; this usually works for a variety of use cases. But the underlying hypothesis that there exists a training dataset matching the use case is not always met. Moreover, the demand for interconnections regardless of the variations of the content calls for increasing generalization and robustness in features. An interesting application characterized by these problematics is the connection of historical and cultural databases of images. Through the seemingly simple task of instance retrieval, we propose to show that it is not trivial to pick features responding well to a panel of variations and semantic content. Introducing a new enhanced version of the Alegoria benchmark, we compare descriptors using the detailed annotations. We further give insights about the core problems in instance retrieval, testing four state-of-the-art additional techniques to increase performance.
On-Chain IoT Data Modification in Blockchains
Venue: arXiv cs.CR 2021
Year: 2021

In recent years, the interest growth in the Blockchains (BC) and Internet-of-Things (IoT) integration -- termed as BIoT -- for more trust via decentralization has led to great potentials in various use cases such as health care, supply chain tracking, and smart cities. A key element of BIoT ecosystems is the data transactions (TX) that include the data collected by IoT devices. BIoT applications face many challenges to comply with the European General Data Protection Regulation (GDPR) i.e., enabling users to hold on to their rights for deleting or modifying their data stored on publicly accessible and immutable BCs. In this regard, this paper identifies the requirements of BCs for being GDPR compliant in BIoT use cases. Accordingly, an on-chain solution is proposed that allows fine-grained modification (update and erasure) operations on TXs' data fields within a BC. The proposed solution is based on a cryptographic primitive called Chameleon Hashing. The novelty of this approach is manifold. BC users have the authority to update their data, which are addressed at the TX level with no side-effects on the block or chain. By performing and storing the data updates, all on-chain, traceability and verifiability of the BC are preserved. Moreover, the compatibility with TX aggregation mechanisms that allow the compression of the BC size is maintained.
CoordiNet: uncertainty-aware pose regressor for reliable vehicle localization
Venue: arXiv cs.CV 2021
Year: 2021

In this paper, we investigate visual-based camera re-localization with neural networks for robotics and autonomous vehicles applications. Our solution is a CNN-based algorithm which predicts camera pose (3D translation and 3D rotation) directly from a single image. It also provides an uncertainty estimate of the pose. Pose and uncertainty are learned together with a single loss function and are fused at test time with an EKF. Furthermore, we propose a new fully convolutional architecture, named CoordiNet, designed to embed some of the scene geometry. Our framework outperforms comparable methods on the largest available benchmark, the Oxford RobotCar dataset, with an average error of 8 meters where previous best was 19 meters. We have also investigated the performance of our method on large scenes for real time (18 fps) vehicle localization. In this setup, structure-based methods require a large database, and we show that our proposal is a reliable alternative, achieving 29cm median error in a 1.9km loop in a busy urban area
Towards Better Adaptive Systems by Combining MAPE, Control Theory, and Machine Learning
Venue: arXiv cs.SE 2021
Year: 2021

Two established approaches to engineer adaptive systems are architecture-based adaptation that uses a Monitor-Analysis-Planning-Executing (MAPE) loop that reasons over architectural models (aka Knowledge) to make adaptation decisions, and control-based adaptation that relies on principles of control theory (CT) to realize adaptation. Recently, we also observe a rapidly growing interest in applying machine learning (ML) to support different adaptation mechanisms. While MAPE and CT have particular characteristics and strengths to be applied independently, in this paper, we are concerned with the question of how these approaches are related with one another and whether combining them and supporting them with ML can produce better adaptive systems. We motivate the combined use of different adaptation approaches using a scenario of a cloud-based enterprise system and illustrate the analysis when combining the different approaches. To conclude, we offer a set of open questions for further research in this interesting area.
GLOWin: A Flow-based Invertible Generative Framework for Learning Disentangled Feature Representations in Medical Images
Venue: arXiv cs.CV 2021
Year: 2021

Disentangled representations can be useful in many downstream tasks, help to make deep learning models more interpretable, and allow for control over features of synthetically generated images that can be useful in training other models that require a large number of labelled or unlabelled data. Recently, flow-based generative models have been proposed to generate realistic images by directly modeling the data distribution with invertible functions. In this work, we propose a new flow-based generative model framework, named GLOWin, that is end-to-end invertible and able to learn disentangled representations. Feature disentanglement is achieved by factorizing the latent space into components such that each component learns the representation for one generative factor. Comprehensive experiments have been conducted to evaluate the proposed method on a public brain tumor MR dataset. Quantitative and qualitative results suggest that the proposed method is effective in disentangling the features from complex medical images.
From Static to Dynamic Prediction: Wildfire Risk Assessment Based on Multiple Environmental Factors
Venue: arXiv cs.LG 2021
Year: 2021

Wildfire is one of the biggest disasters that frequently occurs on the west coast of the United States. Many efforts have been made to understand the causes of the increases in wildfire intensity and frequency in recent years. In this work, we propose static and dynamic prediction models to analyze and assess the areas with high wildfire risks in California by utilizing a multitude of environmental data including population density, Normalized Difference Vegetation Index (NDVI), Palmer Drought Severity Index (PDSI), tree mortality area, tree mortality number, and altitude. Moreover, we focus on a better understanding of the impacts of different factors so as to inform preventive actions. To validate our models and findings, we divide the land of California into 4,242 grids of 0.1 degrees $\times$ 0.1 degrees in latitude and longitude, and compute the risk of each grid based on spatial and temporal conditions. To verify the generalizability of our models, we further expand the scope of wildfire risk assessment from California to Washington without any fine tuning. By performing counterfactual analysis, we uncover the effects of several possible methods on reducing the number of high risk wildfires. Taken together, our study has the potential to estimate, monitor, and reduce the risks of wildfires across diverse areas provided that such environment data is available.
Classification of Motor Imagery EEG Signals by Using a Divergence Based Convolutional Neural Network
Venue: arXiv cs.LG 2021
Year: 2021

Deep neural networks (DNNs) are observed to be successful in pattern classification. However, high classification performances of DNNs are related to their large training sets. Unfortunately, in the literature, the datasets used to classify motor imagery (MI) electroencephalogram (EEG) signals contain a small number of samples. To achieve high performances with small-sized datasets, most of the studies have employed a transformation such as common spatial patterns (CSP) before the classification process. However, CSP is dependent on subjects and introduces computational load in real-time applications. It is observed in the literature that the augmentation process is not applied for increasing the classification performance of EEG signals. In this study, we have investigated the effect of the augmentation process on the classification performance of MI EEG signals instead of using a preceding transformation such as the CSP, and we have demonstrated that by resulting in high success rates for the classification of MI EEGs, the augmentation process is able to compete with the CSP. In addition to the augmentation process, we modified the DNN structure to increase the classification performance, to decrease the number of nodes in the structure, and to be used with less number of hyper parameters. A minimum distance network (MDN) following the last layer of the convolutional neural network (CNN) was used as the classifier instead of a fully connected neural network (FCNN). By augmenting the EEG dataset and focusing solely on CNN's training, the training algorithm of the proposed structure is strengthened without applying any transformation. We tested these improvements on brain-computer interface (BCI) competitions 2005 and 2008 databases with two and four classes, and the high impact of the augmentation on the average performances are demonstrated.
Local Computation Algorithms for Coloring of Uniform Hypergraphs
Venue: arXiv cs.DS 2021
Year: 2021

We present a progress on local computation algorithms for two coloring of $k$-uniform hypergraphs. We focus on instances that satisfy strengthened assumption of Local Lemma of the form $2^{1-αk} (Δ+1) e < 1$, where $Δ$ is the bound on the maximum edge degree of the hypergraph. We discuss how previous works on the subject can be used to obtain an algorithm that works in polylogarithmic time per query for $α$ up to about $0.139$. Then, we present a procedure that, within similar bounds on running time, solves wider range of instances by allowing $α$ at most about $0.227$.
Automatic Quantification of Facial Asymmetry using Facial Landmarks
Venue: arXiv cs.CV 2021
Year: 2021

One-sided facial paralysis causes uneven movements of facial muscles on the sides of the face. Physicians currently assess facial asymmetry in a subjective manner based on their clinical experience. This paper proposes a novel method to provide an objective and quantitative asymmetry score for frontal faces. Our metric has the potential to help physicians for diagnosis as well as monitoring the rehabilitation of patients with one-sided facial paralysis. A deep learning based landmark detection technique is used to estimate style invariant facial landmark points and dense optical flow is used to generate motion maps from a short sequence of frames. Six face regions are considered corresponding to the left and right parts of the forehead, eyes, and mouth. Motion is computed and compared between the left and the right parts of each region of interest to estimate the symmetry score. For testing, asymmetric sequences are synthetically generated from a facial expression dataset. A score equation is developed to quantify symmetry in both symmetric and asymmetric face sequences.
Token-wise Curriculum Learning for Neural Machine Translation
Venue: arXiv cs.CL 2021
Year: 2021

Existing curriculum learning approaches to Neural Machine Translation (NMT) require sampling sufficient amounts of "easy" samples from training data at the early training stage. This is not always achievable for low-resource languages where the amount of training data is limited. To address such limitation, we propose a novel token-wise curriculum learning approach that creates sufficient amounts of easy samples. Specifically, the model learns to predict a short sub-sequence from the beginning part of each target sentence at the early stage of training, and then the sub-sequence is gradually expanded as the training progresses. Such a new curriculum design is inspired by the cumulative effect of translation errors, which makes the latter tokens more difficult to predict than the beginning ones. Extensive experiments show that our approach can consistently outperform baselines on 5 language pairs, especially for low-resource languages. Combining our approach with sentence-level methods further improves the performance on high-resource languages.
Beam-Squint Mitigating for Reconfigurable Intelligent Surface Aided Wideband MmWave Communications
Venue: arXiv cs.IT 2021
Year: 2021

In this paper, we focus our attention on the mitigation of beam squint for reconfigurable intelligent surface (RIS) aided wideband millimeter wave (mmWave) communications. Due to the intrinsic passive property, the phase shifts of all elements in RIS should be the same for all frequencies. However, in the wideband scenario, beam squint induced distinct path phases require designing different phase shifts for different frequencies. The above irreconcilable contradiction will dramatically affect the system performance, considering the RIS usually consists of enormous elements and the bandwidth of wideband mmWave communications may be up to several GHz. Therefore, we propose some novel phase shift design schemes for mitigating the effect of beam squint for both line-of-sight (LoS) and non-Los (NLoS) scenarios. Specifically, for the LoS scenario, we firstly derive the optimal phase shift for each frequency and obtain the common phase shift by maximizing the upper bound of achievable rate. Then, for the NLoS scenario, a mean channel covariance matrix (MCCM) based scheme is proposed by fully exploiting the correlations between both the paths and the subcarriers. Our extensive numerical experiments confirm the effectiveness of the proposed phase shift design schemes.
A novel multimodal fusion network based on a joint coding model for lane line segmentation
Venue: arXiv cs.CV 2021
Year: 2021

There has recently been growing interest in utilizing multimodal sensors to achieve robust lane line segmentation. In this paper, we introduce a novel multimodal fusion architecture from an information theory perspective, and demonstrate its practical utility using Light Detection and Ranging (LiDAR) camera fusion networks. In particular, we develop, for the first time, a multimodal fusion network as a joint coding model, where each single node, layer, and pipeline is represented as a channel. The forward propagation is thus equal to the information transmission in the channels. Then, we can qualitatively and quantitatively analyze the effect of different fusion approaches. We argue the optimal fusion architecture is related to the essential capacity and its allocation based on the source and channel. To test this multimodal fusion hypothesis, we progressively determine a series of multimodal models based on the proposed fusion methods and evaluate them on the KITTI and the A2D2 datasets. Our optimal fusion network achieves 85%+ lane line accuracy and 98.7%+ overall. The performance gap among the models will inform continuing future research into development of optimal fusion algorithms for the deep multimodal learning community.
Recognizing Predictive Substructures with Subgraph Information Bottleneck
Venue: arXiv cs.LG 2021
Year: 2021

The emergence of Graph Convolutional Network (GCN) has greatly boosted the progress of graph learning. However, two disturbing factors, noise and redundancy in graph data, and lack of interpretation for prediction results, impede further development of GCN. One solution is to recognize a predictive yet compressed subgraph to get rid of the noise and redundancy and obtain the interpretable part of the graph. This setting of subgraph is similar to the information bottleneck (IB) principle, which is less studied on graph-structured data and GCN. Inspired by the IB principle, we propose a novel subgraph information bottleneck (SIB) framework to recognize such subgraphs, named IB-subgraph. However, the intractability of mutual information and the discrete nature of graph data makes the objective of SIB notoriously hard to optimize. To this end, we introduce a bilevel optimization scheme coupled with a mutual information estimator for irregular graphs. Moreover, we propose a continuous relaxation for subgraph selection with a connectivity loss for stabilization. We further theoretically prove the error bound of our estimation scheme for mutual information and the noise-invariant nature of IB-subgraph. Extensive experiments on graph learning and large-scale point cloud tasks demonstrate the superior property of IB-subgraph.
Nonterminal complexity of some families of infinite regular languages
Venue: arXiv cs.FL 2021
Year: 2021

Nonterminal complexity of a context-free language is the smallest possible number of nonterminals in its generating grammar. While in general case nonterminal complexity computation problem is unsolvable, it can be computed for different families of regular languages. In this paper we study nonterminal complexity of some families of infinite regular languages.