OpenRaiser/Intern-Atlas huggingface dataset:
Graph Neural Network Architecture Search for Molecular Property Prediction
Venue: arXiv cs.LG 2020
Year: 2020

Predicting the properties of a molecule from its structure is a challenging task. Recently, deep learning methods have improved the state of the art for this task because of their ability to learn useful features from the given data. By treating molecule structure as graphs, where atoms and bonds are modeled as nodes and edges, graph neural networks (GNNs) have been widely used to predict molecular properties. However, the design and development of GNNs for a given data set rely on labor-intensive design and tuning of the network architectures. Neural architecture search (NAS) is a promising approach to discover high-performing neural network architectures automatically. To that end, we develop an NAS approach to automate the design and development of GNNs for molecular property prediction. Specifically, we focus on automated development of message-passing neural networks (MPNNs) to predict the molecular properties of small molecules in quantum mechanics and physical chemistry data sets from the MoleculeNet benchmark. We demonstrate the superiority of the automatically discovered MPNNs by comparing them with manually designed GNNs from the MoleculeNet benchmark. We study the relative importance of the choices in the MPNN search space, demonstrating that customizing the architecture is critical to enhancing performance in molecular property prediction and that the proposed approach can perform customization automatically with minimal manual effort.
CACHE SNIPER : Accurate timing control of cache evictions
Venue: arXiv cs.CR 2020
Year: 2020

Microarchitectural side channel attacks have been very prominent in security research over the last few years. Caches have been an outstanding covert channel, as they provide high resolution and generic cross-core leakage even with simple user-mode code execution privileges. To prevent these generic cross-core attacks, all major cryptographic libraries now provide countermeasures to hinder key extraction via cross-core cache attacks, for instance avoiding secret dependent access patterns and prefetching data. In this paper, we show that implementations protected by 'good-enough' countermeasures aimed at preventing simple cache attacks are still vulnerable. We present a novel attack that uses a special timing technique to determine when an encryption has started and then evict the data precisely at the desired instant. This new attack does not require special privileges nor explicit synchronization between the attacker and the victim. One key improvement of our attack is a method to evict data from the cache with a single memory access and in absence of shared memory by leveraging the transient capabilities of TSX and relying on the recently reverse-engineered L3 replacement policy. We demonstrate the efficiency by performing an asynchronous last level cache attack to extract an RSA key from the latest wolfSSL library, which has been especially adapted to avoid leaky access patterns, and by extracting an AES key from the S-Box implementation included in OpenSSL bypassing the per round prefetch intended as a protection against cache attacks.
Two-stage optimization of urban rail transit formation and real-time station control at comprehensive transportation hub
Venue: arXiv cs.OH 2020
Year: 2020

This paper tries to discuss two strategies of dealing with this complex passenger demand from two aspects: transit train formation and real-time holding control. The genetic algorithm (GA) is designed to solve the integrated two-stage model of optimizing the number, timetable and real-time holding control of the multi-marshalling operated trains. The numerical results show that the combined two-stage model of multi-marshalling operation and holding control at stations can better deal with the demand fluctuation of urban rail transit connecting with the comprehensive transportation hub.
Understanding and Detecting Convergence for Stochastic Gradient Descent with Momentum
Venue: arXiv cs.LG 2020
Year: 2020

Convergence detection of iterative stochastic optimization methods is of great practical interest. This paper considers stochastic gradient descent (SGD) with a constant learning rate and momentum. We show that there exists a transient phase in which iterates move towards a region of interest, and a stationary phase in which iterates remain bounded in that region around a minimum point. We construct a statistical diagnostic test for convergence to the stationary phase using the inner product between successive gradients and demonstrate that the proposed diagnostic works well. We theoretically and empirically characterize how momentum can affect the test statistic of the diagnostic, and how the test statistic captures a relatively sparse signal within the gradients in convergence. Finally, we demonstrate an application to automatically tune the learning rate by reducing it each time stationarity is detected, and show the procedure is robust to mis-specified initial rates.
Towards General and Autonomous Learning of Core Skills: A Case Study in Locomotion
Venue: arXiv cs.RO 2020
Year: 2020

Modern Reinforcement Learning (RL) algorithms promise to solve difficult motor control problems directly from raw sensory inputs. Their attraction is due in part to the fact that they can represent a general class of methods that allow to learn a solution with a reasonably set reward and minimal prior knowledge, even in situations where it is difficult or expensive for a human expert. For RL to truly make good on this promise, however, we need algorithms and learning setups that can work across a broad range of problems with minimal problem specific adjustments or engineering. In this paper, we study this idea of generality in the locomotion domain. We develop a learning framework that can learn sophisticated locomotion behavior for a wide spectrum of legged robots, such as bipeds, tripeds, quadrupeds and hexapods, including wheeled variants. Our learning framework relies on a data-efficient, off-policy multi-task RL algorithm and a small set of reward functions that are semantically identical across robots. To underline the general applicability of the method, we keep the hyper-parameter settings and reward definitions constant across experiments and rely exclusively on on-board sensing. For nine different types of robots, including a real-world quadruped robot, we demonstrate that the same algorithm can rapidly learn diverse and reusable locomotion skills without any platform specific adjustments or additional instrumentation of the learning setup.
Forming better stable solutions in Group Formation Games inspired by Internet Exchange Points (IXPs)
Venue: arXiv cs.GT 2020
Year: 2020

We study a coordination game motivated by the formation of Internet Exchange Points (IXPs), in which agents choose which facilities to join. Joining the same facility as other agents you communicate with has benefits, but different facilities have different costs for each agent. Thus, the players wish to join the same facilities as their "friends", but this is balanced by them not wanting to pay the cost of joining a facility. We first show that the Price of Stability ($PoS$) of this game is at most 2, and more generally there always exists an $α$-approximate equilibrium with cost at most $\frac{2}α$ of optimum. We then focus on how better stable solutions can be formed. If we allow agents to pay their neighbors to prevent them from deviating (i.e., a player $i$ voluntarily pays another player $j$ so that $j$ joins the same facility), then we provide a payment scheme which stabilizes the solution with minimum social cost $s^*$, i.e. PoS is 1. In our main technical result, we consider how much a central coordinator would have to pay the players in order to form good stable solutions. Let $Δ$ denote the total amount of payments needed to be paid to the players in order to stabilize $s^*$, i.e., these are payments that a player would lose if they changed their strategy from the one in $s^*$. We prove that there is a tradeoff between $Δ$ and the Price of Stability: $\fracΔ{cost(s^*)} \le 1 - \frac{2}{5} PoS$. Thus when there are no good stable solutions, only a small amount of extra payment is needed to stabilize $s^*$; and when good stable solutions already exist (i.e., $PoS$ is small), then we should be happy with those solutions instead. Finally, we consider the computational complexity of finding the optimum solution $s^*$, and design a polynomial time $O(\log n)$ approximation algorithm for this problem.
A transprecision floating-point cluster for efficient near-sensor data analytics
Venue: arXiv cs.DC 2020
Year: 2020

Recent applications in the domain of near-sensor computing require the adoption of floating-point arithmetic to reconcile high precision results with a wide dynamic range. In this paper, we propose a multi-core computing cluster that leverages the fined-grained tunable principles of transprecision computing to provide support to near-sensor applications at a minimum power budget. Our design - based on the open-source RISC-V architecture - combines parallelization and sub-word vectorization with near-threshold operation, leading to a highly scalable and versatile system. We perform an exhaustive exploration of the design space of the transprecision cluster on a cycle-accurate FPGA emulator, with the aim to identify the most efficient configurations in terms of performance, energy efficiency, and area efficiency. We also provide a full-fledged software stack support, including a parallel runtime and a compilation toolchain, to enable the development of end-to-end applications. We perform an experimental assessment of our design on a set of benchmarks representative of the near-sensor processing domain, complementing the timing results with a post place-&-route analysis of the power consumption. Finally, a comparison with the state-of-the-art shows that our solution outperforms the competitors in energy efficiency, reaching a peak of 97 Gflop/s/W on single-precision scalars and 162 Gflop/s/W on half-precision vectors.
Sum Rate Maximization for Intelligent Reflecting Surface Assisted Terahertz Communications
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, an intelligent reflecting surface (IRS) is deployed to assist the terahertz (THz) communications. The molecular absorption causes path loss peaks to appear in the THz frequency band, and the fading peak is greatly affected by the transmission distance. In this paper, we aim to maximize the sum rate with individual rate constraints, in which the IRS location, IRS phase shift, the allocation of sub-bands of the THz spectrum, and power control for UEs are jointly optimized. For the special case of a single user equipment (UE) with a single sub-band, the globally optimal solution is provided. For the general case with multiple UEs, the block coordinate searching (BCS) based algorithm is proposed to solve the non-convex problem. Simulation results show that the proposed scheme can significantly enhance system performance.
A Fast and Robust BERT-based Dialogue State Tracker for Schema-Guided Dialogue Dataset
Venue: arXiv cs.LG 2020
Year: 2020

Dialog State Tracking (DST) is one of the most crucial modules for goal-oriented dialogue systems. In this paper, we introduce FastSGT (Fast Schema Guided Tracker), a fast and robust BERT-based model for state tracking in goal-oriented dialogue systems. The proposed model is designed for the Schema-Guided Dialogue (SGD) dataset which contains natural language descriptions for all the entities including user intents, services, and slots. The model incorporates two carry-over procedures for handling the extraction of the values not explicitly mentioned in the current user utterance. It also uses multi-head attention projections in some of the decoders to have a better modelling of the encoder outputs. In the conducted experiments we compared FastSGT to the baseline model for the SGD dataset. Our model keeps the efficiency in terms of computational and memory consumption while improving the accuracy significantly. Additionally, we present ablation studies measuring the impact of different parts of the model on its performance. We also show the effectiveness of data augmentation for improving the accuracy without increasing the amount of computational resources.
Optimal minimal-perturbation university timetabling with faculty preferences
Venue: arXiv cs.AI 2020
Year: 2020

In the university timetabling problem, sometimes additions or cancellations of course sections occur shortly before the beginning of the academic term, necessitating last-minute teaching staffing changes. We present a decision-making framework that both minimizes the number of course swaps, which are inconvenient to faculty members, and maximizes faculty members' preferences for times they wish to teach. The model is formulated as an integer linear program (ILP). Numerical simulations for a hypothetical mid-sized academic department are presented.
Power Control for ISI Mitigation in Mobile Molecular Communication
Venue: arXiv cs.IT 2020
Year: 2020

In mobile molecular communication (MC), inter-symbol interference (ISI) can be mitigated by power control, requiring accurate estimates of the distance from transmitter to receiver. We present two power control strategies based on binary concentration shift keying (BCSK), namely BCSK with power control based on distance (BCSK-d), and BCSK with power control jointly considering distance and residual molecules in the channel (BCSK-d-RM). Performance of BCSK-d and BCSK-d-RM are analyzed in terms of the bit error rate (BER), the average energy consumption per bit, and the optimal detection threshold. Simulation results show that BCSK-d-RM and BCSK-d outperform BCSK in BER with the varying distance and the number of transmitted molecules. As BCSK-d-RM has higher performance, while BCSK-d has lower complexity, these schemes present a useful design tradeoff for mobile MC.
Adversarial Training for Multi-Channel Sign Language Production
Venue: arXiv cs.CV 2020
Year: 2020

Sign Languages are rich multi-channel languages, requiring articulation of both manual (hands) and non-manual (face and body) features in a precise, intricate manner. Sign Language Production (SLP), the automatic translation from spoken to sign languages, must embody this full sign morphology to be truly understandable by the Deaf community. Previous work has mainly focused on manual feature production, with an under-articulated output caused by regression to the mean. In this paper, we propose an Adversarial Multi-Channel approach to SLP. We frame sign production as a minimax game between a transformer-based Generator and a conditional Discriminator. Our adversarial discriminator evaluates the realism of sign production conditioned on the source text, pushing the generator towards a realistic and articulate output. Additionally, we fully encapsulate sign articulators with the inclusion of non-manual features, producing facial features and mouthing patterns. We evaluate on the challenging RWTH-PHOENIX-Weather-2014T (PHOENIX14T) dataset, and report state-of-the art SLP back-translation performance for manual production. We set new benchmarks for the production of multi-channel sign to underpin future research into realistic SLP.
Pixel-Face: A Large-Scale, High-Resolution Benchmark for 3D Face Reconstruction
Venue: arXiv cs.CV 2020
Year: 2020

3D face reconstruction is a fundamental task that can facilitate numerous applications such as robust facial analysis and augmented reality. It is also a challenging task due to the lack of high-quality datasets that can fuel current deep learning-based methods. However, existing datasets are limited in quantity, realisticity and diversity. To circumvent these hurdles, we introduce Pixel-Face, a large-scale, high-resolution and diverse 3D face dataset with massive annotations. Specifically, Pixel-Face contains 855 subjects aging from 18 to 80. Each subject has more than 20 samples with various expressions. Each sample is composed of high-resolution multi-view RGB images and 3D meshes with various expressions. Moreover, we collect precise landmarks annotation and 3D registration result for each data. To demonstrate the advantages of Pixel-Face, we re-parameterize the 3D Morphable Model (3DMM) into Pixel-3DM using the collected data. We show that the obtained Pixel-3DM is better in modeling a wide range of face shapes and expressions. We also carefully benchmark existing 3D face reconstruction methods on our dataset. Moreover, Pixel-Face serves as an effective training source. We observe that the performance of current face reconstruction models significantly improves both on existing benchmarks and Pixel-Face after being fine-tuned using our newly collected data. Extensive experiments demonstrate the effectiveness of Pixel-3DM and the usefulness of Pixel-Face.
Accelerated WGAN update strategy with loss change rate balancing
Venue: arXiv cs.CV 2020
Year: 2020

Optimizing the discriminator in Generative Adversarial Networks (GANs) to completion in the inner training loop is computationally prohibitive, and on finite datasets would result in overfitting. To address this, a common update strategy is to alternate between k optimization steps for the discriminator D and one optimization step for the generator G. This strategy is repeated in various GAN algorithms where k is selected empirically. In this paper, we show that this update strategy is not optimal in terms of accuracy and convergence speed, and propose a new update strategy for Wasserstein GANs (WGAN) and other GANs using the WGAN loss(e.g. WGAN-GP, Deblur GAN, and Super-resolution GAN). The proposed update strategy is based on a loss change ratio comparison of G and D. We demonstrate that the proposed strategy improves both convergence speed and accuracy.
An Intelligent CNN-VAE Text Representation Technology Based on Text Semantics for Comprehensive Big Data
Venue: arXiv cs.LG 2020
Year: 2020

In the era of big data, a large number of text data generated by the Internet has given birth to a variety of text representation methods. In natural language processing (NLP), text representation transforms text into vectors that can be processed by computer without losing the original semantic information. However, these methods are difficult to effectively extract the semantic features among words and distinguish polysemy in language. Therefore, a text feature representation model based on convolutional neural network (CNN) and variational autoencoder (VAE) is proposed to extract the text features and apply the obtained text feature representation on the text classification tasks. CNN is used to extract the features of text vector to get the semantics among words and VAE is introduced to make the text feature space more consistent with Gaussian distribution. In addition, the output of the improved word2vec model is employed as the input of the proposed model to distinguish different meanings of the same word in different contexts. The experimental results show that the proposed model outperforms in k-nearest neighbor (KNN), random forest (RF) and support vector machine (SVM) classification algorithms.
The Adapter-Bot: All-In-One Controllable Conversational Model
Venue: arXiv cs.CL 2020
Year: 2020

Considerable progress has been made towards conversational models that generate coherent and fluent responses by training large language models on large dialogue datasets. These models have little or no control of the generated responses and miss two important features: continuous dialogue skills integration and seamlessly leveraging diverse knowledge sources. In this paper, we propose the Adapter-Bot, a dialogue model that uses a fixed backbone conversational model such as DialGPT (Zhang et al., 2019) and triggers on-demand dialogue skills (e.g., emphatic response, weather information, movie recommendation) via different adapters (Houlsby et al., 2019). Each adapter can be trained independently, thus allowing a continual integration of skills without retraining the entire model. Depending on the skills, the model is able to process multiple knowledge types, such as text, tables, and graphs, in a seamless manner. The dialogue skills can be triggered automatically via a dialogue manager, or manually, thus allowing high-level control of the generated responses. At the current stage, we have implemented 12 response styles (e.g., positive, negative etc.), 8 goal-oriented skills (e.g. weather information, movie recommendation, etc.), and personalized and emphatic responses. We evaluate our model using automatic evaluation by comparing it with existing state-of-the-art conversational models, and we have released an interactive system at adapter.bot.ust.hk.
The Effects of Skin Lesion Segmentation on the Performance of Dermatoscopic Image Classification
Venue: arXiv cs.CV 2020
Year: 2020

Malignant melanoma (MM) is one of the deadliest types of skin cancer. Analysing dermatoscopic images plays an important role in the early detection of MM and other pigmented skin lesions. Among different computer-based methods, deep learning-based approaches and in particular convolutional neural networks have shown excellent classification and segmentation performances for dermatoscopic skin lesion images. These models can be trained end-to-end without requiring any hand-crafted features. However, the effect of using lesion segmentation information on classification performance has remained an open question. In this study, we explicitly investigated the impact of using skin lesion segmentation masks on the performance of dermatoscopic image classification. To do this, first, we developed a baseline classifier as the reference model without using any segmentation masks. Then, we used either manually or automatically created segmentation masks in both training and test phases in different scenarios and investigated the classification performances. Evaluated on the ISIC 2017 challenge dataset which contained two binary classification tasks (i.e. MM vs. all and seborrheic keratosis (SK) vs. all) and based on the derived area under the receiver operating characteristic curve scores, we observed four main outcomes. Our results show that 1) using segmentation masks did not significantly improve the MM classification performance in any scenario, 2) in one of the scenarios (using segmentation masks for dilated cropping), SK classification performance was significantly improved, 3) removing all background information by the segmentation masks significantly degraded the overall classification performance, and 4) in case of using the appropriate scenario (using segmentation for dilated cropping), there is no significant difference of using manually or automatically created segmentation masks.