Reviewerly/exHarmony huggingface dataset:
Behavior Sequence Transformer Applied on SERP Evaluation and Model Interpretation

SERP (Search Engine Result Page) quality evaluation plays a vital role in industrial practice. With the rapid iterations of search engine, traditional page-level metrics like click ratio, dwell time are no longer suitable to evaluate user experience on various templates of results. To promote evaluation accuracy, we implement Transformer to capture the sequential patterns from behavior sequence data. In recent studies, approaches focusing on modeling behavior sequences have emerged. Some studies concentrate on feature engineering by extracting subsequence patterns, others focus on end-to-end deep learning models. While widely used, these two methods both have drawbacks, either a risk of distortion of true subsequence patterns or difficulty for interpretation. Here we implement Transformer to give considerations to both completeness of sequential patterns and model interpretation. To find the best way of modeling behavior sequence data with continuous features, we adopt two embedding methods to predict SERP quality evaluation, and the second one achieves good promotion. What’s more, we develop a novel interpretation method for transformer models and demonstrate its ability to make interpretations for subsequence patterns.
BlinkNet: Software-Defined Deep Learning Analytics with Bounded Resources

Deep neural networks (DNNs) have recently gained unprecedented success in various domains. In resource-constrained edge systems (e.g., mobile devices and IoT devices) QoS-aware DNNs are required to meet latency and memory/storage requirements of mission-critical deep learning applications. However, none of the existing DNNs has been de-signed to satisfy both latency and memory bounds simultaneously as specified by end-users in the resource-constrained systems. This paper proposes a runtime system, BlinkNet, which can guarantee both latency and memory/storage bounds for one or multiple DNNs via efficient QoS-aware per-layer approximation. We implement BlinkNet in Apache TVM and evaluate it using CaffeNet, CIFAR-10-quick, and VGG16 network models on both CPU and GPU platforms. Our experimental results show that BlinkNet can enforce various latency and memory bounds set by end-users with real-world datasets.
Connecting the Semantic Dots: Zero-shot Learning with Self-Aligning Autoencoders and a New Contrastive-Loss for Negative Sampling

We introduce a novel zero-shot learning (ZSL) method, known as ‘self-alignment training’, and use it to train a vanilla autoencoder which is then evaluated on four prominent ZSL Tasks CUB, SUN, AWA1&2. Despite being a far simpler model than the competition, our method achieved results on par with SOTA. In addition, we also present a novel ‘contrastive-loss’ objective to allow autoencoders to learn from negative samples. In particular, we achieve new SOTA of 64.5 on AWA2 for Generalised ZSL and a new SOTA for standard ZSL of 47.7 on SUN. The code is publicly accessible on https://github.com/Wluper/satae.
Benchmarking Offline Reinforcement Learning

Offline reinforcement learning promises to enable control policy learning directly from previously collected data. It is a challenging problem because data collected from previous policies may not well-represent the distribution of the data when following the learned policy. This work evaluates the performance of recent offline RL algorithms using the library D3RLpy, which includes recent state-of-the-art algorithms as well as relevant datasets to evaluate them. We evaluate the performance of algorithms provided by the library without hyperparameter tuning. We compare performance both in terms of algorithm performance on the environments, as well as considering the effect of the dataset on the performance a few select algorithms.
SCGraph: Accelerating Sample-based GNN Training by Staged Caching of Features on GPUs

Graph neural networks (GNNs) have been becoming important tools for processing structured graph data and successfully applied to multiple graph-based application scenarios. The existing GNN systems adopt sample-based training on large-scale graphs over multiple GPUs. Although they support large-scale graph training, large data loading overhead of transferring vertex features between CPUs and GPUs is still a bottleneck. In this work, we propose SCGraph, a method that supports GPU high-speed feature caching. SCGraph classifies the graph vertices sorted by out-degrees. For high out-degree vertices, SCGraph sets grading caches via different GPUs to increase the overall cache content through NVLink high-speed data transmission between them. For low out-degree vertices, SCGraph expands training vertices' neighborhood in advance to regenerate cache. We evaluate SCGraph against two state-of-the-art industrial GNN frameworks, i.e., DGL and PaGraph on various benchmarks. Experimental results show that SCGraph improves the cache hit rate over GPUs up to 23.6%, and achieves up to 1.71x performance speedup over the state-of-the-art baselines while the convergence almost constant.
CopGAT: Co-propagation Self-supervised Graph Attention Network

Graph Attention Network (GAT) is one of the state-of-the-art architectures for Graph Neural Networks (GNNs). In this paper, we first propose Label Purity to explore the relationship between the graph attention and the node labels. By tracking the label purity of graph attention, we observe that graph attention suppresses message passing between inter-class nodes in the graph with homophily. Upon such basis, we further improve graph attention's capability to capture label information by designing a self-supervised graph attention loss. We propose a novel approach based on self-supervised graph attention loss, Co-propagation Self-supervised Graph Attention Network (CopGAT), which explicitly restrains message passing between inter-class nodes. The Co-propagation means propagating node representations in Euclidean space and pseudo labels in probability simplex via the shared attention matrix, combining Message Passing Neural Networks and Label Propagation Algorithm under the graph attention framework. Constrained with the self-supervised graph attention loss generated by co-propagation, our CopGAT enhances the ability of GAT to attenuate the influence of structural noise. Furthermore, our self-supervised graph attention loss can be extended to other graph attention models in a plug-and-play manner. Experimental results demonstrate that our Cop GAT outperforms previous state-of-the-art methods on five benchmark datasets for the semi-supervised node classification task.
What If Kidney Tumor Segmentation Challenge (KiTS19) Never Happened

Federated Learning (FL) is an efficient distributed machine learning algorithm that promises to reduce data migration costs to a centralized repository, alleviate regulatory data restrictions and maintain data privacy. However, it suffers from data heterogeneity, i.e., data distribution across silos is often non-identical and independent (non-iid), making optimization difficult. Further, a significant but rarely studied challenge in FL is the lack of annotated data for training. This challenge is more pronounced in the medical field since data annotations require precision and are highly labor-intensive and time-consuming. That is why a few sites have minimal or no annotated data, which wastes valuable data resources for ML training. In this work, we investigate these two challenges of Federated Learning on a publicly available realistic federated medical dataset, KiTS19. First, we explore Federated Learning for Tumor Segmentation task on the Federated version of the KiTS19 dataset for the first time. We show that FL can maintain 96% of model accuracy compared to the centralized model accuracy with ten institution collaboration. In addition, we investigate the benefits of transfer learning to address the challenge of data heterogeneity and show that 5% accuracy improvement is achieved by using a pre-trained model in FL. Moreover, we propose a Federated semi-supervised learning (FSSL) framework to address the challenge of the lack of annotations at some silos. We show that unlabelled silos add 11% to the model’s efficiency compared with the model trained on labeled silos alone.
A Novel Multimodal Situated Spoken Dialog System for Human Robot Communication in Emergency Evacuation

Given the need for multimodal autonomous human-robot interactive systems serving complex situations, a noise-robust and context- aware situated multimodal spoken dialogue system (MSSDS) in emergency evacuation missions is presented. The MSSDS system is composed of: (1) a context and noise robust Wake-Up-Word (WUW) (a) to initiate the dialogue, and (b) to detect the context switching by explicit use of WUW during emergency evacuation mission steps, (2) multi-turn user- initiated interactive text-and-spoken dialog communication system, and (3) an interactive voice and text interface for human-robot communication designed based on dialogues used in real-world emergency situations. In an emergency environment, speech is mixed with different noises, and therefore communication using speech in such an environment is challenging. We handle the noise by using Team Connect Ceiling (TCC) beam-forming microphone arrays. Innovative and useful applications of spoken dialogue systems, presented as proof of concept, constitute another contribution. Numerous digital assistants, keyword spotting, and wake-up-words-based technologies have already been developed, but these are mainly used indoors. Our objective is to support communication in complex environments, e.g., indoors and outdoors, in human and machine teaming, via a wake-up-word-based multimodal interactive system. The development of the real-world application to communicate with the robot using multimodalities in complex situations based on the presented approach is in progress while the presented simulated approach is reflecting parts of this development. Numerous machine learning technologies and toolkits have been applied in this ongoing development process. The novelty of WUW-based MSSDS is discussed in this paper. Our Markov Decision Process (MDP) evaluation shows that the WUW-based MSSDS performs better.
A Layer Decomposition Approach to Inference Time Prediction of Deep Learning Architectures

In recent years, deep learning models have been widely adopted in lots of fields. such as computer vision, pattern recognition, and classification problems like plant disease classification. Due to the large diversity among the computing devices that these models may run on, we need to choose between the appropriate device based on cost and performance. Furthermore, finding the suitable optimal device for a given project is a complex process that needs significant time and resources. Prediction of inference latency DNN models is necessary for many tasks where measuring the latency on real devices is either infeasible or too costly. This is a very challenging problem, and most existing approaches fail to achieve high accuracy of prediction. While some research has been carried out to predict the inference time of DNN models – most existing techniques assume that training time is linearly related to the number of floating-point operations. This paper designs and develops a framework to predict the inference time for deep learning models and is generic to be easily extended for a large set of devices. Our key idea is decomposing a given model inference into layers and conducting layer-level prediction. Our experiments demonstrate that this strategy provides significant benefits in terms of prediction accuracy.
ASPGen-D: Automatically Generating Fine-grained Apparmor Policies for Docker

With the development of virtualization technology, Docker has became the most popular containerization technology. Due to its shared kernel characteristics, it is lighter and more effi-ciently than traditional virtualization methods, but it also brings security issues to Docker. Applying Mandatory Access Control (MAC) framework to Docker containers is a common way to enhance Docker. However, it is a highly professional and complex task to generate MAC security policies for Docker containers. In this article, we aim to further optimize Docker's AppArmor security policy generating process and propose ASPGen-D, which is a novel framework for automatically generating Docker's Ap-pArmor security policies. It achieves three benefits: 1) ASPGen-D is able to generate security policies for each Docker automatically based on an Expert System. 2) The security policies generated by the framework are more in line with the characteristics of Docker. 3) The security policies can be dynamically updated according to the runtime status of Docker. Our current experimental results show that ASPGen-D is an efficient and effective security solution for Docker.
Protection of Location Private Privacy with Recommendation from a Decision Tree Algorithm

Location-based services (LBSs) have been widely used in apps on smartphones. They help apps provide better service for us, for example, a navigation app guides us to trip. However, LBSs can also cause the problem of privacy disclosure. For some apps, like weather apps, a coarse location accuracy (LA) is enough to achieve their function, while they get our location within several meters. In this paper, we propose a framework for LBS Management (LBSM) to protect users' personal location message by providing multi-level LA to different apps. The framework will create two decision trees for apps and users to analyze the properties, and then use the decision trees to make the classification. According to the classification result, the LA level of an app is finally determined. For better service, the decision trees will adjust constantly when the cloud servers obtain new data of apps and users or their properties have change. Furthermore, some improvements have been made on the basis of traditional Iterative Dichotomiser 3 (ID3) algorithm to make a more accurate classification result. Experiments show that users' personal information can be largest protected while also obtaining satisfactory service with our proposed LBSM.
IGN : Implicit Generative Networks

In this work, we build recent advances in distributional reinforcement learning to give a state-of-art distributional variant of the model based on the IQN. We achieve this by using the GAN model’s generator and discriminator function with the quantile regression to approximate the full quantile value for the state-action return distribution. We demonstrate improved performance on our baseline dataset - 57 Atari 2600 games in the ALE. Also, we use our algorithm to show the state-of-art training performance of risk-sensitive policies in Atari games with the policy optimization and evaluation.
An Edge-based Real-Time Object Detection

This paper looks at performance bottlenecks of real-time object detection on edge devices. The "You only look once v4" (YOLOv4) is currently one of the leading state-of-the-art models for real-time object detection, and its tiny version: YOLOv4-tiny, is designed for edge devices. To improve object detection accuracy without sacrificing detection speed, we propose an object detection method based on YOLOv4-tiny and VGG-Net. First, we implement the mosaic data augmentation and Mish activation function to increase the generalization ability of the proposed model, making it more robust. Secondly, to enhance the richness of the features extracted, an extra 3x3 convolution layer is added in a way that two successive 3x3 convolutions are used to obtain 5x5 receptive fields. This would enable us to extract global features in the first CSP (Cross Stage Partial Network) Block and restructure the connections of the subsequent layers to have the same effect on the next CSP blocks. Evaluation results show that the proposed model has comparable performance and memory footprint but significantly greater accuracy than YOLOv4-tiny. Also, the proposed tiny model has similar performance to YOLOv4-tiny, and improves accuracy with much lower memory overhead, which makes it an ideal solution for real-time object detection, especially on edge devices.
BHMVD: Binary Code-based Hybrid Neural Network for Multiclass Vulnerability Detection

Precise binary code vulnerability detection is a significant research topic in software security. Currently, the majority of software is released in binary form, and the corresponding vulnerability detection approaches for binary code are desired. Existing deep learning-based detection techniques can only detect binary code vulnerabilities but cannot precisely identify the types of vulnerabilities. This paper proposes a Binary code-based Hybrid neural network for Multiclass Vulnerability Detection, dubbed BHMVD. BHMVD generates binary slices according to the control dependence and data dependence of library/ API function calls, and then extracts syntax features from binary slices to generate type slices, which can help identify vulnerability types. This paper uses a hybrid neural network of CNN-BLSTM to extract vulnerability features from binary and type slices. The former extracts local features, while the latter extracts global features. Experiment results on 19 types of vulnerabilities show that BHMVD is effective for binary code-based multiclass vulnerability detection, and using a hybrid neural network can improve detection ability.
A Real-time Digit Gesture Recognition System Based on mmWave Radar

Gesture communication is one of the most general communication methods in the world, with the obvious advantage of exchanging information without worrying about the borderline of different languages. Therefore, establishing a cost-effective way of capturing and understanding human gestures has long been a popular research topic regarding human-machine interaction, particularly in emerging scenarios such as smart cities, etc. In this paper, we propose a system based on a commercially available mmWave radar to recognize digits represented by the travel path of the human hand using a specially designed convolutional neural network (CNN) algorithm. We illustrate the proposed system is capable of recording the path of the moving hand in real-time at the cost of 1 transmitter, 2 receivers, and 2.78 GHz bandwidth from the mmWave radar. Our experimental results show that an average prediction accuracy of 98.8% is achieved in a validation test based on a 7:3 ratio split from existing dataset and an average prediction accuracy of 95.3% in generalization test using fresh data.
Mixed Time-Frame Training for Reinforcement Learning

Reinforcement learning typically only uses one type of environment during training: episodic or non-episodic. In this paper, we propose a novel training technique, Mixed Time-Frame Training, which combines episodic environments and non-episodic environments through training in parallel. This approach takes advantage of the different exploration-exploitation characteristics of the two environment types. We shall benchmark the proposed technique on 10 Atari games with varying levels of exploration difficulty, showing a significant increase in performance. The increase in performance comes with no increase in training frames, algorithmic complexity or computation.
Probabilistic Approach for Recommendation Systems

In this article, we propose a new probabilistic approach for product recommendations using deep learning framework, combining information from historical observations, similar users and prior knowledge. The deep learning approach is using autoregressive recurrent networks to model the recommendations probabilistically from a Bernoulli distribution. If prior information exists we implement a Pseudo-Bayesian approach, where we obtain posterior samples assuming Bernoulli likelihood on the sampled data from the deep learning model. The proposed approach allows for a very flexible modeling of product recommendations and quantifying uncertainty in predictions. Simulations and experiments were conducted to demonstrate the applicability and performance of the model. Comparisons made to related recommendation models revealed more accurate predictions among the proposed models.
Recycling Material Classification using Convolutional Neural Networks

Recent years have witnessed continuous growth of the abilty of convolutional neural networks (CNNs) on solving image classification problems. More recently, CNNs have started to show promising results on classifying recyclable materials given their images. This work aims to showcase the effectiveness of CNN architectures for such classification task on our dataset combined from multiple public sources, where in total 12,873 images of recyclable materials are collected over four classes: glass, metal, paper, and plastic. To the best of our knowledge, our work is the first attempt to train and test CNN models on this dataset. To this end, we train a selection of CNN models, including a simple 8-layer CNN, AlexNet, VGGNet, and InceptionNet. Our empirical results show that VGG-16 (a VGGNet variant of 16 convolutional and pooling layers), combined with transfer learning, produces the best testing accuracy of 84.6%. Furthermore, we import this best model to implement a Raspberry Pi application and an Android application to demonstrate their potential for consumer and industrial usage.
Collar Data Synthesis and Its Application for Generating Virtual Collar 3D Novel View from a Single-View

In recent years, virtual reality, big data and cloud computing technologies have all made rapid development, and they are also more and more closely combined with the apparel industry. The apparel digital design and virtual fitting techniques are playing increasingly important roles in the field of apparel design, manufacturing and sales. The three-dimensional digital design and virtual fitting make the designers or the users have multi-view and more real impressions and experiences of the apparel. But 3D display is difficult to be popularized in apparel industry because of its high cost. The cost of two-dimensional display is low and easy to popularize. However, 2D system can only provide images of a specific perspective, lacking a multi-perspective experience. If the supplier is required to provide 2D images of several specific perspectives, the personalized needs of users cannot be satisfied. Therefore, how to make 2D system according to an object image and users' needs to provide the object images of other perspectives is an urgent problem to be solved. In this work, we propose a novel view synthesis pipeline based on an enhanced Pix2Pix neural network for a novel view (Pix2Pix-V). It consists of four parts: 1) a new method of generating the labeled virtual collar image data based on its 3D model, and the 2D images can be obtained on any perspective; 2) Pix2Pix-V neural network; 3) a Pix2Pix-V training workflow for predicting the novel view; 4) transfer learning is used in predicting similar views of virtual collars.
Novel Adversarial Defense Techniques for White-Box Attacks

In this paper, we propose a novel machine learning technique to detect and correctly classify adversarially modified images. Our approach takes advantage of the adversarial training objective (to influence the prediction with small perturbations to the original image) by searching for large discrepancies in the model’s output relative to the input image. We find clusters in each space, and for an image whose clusters do not agree with each other, we flag the image as potentially adversarial and use the input space cluster’s prediction in place of the attacked model’s output. We have found that this method tends to find adversarial samples which are more likely to be misclassified by the attacked model, and that simply deferring to the input cluster’s prediction on such samples is enough to increase accuracy on adversarial data, even when the attacked model is adversarially trained.