Reviewerly/exHarmony huggingface dataset:
EdgeNN: Efficient Neural Network Inference for CPU-GPU Integrated Edge Devices

With the development of the architectures and the growth of AIoT application requirements, data processing on edge has become popular. Neural network inference is widely employed for data analytics on edge devices. This paper extensively explores neural network inference on integrated edge devices and proposes EdgeNN, the first neural network inference solution on CPU-GPU integrated edge devices. EdgeNN has three novel characteristics. First, EdgeNN can adaptively utilize the unified physical memory and conduct the zero-copy optimization. Second, EdgeNN involves a novel inference-targeted inter- and intra-kernel CPU-GPU hybrid execution approach, which co-runs the CPU with the GPU to fully utilize the edge device’s computing resources. Third, EdgeNN adopts a fine-grained adaptive inference tuning approach, which can divide the complicated inference structure into sub-tasks mapped to the CPU and the GPU. Experiments show that on six popular neural network inference tasks, EdgeNN brings an average of 3.97×, 3.12×, and 8.80× speedups to inference on the CPU of the integrated device, inference on a mobile phone CPU, and inference on an edge CPU device. Additionally, it achieves 22.02% time benefits to the direct execution of the original programs. Specifically, 9.93% comes from better utilization of unified memory, and 10.76% comes from CPU-GPU hybrid execution. Besides, EdgeNN can deliver 29.14× and 5.70× higher energy efficiency than the edge CPU and the discrete GPU, respectively. We have made EdgeNN available at https://github.com/ChenyangZhang-cs/EdgeNN.
Model Based Smart Communication System Design

The development of natural language processing (NLP) has revealed its huge potential power and wide implementation. However, in the current status of natural language processing and smart communication design, there are some problems and limitations. First, most of the NLP and smart communication research mainly focus on the direction of the algorithm but lacks communication and relationship model design. The second, algorithm and memory implementations of models seldom separate for different tasks and situations. The third, wide use of unsupervised learning makes the model difficult to interpret, uncontrollable, and inflexible. To solve those problems, we designed memory-relationship classification categories to classify and guide the implementation of NLP and smart communication. With the implementation of model basing learning concept and communication model, we designed a sequential controllable semantic dimension and relationship dimension system to classify smart communication.
A High-Performance Index for Real-Time Matrix Retrieval (Extended Abstract)

Embedding techniques can be used to represent words using word embedding [1] , images using image-to-vector techniques [2] , [3] and even database queries [4] . As a result, many more real-world objects can be represented by matrices. For example, a matrix can represent a document where each row (i.e., each vector) of the matrix stands for a word in the document. Figure 1 shows the key steps of representing an object (e.g., a document, a video or an audio stream) by a matrix. The intermediate step is to divide the object into small pieces and to convert the small pieces into vectors. The vectors of the object are then put together to form a matrix. These objects represented by matrices require new data management systems to support efficient indexing and retrieval.
Distribution-Regularized Federated Learning on Non-IID Data

Federated learning (FL) has emerged as a popular machine learning paradigm recently. Compared with traditional distributed learning, its unique challenges mainly lie in communication efficiency and non-IID (heterogeneous data) problem. While the widely adopted framework FedAvg can reduce communication overhead significantly, its effectiveness on non-IID data still lacks exploration. In this paper, we study the non-IID problem of FL from the perspective of domain adaptation. We propose a distribution regularization for FL on non-IID data such that the discrepancy of data distributions between clients is reduced. To further reduce the communication cost, we devise two novel distributed learning algorithms, namely rFedAvg and rFedAvg+, for efficiently learning with the distribution regularization. More importantly, we theoretically establish their convergence for strongly convex objectives. Extensive experiments on 4 datasets with both CNN and LSTM as learning models verify the effectiveness and efficiency of the proposed algorithms.
An Efficient Generative Data Imputation Toolbox with Adversarial Learning

The dramatically increasing volume of incomplete data makes the imputation models computationally infeasible in many real-life applications. In this demonstration, we propose a scalable and extendible data imputation toolbox, SEMI, to deal with large-scale incomplete data imputation efficiently and visually. SEMI consists of three modules: data preprocessing, data imputation, and post-imputation prediction. It is built upon SCIS, a scalable imputation system, to significantly speed up the training of generative adversarial imputation models under accuracy-guarantees for large-scale incomplete data. Using a public real-world large-scale incomplete weather dataset, we demonstrate that, SEMI is capable of assisting users to efficiently address real-life large-scale imputation issues, from the aspects of high-efficient imputation system, user-friendly performance visualization, and easy-to-use interaction operation.
Robust Smart Home Face Recognition Under Starving Federated Data

Over the past few years, the field of adversarial attack received numerous attention from various researchers with the help of successful attack success rate against well-known deep neural networks that were acknowledged to achieve high classification ability in various tasks. However, majority of the experiments were completed under a single model, which we believe it may not be an ideal case in a real-life situation. In this paper, we introduce a novel federated adversarial training method for smart home face recognition, named FLATS, where we observed some interesting findings that may not be easily noticed in a traditional adversarial attack to federated learning experiments. By applying different variations to the hyperparameters, we have spotted that our method can make the global model to be robust given a starving federated environment.
Globally Aware Contextual Embeddings for Named Entity Recognition in Social Media Streams

An important task for Information Extraction from Microblogs is Named Entity Recognition (NER) that extracts mentions of real-world entities from microblog messages and meta-information like entity type for better entity characterization. A lot of microblog NER systems have rightly sought to prioritize modeling the non-literary nature of microblog text. These systems are trained on offline static datasets and extract a combination of surface-level features – orthographic, lexical, and semantic – from individual messages for noisy text modeling and entity extraction. But given the constantly evolving nature of microblog streams, detecting all entity mentions from such varying yet limited context in short messages remains a difficult problem to generalize. In this paper, we propose the NER Globalizer pipeline better suited for NER on microblog streams. It characterizes the isolated message processing by existing NER systems as modeling local contextual embeddings, where learned knowledge from the immediate context of a message is used to suggest seed entity candidates. Additionally, it recognizes that messages within a microblog stream are topically related and often repeat mentions of the same entity. This suggests building NER systems that go beyond localized processing. By leveraging occurrence mining, the proposed system therefore follows up traditional NER modeling by extracting additional mentions of seed entity candidates that were previously missed. Candidate mentions are separated into well-defined clusters which are then used to generate a pooled global embedding drawn from the collective context of the candidate within a stream. The global embeddings are utilized to separate false positives from entities whose mentions are produced in the final NER output. Our experiments show that the proposed NER system exhibits superior effectiveness on multiple NER datasets with an average Macro F1 improvement of 47.04% over the best NER baseline while adding only a small computational overhead.
Iterative Refinement for Multi-Source Visual Domain Adaptation (Extended abstract)

Multi-source domain adaptation (MSDA) aims to leverage the knowledge in multiple source domains to assist the prediction in a target domain, where the source and target domains have different data distributions. This paper presents a MSDA model to investigate both domain discrepancy and domain relevance, whose interactions are also exploited to gradually refine the learning performance. Particularly, the proposed model contains two components, i.e., feature spaces learning and transferred weights learning. The former one minimizes the domain discrepancy and the latter one evaluates the domain relevance. Experimental results on several real-world datasets demonstrate the effectiveness of the proposed model.
A Transformer-based Unsupervised Clustering Method for Vehicle Re-identification

Current unsupervised re-identification methods use a clustering-based neural network for training. In the vehicle re-identification field, the feature information between different vehicles is small, and it is not easy to distinguish the detailed features of different vehicles using only the basic clustering algorithm for unsupervised learning. When clustering is performed, the general clustering methods inevitably put different vehicles together due to the high similarity. We propose a new re-identification method to solve these problems. This method is based on clustering and use the unsupervised learning. First, we employ the vision transformer structure as a feature extractor. The vision transformer structure can obtain more discriminative and correlated features than the general convolution. Second, we use a fine-grained clustering method to subdivide the clustered information into different vehicles. We trained our method on two open-source datasets, and finally obtained better test results without additional labeling information.
Unshackling Database Benchmarking from Synthetic Workloads

Introducing new (learned) features into a DBMS requires considerable experimentation and benchmarking to avoid regressions in production (customer) workloads. Using standard benchmarks such as TPC-H and TCH-DS is common practice, but, unfortunately, these do not represent the complexity of real production workloads. To solve this problem, in this demo, we propose a technique that generates a synthetic dataset from query logs and metadata—without touching the original data. The keystone of our approach is to map the data generation as a SAT problem where constraints, such as runtime cardinalities, are extracted from query logs and metadata. We show that our approach can generate representative benchmarks mirroring the performance of the original data without trading off privacy. The demo will guide the attendees through the various steps involved in the data generation and testing process.
Influential Recommender System

Traditional recommender systems are typically passive in that they try to adapt their recommendations to the user’s historical interests. However, it is highly desirable for commercial applications, such as e-commerce, advertisement placement, and news portals, to be able to expand the users’ interests so that they would accept items that they were not originally aware of or interested in to increase customer interactions. In this paper, we present Influential Recommender System (IRS), a new recommendation paradigm that aims to proactively lead a user to like a given objective item by progressively recommending to the user a sequence of carefully selected items (called an influence path). We propose the Influential Recommender Network (IRN), which is a Transformer-based sequential model to encode the items’ sequential dependencies. Since different people react to external influences differently, we introduce the Personalized Impressionability Mask (PIM) to model how receptive a user is to external influence to generate the most effective influence path for the user. To evaluate IRN, we design several performance metrics to measure whether or not the influence path can smoothly expand the user interest to include the objective item while maintaining the user’s satisfaction with the recommendation. Experimental results show that IRN significantly outperforms the baseline recommenders and demonstrates its capability of influencing users’ interests.
Enhanced Video Caption Generation Based on Multimodal Features

Video caption is the automatically generated of abstract expressions for the content contained in videos. It involves two important fields — computer vision and natural language processing, and has become a considerable research topic in smart life. Deep learning has successfully contributed to this task with good results. As we know, video contains various modals of information, yet most of the existing solutions start from the visual perspective of video, while ignoring the equally important audio modal information. Therefore, how to benefit from additional forms of cues other than visual information is a huge challenge. In this work, we propose a video caption generation method that fuses multimodal features in videos, and adds attention mechanism to improve the quality of generated description sentences. The experimental results demonstrate that the method is well validated on the MSR-VTT dataset.
The Effect of English Text Readability on Speech Duration of Second Language Learners

In order to facilitate active communication among people of various cultural and linguistic backgrounds, it is necessary to learn a second language that is linguistically different from one’s native language. The difficulty in learning second language is thought to depend on readability. Most research on the validity of readability assessment has focused on native speakers. Moreover, most of the few studies of readability with second language learners have focused on textual data. This study analyzed actual speech data to determine the effects of word frequencies, syllable structures, and grammatical functions used to estimate the relation with readability on word duration. We compared English word durations in native English speakers and native Japanese speakers. It is hypothesized that highly readable words, i.e., those that are simple and familiar, would be easier for Japanese to utter and shorter in duration, but the opposite was found in the experiment. This suggests that the features adopted for native English speakers as readability indicators are not fully compatible with Japanese speakers.
SimpleCopy: A Strong Data Augmentation for Microalgae Detection

Marine microalgae detection is of great importance to the environment and ecosystem. In this paper, we consider microalgae detection as a computer vision task and use a two-stage object detection network, Cascade R-CNN, to build our detector and deal with the dataset which contains a variety of small targets. Firstly, We proposed a novel data augmentation strategy called SimpleCopy for microscopic images, which typically have more small targets and sparse target distributions. Secondly, we leverage the strengths of different backbone and employ model ensemble techniques to enhance the performance of our detector. Finally, with carefully designed post-processing methods, the recall and precision of our detector can be further improved. Extensive experiments conducted on the marine dataset show the superiority of our model. We verified the effectiveness of our method by achieving 58.18 mAP and ranked 3/347 on the official leadboard.
The Art of Losing to Win: Using Lossy Image Compression to Improve Data Loading in Deep Learning Pipelines

Training deep learning (DL) models often takes a significant amount of time and is thus typically performed on expensive GPUs to speed up the process. However, data loading has recently been identified as one of the main performance bottlenecks in DL, resulting in GPU under-utilization. Looking forward, the combination of larger datasets and faster GPUs will exacerbate the problem. The data management community has started to address this by proposing data loading optimization techniques, including lossy image compression. While lossy compression is a conceptually promising approach for mitigating data loading bottlenecks in DL, there is only limited understanding of its efficacy in terms of impact on model throughput and accuracy. In this paper, we present an extensive experimental analysis of lossy image compression as a means to improve the performance of neural network training. We find that lossy compression can improve both throughput and accuracy of DL pipelines if resources such as time or storage capacity are limited. Furthermore, the choice of compression quality and codec are important hyperparameters when training deep neural networks.
FASI: FPGA-friendly Subgraph Isomorphism on Massive Graphs

Subgraph isomorphism plays a significant role in many applications, such as social networks and bioinformatics. However, due to the inherent NP-hardness, it becomes challenging to compute matches efficiently in large real-world graphs. Many researchers have attempted to solve this problem with the help of new hardware. Nevertheless, most of them focus on GPU. Due to the dataflow feature and burst I/O optimization, FPGA is a potential competitor to speed up subgraph isomorphism. However, there are very few subgraph matching algorithms on FPGA. In this paper, we present an efficient FPGA-friendly Subgraph Isomorphism algorithm FASI, designed on CPU- FPGA heterogeneous platform which leverages FPGA's features. Unlike the existing FPGA-based method FAST, we adopt the worst-case-optimal-join-based pipeline design. First, we propose an FPGA-friendly data structure LPCSR for efficient access to neighbor lists. Second, we offer a joint parallelized pipeline strategy to accelerate matching process. Third, we propose a memory coalescing mechanism and a space-saving pre-allocated write back strategy. Our experiments on both synthetic and real graphs show that FASI outperforms other state-of-the-art subgraph matching algorithms on CPU, GPU and FPGA.
CORGI: An interactive framework for Customizable and Robust Location Obfuscation

Customizing the location obfuscation functions generated by existing systems can result in weakening the privacy guarantees offered by these functions as they are not robust against such updates. In this demo, we present a new framework called, CORGI, i.e., CustOmizable Robust Geo Indistinguishability. The demonstration platform is a web application which is built on top on a real world dataset (Gowalla). The user-friendly interface of the demo allows participants to easily specify their customization preferences and generate a customizable and robust location obfuscation function. They can also examine the trade-offs among privacy, utility, and customization; visualized on a map for comparison between CORGI and a state of the art baseline.
Attributed Transition-Based Domain Control Knowledge for Domain-Independent Planning (Extended Abstract)

This extended abstract from the area of automated planning discusses work on Attributed Transition-Based Domain Control Knowledge (ATB-DCK). ATB-DCK, roughly speaking, represents the "grammar" of solution plans that guides the search. ATB-DCK is expressed by a finite state automaton with attributed states, referring to specific states of objects, connected by transitions imposing constraints on action applicability. This representation stays on side of the planning domain model, but it can be compiled into a classical planning task and thus it complements domain-independent planning techniques. Results on several benchmark domains from the International Planning Competitions show that the use of ATB-DCK often considerably improves efficiency of existing state-of-the-art planning engines.
A Multi-Hypothesis Tracker with Enhanced Appearance Model for Generic Crowded Scene

Pedestrian tracking studies have been facilitated by a large amount of surveillance apparatus in the city while also raising public privacy concerns. In this paper, we propose X-Tracking, a privacy-aware pedestrian tracking paradigm designed for vision systems in Smart City. It allows low-cost compatibility with existing surveillance architecture. To protect entities’ privacy, X-Tracking uses video pre-processing with desensitization so that identity information is unexposed to the tracking algorithm. We implement system-level privacy protection by redesigning the tracking framework that decouples all services based on a single responsibility principle. Then, we elaborate on the roles, behaviors, and protocols used in the new system and illustrate how the paradigm strikes a favorable balance between privacy protection and convenience services. Furthermore, we propose a new tracking task that aims to track humans in masking surveillance video. It is comparable to previous tracking tasks but considering the target with a distorted appearance poses new challenges for visual tracking. Finally, we evaluate the baseline algorithm on the task with a demo dataset.
Improved Handwritten Numeral Recognition on MNIST Dataset with YOLO and LSTM

With the aging of population and the advance of technology, handwritten numeral recognition system is sophisticated and widely used. However, due to the presence of different writing surfaces, postures and other factors, the performance of handwritten numeral recognition is limited. In this paper, we propose a new supervised recurrent neural network, which combines time and space for target location prediction on handwritten datasets. Our method is based on the YOLO framework, and combines a long and short term memory (LSTM) mechanism. Moreover, our method not only locates handwritten images, but also improves the classification accuracy. Extensive comparison with the state-of-the-art methods demonstrates that our method achieves both accuracy and robustness on handwritten datasets. Meanwhile, our method is effective with low computational cost.
X-Tracking: Tracking Human in Masking Surveillance Video

Pedestrian tracking studies have been facilitated by a large amount of surveillance apparatus in the city while also raising public privacy concerns. In this paper, we propose X-Tracking, a privacy-aware pedestrian tracking paradigm designed for vision systems in Smart City. It allows low-cost compatibility with existing surveillance architecture. To protect entities’ privacy, X-Tracking uses video pre-processing with desensitization so that identity information is unexposed to the tracking algorithm. We implement system-level privacy protection by redesigning the tracking framework that decouples all services based on a single responsibility principle. Then, we elaborate on the roles, behaviors, and protocols used in the new system and illustrate how the paradigm strikes a favorable balance between privacy protection and convenience services. Furthermore, we propose a new tracking task that aims to track humans in masking surveillance video. It is comparable to previous tracking tasks but considering the target with a distorted appearance poses new challenges for visual tracking. Finally, we evaluate the baseline algorithm on the task with a demo dataset.