OpenRaiser/Intern-Atlas huggingface dataset:
Spatial Sharing of GPU for Autotuning DNN models
Venue: arXiv cs.NE 2020
Year: 2020

GPUs are used for training, inference, and tuning the machine learning models. However, Deep Neural Network (DNN) vary widely in their ability to exploit the full power of high-performance GPUs. Spatial sharing of GPU enables multiplexing several DNNs on the GPU and can improve GPU utilization, thus improving throughput and lowering latency. DNN models given just the right amount of GPU resources can still provide low inference latency, just as much as dedicating all of the GPU for their inference task. An approach to improve DNN inference is tuning of the DNN model. Autotuning frameworks find the optimal low-level implementation for a certain target device based on the trained machine learning model, thus reducing the DNN's inference latency and increasing inference throughput. We observe an interdependency between the tuned model and its inference latency. A DNN model tuned with specific GPU resources provides the best inference latency when inferred with close to the same amount of GPU resources. While a model tuned with the maximum amount of the GPU's resources has poorer inference latency once the GPU resources are limited for inference. On the other hand, a model tuned with an appropriate amount of GPU resources still achieves good inference latency across a wide range of GPU resource availability. We explore the causes that impact the tuning of a model at different amounts of GPU resources. We present many techniques to maximize resource utilization and improve tuning performance. We enable controlled spatial sharing of GPU to multiplex several tuning applications on the GPU. We scale the tuning server instances and shard the tuning model across multiple client instances for concurrent tuning of different operators of a model, achieving better GPU multiplexing. With our improvements, we decrease DNN autotuning time by up to 75 percent and increase throughput by a factor of 5.
Lights and Shadows in Evolutionary Deep Learning: Taxonomy, Critical Methodological Analysis, Cases of Study, Learned Lessons, Recommendations and Challenges
Venue: arXiv cs.NE 2020
Year: 2020

Much has been said about the fusion of bio-inspired optimization algorithms and Deep Learning models for several purposes: from the discovery of network topologies and hyper-parametric configurations with improved performance for a given task, to the optimization of the model's parameters as a replacement for gradient-based solvers. Indeed, the literature is rich in proposals showcasing the application of assorted nature-inspired approaches for these tasks. In this work we comprehensively review and critically examine contributions made so far based on three axes, each addressing a fundamental question in this research avenue: a) optimization and taxonomy (Why?), including a historical perspective, definitions of optimization problems in Deep Learning, and a taxonomy associated with an in-depth analysis of the literature, b) critical methodological analysis (How?), which together with two case studies, allows us to address learned lessons and recommendations for good practices following the analysis of the literature, and c) challenges and new directions of research (What can be done, and what for?). In summary, three axes - optimization and taxonomy, critical analysis, and challenges - which outline a complete vision of a merger of two technologies drawing up an exciting future for this area of fusion research.
Big Networks: A Survey
Venue: arXiv cs.SI 2020
Year: 2020

A network is a typical expressive form of representing complex systems in terms of vertices and links, in which the pattern of interactions amongst components of the network is intricate. The network can be static that does not change over time or dynamic that evolves through time. The complication of network analysis is different under the new circumstance of network size explosive increasing. In this paper, we introduce a new network science concept called big network. Big networks are generally in large-scale with a complicated and higher-order inner structure. This paper proposes a guideline framework that gives an insight into the major topics in the area of network science from the viewpoint of a big network. We first introduce the structural characteristics of big networks from three levels, which are micro-level, meso-level, and macro-level. We then discuss some state-of-the-art advanced topics of big network analysis. Big network models and related approaches, including ranking methods, partition approaches, as well as network embedding algorithms are systematically introduced. Some typical applications in big networks are then reviewed, such as community detection, link prediction, recommendation, etc. Moreover, we also pinpoint some critical open issues that need to be investigated further.
Random Walks: A Review of Algorithms and Applications
Venue: arXiv cs.SI 2020
Year: 2020

A random walk is known as a random process which describes a path including a succession of random steps in the mathematical space. It has increasingly been popular in various disciplines such as mathematics and computer science. Furthermore, in quantum mechanics, quantum walks can be regarded as quantum analogues of classical random walks. Classical random walks and quantum walks can be used to calculate the proximity between nodes and extract the topology in the network. Various random walk related models can be applied in different fields, which is of great significance to downstream tasks such as link prediction, recommendation, computer vision, semi-supervised learning, and network embedding. In this paper, we aim to provide a comprehensive review of classical random walks and quantum walks. We first review the knowledge of classical random walks and quantum walks, including basic concepts and some typical algorithms. We also compare the algorithms based on quantum walks and classical random walks from the perspective of time complexity. Then we introduce their applications in the field of computer science. Finally we discuss the open issues from the perspectives of efficiency, main-memory volume, and computing time of existing algorithms. This study aims to contribute to this growing area of research by exploring random walks and quantum walks together.
Fundamental Limits of Caching for Demand Privacy against Colluding Users
Venue: arXiv cs.IT 2020
Year: 2020

This work investigates the problem of demand privacy against colluding users for shared-link coded caching systems, where no subset of users can learn any information about the demands of the remaining users. The notion of privacy used here is stronger than similar notions adopted in past work and is motivated by the practical need to insure privacy regardless of the file distribution. Two scenarios are considered: Single File Retrieval (SFR) and Linear Function Retrieval (LFR), where in the latter case each user demands an arbitrary linear combination of the files at the server. The main contributions of this paper are a novel achievable scheme for LFR, referred as privacy key scheme, and a new information theoretic converse bound for SFR. Clearly, being SFR a special case of LFR, an achievable scheme for LFR works for SFR as well, and a converse for SFR is a valid converse for LFR as well. By comparing the performance of the achievable scheme with the converse bound derived in this paper (for the small cache size regime) and existing converse bounds without privacy constraints (in the remaining memory regime), the communication load of the privacy key scheme turns out to be optimal to within a constant multiplicative gap in all parameter regimes. Numerical results show that the new privacy key scheme outperforms in some regime known schemes based on the idea of virtual users, which also satisfy the stronger notion of user privacy against colluding users adopted here. Moreover, the privacy key scheme enjoys much lower subpacketization than known schemes based on virtual users.
New Constructions of MDS Twisted Reed-Solomon Codes and LCD MDS Codes
Venue: arXiv cs.IT 2020
Year: 2020

Maximum distance separable (MDS) codes are optimal where the minimum distance cannot be improved for a given length and code size. Twisted Reed-Solomon codes over finite fields were introduced in 2017, which are generalization of Reed-Solomon codes. Twisted Reed-Solomon codes can be applied in cryptography which prefer the codes with large minimum distance. MDS codes can be constructed from twisted Reed-Solomon codes, and most of them are not equivalent to Reed-Solomon codes. In this paper, we first generalize twisted Reed-Solomon codes to generalized twisted Reed-Solomon codes, then we give some new explicit constructions of MDS (generalized) twisted Reed-Solomon codes. In some cases, our constructions can get MDS codes with the length longer than the constructions of previous works. Linear complementary dual (LCD) codes are linear codes that intersect with their duals trivially. LCD codes can be applied in cryptography. This application of LCD codes renewed the interest in the construction of LCD codes having a large minimum distance. We also provide new constructions of LCD MDS codes from generalized twisted Reed-Solomon codes.
Agricultural Knowledge Management Using Smart Voice Messaging Systems: Combination of Physical and Human Sensors
Venue: arXiv cs.CY 2020
Year: 2020

The use of the Internet of Things (IoT) in agricultural knowledge management systems is one of the most promising approaches to increasing the efficiency of agriculture. However, the existing physical sensors in agriculture are limited for monitoring various changes in the characteristics of crops and may be expensive for the average farmer. We propose a combination of physical and human sensors (the five human senses). By using their own eyes, ears, noses, tongues, and fingers, farmers could check the various changes in the characteristics and conditions (colors of leaves, diseases, pests, faulty or malfunctioning equipment) of their crops and equipment, verbally describe their observations, and capture the descriptions with audio recording devices, such as smartphones. The voice recordings could be transcribed into text by web servers. The data captured by the physical and human sensors (voice messages) are analyzed by data and text mining to create and improve agricultural knowledge. An agricultural knowledge management system using physical and human sensors encourages to share and transfer knowledge among farmers for the purpose of improving the efficiency and productivity of agriculture. We applied one such agricultural knowledge management system (smart voice messaging system) to a greenhouse vegetable farm in Hokkaido. A qualitative analysis of accumulated voice messages and an interview with the farmer demonstrated the effectiveness of this system. The contributions of this study include a new and practical approach to an "agricultural Internet of Everything (IoE)" and evidence of its effectiveness as a result of our trial experiment at a real vegetable farm.
SEALing Neural Network Models in Secure Deep Learning Accelerators
Venue: arXiv cs.AR 2020
Year: 2020

Deep learning (DL) accelerators are increasingly deployed on edge devices to support fast local inferences. However, they suffer from a new security problem, i.e., being vulnerable to physical access based attacks. An adversary can easily obtain the entire neural network (NN) model by physically snooping the GDDR memory bus that connects the accelerator chip with DRAM memory. Therefore, memory encryption becomes important for DL accelerators on edge devices to improve the security of NN models. Nevertheless, we observe that traditional memory encryption solutions that have been efficiently used in CPU systems cause significant performance degradation when directly used in DL accelerators. The main reason comes from the big bandwidth gap between the GDDR memory bus and the encryption engine. To address this problem, our paper proposes SEAL, a Secure and Efficient Accelerator scheme for deep Learning. SEAL enhances the performance of the encrypted DL accelerator from two aspects, i.e., improving the data access bandwidth and the efficiency of memory encryption. Specifically, to improve the data access bandwidth, SEAL leverages a criticality-aware smart encryption scheme which identifies partial data that have no impact on the security of NN models and allows them to bypass the encryption engine, thus reducing the amount of data to be encrypted. To improve the efficiency of memory encryption, SEAL leverages a colocation mode encryption scheme to eliminate memory accesses from counters used for encryption by co-locating data and their counters. Our experimental results demonstrate that, compared with traditional memory encryption solutions, SEAL achieves 1.4 ~ 1.6 times IPC improvement and reduces the inference latency by 39% ~ 60%. Compared with a baseline accelerator without memory encryption, SEAL compromises only 5% ~ 7% IPC for significant security improvement.
Tactics for Internal Compliance: A Literature Review
Venue: arXiv cs.CY 2020
Year: 2020

Compliance of organizations with internal and external norms is a highly relevant topic for both practitioners and academics nowadays. However, the substantive, elementary compliance tactics that organizations can use for achieving internal compliance have been described in a fragmented manner and in the literatures of distinct academic disciplines. Using a multidisciplinary structured literature review of 134 publications, this study offers three contributions. First, we present a typology of 45 compliance tactics, which constitutes a comprehensive and rich overview of elementary ways for bringing the organization into compliance. Secondly, we provide an overview of fundamental concepts in the theory of compliance, which forms the basis for the framework we developed for positioning compliance tactics and for analyzing or developing compliance strategies. Thirdly, we present insights for moving from compliance tactics to compliance strategies. In the process, and using the multidisciplinary literature review to take a bird's-eye view, we demonstrate that compliance strategies need to be regarded as a richer concept than perceived hitherto. We also show that opportunities for innovation exist.
A Flow-Guided Mutual Attention Network for Video-Based Person Re-Identification
Venue: arXiv cs.CV 2020
Year: 2020

Person Re-Identification (ReID) is a challenging problem in many video analytics and surveillance applications, where a person's identity must be associated across a distributed non-overlapping network of cameras. Video-based person ReID has recently gained much interest because it allows capturing discriminant spatio-temporal information from video clips that is unavailable for image-based ReID. Despite recent advances, deep learning (DL) models for video ReID often fail to leverage this information to improve the robustness of feature representations. In this paper, the motion pattern of a person is explored as an additional cue for ReID. In particular, a flow-guided Mutual Attention network is proposed for fusion of image and optical flow sequences using any 2D-CNN backbone, allowing to encode temporal information along with spatial appearance information. Our Mutual Attention network relies on the joint spatial attention between image and optical flow features maps to activate a common set of salient features across them. In addition to flow-guided attention, we introduce a method to aggregate features from longer input streams for better video sequence-level representation. Our extensive experiments on three challenging video ReID datasets indicate that using the proposed Mutual Attention network allows to improve recognition accuracy considerably with respect to conventional gated-attention networks, and state-of-the-art methods for video-based person ReID.
Anxiety Detection Leveraging Mobile Passive Sensing
Venue: arXiv cs.CY 2020
Year: 2020

Anxiety disorders are the most common class of psychiatric problems affecting both children and adults. However, tools to effectively monitor and manage anxiety are lacking, and comparatively limited research has been applied to addressing the unique challenges around anxiety. Leveraging passive and unobtrusive data collection from smartphones could be a viable alternative to classical methods, allowing for real-time mental health surveillance and disease management. This paper presents eWellness, an experimental mobile application designed to track a full-suite of sensor and user-log data off an individual's device in a continuous and passive manner. We report on an initial pilot study tracking ten people over the course of a month that showed a nearly 76% success rate at predicting daily anxiety and depression levels based solely on the passively monitored features.
An Overview on Evaluating and Predicting Scholarly Article Impact
Venue: arXiv cs.DL 2020
Year: 2020

Scholarly article impact reflects the significance of academic output recognised by academic peers, and it often plays a crucial role in assessing the scientific achievements of researchers, teams, institutions and countries. It is also used for addressing various needs in the academic and scientific arena, such as recruitment decisions, promotions, and funding allocations. This article provides a comprehensive review of recent progresses related to article impact assessment and prediction. The~review starts by sharing some insight into the article impact research and outlines current research status. Some core methods and recent progress are presented to outline how article impact metrics and prediction have evolved to consider integrating multiple networks. Key techniques, including statistical analysis, machine learning, data mining and network science, are discussed. In particular, we highlight important applications of each technique in article impact research. Subsequently, we discuss the open issues and challenges of article impact research. At the same time, this review points out some important research directions, including article impact evaluation by considering Conflict of Interest, time and location information, various distributions of scholarly entities, and rising stars.
Behavioral Modeling of Persian Instagram Users to detect Bots
Venue: arXiv cs.SI 2020
Year: 2020

Bots are user accounts in social media which are controlled by computer programs. Similar to many other things, they are used for both good and evil purposes. One nefarious use-case for them is to spread misinformation or biased data in the networks. There are many pieces of research being performed based on social media data and their results validity is extremely threatened by the harmful data bots spread. Consequently, effective methods and tools are required for detecting bots and then removing misleading data spread by the bots. In the present research, a method for detecting Instagram bots is proposed. There is no data set including samples of Instagram bots and genuine accounts, thus the current research has begun with gathering such a data set with respect to generality concerns such that it includes 1,000 data points in each group. The main approach is supervised machine learning and classic models are preferred compared to deep neural networks. The final model is evaluated using multiple methods starting with 10-fold cross-validation. After that, confidence in classification studies and is followed by feature importance analysis and feature behavior against the target probability computed by the model. In the end, an experiment is designed to measure the models effectiveness in an operational environment. Finally, It is strongly concluded that the model performs very well in all evaluation experiments.
DQI: A Guide to Benchmark Evaluation
Venue: arXiv cs.CL 2020
Year: 2020

A `state of the art' model A surpasses humans in a benchmark B, but fails on similar benchmarks C, D, and E. What does B have that the other benchmarks do not? Recent research provides the answer: spurious bias. However, developing A to solve benchmarks B through E does not guarantee that it will solve future benchmarks. To progress towards a model that `truly learns' an underlying task, we need to quantify the differences between successive benchmarks, as opposed to existing binary and black-box approaches. We propose a novel approach to solve this underexplored task of quantifying benchmark quality by debuting a data quality metric: DQI.
KR-BERT: A Small-Scale Korean-Specific Language Model
Venue: arXiv cs.CL 2020
Year: 2020

Since the appearance of BERT, recent works including XLNet and RoBERTa utilize sentence embedding models pre-trained by large corpora and a large number of parameters. Because such models have large hardware and a huge amount of data, they take a long time to pre-train. Therefore it is important to attempt to make smaller models that perform comparatively. In this paper, we trained a Korean-specific model KR-BERT, utilizing a smaller vocabulary and dataset. Since Korean is one of the morphologically rich languages with poor resources using non-Latin alphabets, it is also important to capture language-specific linguistic phenomena that the Multilingual BERT model missed. We tested several tokenizers including our BidirectionalWordPiece Tokenizer and adjusted the minimal span of tokens for tokenization ranging from sub-character level to character-level to construct a better vocabulary for our model. With those adjustments, our KR-BERT model performed comparably and even better than other existing pre-trained models using a corpus about 1/10 of the size.
2nd Place Scheme on Action Recognition Track of ECCV 2020 VIPriors Challenges: An Efficient Optical Flow Stream Guided Framework
Venue: arXiv cs.CV 2020
Year: 2020

To address the problem of training on small datasets for action recognition tasks, most prior works are either based on a large number of training samples or require pre-trained models transferred from other large datasets to tackle overfitting problems. However, it limits the research within organizations that have strong computational abilities. In this work, we try to propose a data-efficient framework that can train the model from scratch on small datasets while achieving promising results. Specifically, by introducing a 3D central difference convolution operation, we proposed a novel C3D neural network-based two-stream (Rank Pooling RGB and Optical Flow) framework for the task. The method is validated on the action recognition track of the ECCV 2020 VIPriors challenges and got the 2nd place (88.31%). It is proved that our method can achieve a promising result even without a pre-trained model on large scale datasets. The code will be released soon.