OpenRaiser/Intern-Atlas huggingface dataset:
Layer-specific Optimization for Mixed Data Flow with Mixed Precision in FPGA Design for CNN-based Object Detectors
Venue: arXiv cs.CV 2020
Year: 2020

Convolutional neural networks (CNNs) require both intensive computation and frequent memory access, which lead to a low processing speed and large power dissipation. Although the characteristics of the different layers in a CNN are frequently quite different, previous hardware designs have employed common optimization schemes for them. This paper proposes a layer-specific design that employs different organizations that are optimized for the different layers. The proposed design employs two layer-specific optimizations: layer-specific mixed data flow and layer-specific mixed precision. The mixed data flow aims to minimize the off-chip access while demanding a minimal on-chip memory (BRAM) resource of an FPGA device. The mixed precision quantization is to achieve both a lossless accuracy and an aggressive model compression, thereby further reducing the off-chip access. A Bayesian optimization approach is used to select the best sparsity for each layer, achieving the best trade-off between the accuracy and compression. This mixing scheme allows the entire network model to be stored in BRAMs of the FPGA to aggressively reduce the off-chip access, and thereby achieves a significant performance enhancement. The model size is reduced by 22.66-28.93 times compared to that in a full-precision network with a negligible degradation of accuracy on VOC, COCO, and ImageNet datasets. Furthermore, the combination of mixed dataflow and mixed precision significantly outperforms the previous works in terms of both throughput, off-chip access, and on-chip memory requirement.
Data Series Indexing Gone Parallel
Venue: arXiv cs.DB 2020
Year: 2020

Data series similarity search is a core operation for several data series analysis applications across many different domains. However, the state-of-the-art techniques fail to deliver the time performance required for interactive exploration, or analysis of large data series collections. In this Ph.D. work, we present the first data series indexing solutions, for both on-disk and in-memory data, that are designed to inherently take advantage of multi-core architectures, in order to accelerate similarity search processing times. Our experiments on a variety of synthetic and real data demonstrate that our approaches are up to orders of magnitude faster than the alternatives. More specifically, our on-disk solution can answer exact similarity search queries on 100GB datasets in a few seconds, and our in-memory solution in a few milliseconds, which enables real-time, interactive data exploration on very large data series collections.
Few-shot Object Detection with Feature Attention Highlight Module in Remote Sensing Images
Venue: arXiv cs.CV 2020
Year: 2020

In recent years, there are many applications of object detection in remote sensing field, which demands a great number of labeled data. However, in many cases, data is extremely rare. In this paper, we proposed a few-shot object detector which is designed for detecting novel objects based on only a few examples. Through fully leveraging labeled base classes, our model that is composed of a feature-extractor, a feature attention highlight module as well as a two-stage detection backend can quickly adapt to novel classes. The pre-trained feature extractor whose parameters are shared produces general features. While the feature attention highlight module is designed to be light-weighted and simple in order to fit the few-shot cases. Although it is simple, the information provided by it in a serial way is helpful to make the general features to be specific for few-shot objects. Then the object-specific features are delivered to the two-stage detection backend for the detection results. The experiments demonstrate the effectiveness of the proposed method for few-shot cases.
Optimal Wireless Streaming of Multi-Quality 360 VR Video by Exploiting Natural, Relative Smoothness-enabled and Transcoding-enabled Multicast Opportunities
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, we would like to investigate optimal wireless streaming of a multi-quality tiled 360 virtual reality (VR) video from a server to multiple users. To this end, we propose to maximally exploit potential multicast opportunities by effectively utilizing characteristics of multi-quality tiled 360 VR videos and computation resources at the users' side. In particular, we consider two requirements for quality variation in one field-of-view (FoV), i.e., the absolute smoothness requirement and the relative smoothness requirement, and two video playback modes, i.e., the direct-playback mode (without user transcoding) and transcode-playback mode (with user transcoding). Besides natural multicast opportunities, we introduce two new types of multicast opportunities, namely, relative smoothness-enabled multicast opportunities, which allow flexible tradeoff between viewing quality and communications resource consumption, and transcoding-enabled multicast opportunities, which allow flexible tradeoff between computation and communications resource consumptions. Then, we establish a novel mathematical model that reflects the impacts of natural, relative smoothness-enabled and transcoding-enabled multicast opportunities on the average transmission energy and transcoding energy. Based on this model, we optimize the transmission resource allocation, playback quality level selection and transmission quality level selection to minimize the energy consumption in the four cases with different requirements for quality variation and video playback modes. By comparing the optimal values in the four cases, we prove that the energy consumption reduces when more multicast opportunities can be utilized. Finally, numerical results show substantial gains of the proposed solutions over existing schemes, and demonstrate the importance of effective exploitation of the three types of multicast opportunities.
A Predictive Strategy for the Iterated Prisoner's Dilemma
Venue: arXiv cs.GT 2020
Year: 2020

The iterated prisoner's dilemma is a game that produces many counter-intuitive and complex behaviors in a social environment, based on very simple basic rules. It illustrates that cooperation can be a good thing even in a competitive world, that individual fitness needs not to be the most important criteria of success, and that some strategies are very strong in a direct confrontation but could still perform poorly on average or are evolutionarily unstable. In this contribution, we present a strategy -- PREDICTOR -- which appears to be "sentient" and chooses to cooperate when playing against some strategies, but defects when playing against others, without the need to record "tags" for its opponents or an involved decision-making mechanism. To be able to operate in the highly-contextual environment, as modeled by the iterated prisoner's dilemma, PREDICTOR learns from its experience to choose optimal actions by modeling its opponent and predicting a (fictive) future. It is shown that PREDICTOR is an efficient strategy for playing the iterated prisoner's dilemma and is simple to implement. In a simulated and representative tournament, it achieves high average scores and wins the tournament for various parameter settings. PREDICTOR thereby relies on a brief phase of exploration to improve its model, and it can evolve morality from intrinsically selfish behavior.
CAGNN: Cluster-Aware Graph Neural Networks for Unsupervised Graph Representation Learning
Venue: arXiv cs.LG 2020
Year: 2020

Unsupervised graph representation learning aims to learn low-dimensional node embeddings without supervision while preserving graph topological structures and node attributive features. Previous graph neural networks (GNN) require a large number of labeled nodes, which may not be accessible in real-world graph data. In this paper, we present a novel cluster-aware graph neural network (CAGNN) model for unsupervised graph representation learning using self-supervised techniques. In CAGNN, we perform clustering on the node embeddings and update the model parameters by predicting the cluster assignments. Moreover, we observe that graphs often contain inter-class edges, which mislead the GNN model to aggregate noisy information from neighborhood nodes. We further refine the graph topology by strengthening intra-class edges and reducing node connections between different classes based on cluster labels, which better preserves cluster structures in the embedding space. We conduct comprehensive experiments on two benchmark tasks using real-world datasets. The results demonstrate the superior performance of the proposed model over existing baseline methods. Notably, our model gains over 7% improvements in terms of accuracy on node clustering over state-of-the-arts.
Multi-Loss Weighting with Coefficient of Variations
Venue: arXiv cs.CV 2020
Year: 2020

Many interesting tasks in machine learning and computer vision are learned by optimising an objective function defined as a weighted linear combination of multiple losses. The final performance is sensitive to choosing the correct (relative) weights for these losses. Finding a good set of weights is often done by adopting them into the set of hyper-parameters, which are set using an extensive grid search. This is computationally expensive. In this paper, we propose a weighting scheme based on the coefficient of variations and set the weights based on properties observed while training the model. The proposed method incorporates a measure of uncertainty to balance the losses, and as a result the loss weights evolve during training without requiring another (learning based) optimisation. In contrast to many loss weighting methods in literature, we focus on single-task multi-loss problems, such as monocular depth estimation and semantic segmentation, and show that multi-task approaches for loss weighting do not work on those single-tasks. The validity of the approach is shown empirically for depth estimation and semantic segmentation on multiple datasets.
Synthetic-to-Real Unsupervised Domain Adaptation for Scene Text Detection in the Wild
Venue: arXiv cs.CV 2020
Year: 2020

Deep learning-based scene text detection can achieve preferable performance, powered with sufficient labeled training data. However, manual labeling is time consuming and laborious. At the extreme, the corresponding annotated data are unavailable. Exploiting synthetic data is a very promising solution except for domain distribution mismatches between synthetic datasets and real datasets. To address the severe domain distribution mismatch, we propose a synthetic-to-real domain adaptation method for scene text detection, which transfers knowledge from synthetic data (source domain) to real data (target domain). In this paper, a text self-training (TST) method and adversarial text instance alignment (ATA) for domain adaptive scene text detection are introduced. ATA helps the network learn domain-invariant features by training a domain classifier in an adversarial manner. TST diminishes the adverse effects of false positives~(FPs) and false negatives~(FNs) from inaccurate pseudo-labels. Two components have positive effects on improving the performance of scene text detectors when adapting from synthetic-to-real scenes. We evaluate the proposed method by transferring from SynthText, VISD to ICDAR2015, ICDAR2013. The results demonstrate the effectiveness of the proposed method with up to 10% improvement, which has important exploration significance for domain adaptive scene text detection. Code is available at https://github.com/weijiawu/SyntoReal_STD
SEDRo: A Simulated Environment for Developmental Robotics
Venue: arXiv cs.AI 2020
Year: 2020

Even with impressive advances in application-specific models, we still lack knowledge about how to build a model that can learn in a human-like way and do multiple tasks. To learn in a human-like way, we need to provide a diverse experience that is comparable to humans. In this paper, we introduce our ongoing effort to build a simulated environment for developmental robotics (SEDRo). SEDRo provides diverse human experiences ranging from those of a fetus to a 12th-month-old. A series of simulated tests based on developmental psychology will be used to evaluate the progress of a learning model. We anticipate SEDRo to lower the cost of entry and facilitate research in the developmental robotics community.
Should a small robot have a small personal space? Investigating personal spatial zones and proxemic behavior in human-robot interaction
Venue: arXiv cs.HC 2020
Year: 2020

This paper presents the first study in a series of proxemics experiments concerned with the role of personal spatial zones in human-robot interaction. In the study 40 participants approached a NAO robot positioned approximately at participants' eye level and entered different social zones around the robot (personal and intimate space). When the robot perceived the approaching person entering its personal space, it started gazing at the participant, and upon the intrusion of its intimate space it leaned back. Our research questions were: (1) given the small size of the robot (58 cm tall), will people expect its social zones to shrink by its size? (2) Will the robot behaviors be interpreted as appropriate social behaviors? We found that the average approach distance of the participants was 48 cm, which represents the inner limit of the human-size personal zone (45-120 cm), but is outside of the personal zone scaled to robot size (16-42 cm). This suggests that most participants did not (fully) scale down the extent of these zones to the robot size. We also found that the leaning back behavior of the robot was correctly interpreted by most participants as the robot's reaction to the intrusion of its personal space; however, our implementation of the behavior was often perceived as "unfriendly". We will discuss this and other limitations of the study in detail. Additionally we found positive correlations between participants' personality traits, Godspeed Questionnaire subscales, and the average approach distance. The technical contribution of this work is the real-time perception of 25 keypoints on the human body using a single compact RGB-D camera and the use of these points for accurate interpersonal distance estimation and as gazing targets for the robot.
Developing Enterprise Cyber Situational Awareness
Venue: arXiv cs.CY 2020
Year: 2020

The topic will focus on the U.S. Department of Defense strategy towards improving their network security defenses for the department and the steps they have taken at the agency level where components under DOD such as The Defense Information Systems Agency are working towards adding tools that provides additional capabilities in the cyber space. This approach will be analyzed to determine if DOD goals address any of their vulnerabilities towards protecting their networks. One of the agencies under the DOD umbrella called The Defense Information Systems Agency provides DOD a template on how to build a network that relies upon layers of security to help it combat cyber attacks against its network. Whether that provides an effective solution to DOD remains a question due to the many components that operate under its direction. Managing these networks is the principle responsibilities for the Department of Defense. Nevertheless, it does demonstrates that there are tools available to help DOD build an strong enterprise cyber network of situational awareness that strengthens the ability to protect their network infrastructure.
A Haystack Full of Needles: Scalable Detection of IoT Devices in the Wild
Venue: arXiv cs.NI 2020
Year: 2020

Consumer Internet of Things (IoT) devices are extremely popular, providing users with rich and diverse functionalities, from voice assistants to home appliances. These functionalities often come with significant privacy and security risks, with notable recent large scale coordinated global attacks disrupting large service providers. Thus, an important first step to address these risks is to know what IoT devices are where in a network. While some limited solutions exist, a key question is whether device discovery can be done by Internet service providers that only see sampled flow statistics. In particular, it is challenging for an ISP to efficiently and effectively track and trace activity from IoT devices deployed by its millions of subscribers --all with sampled network data. In this paper, we develop and evaluate a scalable methodology to accurately detect and monitor IoT devices at subscriber lines with limited, highly sampled data in-the-wild. Our findings indicate that millions of IoT devices are detectable and identifiable within hours, both at a major ISP as well as an IXP, using passive, sparsely sampled network flow headers. Our methodology is able to detect devices from more than 77% of the studied IoT manufacturers, including popular devices such as smart speakers. While our methodology is effective for providing network analytics, it also highlights significant privacy consequences.
Model extraction from counterfactual explanations
Venue: arXiv cs.LG 2020
Year: 2020

Post-hoc explanation techniques refer to a posteriori methods that can be used to explain how black-box machine learning models produce their outcomes. Among post-hoc explanation techniques, counterfactual explanations are becoming one of the most popular methods to achieve this objective. In particular, in addition to highlighting the most important features used by the black-box model, they provide users with actionable explanations in the form of data instances that would have received a different outcome. Nonetheless, by doing so, they also leak non-trivial information about the model itself, which raises privacy issues. In this work, we demonstrate how an adversary can leverage the information provided by counterfactual explanations to build high-fidelity and high-accuracy model extraction attacks. More precisely, our attack enables the adversary to build a faithful copy of a target model by accessing its counterfactual explanations. The empirical evaluation of the proposed attack on black-box models trained on real-world datasets demonstrates that they can achieve high-fidelity and high-accuracy extraction even under low query budgets.
Improving the Usability of Virtual Reality Neuron Tracing with Topological Elements
Venue: arXiv cs.GR 2020
Year: 2020

Researchers in the field of connectomics are working to reconstruct a map of neural connections in the brain in order to understand at a fundamental level how the brain processes information. Constructing this wiring diagram is done by tracing neurons through high-resolution image stacks acquired with fluorescence microscopy imaging techniques. While a large number of automatic tracing algorithms have been proposed, these frequently rely on local features in the data and fail on noisy data or ambiguous cases, requiring time-consuming manual correction. As a result, manual and semi-automatic tracing methods remain the state-of-the-art for creating accurate neuron reconstructions. We propose a new semi-automatic method that uses topological features to guide users in tracing neurons and integrate this method within a virtual reality (VR) framework previously used for manual tracing. Our approach augments both visualization and interaction with topological elements, allowing rapid understanding and tracing of complex morphologies. In our pilot study, neuroscientists demonstrated a strong preference for using our tool over prior approaches, reported less fatigue during tracing, and commended the ability to better understand possible paths and alternatives. Quantitative evaluation of the traces reveals that users' tracing speed increased, while retaining similar accuracy compared to a fully manual approach.
Multi-Perspective Semantic Information Retrieval
Venue: arXiv cs.IR 2020
Year: 2020

Information Retrieval (IR) is the task of obtaining pieces of data (such as documents or snippets of text) that are relevant to a particular query or need from a large repository of information. While a combination of traditional keyword- and modern BERT-based approaches have been shown to be effective in recent work, there are often nuances in identifying what information is "relevant" to a particular query, which can be difficult to properly capture using these systems. This work introduces the concept of a Multi-Perspective IR system, a novel methodology that combines multiple deep learning and traditional IR models to better predict the relevance of a query-sentence pair, along with a standardized framework for tuning this system. This work is evaluated on the BioASQ Biomedical IR + QA challenges.
Compression-aware Continual Learning using Singular Value Decomposition
Venue: arXiv cs.CV 2020
Year: 2020

We propose a compression based continual task learning method that can dynamically grow a neural network. Inspired from the recent model compression techniques, we employ compression-aware training and perform low-rank weight approximations using singular value decomposition (SVD) to achieve network compaction. By encouraging the network to learn low-rank weight filters, our method achieves compressed representations with minimal performance degradation without the need for costly fine-tuning. Specifically, we decompose the weight filters using SVD and train the network on incremental tasks in its factorized form. Such a factorization allows us to directly impose sparsity-inducing regularizers over the singular values and allows us to use fewer number of parameters for each task. We further introduce a novel shared representational space based learning between tasks. This promotes the incoming tasks to only learn residual task-specific information on top of the previously learnt weight filters and greatly helps in learning under fixed capacity constraints. Our method significantly outperforms prior continual learning approaches on three benchmark datasets, demonstrating accuracy improvements of 10.3%, 12.3%, 15.6% on 20-split CIFAR-100, miniImageNet and a 5-sequence dataset, respectively, over state-of-the-art. Further, our method yields compressed models that have ~3.64x, 2.88x, 5.91x fewer number of parameters respectively, on the above mentioned datasets in comparison to baseline individual task models. Our source code is available at https://github.com/pavanteja295/CACL.
Multimodal Autonomous Last Mile Delivery System Design and Application
Venue: arXiv cs.CY 2020
Year: 2020

With the rapid increase in congestion, alternative solutions are needed to efficiently use the capacity of our existing networks. This paper focuses on exploring the emerging autonomous technologies for on-demand food delivery in congested urban cities. Three different last mile food delivery systems are proposed in this study employing aerial and ground autonomous vehicles technologies. The three proposed systems are: robot delivery system, drone delivery system and a hybrid delivery system. In the hybrid system the concept of hub-and-spoke network is explored in order to consolidate orders and reach more destinations in less time. To investigate the performance of the three proposed delivery systems, they are applied to the city of Mississauga network, in an in-house agent-based simulation in MATLAB. 18 Scenarios are tested differing in terms of demand and fleet size. The results show that the hybrid robot-drone delivery system performs the best with a fleet side of 25 robots and 15 drones and with an average preparation and delivery time less than the individual robot and drone system by 48% and 42% respectively.