OpenRaiser/Intern-Atlas huggingface dataset:
Internet of things: a multiprotocol gateway as solution of the interoperability problem
Venue: arXiv cs.NI 2021
Year: 2021

One of the main challenges of the Internet of Things is the interoperability of highly heterogeneous devices, mainly in terms of the communication capabilities and network protocols used. As consequence, the interconnection model of the different devices involves an intermediary device, known as gateway. This gateway is a centralized element for the management of the devices that make up an IoT application. In addition, it is essential for the transmission of information to the Internet, especially when many IoT devices are not IP-based. This chapter describes a proposed model for an IoT gateway that allows the exchange of data through different wireless technologies and forwarding of such data to the Internet. The proposed gateway has important advantages such as: supporting for multiprotocol interconnectivity, the remote configuration of wireless nodes for sensor and actuators management; a flexible algorithm totranslate the data obtained by sensors into a uniform format for transmission to a cloud server; low energy consumption due to efficient data transfer over MQTT protocol. In order to demonstrate the usefulness of the developed gateway, a proof of concept test was implemented. The implemented scenario consists of 2 wireless nodes responsible for sensing environmental variables and transmitting data to the gateway node through different communication protocols. The obtained results show the feasibility for simultaneous data transmission from the remote wireless nodes to the gateway. Metrics on energy consumption in the devices are also presented.
On The State of Data In Computer Vision: Human Annotations Remain Indispensable for Developing Deep Learning Models
Venue: arXiv cs.CV 2021
Year: 2021

High-quality labeled datasets play a crucial role in fueling the development of machine learning (ML), and in particular the development of deep learning (DL). However, since the emergence of the ImageNet dataset and the AlexNet model in 2012, the size of new open-source labeled vision datasets has remained roughly constant. Consequently, only a minority of publications in the computer vision community tackle supervised learning on datasets that are orders of magnitude larger than Imagenet. In this paper, we survey computer vision research domains that study the effects of such large datasets on model performance across different vision tasks. We summarize the community's current understanding of those effects, and highlight some open questions related to training with massive datasets. In particular, we tackle: (a) The largest datasets currently used in computer vision research and the interesting takeaways from training on such datasets; (b) The effectiveness of pre-training on large datasets; (c) Recent advancements and hurdles facing synthetic datasets; (d) An overview of double descent and sample non-monotonicity phenomena; and finally, (e) A brief discussion of lifelong/continual learning and how it fares compared to learning from huge labeled datasets in an offline setting. Overall, our findings are that research on optimization for deep learning focuses on perfecting the training routine and thus making DL models less data hungry, while research on synthetic datasets aims to offset the cost of data labeling. However, for the time being, acquiring non-synthetic labeled data remains indispensable to boost performance.
Greedy Network Enlarging
Venue: arXiv cs.CV 2021
Year: 2021

Recent studies on deep convolutional neural networks present a simple paradigm of architecture design, i.e., models with more MACs typically achieve better accuracy, such as EfficientNet and RegNet. These works try to enlarge all the stages in the model with one unified rule by sampling and statistical methods. However, we observe that some network architectures have similar MACs and accuracies, but their allocations on computations for different stages are quite different. In this paper, we propose to enlarge the capacity of CNN models by improving their width, depth and resolution on stage level. Under the assumption that the top-performing smaller CNNs are a proper subcomponent of the top-performing larger CNNs, we propose an greedy network enlarging method based on the reallocation of computations. With step-by-step modifying the computations on different stages, the enlarged network will be equipped with optimal allocation and utilization of MACs. On EfficientNet, our method consistently outperforms the performance of the original scaling method. In particular, with application of our method on GhostNet, we achieve state-of-the-art 80.9% and 84.3% ImageNet top-1 accuracies under the setting of 600M and 4.4B MACs, respectively.
Adversarial Robustness of Deep Code Comment Generation
Venue: arXiv cs.SE 2021
Year: 2021

Deep neural networks (DNNs) have shown remarkable performance in a variety of domains such as computer vision, speech recognition, or natural language processing. Recently they also have been applied to various software engineering tasks, typically involving processing source code. DNNs are well-known to be vulnerable to adversarial examples, i.e., fabricated inputs that could lead to various misbehaviors of the DNN model while being perceived as benign by humans. In this paper, we focus on the code comment generation task in software engineering and study the robustness issue of the DNNs when they are applied to this task. We propose ACCENT, an identifier substitution approach to craft adversarial code snippets, which are syntactically correct and semantically close to the original code snippet, but may mislead the DNNs to produce completely irrelevant code comments. In order to improve the robustness, ACCENT also incorporates a novel training method, which can be applied to existing code comment generation models. We conduct comprehensive experiments to evaluate our approach by attacking the mainstream encoder-decoder architectures on two large-scale publicly available datasets. The results show that ACCENT efficiently produces stable attacks with functionality-preserving adversarial examples, and the generated examples have better transferability compared with baselines. We also confirm, via experiments, the effectiveness in improving model robustness with our training method.
Debiasing Samples from Online Learning Using Bootstrap
Venue: arXiv cs.LG 2021
Year: 2021

It has been recently shown in the literature that the sample averages from online learning experiments are biased when used to estimate the mean reward. To correct the bias, off-policy evaluation methods, including importance sampling and doubly robust estimators, typically calculate the conditional propensity score, which is ill-defined for non-randomized policies such as UCB. This paper provides a procedure to debias the samples using bootstrap, which doesn't require the knowledge of the reward distribution and can be applied to any adaptive policies. Numerical experiments demonstrate the effective bias reduction for samples generated by popular multi-armed bandit algorithms such as Explore-Then-Commit (ETC), UCB, Thompson sampling (TS) and $ε$-greedy (EG). We analyze and provide theoretical justifications for the procedure under the ETC algorithm, including the asymptotic convergence of the bias decay rate in the real and bootstrap worlds.
HR-Crime: Human-Related Anomaly Detection in Surveillance Videos
Venue: arXiv cs.CV 2021
Year: 2021

The automatic detection of anomalies captured by surveillance settings is essential for speeding the otherwise laborious approach. To date, UCF-Crime is the largest available dataset for automatic visual analysis of anomalies and consists of real-world crime scenes of various categories. In this paper, we introduce HR-Crime, a subset of the UCF-Crime dataset suitable for human-related anomaly detection tasks. We rely on state-of-the-art techniques to build the feature extraction pipeline for human-related anomaly detection. Furthermore, we present the baseline anomaly detection analysis on the HR-Crime. HR-Crime as well as the developed feature extraction pipeline and the extracted features will be publicly available for further research in the field.
Enhanced Regular Corecursion for Data Streams
Venue: arXiv cs.PL 2021
Year: 2021

We propose a simple calculus for processing data streams (infinite flows of data series), represented by finite sets of equations built on stream operators. Furthermore, functions defining streams are regularly corecursive, that is, cyclic calls are detected, avoiding non-termination as happens with ordinary recursion in the call-by-value evaluation strategy. As we illustrate by several examples, the combination of such two mechanisms provides a good compromise between expressive power and decidability. Notably, we provide an algorithm to check that the stream returned by a function call is represented by a well-formed set of equations which actually admits a unique solution, hence access to an arbitrary element of the returned stream will never diverge.
Human Evaluation of Creative NLG Systems: An Interdisciplinary Survey on Recent Papers
Venue: arXiv cs.CL 2021
Year: 2021

We survey human evaluation in papers presenting work on creative natural language generation that have been published in INLG 2020 and ICCC 2020. The most typical human evaluation method is a scaled survey, typically on a 5 point scale, while many other less common methods exist. The most commonly evaluated parameters are meaning, syntactic correctness, novelty, relevance and emotional value, among many others. Our guidelines for future evaluation include clearly defining the goal of the generative system, asking questions as concrete as possible, testing the evaluation setup, using multiple different evaluation setups, reporting the entire evaluation process and potential biases clearly, and finally analyzing the evaluation results in a more profound way than merely reporting the most typical statistics.
Compressed sensing in the presence of speckle noise
Venue: arXiv cs.IT 2021
Year: 2021

The problem of recovering a structured signal from its linear measurements in the presence of speckle noise is studied. This problem appears in many imaging systems such as synthetic aperture radar and optical coherence tomography. The current acquisition technology oversamples signals and converts the problem into a denoising problem with multiplicative noise. However, this paper explores the possibility of reducing the number of measurements below the ambient dimension of the signal. The sophistications that appear in the study of multiplicative noises have so far impeded theoretical analysis of such problems. This paper aims to present the first theoretical result regarding the recovery of signals from their undersampled measurements under the speckle noise. It is shown that if the signal class is structured, in the sense that the signals can be compressed efficiently, then one can obtain accurate estimates of the signal from fewer measurements than the ambient dimension. We demonstrate the effectiveness of the methods we propose through simulation results.
Towards Adversarially Robust and Domain Generalizable Stereo Matching by Rethinking DNN Feature Backbones
Venue: arXiv cs.CV 2021
Year: 2021

Stereo matching has recently witnessed remarkable progress using Deep Neural Networks (DNNs). But, how robust are they? Although it has been well-known that DNNs often suffer from adversarial vulnerability with a catastrophic drop in performance, the situation is even worse in stereo matching. This paper first shows that a type of weak white-box attacks can overwhelm state-of-the-art methods. The attack is learned by a proposed stereo-constrained projected gradient descent (PGD) method in stereo matching. This observation raises serious concerns for the deployment of DNN-based stereo matching. Parallel to the adversarial vulnerability, DNN-based stereo matching is typically trained under the so-called simulation to reality pipeline, and thus domain generalizability is an important problem. This paper proposes to rethink the learnable DNN-based feature backbone towards adversarially-robust and domain generalizable stereo matching by completely removing it for matching. In experiments, the proposed method is tested in the SceneFlow dataset and the KITTI2015 benchmark, with promising results. We compute the matching cost volume using the classic multi-scale census transform (i.e., local binary pattern) of the raw input stereo images, followed by a stacked Hourglass head sub-network solving the matching problem. It significantly improves the adversarial robustness, while retaining accuracy performance comparable to state-of-the-art methods. It also shows better generalizability from simulation (SceneFlow) to real (KITTI) datasets when no fine-tuning is used.
BadEncoder: Backdoor Attacks to Pre-trained Encoders in Self-Supervised Learning
Venue: arXiv cs.CR 2021
Year: 2021

Self-supervised learning in computer vision aims to pre-train an image encoder using a large amount of unlabeled images or (image, text) pairs. The pre-trained image encoder can then be used as a feature extractor to build downstream classifiers for many downstream tasks with a small amount of or no labeled training data. In this work, we propose BadEncoder, the first backdoor attack to self-supervised learning. In particular, our BadEncoder injects backdoors into a pre-trained image encoder such that the downstream classifiers built based on the backdoored image encoder for different downstream tasks simultaneously inherit the backdoor behavior. We formulate our BadEncoder as an optimization problem and we propose a gradient descent based method to solve it, which produces a backdoored image encoder from a clean one. Our extensive empirical evaluation results on multiple datasets show that our BadEncoder achieves high attack success rates while preserving the accuracy of the downstream classifiers. We also show the effectiveness of BadEncoder using two publicly available, real-world image encoders, i.e., Google's image encoder pre-trained on ImageNet and OpenAI's Contrastive Language-Image Pre-training (CLIP) image encoder pre-trained on 400 million (image, text) pairs collected from the Internet. Moreover, we consider defenses including Neural Cleanse and MNTD (empirical defenses) as well as PatchGuard (a provable defense). Our results show that these defenses are insufficient to defend against BadEncoder, highlighting the needs for new defenses against our BadEncoder. Our code is publicly available at: https://github.com/jjy1994/BadEncoder.
Knowing When to Quit: Selective Cascaded Regression with Patch Attention for Real-Time Face Alignment
Venue: arXiv cs.CV 2021
Year: 2021

Facial landmarks (FLM) estimation is a critical component in many face-related applications. In this work, we aim to optimize for both accuracy and speed and explore the trade-off between them. Our key observation is that not all faces are created equal. Frontal faces with neutral expressions converge faster than faces with extreme poses or expressions. To differentiate among samples, we train our model to predict the regression error after each iteration. If the current iteration is accurate enough, we stop iterating, saving redundant iterations while keeping the accuracy in check. We also observe that as neighboring patches overlap, we can infer all facial landmarks (FLMs) with only a small number of patches without a major accuracy sacrifice. Architecturally, we offer a multi-scale, patch-based, lightweight feature extractor with a fine-grained local patch attention module, which computes a patch weighting according to the information in the patch itself and enhances the expressive power of the patch features. We analyze the patch attention data to infer where the model is attending when regressing facial landmarks and compare it to face attention in humans. Our model runs in real-time on a mobile device GPU, with 95 Mega Multiply-Add (MMA) operations, outperforming all state-of-the-art methods under 1000 MMA, with a normalized mean error of 8.16 on the 300W challenging dataset.
Relevance ranking for proximity full-text search based on additional indexes with multi-component keys
Venue: arXiv cs.IR 2021
Year: 2021

The problem of proximity full-text search is considered. If a search query contains high-frequently occurring words, then multi-component key indexes deliver an improvement in the search speed compared with ordinary inverted indexes. It was shown that we can increase the search speed by up to 130 times in cases when queries consist of high-frequently occurring words. In this paper, we investigate how the multi-component key index architecture affects the quality of the search. We consider several well-known methods of relevance ranking, where these methods are of different authors. Using these methods, we perform the search in the ordinary inverted index and then in an index enhanced with multi-component key indexes. The results show that with multi-component key indexes we obtain search results that are very close, in terms of relevance ranking, to the search results that are obtained by means of ordinary inverted indexes.
Transformer-based Map Matching Model with Limited Ground-Truth Data using Transfer-Learning Approach
Venue: arXiv cs.LG 2021
Year: 2021

In many spatial trajectory-based applications, it is necessary to map raw trajectory data points onto road networks in digital maps, which is commonly referred to as a map-matching process. While most previous map-matching methods have focused on using rule-based algorithms to deal with the map-matching problems, in this paper, we consider the map-matching task from the data-driven perspective, proposing a deep learning-based map-matching model. We build a Transformer-based map-matching model with a transfer learning approach. We generate trajectory data to pre-train the Transformer model and then fine-tune the model with a limited number of ground-truth data to minimize the model development cost and reduce the real-to-virtual gap. Three metrics (Average Hamming Distance, F-score, and BLEU) at two levels (point and segment level) are used to evaluate the model performance. The results indicate that the proposed model outperforms existing models. Furthermore, we use the attention weights of the Transformer to plot the map-matching process and find how the model matches the road segments correctly.
A Comparison Study of Cellular Deployments in Chicago and Miami Using Apps on Smartphones
Venue: arXiv cs.NI 2021
Year: 2021

Cellular operators have begun deploying 5G New Radio (NR) in all available bands: low (< 1 GHz), mid (1 - 6 GHz), and high (> 24 GHz) to exploit the different capabilities of each. At the same time, traditional 4G Long Term Evolution (LTE) deployments are being enhanced with the addition of bands in the unlicensed 5 GHz (using License Assisted Access, or LAA) and the 3.5 GHz Citizens Broadband Radio Service (CBRS) resulting in throughput performance comparable to 5G in mid-band. We present a detailed study comparing 4G and 5G deployments, in all bands in Chicago, and focused mmWave measurements and analysis in Miami. Our methodology, based on commercial and custom apps, is scalable for crowdsourcing measurements on a large scale and provides detailed data (throughput, latency, signal strength, etc.) on actual deployments. Our main conclusions based on the measurements are (i) optimized 4G networks in mid-band are comparable in both throughput and latency to current deployments of 5G (both standalone (SA) and non-standalone (NSA)) and (ii) mmWave 5G, even in NSA mode, can deliver multi-Gbps throughput reliably if the installation is dense enough, but performance is still brittle due to the propagation limitations imposed by distance and body loss. Thus, while 5G demonstrates significant early promise, further work needs to be done to ensure that the stated goals of 5G are met.
Explainable Deep Few-shot Anomaly Detection with Deviation Networks
Venue: arXiv cs.CV 2021
Year: 2021

Existing anomaly detection paradigms overwhelmingly focus on training detection models using exclusively normal data or unlabeled data (mostly normal samples). One notorious issue with these approaches is that they are weak in discriminating anomalies from normal samples due to the lack of the knowledge about the anomalies. Here, we study the problem of few-shot anomaly detection, in which we aim at using a few labeled anomaly examples to train sample-efficient discriminative detection models. To address this problem, we introduce a novel weakly-supervised anomaly detection framework to train detection models without assuming the examples illustrating all possible classes of anomaly. Specifically, the proposed approach learns discriminative normality (regularity) by leveraging the labeled anomalies and a prior probability to enforce expressive representations of normality and unbounded deviated representations of abnormality. This is achieved by an end-to-end optimization of anomaly scores with a neural deviation learning, in which the anomaly scores of normal samples are imposed to approximate scalar scores drawn from the prior while that of anomaly examples is enforced to have statistically significant deviations from these sampled scores in the upper tail. Furthermore, our model is optimized to learn fine-grained normality and abnormality by top-K multiple-instance-learning-based feature subspace deviation learning, allowing more generalized representations. Comprehensive experiments on nine real-world image anomaly detection benchmarks show that our model is substantially more sample-efficient and robust, and performs significantly better than state-of-the-art competing methods in both closed-set and open-set settings. Our model can also offer explanation capability as a result of its prior-driven anomaly score learning. Code and datasets are available at: https://git.io/DevNet.
A Sequential Supervised Machine Learning Approach for Cyber Attack Detection in a Smart Grid System
Venue: arXiv cs.CR 2021
Year: 2021

Modern smart grid systems are heavily dependent on Information and Communication Technology, and this dependency makes them prone to cyberattacks. The occurrence of a cyberattack has increased in recent years resulting in substantial damage to power systems. For a reliable and stable operation, cyber protection, control, and detection techniques are becoming essential. Automated detection of cyberattacks with high accuracy is a challenge. To address this, we propose a two-layer hierarchical machine learning model having an accuracy of 95.44 % to improve the detection of cyberattacks. The first layer of the model is used to distinguish between the two modes of operation (normal state or cyberattack). The second layer is used to classify the state into different types of cyberattacks. The layered approach provides an opportunity for the model to focus its training on the targeted task of the layer, resulting in improvement in model accuracy. To validate the effectiveness of the proposed model, we compared its performance against other recent cyber attack detection models proposed in the literature.