Reviewerly/exHarmony huggingface dataset:
Rethinking Pseudo-LiDAR Representation

The recently proposed pseudo-LiDAR based 3D detectors greatly improve the benchmark of monocular/stereo 3D detection task. However, the underlying mechanism remains obscure to the research community. In this paper, we perform an in-depth investigation and observe that the efficacy of pseudo-LiDAR representation comes from the coordinate transformation, instead of data representation itself. Based on this observation, we design an image based CNN detector named PatchNet, which is more generalized and can be instantiated as pseudo-LiDAR based 3D detectors. Moreover, the pseudo-LiDAR data in our PatchNet is organized as the image representation, which means existing 2D CNN designs can be easily utilized for extracting deep features from input data and boosting 3D detection performance. We conduct extensive experiments on the challenging KITTI dataset, where the proposed PatchNet outperforms all existing pseudo-LiDAR based counterparts. Code has been made available at: https://github.com/xinzhuma/patchnet .
Two-Stream Consensus Network for Weakly-Supervised Temporal Action Localization

Weakly-supervised Temporal Action Localization (W-TAL) aims to classify and localize all action instances in an untrimmed video under only video-level supervision. However, without frame-level annotations, it is challenging for W-TAL methods to identify false positive action proposals and generate action proposals with precise temporal boundaries. In this paper, we present a Two-Stream Consensus Network (TSCN) to simultaneously address these challenges. The proposed TSCN features an iterative refinement training method, where a frame-level pseudo ground truth is iteratively updated, and used to provide frame-level supervision for improved model training and false positive action proposal elimination. Furthermore, we propose a new attention normalization loss to encourage the predicted attention to act like a binary selection, and promote the precise localization of action instance boundaries. Experiments conducted on the THUMOS14 and ActivityNet datasets show that the proposed TSCN outperforms current state-of-the-art methods, and even achieves comparable results with some recent fully-supervised methods.
Whole-Body Human Pose Estimation in the Wild

This paper investigates the task of 2D human whole-body pose estimation, which aims to localize dense landmarks on the entire human body including face, hands, body, and feet. As existing datasets do not have whole-body annotations, previous methods have to assemble different deep models trained independently on different datasets of the human face, hand, and body, struggling with dataset biases and large model complexity. To fill in this blank, we introduce COCO-WholeBody which extends COCO dataset with whole-body annotations. To our best knowledge, it is the first benchmark that has manual annotations on the entire human body, including 133 dense landmarks with 68 on the face, 42 on hands and 23 on the body and feet. A single-network model, named ZoomNet, is devised to take into account the hierarchical structure of the full human body to solve the scale variation of different body parts of the same person. ZoomNet is able to significantly outperform existing methods on the proposed COCO-WholeBody dataset. Extensive experiments show that COCO-WholeBody not only can be used to train deep models from scratch for whole-body pose estimation but also can serve as a powerful pre-training dataset for many different tasks such as facial landmark detection and hand keypoint estimation. The dataset is publicly available at https://github.com/jin-s13/COCO-WholeBody .
A Survey of Single-Scene Video Anomaly Detection

This article summarizes research trends on the topic of anomaly detection in video feeds of a single scene. We discuss the various problem formulations, publicly available datasets and evaluation criteria. We categorize and situate past research into an intuitive taxonomy and provide a comprehensive comparison of the accuracy of many algorithms on standard test sets. Finally, we also provide best practices and suggest some possible directions for future research.
Semi-supervised task-driven data augmentation for medical image segmentation

Supervised learning-based segmentation methods typically require a large number of annotated training data to generalize well at test time. In medical applications, curating such datasets is not a favourable option because acquiring a large number of annotated samples from experts is time-consuming and expensive. Consequently, numerous methods have been proposed in the literature for learning with limited annotated examples. Unfortunately, the proposed approaches in the literature have not yet yielded significant gains over random data augmentation for image segmentation, where random augmentations themselves do not yield high accuracy. In this work, we propose a novel task-driven data augmentation method for learning with limited labeled data where the synthetic data generator, is optimized for the segmentation task. The generator of the proposed method models intensity and shape variations using two sets of transformations, as additive intensity transformations and deformation fields. Both transformations are optimized using labeled as well as unlabeled examples in a semi-supervised framework. Our experiments on three medical datasets, namely cardiac, prostate and pancreas, show that the proposed approach significantly outperforms standard augmentation and semi-supervised approaches for image segmentation in the limited annotation setting. The code is made publicly available at https://github.com/krishnabits001/task_driven_data_augmentation.
Efficient activity recognition using lightweight CNN and DS-GRU network for surveillance applications

Recognizing human activities has become a trend in smart surveillance that contains several challenges, such as performing effective analyses of huge video data streams, while maintaining low computational complexity, and performing this task in real-time. Current activity recognition techniques are using convolutional neural network (CNN) models with computationally complex classifiers, creating hurdles in obtaining quick responses for abnormal activities. To address these challenges in real-time surveillance, this paper proposes a lightweight deep learning-assisted framework for activity recognition. First, we detect a human in the surveillance stream using an effective CNN model, which is trained on two surveillance datasets. The detected individual is tracked throughout the video stream via an ultra-fast object tracker called the ‘minimum output sum of squared error’ (MOSSE). Next, for each tracked individual, pyramidal convolutional features are extracted from two consecutive frames using the efficient LiteFlowNet CNN. Finally, a novel deep skip connection gated recurrent unit (DS-GRU) is trained to learn the temporal changes in the sequence of frames for activity recognition. Experiments are conducted over five benchmark activity recognition datasets, and the results indicate the efficiency of the proposed technique for real-time surveillance applications compared to the state-of-the-art.
Digital quantum simulation, Trotter errors, and quantum chaos of the kicked top

This work aims at giving Trotter errors in digital quantum simulation (DQS) of collective spin systems an interpretation in terms of quantum chaos of the kicked top. In particular, for DQS of such systems, regular dynamics of the kicked top ensures convergence of the Trotterized time evolution, while chaos in the top, which sets in above a sharp threshold value of the Trotter step size, corresponds to the proliferation of Trotter errors. We show the possibility to analyze this phenomenology in a wide variety of experimental realizations of the kicked top, ranging from single atomic spins to trapped-ion quantum simulators which implement DQS of all-to-all interacting spin-1/2 systems. These platforms thus enable in-depth studies of Trotter errors and their relation to signatures of quantum chaos, including the growth of out-of-time-ordered correlators.
Effects of Hidden Layers on the Efficiency of Neural networks

Hidden layers play a vital role in the performance of Neural network especially in the case of complex problems where the accuracy and the time complexity are the main constraints. The process of deciding the number of hidden layers and number of neurons in each hidden layer is still confusing. In this article, we reviewed different impacts of Hidden layers on the network which provides an overview of using three numbers of hidden layers that were found to be optimal in terms of reducing the time complexity and getting the qualified accuracy. The techniques implementing less than three number of hidden layers mostly had a loss in accuracy while the architecture implementing more than three numbers of hidden layers were found not to be optimal in terms of time complexity. Usually implementing three numbers of hidden layers give the optimal performance in terms of time complexity and accuracy. We had a survey on recent work about the Neural network based on the Empirical observations, in which if the number of hidden layers is reduced it has a direct impact on the accuracy of the network as with the complex problem having less number of hidden layers it might be possible that network will not be trained properly. On the other hand when the number of hidden layers cross the optimal number of hidden layers (three layers), time complexity increases in orders of magnitude as compared to the accuracy gain.
Multi-task learning based pre-trained language model for code completion

Code completion is one of the most useful features in the Integrated Development Environments (IDEs), which can accelerate software development by suggesting the next probable token based on the contextual code in real-time. Recent studies have shown that statistical language modeling techniques can improve the performance of code completion tools through learning from large-scale software repositories. However, these models suffer from two major drawbacks: a) Existing research uses static embeddings, which map a word to the same vector regardless of its context. The differences in the meaning of a token in varying contexts are lost when each token is associated with a single representation; b) Existing language model based code completion models perform poor on completing identifiers, and the type information of the identifiers is ignored in most of these models. To address these challenges, in this paper, we develop a multi-task learning based pre-trained language model for code understanding and code generation with a Transformer-based neural architecture. We pre-train it with hybrid objective functions that incorporate both code understanding and code generation tasks. Then we fine-tune the pre-trained model on code completion. During the completion, our model does not directly predict the next token. Instead, we adopt multi-task learning to predict the token and its type jointly and utilize the predicted type to assist the token prediction. Experiments results on two real-world datasets demonstrate the effectiveness of our model when compared with state-of-the-art methods.
A hybrid method with dynamic weighted entropy for handling the problem of class imbalance with overlap in credit card fraud detection

Class imbalance with overlap is a very challenging problem in electronic fraud transaction detection. Fraudsters have racked their brains to make a fraud transaction as similar as a genuine one in order to avoid being found. Therefore, lots of data of fraud transactions overlap with genuine transactions so that it is hard to distinguish them. However, most attention has been focused on class imbalance rather than overlapping issues for machine-learning-based methods of fraud transaction detection. This paper proposes a novel hybrid method to handle the problem of class imbalance with overlap based on a divide-and-conquer idea. Firstly, an anomaly detection model is trained on the minority samples for excluding both a few outliers of minority class and lots of majority samples from the original dataset. Then the remaining samples form an overlapping subset that has a low imbalance ratio and a reduced learning interference from both minority class and majority class than the original dataset. After that, this difficult overlapping subset is dealt with a non-linear classifier in order to distinguish them well. To achieve good properties of the overlapping subset, we propose a novel assessment criterion, Dynamic Weighted Entropy (DWE), to evaluate its quality. It is a specially designed trade-off between the number of excluded outliers of minority class and the ratio of class imbalance of overlapping subset. With the help of DWE, time consumption on searching good hyper-parameters is dramatically declined. Extensive experiments on Kaggle fraud detection dataset and a large real electronic transaction dataset demonstrate that our method significantly outperforms state-of-the-art ones.
Deep Neural Network Approximation Theory

This paper develops fundamental limits of deep neural network learning by characterizing what is possible if no constraints are imposed on the learning algorithm and on the amount of training data. Concretely, we consider Kolmogorov-optimal approximation through deep neural networks with the guiding theme being a relation between the complexity of the function (class) to be approximated and the complexity of the approximating network in terms of connectivity and memory requirements for storing the network topology and the associated quantized weights. The theory we develop establishes that deep networks are Kolmogorov-optimal approximants for markedly different function classes, such as unit balls in Besov spaces and modulation spaces. In addition, deep networks provide exponential approximation accuracy-i.e., the approximation error decays exponentially in the number of nonzero weights in the network-of the multiplication operation, polynomials, sinusoidal functions, and certain smooth functions. Moreover, this holds true even for one-dimensional oscillatory textures and the Weierstrass function-a fractal function, neither of which has previously known methods achieving exponential approximation accuracy. We also show that in the approximation of sufficiently smooth functions finite-width deep networks require strictly smaller connectivity than finite-depth wide networks.
<scp>ME‐Net</scp>: <scp>Multi‐encoder</scp> net framework for brain tumor segmentation

Abstract MRI plays a vital role to evaluate brain tumor diagnosis and treatment planning. However, the manual segmentation of the MRI image is strenuous. With the development of deep learning, a large number of automatic segmentation methods have been developed, but most of them stay in 2D images, which leads to subpar performance. Aiming at segmenting 3D MRI, we propose a model for brain tumor segmentation with multiple encoders. Our model reduces the difficulty of feature extraction and greatly improves model performance. We also introduced a new loss function named “Categorical Dice,” and set different weights for different segmented regions at the same time, which solved the problem of voxel imbalance. We evaluated our approach using the online BraTS 2020 Challenge verification. Our proposed method can achieve promising results compared to the state‐of‐the‐art approaches with Dice scores of 0.70249, 0.88267, and 0.73864 for the intact tumor, tumor core, and enhancing tumor.
Short-term bitcoin market prediction via machine learning

We analyze the predictability of the bitcoin market across prediction horizons ranging from 1 to 60 min. In doing so, we test various machine learning models and find that, while all models outperform a random classifier, recurrent neural networks and gradient boosting classifiers are especially well-suited for the examined prediction tasks. We use a comprehensive feature set, including technical, blockchain-based, sentiment-/interest-based, and asset-based features. Our results show that technical features remain most relevant for most methods, followed by selected blockchain-based and sentiment-/interest-based features. Additionally, we find that predictability increases for longer prediction horizons. Although a quantile-based long-short trading strategy generates monthly returns of up to 39% before transaction costs, it leads to negative returns after taking transaction costs into account due to the particularly short holding periods.
Hybrid Multiple Attention Network for Semantic Segmentation in Aerial Images

Semantic segmentation in very-high-resolution (VHR) aerial images is one of the most challenging tasks in remote sensing image understanding. Most of the current approaches are based on deep convolutional neural networks (DCNNs). However, standard convolution with local receptive fields fails in modeling global dependencies. Prior research works have indicated that attention-based methods can capture long-range dependencies and further reconstruct the feature maps for better representation. Nevertheless, limited by the mere perspective of spatial and channel attention and huge computation complexity of self-attention (SA) mechanism, it is unlikely to model the effective semantic interdependencies between each pixel pair of remote sensing data with complex spectra. In this work, we propose a novel attention-based framework named hybrid multiple attention network (HMANet) to adaptively capture global correlations from the perspective of space, channel, and category in a more effective and efficient manner. Concretely, a class augmented attention (CAA) module embedded with a class channel attention (CCA) module can be used to compute category-based correlation and recalibrate the class-level information. In addition, we introduce a simple yet effective region shuffle attention (RSA) module to reduce feature redundant and improve the efficiency of SA mechanism via regionwise representations. Extensive experimental results on the ISPRS Vaihingen, Potsdam benchmark, and iSAID data set demonstrate the effectiveness and efficiency of our HMANet over other state-of-the-art methods.
Auto-Pytorch: Multi-Fidelity MetaLearning for Efficient and Robust AutoDL

While early AutoML frameworks focused on optimizing traditional ML pipelines and their hyperparameters, a recent trend in AutoML is to focus on neural architecture search. In this paper, we introduce Auto-PyTorch, which brings together the best of these two worlds by jointly and robustly optimizing the network architecture and the training hyperparameters to enable fully automated deep learning (AutoDL). Auto-PyTorch achieves state-of-the-art performance on several tabular benchmarks by combining multi-fidelity optimization with portfolio construction for warmstarting and ensembling of deep neural networks (DNNs) and common baselines for tabular data. To thoroughly study our assumptions on how to design such an AutoDL system, we additionally introduce a new benchmark on learning curves for DNNs, dubbed LCBench, and run extensive ablation studies of the full Auto-PyTorch on typical AutoML benchmarks, eventually showing that Auto-PyTorch performs better than several state-of-the-art competitors.
HiT: Hierarchical Transformer with Momentum Contrast for Video-Text Retrieval

Video-Text Retrieval has been a hot research topic with the growth of multimedia data on the internet. Transformer for video-text learning has attracted increasing attention due to its promising performance. However, existing cross-modal transformer approaches typically suffer from two major limitations: 1) Exploitation of the transformer architecture where different layers have different feature characteristics is limited; 2) End-to-end training mechanism limits negative sample interactions in a mini-batch. In this paper, we propose a novel approach named Hierarchical Transformer (HiT) for video-text retrieval. HiT performs Hierarchical Cross-modal Contrastive Matching in both feature-level and semantic-level, achieving multi-view and comprehensive retrieval results. Moreover, inspired by MoCo, we propose Momentum Cross-modal Contrast for cross-modal learning to enable large-scale negative sample interactions on-the-fly, which contributes to the generation of more precise and discriminative representations. Experimental results on the three major Video-Text Retrieval benchmark datasets demonstrate the advantages of our method.
Digital learning in schools: What does it take beyond digital technology?

We investigated how often teachers apply digital technology in their teaching and which student learning activities teachers initiate. Further, we analyzed factors relating to technology use. 410 teachers in our sample, representative for the state of Bavaria (Germany), reported that they spend a substantial amount of time using digital technologies in a typical lesson. Results indicated that rather teachers’ basic digital skills and technology-related teaching skills than digital technology resources are crucial. Even though a certain threshold level of digital technology is necessary in school, our results suggest shifting the focus from equipping schools to teachers’ skills using technologies effectively.
Real-Time and Accurate UAV Pedestrian Detection for Social Distancing Monitoring in COVID-19 Pandemic

Coronavirus Disease 2019 (COVID-19) is a highly infectious virus that has created a health crisis for people all over the world. Social distancing has proved to be an effective non-pharmaceutical measure to slow down the spread of COVID-19. As unmanned aerial vehicle (UAV) is a flexible mobile platform, it is a promising option to use UAV for social distance monitoring. Therefore, we propose a lightweight pedestrian detection network to accurately detect pedestrians by human head detection in real-time and then calculate the social distancing between pedestrians on UAV images. In particular, our network follows the PeleeNet as backbone and further incorporates the multi-scale features and spatial attention to enhance the features of small objects, like human heads. The experimental results on Merge-Head dataset show that our method achieves 92.22% AP (average precision) and 76 FPS (frames per second), outperforming YOLOv3 models and SSD models and enabling real-time detection in actual applications. The ablation experiments also indicate that multi-scale feature and spatial attention significantly contribute the performance of pedestrian detection. The test results on UAV-Head dataset show that our method can also achieve high precision pedestrian detection on UAV images with 88.5% AP and 75 FPS. In addition, we have conducted a precision calibration test to obtain the transformation matrix from images (vertical images and tilted images) to real-world coordinate. Based on the accurate pedestrian detection and the transformation matrix, the social distancing monitoring between individuals is reliably achieved.