Reviewerly/exHarmony huggingface dataset:
A Comparative Review of Recent Kinect-Based Action Recognition Algorithms

Video-based human action recognition is currently one of the most active research areas in computer vision. Various research studies indicate that the performance of action recognition is highly dependent on the type of features being extracted and how the actions are represented. Since the release of the Kinect camera, a large number of Kinect-based human action recognition techniques have been proposed in the literature. However, there still does not exist a thorough comparison of these Kinect-based techniques under the grouping of feature types, such as handcrafted versus deep learning features and depth-based versus skeleton-based features. In this paper, we analyze and compare 10 recent Kinect-based algorithms for both cross-subject action recognition and cross-view action recognition using six benchmark datasets. In addition, we have implemented and improved some of these techniques and included their variants in the comparison. Our experiments show that the majority of methods perform better on cross-subject action recognition than cross-view action recognition, that the skeleton-based features are more robust for cross-view recognition than the depth-based features, and that the deep learning features are suitable for large datasets.
Medical image segmentation using deep learning: A survey

Deep learning has been widely used for medical image segmentation and a large number of papers has been presented recording the success of deep learning in the field. A comprehensive thematic survey on medical image segmentation using deep learning techniques is presented. This paper makes two original contributions. Firstly, compared to traditional surveys that directly divide literatures of deep learning on medical image segmentation into many groups and introduce literatures in detail for each group, we classify currently popular literatures according to a multi-level structure from coarse to fine. Secondly, this paper focuses on supervised and weakly supervised learning approaches, without including unsupervised approaches since they have been introduced in many old surveys and they are not popular currently. For supervised learning approaches, we analyse literatures in three aspects: the selection of backbone networks, the design of network blocks, and the improvement of loss functions. For weakly supervised learning approaches, we investigate literature according to data augmentation, transfer learning, and interactive segmentation, separately. Compared to existing surveys, this survey classifies the literatures very differently from before and is more convenient for readers to understand the relevant rationale and will guide them to think of appropriate improvements in medical image segmentation based on deep learning approaches.
Conditional BERT Contextual Augmentation

Data augmentation methods are often applied to prevent overfitting and improve generalization of deep neural network models. Recently proposed contextual augmentation augments labeled sentences by randomly replacing words with more varied substitutions predicted by language model. Bidirectional Encoder Representations from Transformers (BERT) demonstrates that a deep bidirectional language model is more powerful than either an unidirectional language model or the shallow concatenation of a forward and backward model. We propose a novel data augmentation method for labeled sentences called conditional BERT contextual augmentation. We retrofit BERT to conditional BERT by introducing a new conditional masked language model (The term “conditional masked language model” appeared once in original BERT paper, which indicates context-conditional, is equivalent to term “masked language model”. In our paper, “conditional masked language model” indicates we apply extra label-conditional constraint to the “masked language model”.) task. The well trained conditional BERT can be applied to enhance contextual augmentation. Experiments on six various different text classification tasks show that our method can be easily applied to both convolutional or recurrent neural networks classifier to obtain improvement.
A Challenge Dataset and Effective Models for Aspect-Based Sentiment Analysis

Aspect-based sentiment analysis (ABSA) has attracted increasing attention recently due to its broad applications.In existing ABSA datasets, most sentences contain only one aspect or multiple aspects with the same sentiment polarity, which makes ABSA task degenerate to sentence-level sentiment analysis.In this paper, we present a new large-scale Multi-Aspect Multi-Sentiment (MAMS) dataset, in which each sentence contains at least two different aspects with different sentiment polarities.The release of this dataset would push forward the research in this field.In addition, we propose simple yet effective CapsNet and CapsNet-BERT models which combine the strengths of recent NLP advances.Experiments on our new dataset show that the proposed model significantly outperforms the state-of-the-art baseline methods 1 .
What makes a good conversation? How controllable attributes affect human judgments

Abigail See, Stephen Roller, Douwe Kiela, Jason Weston. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019.
MEC-Assisted Immersive VR Video Streaming Over Terahertz Wireless Networks: A Deep Reinforcement Learning Approach

Immersive virtual reality (VR) video is becoming increasingly popular owing to its enhanced immersive experience. To enjoy ultrahigh resolution immersive VR video with wireless user equipments, such as head-mounted displays (HMDs), ultralow-latency viewport rendering, and data transmission are the core prerequisites, which could not be achieved without a huge bandwidth and superior processing capabilities. Besides, potentially very high energy consumption at the HMD may impede the rapid development of wireless panoramic VR video. Multiaccess edge computing (MEC) has emerged as a promising technology to reduce both the task processing latency and the energy consumption for HMD, while bandwidth-rich terahertz (THz) communication is expected to enable ultrahigh-speed wireless data transmission. In this article, we propose to minimize the long-term energy consumption of a THz wireless access-based MEC system for high quality immersive VR video services support by jointly optimizing the viewport rendering offloading and downlink transmit power control. Considering the time-varying nature of wireless channel conditions, we propose a deep reinforcement learning-based approach to learn the optimal viewport rendering offloading and transmit power control policies and an asynchronous advantage actor-critic (A3C)-based joint optimization algorithm is proposed. The simulation results demonstrate that the proposed algorithm converges fast under different learning rates, and outperforms existing algorithms in terms of minimized energy consumption and maximized reward.
On the Effectiveness of Least Squares Generative Adversarial Networks

Unsupervised learning with generative adversarial networks (GANs) has proven to be hugely successful. Regular GANs hypothesize the discriminator as a classifier with the sigmoid cross entropy loss function. However, we found that this loss function may lead to the vanishing gradients problem during the learning process. To overcome such a problem, we propose in this paper the Least Squares Generative Adversarial Networks (LSGANs) which adopt the least squares loss for both the discriminator and the generator. We show that minimizing the objective function of LSGAN yields minimizing the Pearson <inline-formula><tex-math notation="LaTeX">$\chi ^2$</tex-math></inline-formula> divergence. We also show that the derived objective function that yields minimizing the Pearson <inline-formula><tex-math notation="LaTeX">$\chi ^2$</tex-math></inline-formula> divergence performs better than the classical one of using least squares for classification. There are two benefits of LSGANs over regular GANs. First, LSGANs are able to generate higher quality images than regular GANs. Second, LSGANs perform more stably during the learning process. For evaluating the image quality, we conduct both qualitative and quantitative experiments, and the experimental results show that LSGANs can generate higher quality images than regular GANs. Furthermore, we evaluate the stability of LSGANs in two groups. One is to compare between LSGANs and regular GANs without gradient penalty. We conduct three experiments, including Gaussian mixture distribution, difficult architectures, and a newly proposed method — datasets with small variability, to illustrate the stability of LSGANs. The other one is to compare between LSGANs with gradient penalty (LSGANs-GP) and WGANs with gradient penalty (WGANs-GP). The experimental results show that LSGANs-GP succeed in training for all the difficult architectures used in WGANs-GP, including 101-layer ResNet.
Sine Chaotification Model for Enhancing Chaos and Its Hardware Implementation

When chaotic systems are used in different practical applications, such as nonlinear control and cryptography, their complex chaos dynamics are strongly required. However, many existing chaotic systems have simple complexity, and this brings negative effects to chaos-based applications. To address this issue, this paper introduces a sine chaotification model (SCM) as a general framework to enhance the chaos complexity of existing one-dimensional (1-D) chaotic maps. The SCM uses a sine function as a nonlinear chaotification transform and applies it to the output of a 1-D chaotic map. The resulting enhanced chaotic map of the SCM has better chaos complexity and a much larger chaotic range than the seed map. Theoretical analysis verifies the efficiency of the SCM. To show the performance of the SCM, we apply SCM to three existing chaotic maps and analyze the dynamics properties of the obtained enhanced chaotic maps. Performance evaluations prove that the three enhanced chaotic maps have more complicated dynamics behaviors than their seed chaotic maps. To show the implementation simplicity of the SCM, we implement the three enhanced chaotic maps using the field-programmable gate array. To investigate the SCM in practical application, we design pseudorandom number generators using the enhanced chaotic maps.
Learning The Difference That Makes A Difference With Counterfactually-Augmented Data

Despite alarm over the reliance of machine learning systems on so-called spurious patterns in training data, the term lacks coherent meaning in standard statistical frameworks. However, the language of causality offers clarity: spurious associations are those due to a common cause (confounding) vs direct or indirect effects. In this paper, we focus on NLP, introducing methods and resources for training models insensitive to spurious patterns. Given documents and their initial labels, we task humans with revise each document to accord with a counterfactual target label, asking that the revised documents be internally coherent while avoiding any gratuitous changes. Interestingly, on sentiment analysis and natural language inference tasks, classifiers trained on original data fail on their counterfactually-revised counterparts and vice versa. Classifiers trained on combined datasets perform remarkably well, just shy of those specialized to either domain. While classifiers trained on either original or manipulated data alone are sensitive to spurious features (e.g., mentions of genre), models trained on the combined data are insensitive to this signal. We will publicly release both datasets.
ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators

While masked language modeling (MLM) pre-training methods such as BERT produce excellent results on downstream NLP tasks, they require large amounts of compute to be effective. These approaches corrupt the input by replacing some tokens with [MASK] and then train a model to reconstruct the original tokens. As an alternative, we propose a more sample-efficient pre-training task called replaced token detection. Instead of masking the input, our approach corrupts it by replacing some input tokens with plausible alternatives sampled from a small generator network. Then, instead of training a model that predicts the original identities of the corrupted tokens, we train a discriminative model that predicts whether each token in the corrupted input was replaced by a generator sample or not. Thorough experiments demonstrate this new pre-training task is more efficient than MLM because the model learns from all input tokens rather than just the small subset that was masked out. As a result, the contextual representations learned by our approach substantially outperform the ones learned by methods such as BERT and XLNet given the same model size, data, and compute. The gains are particularly strong for small models; for example, we train a model on one GPU for 4 days that outperforms GPT (trained using 30x more compute) on the GLUE natural language understanding benchmark. Our approach also works well at scale, where we match the performance of RoBERTa, the current state-of-the-art pre-trained transformer, while using less than 1/4 of the compute.
Beyond RNNs: Positional Self-Attention with Co-Attention for Video Question Answering

Most of the recent progresses on visual question answering are based on recurrent neural networks (RNNs) with attention. Despite the success, these models are often timeconsuming and having difficulties in modeling long range dependencies due to the sequential nature of RNNs. We propose a new architecture, Positional Self-Attention with Coattention (PSAC), which does not require RNNs for video question answering. Specifically, inspired by the success of self-attention in machine translation task, we propose a Positional Self-Attention to calculate the response at each position by attending to all positions within the same sequence, and then add representations of absolute positions. Therefore, PSAC can exploit the global dependencies of question and temporal information in the video, and make the process of question and video encoding executed in parallel. Furthermore, in addition to attending to the video features relevant to the given questions (i.e., video attention), we utilize the co-attention mechanism by simultaneously modeling “what words to listen to” (question attention). To the best of our knowledge, this is the first work of replacing RNNs with selfattention for the task of visual question answering. Experimental results of four tasks on the benchmark dataset show that our model significantly outperforms the state-of-the-art on three tasks and attains comparable result on the Count task. Our model requires less computation time and achieves better performance compared with the RNNs-based methods. Additional ablation study demonstrates the effect of each component of our proposed model.
A Secure Federated Learning Framework for 5G Networks

Federated Learning (FL) has been recently proposed as an emerging paradigm to build machine learning models using distributed training datasets that are locally stored and maintained on different devices in 5G networks while providing privacy preservation for participants. In FL, the central aggregator accumulates local updates uploaded by participants to update a global model. However, there are two critical security threats: poisoning and membership inference attacks. These attacks may be carried out by malicious or unreliable participants, resulting in the construction failure of global models or privacy leakage of FL models. Therefore, it is crucial for FL to develop security means of defense. In this article, we propose a blockchain-based secure FL framework to create smart contracts and prevent malicious or unreliable participants from involving in FL. In doing so, the central aggregator recognizes malicious and unreliable participants by automatically executing smart contracts to defend against poisoning attacks. Further, we use local differential privacy techniques to prevent membership inference attacks. Numerical results suggest that the proposed framework can effectively deter poisoning and membership inference attacks, thereby improving the security of FL in 5G networks.
Deeply supervised 3D fully convolutional networks with group dilated convolution for automatic <scp>MRI</scp> prostate segmentation

Purpose Reliable automated segmentation of the prostate is indispensable for image‐guided prostate interventions. However, the segmentation task is challenging due to inhomogeneous intensity distributions, variation in prostate anatomy, among other problems. Manual segmentation can be time‐consuming and is subject to inter‐ and intraobserver variation. We developed an automated deep learning‐based method to address this technical challenge. Methods We propose a three‐dimensional (3D) fully convolutional networks ( FCN ) with deep supervision and group dilated convolution to segment the prostate on magnetic resonance imaging ( MRI ). In this method, a deeply supervised mechanism was introduced into a 3D FCN to effectively alleviate the common exploding or vanishing gradients problems in training deep models, which forces the update process of the hidden layer filters to favor highly discriminative features. A group dilated convolution which aggregates multiscale contextual information for dense prediction was proposed to enlarge the effective receptive field of convolutional neural networks, which improve the prediction accuracy of prostate boundary. In addition, we introduced a combined loss function including cosine and cross entropy, which measures similarity and dissimilarity between segmented and manual contours, to further improve the segmentation accuracy. Prostate volumes manually segmented by experienced physicians were used as a gold standard against which our segmentation accuracy was measured. Results The proposed method was evaluated on an internal dataset comprising 40 T2‐weighted prostate MR volumes. Our method achieved a Dice similarity coefficient ( DSC ) of 0.86 ± 0.04, a mean surface distance ( MSD ) of 1.79 ± 0.46 mm, 95% Hausdorff distance (95% HD ) of 7.98 ± 2.91 mm, and absolute relative volume difference ( aRVD ) of 15.65 ± 10.82. A public dataset ( PROMISE 12) including 50 T2‐weighted prostate MR volumes was also employed to evaluate our approach. Our method yielded a DSC of 0.88 ± 0.05, MSD of 1.02 ± 0.35 mm, 95% HD of 9.50 ± 5.11 mm, and aRVD of 8.93 ± 7.56. Conclusion We developed a novel deeply supervised deep learning‐based approach with a group dilated convolution to automatically segment the MRI prostate, demonstrated its clinical feasibility, and validated its accuracy against manual segmentation. The proposed technique could be a useful tool for image‐guided interventions in prostate cancer.
Efficient and Secure Data Sharing for 5G Flying Drones: A Blockchain-Enabled Approach

The drone's open and untrusted environment may create problems for authentication and data sharing.To address this issue, we propose a blockchain-enabled efficient and secure data-sharing model for 5G flying drones.In this model, blockchain and attribute-based encryption (ABE) are applied to ensure the security of instruction issues and data sharing.The authentication mechanism in the model employs a smart contract for authentication and access control, public-key cryptography for providing accounts and ensuring accounts security, and a distributed ledger for security audit.In addition, to speed up outsourced computations and reduce electricity consumption, an ABE model with parallel outsourced computation (ABEM-POC) is constructed, and a generic parallel computation method for ABE is proposed.The analysis of the experimental results shows that parallel computation significantly improves the speed of outsourced encryption and decryption compared with serial computation.
Transforming machine translation: a deep learning system reaches news translation quality comparable to human professionals

Abstract The quality of human translation was long thought to be unattainable for computer translation systems. In this study, we present a deep-learning system, CUBBITT, which challenges this view. In a context-aware blind evaluation by human judges, CUBBITT significantly outperformed professional-agency English-to-Czech news translation in preserving text meaning (translation adequacy). While human translation is still rated as more fluent, CUBBITT is shown to be substantially more fluent than previous state-of-the-art systems. Moreover, most participants of a Translation Turing test struggle to distinguish CUBBITT translations from human translations. This work approaches the quality of human translation and even surpasses it in adequacy in certain circumstances.This suggests that deep learning may have the potential to replace humans in applications where conservation of meaning is the primary aim.
FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection

Monocular 3D object detection is an important task for autonomous driving considering its advantage of low cost. It is much more challenging than conventional 2D cases due to its inherent ill-posed property, which is mainly reflected in the lack of depth information. Recent progress on 2D detection offers opportunities to better solving this problem. However, it is non-trivial to make a general adapted 2D detector work in this 3D task. In this paper, we study this problem with a practice built on a fully convolutional single-stage detector and propose a general framework FCOS3D. Specifically, we first transform the commonly defined 7-DoF 3D targets to the image domain and decouple them as 2D and 3D attributes. Then the objects are distributed to different feature levels with consideration of their 2D scales and assigned only according to the projected 3D-center for the training procedure. Furthermore, the center-ness is redefined with a 2D Gaussian distribution based on the 3D-center to fit the 3D target formulation. All of these make this framework simple yet effective, getting rid of any 2D detection or 2D-3D correspondence priors. Our solution achieves 1st place out of all the vision-only methods in the nuScenes 3D detection challenge of NeurIPS 2020. Code and models are released at https://github.com/open-mmlab/mmdetection3d.
Unsupervised Semantic-Preserving Adversarial Hashing for Image Search

Hashing plays a pivotal role in nearest-neighbor searching for large-scale image retrieval. Recently, deep learning-based hashing methods have achieved promising performance. However, most of these deep methods involve discriminative models, which require large-scale, labeled training datasets, thus hindering their real-world applications. In this paper, we propose a novel strategy to exploit the semantic similarity of the training data and design an efficient generative adversarial framework to learn binary hash codes in an unsupervised manner. Specifically, our model consists of three different neural networks: an encoder network to learn hash codes from images, a generative network to generate images from hash codes, and a discriminative network to distinguish between pairs of hash codes and images. By adversarially training these networks, we successfully learn mutually coherent encoder and generative networks, and can output efficient hash codes from the encoder network. We also propose a novel strategy, which utilizes both feature and neighbor similarities, to construct a semantic similarity matrix, then use this matrix to guide the hash code learning process. Integrating the supervision of this semantic similarity matrix into the adversarial learning framework can efficiently preserve the semantic information of training data in Hamming space. The experimental results on three widely used benchmarks show that our method not only significantly outperforms several state-of-the-art unsupervised hashing methods, but also achieves comparable performance with popular supervised hashing methods.