Reviewerly/exHarmony huggingface dataset:
Quantum Approximate Counting, Simplified

In 1998, Brassard, Høyer, Mosca, and Tapp (BHMT) gave a quantum algorithm for approximate counting. Given a list of N items, K of them marked, their algorithm estimates K to within relative error ∊ by making only queries. Although this speedup is of "Grover" type, the BHMT algorithm has the curious feature of relying on the Quantum Fourier Transform (QFT), more commonly associated with Shor's algorithm. Is this necessary? This paper presents a simplified algorithm, which we prove achieves the same query complexity using Grover iterations only. We also generalize this to a QFT-free algorithm for amplitude estimation. Related approaches to approximate counting were sketched previously by Grover, Abrams and Williams, Suzuki et al., and Wie (the latter two as we were writing this paper), but in all cases without rigorous analysis.
Sentiment Analysis of Text Based on Bidirectional LSTM With Multi-Head Attention

Recent years, many scientists address the research on text sentiment analysis of social media due to the exponential growth of social multimedia content.Natural language ambiguities and indirect sentiments within the social media text have made it hard to classify by using traditional machine learning approaches, such as support vector machines, naive Bayes, hybrid models and so on.This article aims to investigate the sentiment analysis of social media Chinese text by combining Bidirectional Long-Short Term Memory (BiLSTM) networks with a Multi-head Attention (MHAT) mechanism in order to overcome the deficiency of Sentiment Analysis that is performed with traditional machine learning.BiLSTM networks, not only solve the long-term dependency problem, but they also capture the actual context of the text.Due to the fact that the MHAT mechanism can learn the relevant information from a different representation subspace by using multiple distributed calculations, the purpose is to add influence weights to the constructed text sequence.The results of the numerical experiments show that the proposed model achieves better performance than the existing well-established methods.
Super-resolution reconstruction of knee magnetic resonance imaging based on deep learning

With the rapid development of medical imaging and intelligent diagnosis, artificial intelligence methods have become a research hotspot of radiography processing technology in recent years. The low definition of knee magnetic resonance image texture seriously affects the diagnosis of knee osteoarthritis. This paper presents a super-resolution reconstruction method to address this problem. In this paper, we propose an efficient medical image super-resolution (EMISR) method, in which we mainly adopted three hidden layers of super-resolution convolution neural network (SRCNN) and a sub-pixel convolution layer of efficient sub-pixel convolution neural network (ESPCN). The addition of the efficient sub-pixel convolutional layer in the hidden layer and the small network replacement consisting of concatenated convolutions to address low-resolution images but not high-resolution images are important. The EMISR method also uses cascaded small convolution kernels to improve reconstruction speed and deepen the convolution neural network to improve reconstruction quality. The proposed method is tested in the public dataset IDI, and the reconstruction quality of the algorithm is higher than that of the sparse coding-based network (SCN) method, the SRCNN method, and the ESPCN method (+ 2.306 dB, + 2.540 dB, + 1.089 dB improved); moreover, the reconstruction speed is faster than its counterparts (+ 4.272 s, + 1.967 s, and + 0.073 s improved). The experimental results show that our EMISR framework has improved performance and greatly reduces the number of parameters and training time. Furthermore, the reconstructed image presents more details, and the edges are more complete. Therefore, the EMISR technique provides a more powerful medical analysis in knee osteoarthritis examinations.
Predicting daily diffuse horizontal solar radiation in various climatic regions of China using support vector machine and tree-based soft computing models with local and extrinsic climatic data

Knowledge of diffuse horizontal solar radiation (Rd) on horizontal surfaces is a prerequisite for the design and optimization of active and passive solar energy systems such as the solar illumination system within a building, but it is unavailable in many worldwide locations and commonly predicted by readily available climatic variables. However, reliable prediction of Rd is difficult when lack of complete or previous climatic data at the target station. This study evaluated the performance of support vector machine (SVM) and four tree-based soft computing models, i.e. M5 model tree (M5Tree), random forest (RF), extreme gradient boosting (XGBoost) and gradient boosting with categorical features support (CatBoost), for prediction of daily horizontal Rd when using limited local (Scenario 1) and extrinsic (Scenarios 2 and 3) climatic data. Six input combinations of daily global solar radiation (Rs), sunshine hour (n), maximum/minimum temperature (Tmax/Tmin) and relative humidity (RH) during 1996–2015 at 15 weather stations across various climatic rons of China were considered. The results demonstrated that, when lack of Rs, the average root mean square error (RMSE) was considerably increased across China (42.4%) in Scenario 1, especially in the (sub)tropical monsoon ron (68.3%). SVM offered the best combination of prediction accuracy and generalization capability in all scenarios, followed by CatBoost. CatBoost produced the closest daily Rd estimates to SVM and satisfactory generalization capability. In Scenario 2, CatBoost and SVM models developed with climatic data from Beijing gave the overall best daily Rd estimates over the 15 stations, while models developed with data from 14 weather stations in Scenario 3 produced even better and steadier Rd estimates across China compared with those in Scenario 2. The average computational time of SVM (6.6 s) for a single sample was approximately 1.9 times that of CatBoost (3.5 s) in Scenarios 1 and 2, while the corresponding value (842.6 s) was approximately 33.9 times that of CatBoost (24.9 s) in Scenario 3. Comprehensively considering prediction accuracy, generalization capability and computational efficiency, CatBoost is highly recommended to develop general models for daily Rd prediction in various climatic rons of China, particularly when lack of previous local climatic data.
Breast Cancer Detection in the IOT Health Environment Using Modified Recursive Feature Selection

The accurate and efficient diagnosis of breast cancer is extremely necessary for recovery and treatment in early stages in IoT healthcare environment. Internet of Things has witnessed the transition in life for the last few years which provides a way to analyze both the real-time data and past data by the emerging role of artificial intelligence and data mining techniques. The current state-of-the-art method does not effectively diagnose the breast cancer in the early stages, and most of the ladies suffered from this dangerous disease. Thus, the early detection of breast cancer significantly poses a great challenge for medical experts and researchers. To solve the problem of early-stage detection of breast cancer, we proposed machine learning-based diagnostic system which effectively classifies the malignant and benign people in the environment of IoT. In the development of our proposed system, a machine learning classifier support vector machine is used to classify the malignant and benign people. To improve the classification performance of the classification system, we used a recursive feature selection algorithm to select more suitable features from the breast cancer dataset. The training/testing splits method is applied for training and testing of the classifier for the best predictive model. Additionally, the classifier performance has been checked on by using performance evaluation metrics such as classification, specificity, sensitivity, Matthews’s correlation coefficient, F1-score, and execution time. To test the proposed method, the dataset “Wisconsin Diagnostic Breast Cancer” has been used in this research study. The experimental results demonstrate that the recursive feature selection algorithm selects the best subset of features, and the classifier SVM achieved optimal classification performance on this best subset of features. The SVM kernel linear achieved high classification accuracy (99%), specificity (99%), and sensitivity (98%), and the Matthews’s correlation coefficient is 99%. From these experimental results, we concluded that the proposed system performance is excellent due to the selection of more appropriate features that are selected by the recursive feature selection algorithm. Furthermore, we suggest this proposed system for effective and efficient early stages diagnosis of breast cancer. Thus, through this system, the recovery and treatment will be more effective for breast cancer. Lastly, the implementation of the proposed system is very reliable in all aspects of IoT healthcare for breast cancer.
Drawing Early-Bird Tickets: Toward More Efficient Training of Deep Networks

(Frankle & Carbin, 2019) shows that there exist winning tickets (small but critical subnetworks) for dense, randomly initialized networks, that can be trained alone to achieve comparable accuracies to the latter in a similar number of iterations. However, the identification of these winning tickets still requires the costly train-prune-retrain process, limiting their practical benefits. In this paper, we discover for the first time that the winning tickets can be identified at the very early training stage, which we term as early-bird (EB) tickets, via low-cost training schemes (e.g., early stopping and low-precision training) at large learning rates. Our finding of EB tickets is consistent with recently reported observations that the key connectivity patterns of neural networks emerge early. Furthermore, we propose a mask distance metric that can be used to identify EB tickets with low computational overhead, without needing to know the true winning tickets that emerge after the full training. Finally, we leverage the existence of EB tickets and the proposed mask distance to develop efficient training methods, which are achieved by first identifying EB tickets via low-cost schemes, and then continuing to train merely the EB tickets towards the target accuracy. Experiments based on various deep networks and datasets validate: 1) the existence of EB tickets, and the effectiveness of mask distance in efficiently identifying them; and 2) that the proposed efficient training via EB tickets can achieve up to 4.7x energy savings while maintaining comparable or even better accuracy, demonstrating a promising and easily adopted method for tackling cost-prohibitive deep network training.
On Identifiability in Transformers

In this work we contribute towards a deeper understanding of the Transformer architecture by investigating two of its core components: self-attention and contextual embeddings. In particular, we study the identifiability of attention weights and token embeddings, and the aggregation of context into hidden tokens. We show that attention weights are not unique and propose effective attention as an alternative for better interpretability. Furthermore, we show that input tokens retain their identity in the first hidden layers and then progressively become less identifiable. We also provide evidence for the role of non-linear activations in preserving token identity. Finally, we demonstrate strong mixing of input information in the generation of contextual embeddings by means of a novel quantification method based on gradient attribution. Overall, we show that self-attention distributions are not directly interpretable and present tools to further investigate Transformer models.
Robust Adversarial Objects against Deep Learning Models

Previous work has shown that Deep Neural Networks (DNNs), including those currently in use in many fields, are extremely vulnerable to maliciously crafted inputs, known as adversarial examples. Despite extensive and thorough research of adversarial examples in many areas, adversarial 3D data, such as point clouds, remain comparatively unexplored. The study of adversarial 3D data is crucial considering its impact in real-life, high-stakes scenarios including autonomous driving. In this paper, we propose a novel adversarial attack against PointNet++, a deep neural network that performs classification and segmentation tasks using features learned directly from raw 3D points. In comparison to existing works, our attack generates not only adversarial point clouds, but also robust adversarial objects that in turn generate adversarial point clouds when sampled both in simulation and after construction in real world. We also demonstrate that our objects can bypass existing defense mechanisms designed especially against adversarial 3D data.
Adaptive Trust Region Policy Optimization: Global Convergence and Faster Rates for Regularized MDPs

Trust region policy optimization (TRPO) is a popular and empirically successful policy search algorithm in Reinforcement Learning (RL) in which a surrogate problem, that restricts consecutive policies to be ‘close’ to one another, is iteratively solved. Nevertheless, TRPO has been considered a heuristic algorithm inspired by Conservative Policy Iteration (CPI). We show that the adaptive scaling mechanism used in TRPO is in fact the natural “RL version” of traditional trust-region methods from convex analysis. We first analyze TRPO in the planning setting, in which we have access to the model and the entire state space. Then, we consider sample-based TRPO and establish Õ(1/√N) convergence rate to the global optimum. Importantly, the adaptive scaling mechanism allows us to analyze TRPO in regularized MDPs for which we prove fast rates of Õ(1/N), much like results in convex optimization. This is the first result in RL of better rates when regularizing the instantaneous cost or reward.
Entropy and Confidence-Based Undersampling Boosting Random Forests for Imbalanced Problems

In this article, we propose a novel entropy and confidence-based undersampling boosting (ECUBoost) framework to solve imbalanced problems. The boosting-based ensemble is combined with a new undersampling method to improve the generalization performance. To avoid losing informative samples during the data preprocessing of the boosting-based ensemble, both confidence and entropy are used in ECUBoost as benchmarks to ensure the validity and structural distribution of the majority samples during the undersampling. Furthermore, different from other iterative dynamic resampling methods, ECUBoost based on confidence can be applied to algorithms without iterations such as decision trees. Meanwhile, random forests are used as base classifiers in ECUBoost. Furthermore, experimental results on both artificial data sets and KEEL data sets prove the effectiveness of the proposed method.
A Comparison of End-to-End Models for Long-Form Speech Recognition

End-to-end automatic speech recognition (ASR) models, including both attention-based models and the recurrent neural network transducer (RNN-T), have shown superior performance compared to conventional systems [1], [2]. However, previous studies have focused primarily on short utterances that typically last for just a few seconds or, at most, a few tens of seconds. Whether such architectures are practical on long utterances that last from minutes to hours remains an open question. In this paper, we both investigate and improve the performance of end-to-end models on long-form transcription. We first present an empirical comparison of different end-to-end models on a real world long-form task and demonstrate that the RNN-T model is much more robust than attention-based systems in this regime. We next explore two improvements to attention-based systems that significantly improve its performance: restricting the attention to be monotonic, and applying a novel decoding algorithm that breaks long utterances into shorter overlapping segments. Combining these two improvements, we show that attention-based end-to-end models can be very competitive to RNN-T on long-form speech recognition.
Real-Time Detection Method for Small Traffic Signs Based on Yolov3

It is very challenging to detect traffic signs using a high-precision real-time approach in realistic scenes with respect to driver-assistance systems for driving vehicles and autonomous driving. To address this challenge, in this paper, a new detection scheme (named MSA_YOLOv3) is proposed to accurately achieve real-time localization and classification of small traffic signs. First, data augmentation is achieved using image mixup technology. Second, a multi-scale spatial pyramid pooling block is introduced into the Darknet53 network to enable the network to learn object features more comprehensively. Finally, a bottom-up augmented path is designed to enhance the feature pyramid in YOLOv3, and the result is to achieve accurate localization of objects by utilizing fine-grained features effectively in the lower layers. According to the tests on the TT100K dataset (which is a dataset for traffic sign detection), the performance of the proposed MSA_YOLOv3 is better than that of YOLOv3 in detecting small traffic signs. The detection speed of MSA_YOLOv3 is 23.81 FPS, and the mAP (mean Average Precision) reaches up to 86%.
Hybrid Watermarking Algorithm Using Clifford Algebra With Arnold Scrambling and Chaotic Encryption

With the widespread use of color images, the copyright protection of those images using watermarks is one of the latest research topics.The use of color images as watermarks has advantages over binary and irreplaceable grayscale images.Color images are intuitive, rich, and lively; they have large amounts of copyright protection information and more easily recognized by human vision.To improve the security of watermark information and embedding positions and improve the algorithm's robustness against various attacks, a Quaternion Fourier transform (QFT) based algorithm, based on Arnold transform and chaotic encryption, is proposed in this paper.Geometric algebra (GA) can deal with color images in vector form with each component of RGB handled individually.We used Quaternion, which is a sub-algebra of GA, and effectively handled color image processing by using Fourier transformation.After deriving the calculation process of the QFT with strong security by Arnold scrambling and chaotic encryption, this paper proposes a digital watermarking algorithm that resists geometric attacks by using color images as carriers.The robustness and quality of the proposed watermarking algorithm is tested with different with many statistical measures.Experimental outcomes show that the proposed approach is the best to solve conflict problems between quality and robustness.Also, the proposed approach exhibits worthy robustness against many attacks, such as, conventional attacks, and geometrical attacks.
Classical and deep learning methods for recognizing human activities and modes of transportation with smartphone sensors

The Sussex-Huawei Locomotion-Transportation Recognition Challenge presented a unique opportunity to the activity-recognition community to test their approaches on a large, real-life benchmark dataset with activities different from those typically recognized. The goal of the challenge was to recognize, as accurately as possible, eight locomotion activities (Still, Walk, Run, Bike, Car, Bus, Train, Subway) using smartphone sensor data. This paper describes the method we developed to win this challenge, and provides an analysis of the effectiveness of its components. We used complex feature extraction and selection methods to train classical machine learning models. In addition, we trained deep learning models using a novel end-to-end architecture for deep multimodal spectro-temporal fusion. All the models were fused into an ensemble with the final predictions smoothed by a hidden Markov model to account for temporal dependencies of the activities. The presented method achieved an F1 score of 94.9% on the challenge test data. We tested different sampling frequencies, window sizes, feature types, classification models and the importance of stand-alone sensors and their fusion for the task. Finally, we present an energy-efficient smartphone implementation of the method.
Sense and Sensibility: Characterizing Social Media Users Regarding the Use of Controversial Terms for COVID-19

With the world-wide development of 2019 novel coronavirus, although WHO has officially announced the disease as COVID-19, one controversial term - "Chinese Virus" is still being used by a great number of people. In the meantime, global online media coverage about COVID-19-related racial attacks increases steadily, most of which are anti-Chinese or anti-Asian. As this pandemic becomes increasingly severe, more people start to talk about it on social media platforms such as Twitter. When they refer to COVID-19, there are mainly two ways: using controversial terms like "Chinese Virus" or "Wuhan Virus", or using non-controversial terms like "Coronavirus". In this article, we attempt to characterize the Twitter users who use controversial terms and those who use non-controversial terms. We use the Tweepy API to retrieve 17 million related tweets and the information of their authors. We find the significant differences between these two groups of Twitter users across their demographics, user-level features like the number of followers, political following status, as well as their geo-locations. Moreover, we apply classification models to predict Twitter users who are more likely to use controversial terms. To our best knowledge, this is the first large-scale social media-based study to characterize users with respect to their usage of controversial terms during a major crisis.
CoSDA-ML: Multi-Lingual Code-Switching Data Augmentation for Zero-Shot Cross-Lingual NLP

Multi-lingual contextualized embeddings, such as multilingual-BERT (mBERT), have shown success in a variety of zero-shot cross-lingual tasks. However, these models are limited by having inconsistent contextualized representations of subwords across different languages. Existing work addresses this issue by bilingual projection and fine-tuning technique. We propose a data augmentation framework to generate multi-lingual code-switching data to fine-tune mBERT, which encourages model to align representations from source and multiple target languages once by mixing their context information. Compared with the existing work, our method does not rely on bilingual sentences for training, and requires only one training process for multiple target languages. Experimental results on five tasks with 19 languages show that our method leads to significantly improved performances for all the tasks compared with mBERT.
Bridging the Structural Gap Between Encoding and Decoding for Data-To-Text Generation

Generating sequential natural language descriptions from graph-structured data (e.g., knowledge graph) is challenging, partly because of the structural differences between the input graph and the output text.Hence, popular sequence-to-sequence models, which require serialized input, are not a natural fit for this task.Graph neural networks, on the other hand, can better encode the input graph but broaden the structural gap between the encoder and decoder, making faithful generation difficult.To narrow this gap, we propose DUA-LENC, a dual encoding model that can not only incorporate the graph structure, but can also cater to the linear structure of the output text.Empirical comparisons with strong single-encoder baselines demonstrate that dual encoding can significantly improve the quality of the generated text.