Reviewerly/exHarmony huggingface dataset:
Towards Transparent and Explainable Attention Models

Recent studies on interpretability of attention distributions have led to notions of faithful and plausible explanations for a model's predictions.Attention distributions can be considered a faithful explanation if a higher attention weight implies a greater impact on the model's prediction.They can be considered a plausible explanation if they provide a humanunderstandable justification for the model's predictions.In this work, we first explain why current attention mechanisms in LSTM based encoders can neither provide a faithful nor a plausible explanation of the model's predictions.We observe that in LSTM based encoders the hidden representations at different time-steps are very similar to each other (high conicity) and attention weights in these situations do not carry much meaning because even a random permutation of the attention weights does not affect the model's predictions.Based on experiments on a wide variety of tasks and datasets, we observe attention distributions often attribute the model's predictions to unimportant words such as punctuation and fail to offer a plausible explanation for the predictions.To make attention mechanisms more faithful and plausible, we propose a modified LSTM cell with a diversity-driven training objective that ensures that the hidden representations learned at different time steps are diverse.We show that the resulting attention distributions offer more transparency as they (i) provide a more precise importance ranking of the hidden states (ii) are better indicative of words important for the model's predictions (iii) correlate better with gradient-based attribution methods.Human evaluations indicate that the attention distributions learned by our model offer a plausible explanation of the model's predictions.Our code has been made publicly available at https://github.com/akashkm99/Interpretable-Attention IntroductionQuestion 1: What is the best way to improve my spoken English soon ?
Impact of Stemming and Word Embedding on Deep Learning-Based Arabic Text Categorization

Document classification is a classical problem in information retrieval, and plays an important role in a variety of applications. Automatic document classification can be defined as content-based assignment of one or more predefined categories to documents. Many algorithms have been proposed and implemented to solve this problem in general, however, classifying Arabic documents is lagging behind similar works in other languages. In this paper, we present seven deep learning-based algorithms to classify the Arabic documents. These are: Convolutional Neural Network (CNN), CNN-LSTM (LSTM = Long Short-Term Memory), CNN-GRU (GRU = Gated Recurrent Units), BiLSTM (Bidirectional LSTM), BiGRU, Att-LSTM (Attention-based LSTM), and Att-GRU. And for word representation, we applied the word embedding technique (Word2Vec). We tested our approach on two large datasets-with six and eight categories-using ten-fold cross-validation. Our objective was to study how the classification is affected by the stemming strategies and word embedding. First, we looked into the effects of different stemming algorithms on the document classification with different deep learning models. We experimented with eleven different stemming algorithms, broadly falling into: root-based and stem-based, and no stemming. We performed ANOVA test on the classification results using the different stemmers, which helps assure if the results are significant. The results of our study indicate that stem-based algorithms perform slightly better compared to root-based algorithms. Among the deep learning models, the Attention mechanism and the Bidirectional learning gave outstanding performance with Arabic text categorization. Our best performance is F-score = 97.96%, achieved using the Att-GRU model with stem-based algorithm. Next, we looked into different controlling parameters for word embedding. For Word2Vec, both skip-gram and bag-of-words (CBOW) perform well with either stemming strategies. However, when using a stem-based algorithm, skip-gram achieves good results with a vector of smaller dimension, while CBOW requires a larger dimension vector to achieve a similar performance.
High-Fidelity Controlled- <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"><mml:mi>Z</mml:mi></mml:math> Gate with Maximal Intermediate Leakage Operating at the Speed Limit in a Superconducting Quantum Processor

We introduce the sudden variant (SNZ) of the Net Zero scheme realizing controlled-$Z$ (CZ) gates by baseband flux control of transmon frequency. SNZ CZ gates operate at the speed limit of transverse coupling between computational and non-computational states by maximizing intermediate leakage. The key advantage of SNZ is tuneup simplicity, owing to the regular structure of conditional phase and leakage as a function of two control parameters. We realize SNZ CZ gates in a multi-transmon processor, achieving $99.93\pm0.24\%$ fidelity and $0.10\pm0.02\%$ leakage. SNZ is compatible with scalable schemes for quantum error correction and adaptable to generalized conditional-phase gates useful in intermediate-scale applications.
SSD vs. YOLO for Detection of Outdoor Urban Advertising Panels under Multiple Variabilities

This work compares Single Shot MultiBox Detector (SSD) and You Only Look Once (YOLO) deep neural networks for the outdoor advertisement panel detection problem by handling multiple and combined variabilities in the scenes. Publicity panel detection in images offers important advantages both in the real world as well as in the virtual one. For example, applications like Google Street View can be used for Internet publicity and when detecting these ads panels in images, it could be possible to replace the publicity appearing inside the panels by another from a funding company. In our experiments, both SSD and YOLO detectors have produced acceptable results under variable sizes of panels, illumination conditions, viewing perspectives, partial occlusion of panels, complex background and multiple panels in scenes. Due to the difficulty of finding annotated images for the considered problem, we created our own dataset for conducting the experiments. The major strength of the SSD model was the almost elimination of False Positive (FP) cases, situation that is preferable when the publicity contained inside the panel is analyzed after detecting them. On the other side, YOLO produced better panel localization results detecting a higher number of True Positive (TP) panels with a higher accuracy. Finally, a comparison of the two analyzed object detection models with different types of semantic segmentation networks and using the same evaluation metrics is also included.
Deep Learning Based Abstractive Text Summarization: Approaches, Datasets, Evaluation Measures, and Challenges

In recent years, the volume of textual data has rapidly increased, which has generated a valuable resource for extracting and analysing information. To retrieve useful knowledge within a reasonable time period, this information must be summarised. This paper reviews recent approaches for abstractive text summarisation using deep learning models. In addition, existing datasets for training and validating these approaches are reviewed, and their features and limitations are presented. The Gigaword dataset is commonly employed for single-sentence summary approaches, while the Cable News Network (CNN)/Daily Mail dataset is commonly employed for multisentence summary approaches. Furthermore, the measures that are utilised to evaluate the quality of summarisation are investigated, and Recall-Oriented Understudy for Gisting Evaluation 1 (ROUGE1), ROUGE2, and ROUGE-L are determined to be the most commonly applied metrics. The challenges that are encountered during the summarisation process and the solutions proposed in each approach are analysed. The analysis of the several approaches shows that recurrent neural networks with an attention mechanism and long short-term memory (LSTM) are the most prevalent techniques for abstractive text summarisation. The experimental results show that text summarisation with a pretrained encoder model achieved the highest values for ROUGE1, ROUGE2, and ROUGE-L (43.85, 20.34, and 39.9, respectively). Furthermore, it was determined that most abstractive text summarisation models faced challenges such as the unavailability of a golden token at testing time, out-of-vocabulary (OOV) words, summary sentence repetition, inaccurate sentences, and fake facts.
The transformer earthquake alerting model: a new versatile approach to earthquake early warning

SUMMARY Earthquakes are major hazards to humans, buildings and infrastructure. Early warning methods aim to provide advance note of incoming strong shaking to enable preventive action and mitigate seismic risk. Their usefulness depends on accuracy, the relation between true, missed and false alerts and timeliness, the time between a warning and the arrival of strong shaking. Current approaches suffer from apparent aleatoric uncertainties due to simplified modelling or short warning times. Here we propose a novel early warning method, the deep-learning based transformer earthquake alerting model (TEAM), to mitigate these limitations. TEAM analyses raw, strong motion waveforms of an arbitrary number of stations at arbitrary locations in real-time, making it easily adaptable to changing seismic networks and warning targets. We evaluate TEAM on two regions with high seismic hazard, Japan and Italy, that are complementary in their seismicity. On both data sets TEAM outperforms existing early warning methods considerably, offering accurate and timely warnings. Using domain adaptation, TEAM even provides reliable alerts for events larger than any in the training data, a property of highest importance as records from very large events are rare in many regions.
HEDGE: Efficient Traffic Classification of Encrypted and Compressed Packets

As the size and source of network traffic increase, so does the challenge of monitoring and analysing network traffic. Therefore, sampling algorithms are often used to alleviate these scalability issues. However, the use of high entropy data streams, through the use of either encryption or compression, further compounds the challenge as current state of the art algorithms cannot accurately and efficiently differentiate between encrypted and compressed packets. In this work, we propose a novel traffic classification method named HEDGE (High Entropy DistinGuishEr) to distinguish between compressed and encrypted traffic. HEDGE is based on the evaluation of the randomness of the data streams and can be applied to individual packets without the need to have access to the entire stream. Findings from the evaluation show that our approach outperforms current state of the art. We also make available our statistically sound dataset, based on known benchmarks, to the wider research community.
With Little Power Comes Great Responsibility

Despite its importance to experimental design, statistical power (the probability that, given a real effect, an experiment will reject the null hypothesis) has largely been ignored by the NLP community.Underpowered experiments make it more difficult to discern the difference between statistical noise and meaningful model improvements, and increase the chances of exaggerated findings.By metaanalyzing a set of existing NLP papers and datasets, we characterize typical power for a variety of settings and conclude that underpowered experiments are common in the NLP literature.In particular, for several tasks in the popular GLUE benchmark, small test sets mean that most attempted comparisons to state of the art models will not be adequately powered.Similarly, based on reasonable assumptions, we find that the most typical experimental design for human rating studies will be underpowered to detect small model differences, of the sort that are frequently studied.For machine translation, we find that typical test sets of 2000 sentences have approximately 75% power to detect differences of 1 BLEU point.To improve the situation going forward, we give an overview of best practices for power analysis in NLP and release a series of notebooks to assist with future power analyses. 1
Multi-attention fusion modeling for sentiment analysis of educational big data

As an important branch of natural language processing, sentiment analysis has received increasing attention. In teaching evaluation, sentiment analysis can help educators discover the true feelings of students about the course in a timely manner and adjust the teaching plan accurately and timely to improve the quality of education and teaching. Aiming at the inefficiency and heavy workload of college curriculum evaluation methods, a Multi-Attention Fusion Modeling (Multi-AFM) is proposed, which integrates global attention and local attention through gating unit control to generate a reasonable contextual representation and achieve improved classification results. Experimental results show that the Multi-AFM model performs better than the existing methods in the application of education and other fields.
Incorporating Virtual Reality Technology in Safety Training Solution for Construction Site of Urban Cities

The lack of experiential training has become the primary cause of incidents that could have been easily avoided in construction areas, and the implementation of experimental training is crucial in minimizing incidents at construction sites. The limitation on the available training tools has made it impossible to build up a real test site or reproduce the constructing scenarios. This research aims to develop an immersive and interactive multiplayer-based training platform that incorporates virtual reality (VR) technology to improve the safety awareness of workers. The developed simulation platform serves as a training solution, enabling the provision of repeatable and flexible procedures within a secure environment. An evaluation survey was conducted to make a comparison between traditional training methods and the proposed VR solution. Promisingly, the results indicate that workers were better trained under the developed immersive environment, and they could memorize critical points more effectively because the implementation of VR technology can allow people to experience hazardous situations without being physically injured, thus creating a safer and more efficient training environment. This study reveals that the nomination of the proposed VR platform could reap many benefits and become an advantageous tool for construction training, as well as stimulate human-machine interaction research.
Bilateral Grid Learning for Stereo Matching Networks

Real-time performance of stereo matching networks is important for many applications, such as automatic driving, robot navigation and augmented reality (AR). Although significant progress has been made in stereo matching networks in recent years, it is still challenging to balance real-time performance and accuracy. In this paper, we present a novel edge-preserving cost volume upsampling module based on the slicing operation in the learned bilateral grid. The slicing layer is parameter-free, which allows us to obtain a high quality cost volume of high resolution from a low-resolution cost volume under the guide of the learned guidance map efficiently. The proposed cost volume upsampling module can be seamlessly embedded into many existing stereo matching networks, such as GCNet, PSMNet, and GANet. The resulting networks are accelerated several times while maintaining comparable accuracy. Furthermore, we design a real-time network (named BGNet) based on this module, which outperforms existing published real-time deep stereo matching networks, as well as some complex networks on the KITTI stereo datasets. The code is available at https://github.com/YuhuaXu/BGNet.
Online Class-Incremental Continual Learning with Adversarial Shapley Value

As image-based deep learning becomes pervasive on every device, from cell phones to smart watches, there is a growing need to develop methods that continually learn from data while minimizing memory footprint and power consumption. While memory replay techniques have shown exceptional promise for this task of continual learning, the best method for selecting which buffered images to replay is still an open question. In this paper, we specifically focus on the online class-incremental setting where a model needs to learn new classes continually from an online data stream. To this end, we contribute a novel Adversarial Shapley value scoring method that scores memory data samples according to their ability to preserve latent decision boundaries for previously observed classes (to maintain learning stability and avoid forgetting) while interfering with latent decision boundaries of current classes being learned (to encourage plasticity and optimal learning of new class boundaries). Overall, we observe that our proposed ASER method provides competitive or improved performance compared to state-of-the-art replay-based continual learning methods on a variety of datasets.
Variational quantum algorithm for the Poisson equation

The Poisson equation has wide applications in many areas of science and engineering. Although there are some quantum algorithms that can efficiently solve the Poisson equation, they generally require a fault-tolerant quantum computer, which is beyond the current technology. We propose a variational quantum algorithm (VQA) to solve the Poisson equation, which can be executed on noisy intermediate-scale quantum devices. In detail, we first adopt the finite-difference method to transform the Poisson equation into a linear system. Then, according to the special structure of the linear system, we find an explicit tensor product decomposition, with only $(2{log}_{2}n+1)$ items, of its coefficient matrix under a specific set of simple operators, where $n$ is the dimension of the coefficient matrix. This implies that the proposed VQA needs fewer quantum measurements, which dramatically reduces the required quantum resources. Additionally, we design observables to efficiently evaluate the expectation values of the simple operators on a quantum computer. Numerical experiments demonstrate that our algorithm can solve the Poisson equation.
Least squares support vector machine for class imbalance learning and their applications to fault detection of aircraft engine

Imbalanced problems often occur when the size of majority class is bigger than that of the minority one. The Least squares support vector machine (LSSVM) is an effective method for solving classification problem on balanced datasets. However, LSSVM has bad performance on minority class facing with class imbalance learning for the classification boundary skewing toward the majority class. In order to overcome the drawback, LSSVM for class imbalance learning (LSSVM-CIL) is proposed. LSSVM-CIL utilizes two different regularization parameters C+ and C− that evaluate different misclassification costs. Furthermore, a method of combining reduced technique and recursive strategy is proposed to reduce the size of support vectors and retain representative samples. In addition, decomposition of the matrices via Cholesky factorization is employed as a solution to enhance the computational stability. Furthermore, the effectiveness of the two algorithms presented in this paper is confirmed with experimental results on various real-world imbalanced datasets. Fault detection of aircraft engine can be regarded as a CIL problem and has the demand for the real time. Finally, experiments on aircraft engine indicate that the two algorithms can be selected as candidate techniques for fault detection of aircraft engine.
Cloud height and tracking accuracy of three all sky imager systems for individual clouds

Solar irradiance nowcasts can be derived with sky images from all sky imagers (ASI) by detecting and analyzing transient clouds, which are the main contributor of intra-hour solar irradiance variability. The accuracy of ASI based solar irradiance nowcasting systems depends on various processing steps. Two vital steps are the cloud height detection and cloud tracking. This task is challenging, due to the atmospheric conditions that are often complex, including various cloud layers moving in different directions simultaneously. This challenge is addressed by detecting and tracking individual clouds. For this, we developed two distinct ASI nowcasting approaches with four or two cameras and a third hybridized approach. These three systems create individual 3-D cloud models with unique attributes including height, position, size, optical properties and motion. This enables us to describe complex multi-layer conditions. In this paper, derived cloud height and motion vectors are compared with a reference ceilometer (height) and shadow camera system (motion) over a 30 day validation period. The validation data set includes a wide range of cloud heights, cloud motion patterns and atmospheric conditions. Furthermore, limitations of ASI based nowcasting systems due to image resolution and image perspective constrains are discussed. The most promising system is found to be the hybridized approach. This approach uses four ASIs and a voxel carving based cloud modeling combined with a cloud segmentation independent stereoscopic cloud height and tracking detection. We observed for this approach an overall mean absolute error of 648 m for the height, 1.3 m/s for the cloud speed and 16.2° for the motion direction.
A Novel GRU-RNN Network Model for Dynamic Path Planning of Mobile Robot

A dynamic path planning method based on a gated recurrent unit-recurrent neural network model is proposed for the problem of path planning of a mobile robot in an unknown space.A deep neural network with sensor input is used to generate a new control strategy output to the physical model to control the movement of the robot and thus achieve collision avoidance behavior.Inputs and tags are derived from sample sets generated by an improved artificial potential field and an improved ant colony optimization algorithm.In order to make the ant colony algorithm converge quickly, the pheromone trail and the state transition probability are improved.The field function of the artificial potential field method is modified.Using the end-to-end network model to learn the mapping between input and output in the sample data, the direction and speed of the mobile robot are obtained.The simulation experiments and realistic simulations show that the network model can plan a reasonable path in an unknown environment.Compared with other traditional path planning algorithms, the proposed method is more robust than the traditional path planning algorithms to differences in the robot structure.
Regimes of Classical Simulability for Noisy Gaussian Boson Sampling

As a promising candidate for exhibiting quantum computational supremacy, Gaussian boson sampling (GBS) is designed to exploit the ease of experimental preparation of Gaussian states. However, sufficiently large and inevitable experimental noise might render GBS classically simulable. In this work, we formalize this intuition by establishing a sufficient condition for approximate polynomial-time classical simulation of noisy GBS-in the form of an inequality between the input squeezing parameter, the overall transmission rate, and the quality of photon detectors. Our result serves as a nonclassicality test that must be passed by any quantum computational supremacy demonstration based on GBS. We show that, for most linear-optical architectures, where photon loss increases exponentially with the circuit depth, noisy GBS loses its quantum advantage in the asymptotic limit. Our results thus delineate intermediate-sized regimes where GBS devices might considerably outperform classical computers for modest noise levels. Finally, we find that increasing the amount of input squeezing is helpful to evade our classical simulation algorithm, which suggests a potential route to mitigate photon loss.