Reviewerly/exHarmony huggingface dataset:
Stochastic Attention Head Removal: A Simple and Effective Method for Improving Transformer Based ASR Models

Recently, Transformer based models have shown competitive automatic speech recognition (ASR) performance. One key factor in the success of these models is the multi-head attention mechanism. However, for trained models, we have previously observed that many attention matrices are close to diagonal, indicating the redundancy of the corresponding attention heads. We have also found that some architectures with reduced numbers of attention heads have better performance. Since the search for the best structure is time prohibitive, we propose to randomly remove attention heads during training and keep all attention heads at test time, thus the final model is an ensemble of models with different architectures. The proposed method also forces each head independently learn the most useful patterns. We apply the proposed method to train Transformer based and Convolution-augmented Transformer (Conformer) based ASR models. Our method gives consistent performance gains over strong baselines on the Wall Street Journal, AISHELL, Switchboard and AMI datasets. To the best of our knowledge, we have achieved state-of-the-art end-to-end Transformer based model performance on Switchboard and AMI.
Adjusting the Block Interval in PoW Consensus by Block Interval Process Improvement

Blockchain is not widely applied in various fields due to the critical issue of scalability as part of the blockchain trilemma. This issue arises during consensus among the nodes in a public blockchain. To address the issue of low scalability with proof-of-work (PoW) consensus, various methods have been proposed for transaction per second (TPS) improvement. However, no such methods include an improvement in the consensus step. Therefore, to improve PoW public blockchain scalability, it is important to shorten the time required for PoW consensus. This paper proposes a method for minimizing the block intervals that occur during consensus over a PoW blockchain network. A shortened block interval leads to an increase in the probability of three different attacks: selfish mining, double-spending, and eclipse attacks. According to an experiment using Ethereum, with a typical PoW blockchain, it is inevitable to provide rewards for stable block mining in competition between mining pools. To find an optimal block interval in the PoW consensus algorithm, we conducted a four-step experiment. The purpose of this experiment was to verify the difficulty level and issues with Mainnet security. Therefore, considering stale block mining rewards, an optimal block interval is proposed. The Ethereum TPS was improved by at least 200%. Given this finding, it is considered possible to achieve a similar improvement in a different PoW blockchain. On balance, even if the block interval is shorter than that of the PoW Mainnet, network security falls by only 1.21% in Testnet, even with a rise in the stale block rate, while performance is increased at up to 120 TPS, which is three times higher than that in Mainnet.
Research on Migrant Works' Concern Recognition and Emotion Analysis Based on Web Text Data

Based on the characteristics of convenience, autonomy, and equality, online self-media has become an important way for contemporary migrant workers to observe the world, understand society, examine themselves and express their demands. On the basis of the analysis of the domestic migrant works' concerns and their emotion analysis, we crawl data on Weibo about migrant works' topics as the basic corpus of migrant works' concerns, and then uses a combination of TF-IDF and Word2Vec methods to construct a recognition model of migrant workers' concerns. We found that wages, children's education, medical care and returning home are the main concerns of migrant workers. Meanwhile, further emotion analysis of the migrant works' concerns of using a deep learning model fused with Bi-LSTM and CNN was conducted. The results show that the proportion of negative emotion such as worries, complaints and impetuosity was significantly higher than that of other positive and neutral emotion like encourage and comfort. And the time when the negative emotion are concentrated is significantly related to the social events that occur in the corresponding time period. On the one hand, it shows that the concerns and emotion of migrant workers can be effectively observed and predicted through web text data. On the other hand, it also shows that the core well-being issues of migrant workers in the process of urban integration have not been effectively solved, and the government and relevant departments need to take targeted measures and give priority attention.
Multi-wavelet guided deep mean-shift prior for image restoration

Image restoration is essentially recognized as an ill-posed problem. A promising solution in recent years is incorporating deep network-driven priors into the iterative restoration procedure as constrained conditions. Among them, deep mean-shift prior utilizes the denoising autoencoder to play the role of prior updating. In this study, we present multiple wavelets guided deep mean-shift prior, which integrates the advantages of structural representation in the wavelet transform and learning ability in deep network. Specifically, by re-arranging the multi-view and multi-resolution features generated by multiple wavelet transforms as the input of denoising autoencoder, a more powerful prior information is learned. It benefits from the recurrent structure-preserving and multi-view complementary aggregation properties. We embed the learned prior information into the iterative recovery process and adopt proximal gradient descent to tackle it. Extensive experiments on image deblurring and compressed sensing tasks demonstrated significantly improved performances both visually and quantitatively.
Modal Regression-Based Graph Representation for Noise Robust Face Hallucination

Manifold learning-based face hallucination technologies have been widely developed during the past decades. However, the conventional learning methods always become ineffective in noise environment due to the least-square regression, which usually generates distorted representations for noisy inputs they employed for error modeling. To solve this problem, in this article, we propose a modal regression-based graph representation (MRGR) model for noisy face hallucination. In MRGR, the modal regression-based function is incorporated into graph learning framework to improve the resolution of noisy face images. Specifically, the modal regression-induced metric is used instead of the least-square metric to regularize the encoding errors, which admits the MRGR to robust against noise with uncertain distribution. Moreover, a graph representation is learned from feature space to exploit the inherent typological structure of patch manifold for data representation, resulting in more accurate reconstruction coefficients. Besides, for noisy color face hallucination, the MRGR is extended into quaternion (MRGR-Q) space, where the abundant correlations among different color channels can be well preserved. Experimental results on both the grayscale and color face images demonstrate the superiority of MRGR and MRGR-Q compared with several state-of-the-art methods.
Towards heterogeneous multi-dimensional variability modeling in cyber-physical production systems

Cyber-Physical Production Systems (CPPSs) are complex systems interacting with their environment by sensors and actuators. Such systems typically have a long lifespan, over which a plethora of variants are developed and maintained. The heterogeneity of hardware and software components used in CPPSs and the multiple disciplines (mechanical, electrical, software engineering) involved in the development and maintenance of CPPSs, however, make it difficult to manage their variability. Specifically, variability needs to be expressed in and across multiple disciplines, which use heterogeneous methods and tools. This also affects configuration as well as co-evolution of models and artifacts. In this short paper, we discuss our first ideas towards a Heterogeneous Multi-Dimensional Variability Modeling approach for CPPSs. Our approach builds on and extends existing work to address the challenges of modeling the variability of CPPSs and supporting their configuration and evolution. We showcase our idea using a case study system and outline a research agenda.
Video Quality Assessment With Serial Dependence Modeling

Video quality assessment (VQA) is much more challenging than image quality assessment, due to the difficulty of modeling temporal influence among frames. Most of the existing VQA methods usually isolate each moment within the video (i.e., it neglects the sequential nature), leading to a large gap from the subjective perception. Recent research on neuroscience suggests a serially dependent perception (SDP) mechanism in the human visual system (HVS). Namely, the HVS tends to incorporate the recent past visual experience to predict the present perception. Inspired by the SDP, we suggest that the HVS prefers stable and continuous degradations in videos due to their predictability, and exhibits less tolerance to interrupted and unpredictable disturbances. Thus, we introduce a novel serial dependence modeling (SDM) framework for full-reference VQA in this paper. Firstly, the instantaneous degradation is measured on both the static appearance and motion information for each glimpse of scenes. Since motion plays an important role in videos, two types of structures are extracted for motion representation, namely, an explicit content-based 3D structure and an implicit feature-based 2D structure. Next, an assessment-directed long-short term memory (A-LSTM) is proposed to capture the serial dependence among instantaneous degradations. With the consideration of the perceptual effect from the previous moment on the current one, especially the effect from the perceptually worst moment, the serially dependent degradation is characterized. Finally, by mimicking the subjective rating for video-viewing, an attention-based quality decision procedure is presented to acquire the final video quality. Experimental results on publicly available VQA databases demonstrate that the proposed method maintains good consistency with the subjective perception.
HSGMP

Semantic relationship information is important to the image-text retrieval task. Existing work usually extract relationship information by calculating the relationship value pairwise, which is hardly to find out a meaningful semantic relationship. A more reasonable method is to convert the modal to a scene graph, thereby explicitly modeling the relationship. Scene graph is a kind of graph data structure modeling the scene of modality. There are two concept in a scene graph, object and relationship. In image modal, object indicates the image region and relationship represents the predicate of the image regions. In text modal, object indicates the entity and relationship represents the association between entities, also known as semantic relationship. In image-text retrieval task, both object and relationship are important, and a key challenge is to obtain semantic information. In this paper, image and text are represented as two kinds of scene graphs: visual scene graph and textual scene graph, and then they are combined into Heterogeneous Scene Graph(HSG). By explicitly modeling relationships using directed graph, the information can be passed edge-wise. To further extract semantic information, we introduce the metapath, which can extract specific semantic information on specified path. Moreover, we propose Heterogeneous Message Passing(HMP) to communicate information on the metapath. After the message passing, the similarity of two modalities can be represented as the similarity of the graphs. Experiment shows that the model achieve competitive results on Flickr30K and MSCOCO, which indicates that our approach has advantages in image-text retrieval.
Identification of Edge Disconnections in Networks Based on Graph Filter Outputs

Graphs are fundamental mathematical structures used in various fields to model statistical and physical relationships between data, signals, and processes. In some applications, such as data processing in graphs that represent physical networks, the initial network topology is known. However, disconnections of edges in the network change the topology and may affect the signals and processes over the network. In this paper, we consider the problem of edge disconnection identification in networks by using concepts from graph signal processing (GSP). We assume that the graph signals measured over the vertices of the network can be represented as white noise that has been filtered on the graph topology by a smooth graph filter. We develop the likelihood ratio test (LRT) to detect a specific set of edge disconnections. Then, we provide the maximum likelihood (ML) decision rule for identifying general scenarios of edge disconnections in the network. It is shown that the sufficient statistics of the LRT and ML decision rule are the graph-frequency energy levels in the graph spectral domain. However, the ML decision rule leads to a high-complexity exhaustive search over the edges in the network and is practically infeasible. Thus, we propose a low-complexity greedy method that identifies a single disconnected edge at each iteration. Moreover, by using the smoothness of the considered graph filter, we suggest a local implementation of the decision rule, which is based solely on the measurements at neighboring vertices. Simulation results demonstrate that the proposed methods outperform existing detection and identification methods on a synthetic dataset and for line outage identification in power systems from the IEEE 118-bus test case.
The extraction of the training subset for the spatial distribution modelling of the heavy metals in topsoil

The choice of the method of raw data dividing into train and test subsets in the models based on the artificial neural networks (ANN) is one of the underexplored problems of continuous Spatio-temporal field interpolation. Selecting the best training subset for modelling the spatial distribution of elements in the topsoil is not a trivial task since the sampling points are not equivalent. Errors and outliers, which are present in the distribution, can be misleading. Most of the points, which contain utility information, should be involved in modelling. The raw data were Chromium (Cr) and Manganese (Mn) contents in the topsoil on the residential areas in Noyabrsk and Novy Urengoy (South Part) cities (Russian subarctic zone). A three-step algorithm for extraction of the raw data dividing into training and test subsets for modelling the spatial distribution of the feature presented. The spatial distributions of the element contents in the topsoil, which constructed by the multilayer perceptron (MLP), consider spatial heterogeneity and training rules. The MLP structure was chosen by the minimization of the root mean squared error (RMSE). For each territory, according to the number of hits in the training subset, the points divided into three classes: the «elite», «middle», and «useless». Considering this information at the stage of dividing the raw sample makes it possible to increase the accuracy of the predictive model.
Graph-Time Convolutional Neural Networks

Spatiotemporal data can be represented as a process over a graph, which captures their spatial relationships either explicitly or implicitly. How to leverage such a structure for learning representations is one of the key challenges when working with graphs. In this paper, we represent the spatiotemporal relationships through product graphs and develop a first principle graph-time convolutional neural network (GTCNN). The GTCNN is a compositional architecture with each layer comprising a graph-time convolutional module, a graphtime pooling module, and a nonlinearity. We develop a graph-time convolutional filter by following the shift-and-sum principles of the convolutional operator to learn higher-level features over the product graph. The product graph itself is parametric so that we can learn also the spatiotemporal coupling from data. We develop a zero-pad pooling that preserves the spatial graph (the prior about the data) while reducing the number of active nodes and the parameters. Experimental results with synthetic and real data corroborate the different components and compare with baseline and state-of-the-art solutions.
Learning to combine the modalities of language and video for temporal moment localization

Temporal moment localization aims to retrieve the best video segment matching a moment specified by a query. The existing methods generate the visual and semantic embeddings independently and fuse them without full consideration of the long-term temporal relationship between them. To address these shortcomings, we introduce a novel recurrent unit, cross-modal long short-term memory (CM-LSTM), by mimicking the human cognitive process of localizing temporal moments that focuses on the part of a video segment related to the part of a query, and accumulates the contextual information across the entire video recurrently. In addition, we devise a two-stream attention mechanism for both attended and unattended video features by the input query to prevent necessary visual information from being neglected. To obtain more precise boundaries, we propose a two-stream attentive cross-modal interaction network (TACI) that generates two 2D proposal maps obtained globally from the integrated contextual features, which are generated by using CM-LSTM, and locally from boundary score sequences and then combines them into a final 2D map in an end-to-end manner. On the TML benchmark dataset, ActivityNet-Captions, the TACI outperforms state-of-the-art TML methods with [email protected] of 45.50% and 27.23% for [email protected] and [email protected], respectively. In addition, we show that the revised state-of-the-arts methods by replacing original LSTM with our CM-LSTM achieves performance gains.
Time-Series Deep-Learning Classifier for Human Activity Recognition Based On Smartphone Built-in Sensors

Abstract Human Activity Recognition (HAR) is gaining more interest in recent years due to its growing role in many human-related sectors such as the health sector especially with elderly people and motion restricted patients. In recent years, there has been great progress in identifying human activity using various machine learning approaches. However, traditional methods of feature extraction are the most challenging in the feature selection process. Deep learning is a promising approach in the human activity recognition research area and has overcome the feature selection problem. However, several challenges are still open to research issues such as classification performance. This paper describes how to identify specific types of human physical activities using the accelerator and gyroscope data generated by the smartphone user. A deep convolutional neural network architecture has been proposed to perform HAR efficiently and effectively the system has been trained and tested over a dataset generated with the aid of 50 volunteers with four activities (walking, running, walking up-down stairs finally sitting-standing on the chair) events in real-world conditions. We chose four classes, each of which performs well, get to know our range of activities achieving 99% for validation and 99.8% for testing overall accuracy
Partial Key Exposure Attack on Short Secret Exponent CRT-RSA

Let (N, e) be an RSA public key, where \(N=pq\) is the product of equal bitsize primes p, q. Let \(d_p, d_q\) be the corresponding secret CRT-RSA exponents.
Analogical Reasoning With Deep Learning-Based Symbolic Processing

The authors propose analogical reasoning systems based on first-order predicate logic using deep learning. The proposed systems consist of a model combining recursive neural networks and Word2Vec. When unknown data is input in this trained model, similar outputs to those of the trained data are obtained. Previous studies on symbolic inference using deep learning have focused on deductive and inductive inferences, and they require rule templates that are manually created and provided to the models in advance. However, it is not enough to infer unknown rules from a large amount of data on the Internet. Analogical reasoning is a mode of inference that attempts to solve a new case based on similarity. Thus, it enables inference of rules that were challenging to comprehend in previous studies. As a result of the experiments, unknown rules can be efficiently inferred from known rules contained in the knowledge bases. Using the proposed systems, the authors can extract unknown rules from five of the eight datasets consisting of three types of knowledge bases described in Prolog. The proposed method is the first case of analogical reasoning based on the first-order predicate logic using deep learning. Furthermore, the models used in the proposed systems have shown high robustness when using Word2Vec. For this reason, it is suggested that the practical applications of the proposed systems enable efficient inferences from a large amount of data that include noisy data on the Internet.
Algorithmic Problems in the Symbolic Approach to the Verification of Automatically Synthesized Cryptosystems

Automated methods can be used to generate cryptosystems by combining the primitives in an arbitrary fashion, to weed out insecure cryptosystems, and to prove the security of those that survive. In this paper, we study several algorithmic problems arising from the verification of automatically synthesized cryptosystems built from block ciphers, in a theory that includes ACUN. One of these is static equivalence to an algorithm that produces a sequence of random terms. The other is invertibility, the problem of determining whether, given an automatically synthesized cryptosystem, built from block ciphers, and the ability to compute inverses, is it always possible to compute the original plaintext from the ciphertext? We show that static equivalence to random in this theory is undecidable in general. In addition, we identify a reasonable special case for which there is a decidable condition implying security, along with an algorithm for verifying it. For invertibility, we identify a reasonable class of cryptosystems for which invertibility is equivalent to a simple syntactic condition that can be easily verified.
Class Imbalance Issue in Software Defect Prediction Models by various Machine Learning Techniques: An Empirical Study

Software practitioners are continuing to build advanced software defect prediction (SDP) models to help the tester find fault-prone modules. However, the Class Imbalance (CI) problem consists of uncommonly few defective instances, and more non-defective instances cause inconsistency in the performance. We have conducted 880 experiments to analyze the variation in the performance of 10 SDP models by concerning the class imbalance problem. In our experiments, we have used 22 public datasets consists of 41 software metrics, 10 baseline SDP methods, and 4 sampling techniques. We used Mathews Correlation Coefficient (MCC), which is more useful when a dataset is highly imbalanced. We have also compared the predictive performance of various ML models by applying 4 sampling techniques. To examine the performance of different SDP models, we have used the F-measure. We found the performance of the learning models is unsatisfactory, which needs to mitigate. We have also found a few surprising results, some logical patterns between classifier and sampling technique. It provides a connection between sampling technique, software matrices, and a classifier.