Reviewerly/exHarmony huggingface dataset:
Effect of Logistic Activation Function and Multiplicative Input Noise on DNN-kWTA Model

The dual neural network-based (DNN) k-winner-take-all (kWTA) model is one of the simplest analog neural network models for the kWTA process. This paper analyzes the behaviors of the DNN-kWTA model under these two imperfections. The two imperfections are, (1) the activation function of IO neurons is a logistic function rather than an ideal step function, and (2) there are multiplicative Gaussian noise in the inputs. With the two imperfections, the model may not be able to perform correctly. Hence it is important to estimate the probability of the imperfection model performing correctly. We first derive the equivalent activation function of IO neurons under the two imperfections. Next, we derive the sufficient conditions for the imperfect model to operate correctly. These results enable us to efficiently estimate the probability of the imperfect model generating correct outputs. Additionally, we derive a bound on the probability that the imperfect model generates the desired outcomes for inputs with a uniform distribution. Finally, we discuss how to generalize our findings to handle non-Gaussian multiplicative input noise.
MRCE: A Multi-Representation Collaborative Enhancement Model for Aspect-Opinion Pair Extraction

Aspect-Opinion Pair Extraction (AOPE) is an emerging combination task of fine-grained sentiment analysis. Traditional works devise pipeline frameworks for AOPE, which potentially suffer from error propagation. To solve this problem, numerous joint methods have been proposed recently. However, these joint methods have not simultaneously considered the following three points: (1) hierarchically linguistic information captured by pre-trained language models (PLMs), (2) comprehensive word-level semantic relation learning, (3) explicit and overall subtask interaction modeling. In this paper, we propose a Multi-Representation Collaborative Enhancement (MRCE) model, which can address the above three issues with a joint framework. Extensive experiments on three widely used datasets demonstrate that our model outperforms the state-of-the-art methods.
Multi-level Network Based on Text Attention and Pose-Guided for Person Re-ID

Text-based person Re-ID aims to find the target person’s image from the image gallery under the condition that the text description about the target person is known. Since there is vast modal difference between the image and text, how to effectively match the semantic features of the image-text is extremely important. Existing schemes mainly consider how to extract more accurate text representation or more complete image representation but ignore multi-granularity feature matching. In this paper, we proposed a Text Attention to Multi-level Network model by Pose and attention-guided for Text-based Person Re-ID(PAMN). Specifically, we firstly design a pose-guided image feature extracting model and an attention-driven text semantics representation model to respectively learn multi-granularity features of image and text, and then employ cross-modal projection matching to align them from different granularities so as to obtain high matching accuracy. Experimental validation is performed on the standard datasets CUHK-pedes and the newly proposed datasets ICFG-pedes, and the experimental results show that the performance of our PAMN is better than other existing methods.
Automatic Firearm Detection in Images and Videos Using YOLO-Based Model

In this day and age, we are witness to ever increasing gun violence all around the world. Technology has surpassed all human beliefs where each person can be easily tracked through their mobiles or through the fortitude of CCTV cameras available all across public properties and areas. There is a need to stop gun violence to protect people’s Right to Live. There are several instances appearing in the news daily about deaths caused due to gun violence. An alarm based system can be introduced which tracks the publicly available CCTV footage to look for guns in the open and raise appropriate alarms. In order to achieve this a robust model to identify and classify firearms automatically from videos is required. The aim of this paper is to describe a YOLO-based model which is highly effective in recognizing firearms in videos and mark them in the video such that the model can be further used for further applications such as raising alarms, tracking human beings with firearms etc.
Saliency-Guided Learned Image Compression for Object Detection

With recent advance of computer vision techniques, an increasing amount of image and video content is consumed by machines. However, existing image and video compression schemes are mainly designed for human vision, which are not optimized concerning machine vision. In this paper, we propose a saliency guided learned image compression scheme for machines, where object detection is considered as an example task. To obtain salient regions for machine vision, a saliency map is obtained for each detected object using an existing black-box explanation of neural networks, and maps for multiple objects are merged sophistically into one. Based on a neural network-based image codec, a bitrate allocation scheme has been designed which prunes the latent representation of the image according to the saliency map. During the training of end-to-end image codec, both pixel fidelity and machine vision fidelity are used for performance evaluation, where the degradation in detection accuracy is measured without ground-truth annotation. Experimental results demonstrate that the proposed scheme can achieve up to 14.1% reduction in bitrate with the same detection accuracy compared with the baseline learned image codec.
Improving Knowledge Graph Embedding Using Dynamic Aggregation of Neighbor Information

Knowledge graph embedding represents the embedding of entities and relations in the knowledge graph into a low-dimensional vector space to accomplish the knowledge graph complementation task. Most existing knowledge graph embedding models such as TransE and RotatE based on translational distance models only consider triple-level information. Considering the rich contextual information of entities in the graph, we propose a new approach named DAN, which is able to dynamically aggregate neighbor information. Firstly, the relation is regarded as a transformation operation on heterogeneous neighbors. Neighborhood information is transformed into an homogeneous space with the central node. This solves the problem of transformation between heterogeneous nodes in the knowledge graph. Then, by dynamically aggregating neighbor information, the same entity owns different information embedding in different triples. This method enriches the representation of entity information and can be combined with other embedding models as a general method. We combine our approach with TransE and RotatE models. The experimental results show that DAN can improve the accuracy of the original model. Our best result outperforms the existing state-of-the-art models in link prediction. While our method has a faster convergence rate and higher accuracy with a slightly increased number of parameters.
Wide Ensemble of Interpretable TSK Fuzzy Classifiers with Application to Smartphone Sensor-Based Human Activity Recognition

In this study, by a wide ensemble of Takagi-Sugeno-Kang (TSK) fuzzy sub-classifiers without an individual aggregation step, an interpretable fuzzy classifier (WEIFC) is developed from a novel perspective. WEIFC has two merits: (1) the antecedent parts of fuzzy rules in each sub-classifier of WEIFC are built on a different bootstrapped data subspace, which guarantees both the diversity among the sub-classifiers and the avoidance of the curse of dimensionality caused by the multi-dimensional and even high-dimensional input data. (2) all sub-classifiers on their respective bootstrapped data subspaces can be trained in parallel without an individual aggregation step, which guarantees both the fast training procedure and good interpretability. According to the novel ensemble structure of WEIFC, it becomes very suitable for applications where the classification testing accuracy and interpretability should be well balanced. As a result, WEIFC is applied to the smartphone sensor-based human activity recognition (SSHAR) task to evaluate its practicability. Compared with the thirteen state-of-the-art classification methods for SSHAR, the experimental results on the benchmarking dataset human activity recognition (HAR) show that WEIFC has better classification testing accuracy than most of them.
The EsnTorch Library: Efficient Implementation of Transformer-Based Echo State Networks

Transformer-based models have revolutionized NLP. But in general, these models are highly resource consuming. Based on this consideration, several reservoir computing approaches to NLP have shown promising results. In this context, we propose EsnTorch, a library that implements echo state networks (ESNs) with transformer-based embeddings for text classification. EsnTorch is developed in PyTorch, optimized to work on GPU, and compatible with the transformers and datasets libraries from Hugging Face: the major data science platform for NLP. Accordingly, our library can make use of all the models and datasets available from Hugging Face. A transformer-based ESN implemented in EsnTorch consists of four building blocks: (1) An embedding layer, which uses a transformer-based model to embed the input texts; (2) A reservoir layer, which can implements three kinds of reservoirs: recurrent, linear or null; (3) A pooling layer, which offers three kinds of pooling strategies: mean, last, or None; (4) And a learning algorithm block, which provides six different supervised learning algorithms. Overall, this work falls within the context of sustainable models for NLP.
SMART: A Robustness Evaluation Framework for Neural Networks

Robustness is urgently needed when neural network models are deployed under adversarial environments. Typically, a model learns to separate data points into different classes while training. A more robust model is more resistant to small perturbations within the local microsphere space of a given data point. In this paper, we try to measure the model’s robustness from the perspective of data separability. We propose a modified data separability index Mahalanobis Distance-based Separability Index (MDSI), and present a new robustness evaluation framework Separability in Matrix-form for Adversarial Robustness of neTwork (SMART). Specifically, we use multiple attacks to find adversarial inputs, and incorporate them with clean data points. We use MDSI to evaluate the separability of the new dataset with correct labels and the model’s prediction, and then compute a SMART score to show the model’s robustness. Compared with existing robustness measurement, our framework builds up a connection between data separability and the model’s robustness, showing openness, scalability, and pluggability in architecture. The effectiveness of our method is verified in experiments.
Multi-label Learning with Data Self-augmentation

Multi-label learning (MLL) deals with the problem where each training example is associated with multiple labels. Existing MLL approaches focus on manipulating feature space and modeling label dependencies among labels. However, both of them require additional burden assumps and cannot easily be embedded into existing algorithms, while data augmentation is the more intuitive way to facilitate MLL. Therefore, in this paper, we propose a novel data augmentation method named MLAUG, i.e. Multi-Label learning with data self-Augmentation for MLL. Specifically, to achieve data augmentation, we learn feature correlation and label correlation matrices in feature space and label space simultaneously in an adaptive manner. Guided by the learned correlation matrices, MLAUG is able to refine the original feature and label spaces by linear combinations of other data vectors. In this way, we could obtain semantic-richer feature distribution and smoother label distributions, thus facilitating multi-label predictive performance. Besides, to further improve the model’s performance, we introduce feature and label graph Laplacian regularization for guaranteeing the discriminability and capturing more adequate label correlation, respectively. Extensive experiment results demonstrate the effectiveness of our proposed MLAUG.
Towards a Unified Benchmark for Reinforcement Learning in Sparse Reward Environments

Reinforcement learning in sparse reward environments is challenging and has recently received increasing attention, with dozens of new algorithms proposed every year. Despite promising results demonstrated in various sparse reward environments, this domain lacks a unified definition of a sparse reward environment and an experimentally fair way to compare existing algorithms. These issues significantly affect the in-depth analysis of the underlying problem and hinder further studies. This paper proposes a benchmark to unify the selection of environments and the comparison of algorithms. We first define sparsity to describe the proportion of rewarded states in the entire state space and select environments by this sparsity. Inspired by the sparsity concept, we categorize the existing algorithms into two classes. To provide a fair comparison of different algorithms, we propose a new metric along with a standard protocol for performance evaluation. Primary experimental evaluations of seven algorithms in ten environments provide a startup user guide of the proposed benchmark. We hope the proposed benchmark will promote the research of reinforcement learning algorithms in sparse reward environments. The source code of this work is published on https://github.com/simayuhe/ICONIP_Benchmark.git .
Unsupervised Domain Adaptation Supplemented with Generated Images

With Domain Adaptation we aim to leverage a given source dataset to model a classifier on the target domain. In an unsupervised setting, the goal is to derive class-based features and adapt it to a different domain. Minimizing discrepancy between these domains is one of the most promising directions to training a common classifier. There have been several algorithmic and deep modelling approaches to this problem, all trying to combat the inherent problem of finding an ideal objective function. Recent algorithmic approaches focus on extracting domain-invariant features by minimizing the distributional and geometrical divergence between domains simultaneously. In this work, we present the results of adding generated synthetic images to some renowned shallow algorithmic approaches. Using Generative Adversarial Networks (GANs), we generate images using categorical information from source domain, thus, adding variance and variety to the source data. We present the impact of synthetic data on notable unsupervised domain adaptation algorithms and show an improvement in about 62% of the 80 task results.
Diverse and High-Quality Data Augmentation Using GPT for Named Entity Recognition

Data augmentation technology has been widely used in computer vision and speech with good results. In computer vision and speech, simple manipulation of gold data can achieve great data augmentation effects. However, in NLP (natural language processing), simple operations on data, such as "randomly delete words", "swap word positions", can have a huge impact on the semantics of sentences. This impact on semantics can be devastating for fine-grained tasks like NER (named entity recognition). In this work, we propose a novel model to generate diverse and high-quality data for NER, which is called DHQDA (Diverse and High-Quality Data Augmentation). Our model outputs the data by using a small-scale neural network to prompt the key and value in the transformer block of the PLM (pre-trained language model). Our experimental results demonstrate that DHQDA performs more stable and better than baseline methods on both Chinese and English datasets, whether in rich or low-resource situations.
Deep Learning Based Automobile Identification Application

Today, the high competition among domestic automobile manufacturers is intense situation than previous years. This result gives advantages in a good variety of brands, models, engine sizes and appearances. This can cause some critical issues in recognizing and recalling a car by manufacturer. In addition, an owner may modify some parts of original vehicle such as the head bumper, the rear bumper, and the head light. This modification also affects the people who are looking for pre-owned cars. Despite the fact, the details are mismatch with the vehicle registration book that issued by the Department of Land Transport. From this incident, the researchers implemented a convolutional neural network (CNN) in the identification of vehicle characteristics to reduce the ambiguity for each car’s models. The researchers conducted experiments using five algorithms. SVM, ResNet34, ResNet50 and Inception-ResNetV2. The researchers set up a library of two car models, Toyota Hilux and Honda Civic sedan and Civic Hatchback, including models from past ten years ago until the present. The images are of 224 × 224 pixels. The data are categorized into two sets, a training set has 1,449 images which is counted as 80% of total images and a testing set is having 362 images which is about 20% of total. The total images are 1,811 and 26 Classes. Our experiments compared the accuracies of SVM, ResNet34, ResNet50, and Inception-ResNetV2, which came out to be 21.4%, 55.5%, 66.6%, and 92.8% respectively. As a result, Inception-ResNetV2 outperforms among all other methods.
ReGAE: Graph Autoencoder Based on Recursive Neural Networks

Invertible transformation of large graphs into fixed dimensional vectors (embeddings) remains a challenge. Its overcoming would reduce any operation on graphs to an operation in a vector space. However, most existing methods are limited to graphs with tens of vertices. In this paper we address the above challenge with recursive neural networks – the encoder and the decoder. The encoder network transforms embeddings of subgraphs into embeddings of larger subgraphs, and eventually into the embedding of the input graph. The decoder does the opposite. The dimension of the embeddings is constant regardless of the size of the (sub)graphs. Simulation experiments presented in this paper confirm that our proposed graph autoencoder, ReGAE, can handle graphs with even thousands of vertices.
Generalization Bounds for Set-to-Set Matching with Negative Sampling

The problem of matching two sets of multiple elements, namely set-to-set matching, has received a great deal of attention in recent years. In particular, it has been reported that good experimental results can be obtained by preparing a neural network as a matching function, especially in complex cases where, for example, each element of the set is an image. However, theoretical analysis of set-to-set matching with such black-box functions is lacking. This paper aims to perform a generalization error analysis in set-to-set matching to reveal the behavior of the model in that task.
Recovering Missing Key Information: An Aspect-Guided Generator for Abstractive Multi-document Summarization

Abstractive multi-document summarization (MDS) paraphrases the salient key information scattered across multiple documents. Due to the large length of the documents, most previous methods opt to first extract salient sentence-level information and then summarize it. However, they neglect the aspect information: documents are often well-organized and written down according to certain aspects. The absence of aspects renders the generated summaries not comprehensive and wastes the prior aspect knowledge. To solve the issue, we propose a novel aspect-guided joint learning framework to detect aspect information for guiding the generating process. Specifically, our proposed method adopts feed-forward networks to detect the aspects in the given context. The detected aspect information serves as both constraints of the objective function and supplement information expressed in the context representations. Aspect information is explicitly discovered and exploited to facilitate generating comprehensive summaries. We conduct extensive experiments on the public dataset. The experimental results demonstrate that our proposed method outperforms previous state-of-the-art (SOTA) baselines, achieving a new SOTA performance on the dataset.
Searching for Textual Adversarial Examples with Learned Strategy

Adversarial attacks can help to reveal the vulnerability of neural networks. In the text classification domain, synonym replacement is an effective way to generate adversarial examples. However, the number of replacement combinations grows exponentially with the text length, making the search difficult. In this work, we propose an attack method which combines a synonym selection network and search strategies of beam search and Monte Carlo tree search (MCTS). The synonym selection network learns the patterns of synonyms which have high attack effect. We combine the network with beam search to gain a broader view by multiple search paths, and with MCTS to gain a deeper view by the exploration feedback, so as to effectively avoid local optimum. We evaluate our method with four datasets in a challenging black box setting which requires no access to the victim model’s parameters. Experimental results show that our method can generate high-quality adversarial examples with higher attack success rate and fewer number of victim model queries, and further experiments show that our method has higher transferability on the victim models. The code and data can be obtained via https://github.com/CMACH508/SearchTextualAdversarialExamples .
Adversarial Training with Knowledge Distillation Considering Intermediate Representations in CNNs

A main challenge for training convolutional neural networks (CNNs) is improving the robustness against adversarial examples, which are images with added the artificial perturbations to induce misclassification in a CNNs. This challenge can be solved only by adversarial training, which uses adversarial examples rather than natural images for CNN training. Since its introduction, adversarial training has been continuously refined from various points of view. Some methods focus on constraining CNN outputs between adversarial examples and natural images, resembling knowledge distillation training. Knowledge distillation was originally intended to constrain the outputs of teacher–student CNNs to promote generalization of the student CNN. However, recent methods for knowledge distillation constrain intermediate representations rather than outputs to improve performance for natural images because it directly works well to preserve intraclass cohesiveness. To further investigate adversarial training using recent knowledge distillation methodology (i.e., constraining intermediate representations), we attempted to evaluate this method and compared it with conventional ones. We first visualized intermediate representations and experimentally found that cohesiveness is essential to properly classify not only natural images but also adversarial examples. Then, we devised knowledge distillation using intermediate representations for adversarial training and demonstrated its improved accuracy compared with output constraining for classifying both natural images and adversarial examples.