Reviewerly/exHarmony huggingface dataset:
Bring Ancient Murals Back to Life

Digital inpainting of murals has always been a challenging problem. The damage forms in real murals are complex, such as cracks, flaking, and fading. There are many difficulties in applying deep learning technology to mural inpainting. First, data sets are often difficult to obtain. Second, the network based on supervised learning is unfit to be applied to the real multiple mural damages, which makes the network unpromotable. Third, the output of deep neural network is the combination of the unmasked area in the label image and the corresponding masked area in the generated image, so there is no change in the unmasked area. Murals often fade or change color after a hundred years or more, which leads to the lack of aesthetic feeling in the repaired images. We propose a mural inpainting model based on the translation method with three domains, including a SVD block and a dense spatial attention with mask block. Specifically, the model trains two Variational Auto-Encoders to respectively map the real mural images and the clean mural images to two deep spaces, the mapping network learns the transformation between the two deep spaces by paired data. This transformation can well extend to real mural images. Experiments show that the performance of our model is better than the comparative methods, and the visual quality is improved.
Deps-SAN: Neural Machine Translation with Dependency-Scaled Self-Attention Network

Syntax knowledge contributes its powerful strength in Neural machine translation (NMT) tasks. Early NMT works supposed that syntax details can be automatically learned from numerous texts via attention networks. However, succeeding researches pointed out that limited by the uncontrolled nature of attention computation, the NMT model requires an external syntax to capture the deep syntactic awareness. Although existing syntax-aware NMT methods have born great fruits in combining syntax, the additional workloads they introduced render the model heavy and slow. Particularly, these efforts scarcely involve the Transformer-based NMT and modify its core self-attention network (SAN). To this end, we propose a parameter-free, Dependency-scaled Self-Attention Network (Deps-SAN) for syntax-aware Transformer-based NMT. A quantified matrix of dependency closeness between tokens is constructed to impose explicit syntactic constraints into the SAN for learning syntactic details and dispelling the dispersion of attention distributions. Two knowledge sparsing techniques are further integrated to avoid the model overfitting the dependency noises introduced by the external parser. Experiments and analyses on IWSLT14 German-to-English and WMT16 German-to-English benchmark NMT tasks verify the effectiveness of our approach.
Universal Distributional Decision-Based Black-Box Adversarial Attack with Reinforcement Learning

The vulnerability of the high-performance machine learning models implies a security risk in applications with real-world consequences. Research on adversarial attacks is beneficial in guiding the development of machine learning models on the one hand and finding targeted defenses on the other. However, most of the adversarial attacks today leverage the gradient or logit information from the models to generate adversarial perturbation. Works in the more realistic domain: decision-based attacks, which generate adversarial perturbation solely based on observing the output label of the targeted model, are still relatively rare and mostly use gradient-estimation strategies. In this work, we propose a pixel-wise decision-based attack algorithm that finds a distribution of adversarial perturbation through a reinforcement learning algorithm. We call this method Decision-based Black-box Attack with Reinforcement learning (DBAR). Experiments show that the proposed approach outperforms state-of-the-art decision-based attacks with a higher attack success rate and greater transferability.
Hierarchical Down-Sampling Based Ultra High-Resolution Image Inpainting

In recent years, image inpainting techniques have received a great deal of attention in the field of image processing. Many useful inpainting algorithms have been proposed, such as automatically removing some objects or repairing damaged images. Since the image inpainting prediction hole part from an already known area, the original input information was needed, then down-sampling it easy to tract. But most methods suffer from image quality degradation at the down-sampling part because the general average or maximum value-based down-sampling loses pixel information. In addition, Performing a large calculation at once time costs a lot of memory that makes them only can handle the input smaller than 1 K. In this paper, we propose a new ultra-high resolution image inpainting method that fully utilizes and densifies pixels by down-sampling the images without extra processing such as calculation, adapting it to existing models resulting in higher quality inpainting with ultra high-resolution images (more than 2 K images), and also can adjust the down-sampling to an arbitrary level for your need. The experiments demonstrate that the proposed HdCRAE quantitatively and qualitatively outperforms state-of-the-art.
A Comparative Analysis of Loss Functions for Handling Foreground-Background Imbalance in Image Segmentation

Foreground-Background (F-B) imbalance problem has emerged as a fundamental challenge to building accurate image segmentation models in computer vision. F-B imbalance problem occurs due to a disproportionate ratio of observations of foreground and background samples. It degrades the model performance of image segmentation applications. Several loss functions are proposed to improve the classification performance of models on the traditional imbalanced data. But there is no sufficient information on whether these loss functions can improve the accuracy performance of the image segmentation models in the presence of F-B imbalance problem. In this paper, we perform a comparative analysis between four loss functions, namely Focal loss, Dice loss, Tversky and Mixed Focal loss to handle the F-B imbalance problem on datasets from various domains. We embed each loss function in the deep learning model using U-Net architecture and experimentally perform the evaluation in terms of accuracy, precision, F1-score and recall metrics. Then, we present a comparative discussion to the researchers to identify the appropriate loss function that achieves better detection accuracy in different domains.
Detecting and Mitigating Backdoor Attacks with Dynamic and Invisible Triggers

When a deep learning-based model is attacked by backdoor attacks, it behaves normally for clean inputs, whereas outputs unexpected results for inputs with specific triggers. This causes serious threats to deep learning-based applications. Many backdoor detection methods have been proposed to address these threats. However, these defenses can only work on the backdoored models attacked by static trigger(s). Recently, some backdoor attacks with dynamic and invisible triggers have been developed, and existing detection methods cannot defend against these attacks. To address this new threat, in this paper, we propose a new defense mechanism that can detect and mitigate backdoor attacks with dynamic and invisible triggers. We reverse engineer generators that transform clean images into backdoor images for each label. The generated images by the generator can help to detect the existence of a backdoor and further remove it. To the best of our knowledge, our work is the first work to defend against backdoor attacks with dynamic and invisible triggers. Experiments on multiple datasets show that the proposed method can effectively detect and mitigate the backdoor with dynamic and invisible triggers in deep learning-based models.
Image Super-Resolution Based on Adaptive Feature Fusion Channel Attention

Since the advent of SENet, existing image super-resolution models based on deep learning have been keen to improve networks’ cross-channel understanding and characterization capabilities by introducing channel attention into the network. However, we note that previous channel attention uses one or two global pooling methods as preprocessing methods to extract channel description scalars and treat each channel description scalar equally. Doing so will undoubtedly lose some of the feature information and bring some inflexibility, resulting in the network’s inability to distinguish each feature channel very well. To address it, we propose adaptive feature fusion channel attention(AFFCA). Specifically, we use diverse global pooling methods to extract feature description scalars and adaptively adjust each scalar’s weights to merge them into a representative feature description scalar. Our method can extract enough information from the feature map to further enhance the ability to distinguish between channels so as to optimize the channel attention performance. We have done some experiments and compared them with the existing methods. The results show that our AFFCA achieves better accuracy and visual improvements against current methods.
SATNet: Captioning with Semantic Alignment and Feature Enhancement

The fusion of region and grid features based on location alignment can make the utilization of image features better to a certain extent, thus improving the accuracy of image captioning. However, it still inevitably introduces semantic noise because of spatial misalignment. To address the problem, this paper proposes a novel image captioning model based on semantic alignment and feature enhancement, which contains a Visual Features Adaptive Alignment Module (VFAA) and a Features Enhancement Module (FEM). The VFAA module, at the encoder layer, utilizes Visual Semantic Graph (VSG) to generate pure semantic information for more accurately guiding the alignment and fusion of the region and grid features, thus further reducing the semantic noise caused by spatial dislocation. In addition, to ensure that the features that eventually enter the decoder layer do not lose their specific attributes, we design the FEM module to fuse the original region and grid features. To validate the effectiveness of the proposed model, we conduct extensive experiments on the MS-COCO dataset and test it on the online test server. The experimental results show that our model is superior to many state-of-the-art methods.
Understanding Graph and Understanding Map and Their Potential Applications

Based on a previously proposed data structure, Understanding Tree, this paper introduces two data structures: Understanding Graph and Understanding Map, and explores their potential applications. Understanding Graph and Understanding Map are special cases of semantic network. They are constructed according to concept definitions. Potential applications of them include quantitatively measuring a concept’s complexity degree, measuring a concept’s importance degree in a domain, and computing an optimized learning sequence for comprehending a concept etc.
Incorporating Generation Method and Discourse Structure to Event Coreference Resolution

Event coreference resolution is an important task of information extraction. Previous work often focused on modeling the sentence structure, ignoring the structure between paragraphs which is also important to event coreference resolution. Moreover, almost all previous work modeled event coreference resolution as a classification task. In this paper, we introduce macro discourse structure to help event coreference resolution through a Relational Graph Convolutional Network (R-GCN), which can take advantage of structure and relations between paragraphs. Moreover, we are the first to introduce an encoder-decoder style generation model to further boost event coreference resolution task. The experimental results on the English KBP2016 and KBP2017 datasets show that our model CGECR (Classification and Generation models for Event Coreference Resolution) outperforms the SOTA baselines.
Evolutionary Action Selection for Gradient-Based Policy Learning

Evolutionary Algorithms (EAs) and Deep Reinforcement Learning (DRL) have recently been integrated to take advantage of both methods for better exploration and exploitation. The evolutionary part of these hybrid methods maintains a population of policy networks. However, existing methods focus on optimizing the parameters of policy network, which is usually high-dimensional and tricky for EA. In this paper, we shift the target of evolution from high-dimensional parameter space to low-dimensional action space. We propose Evolutionary Action Selection-Twin Delayed Deep Deterministic Policy Gradient (EAS-TD3), a novel hybrid method of EA and DRL. In EAS, we focus on optimizing the action chosen by the policy network and attempt to obtain high-quality actions to promote policy learning through an evolutionary algorithm. We conduct several experiments on challenging continuous control tasks. The result shows that EAS-TD3 shows superior performance over other state-of-art methods.
BBSN: Bilateral-Branch Siamese Network for Imbalanced Multi-label Text Classification

In multi-label text classification, the numbers of instances in different categories are usually extremely imbalanced. How to learn good models from imbalanced data is a challenging task. Some existing works tackle it through class re-balancing strategies or imbalanced loss objectives, but their performance remains limited in the cases of imbalanced distributed data. In this work, we propose a model, which combined Siamese Network and Bilateral-Branch Network to deal with both representation learning and classifier learning simultaneously. In the siamese network component, we propose a category-specific similarity strategy to improve the representation learning and adapt a novelty dynamic learning mechanism to make the model end-to-end trainable, and in the bilateral-branch network, we adopt the cumulative learning strategy to shift the learning focus from universal pattern to tail learning. In general, we adopt a multi-task architecture to ensure that both the head categories and the tail categories are adequately trained. The experiments on two benchmark datasets show that our method can improve the performance on the entire and tail categories, and achieves competitive performance compared with existing approaches.
A Measurement-Based Quantum-Like Language Model for Text Matching

Quantum language models have attracted extensive attention in natural language processing tasks. Nevertheless, in this field measurement operators are mostly generated by randomly initialized parameter matrices, which cannot well explain the role of measurement operators in quantum theory. In this paper, we propose a Measurement-Based Quantum-like Language Model (MBQLM). Specifically, each word is considered a fundamental event in quantum probability space, which is a quantum state represented by a density matrix. We take the word density matrix in one sentence as a set of measurement operators to measure another sentence, which is consistent with the definition of measurement operators in quantum theory and has a specific semantic interpretation. The measured sentence state matrix representation effectively interact the feature information between sentences. To evaluate the performance of our model, we conduct experiments on WikiQA and TREC-QA datasets. The results show that our model achieves better performance than all benchmarks.
Towards Accurate Alignment and Sufficient Context in Scene Text Recognition

Encoder-decoder framework has recently become cutting-edge in scene text recognition (STR), where most decoder networks consist of two parts: an attention model to align visual features from the encoder for each character, and a linear or LSTM-based model to predict label sequence. However, it is difficult for these attention models to obtain accurate alignment, and linear or LSTM model usually captures limited context. To emphasize the role of character feature alignment, we separate the attention alignment module from the decoder network in this work, forming an Encoder-Alignment-Decoder framework. Under this framework, we propose a deformable attention based model to accurately align visual features of each character. In this alignment model, we explicitly learn the spatial coordinate information of each character from the input reading order sequence and optimize it with learnable sampled offsets in the attention block to obtain accurate aligned features. To address the lack of context, we explore transformer-based decoder to capture global context by multi-head attention, where a mask matrix is integrated to keep attention weights focused on the relevant context during the decoding. Extensive experiments demonstrate the effectiveness of the Encoder-Alignment-Decoder framework in STR, achieving better performance than other language free methods with significant improvement on most benchmark STR datasets, and obtain the state-of-the-art performance on several datasets by integrating a language model.
Rethinking Image Inpainting with Attention Feature Fusion

Recent image inpainting models have archived significant progress through learning from large-scale data. However, restoring images under complicated scenarios (e.g. large masks or complex textures) remains challenging. We argue that the inadequate learning of global structure and local texture could lead to the artifacts and blur of current models. Inspired by feature fusion methods, we utilize Attention Feature Fusion (AFF) to better aggregate the different levels of features within our inpainting model from two perspectives. 1) We insert AFF through skip connections to pass long-distance textures to late semantics; 2) Our modified multi-dilated blocks with AFF residual could fuse features in different receptive fields. Both strategies aim to strengthen the texture and structure aggregation and reduce the inconsistency of semantics during learning. We show quantitatively and qualitatively that our approach outperforms current methods on benchmark datasets.
Lee metrics on groups

In this work we consider interval metrics on groups; that is, integral invariant metrics whose associated weight functions do not have gaps.We give conditions for a group to have and not to have interval metrics.Then we study Lee metrics on general groups, that is interval metrics having the finest unitary symmetric associated partition.These metrics generalize the classic Lee metric on cyclic groups.In the case that G is a torsion-free group or a finite group of odd order, we prove that G has a Lee metric if and only if G is cyclic.Also, if G is a group admitting Lee metrics then G × Z k 2 always has Lee metrics for every k ∈ N.Then, we show that some families of metacyclic groups, such as cyclic, dihedral, and dicyclic groups, always have Lee metrics.Finally, we give conditions for non-cyclic groups such that they do not have Lee metrics.We end with tables of all groups of order ≤ 31 indicating which of them have (or have not) Lee metrics and why (not).
Angelo Sotira

DeviantART is the world’s largest virtual art community , in which artists and enthusiasts can meet, share, and sell their art. Founded in 2000 by Angelo Sotira, Matthew Stephens, and Scott Jarkoff, it was one of the first social networks, pioneering in permitting users to create free profiles, a space to post artworks and converse. Innovative features of DeviantART include the ability to comment with a drawing tool, instead of words. Llamas became an important mascot to the DeviantART community, added in response to an Internet meme as something that could be gifted between users. Sotira bootstrap funded DeviantART in 2000 with $15k, and it grew on cash flow until 2007 when it raised $3.5 million Series A from software developer DivX. In 2009, DeviantART raised a further $10 million and launched a world tour to ten cities globally, connecting the community in real life. The platform now sustains 30 million members, 65 million unique visitors per month, and 350 million artworks. In 2017, Wix acquired DeviantART for $36 million.
An Integrated INS/GNSS System With an Attention-Based Deep Network for Drones in GNSS Denied Environments

We propose a neural network-based approach to assist positioning in an integrated inertial navigation system/global navigation satellite system (INS/GNSS) during GNSS interruptions. One can aid a navigation system during GNSS outages by substituting the GNSS measurements with a well-trained neural network (NN). Outputs of this NN can then be used in a Kalman filtering scheme to acquire the best consistent and concise estimates according to the INS measurements. Since this problem has inherent spatial and temporal aspects, the proposed NN should account for both aspects simultaneously. Convolutional long short-term memory (CLSTM) structure is an excellent candidate to satisfy the aforementioned requirement. The designed CLSTM architecture uses the angular rates and specific forces measured by INS to output the pseudo GNSS position increment covering for the lost GNSS signal. An attention mechanism is added to the final layer of the CLSTM to counter the gradient vanishing problem in long time-series prediction. A field test utilizing a fixed-wing unmanned aerial vehicle is arranged to evaluate the proposed architecture's performance. The field test result shows a significant 3D positioning improvement during GNSS outages.
Intuitionistic Fuzzy Universum Support Vector Machine

The classical support vector machine is an effective classification technique. It solves a convex optimization problem to give a global solution. But it suffers from noise and outliers. To deal with this, an intuitionistic fuzzy number (IFN) is assigned to the training samples which reduces the effect of noise. In this paper, we propose intuitionistic fuzzy universum support vector machine (IFUSVM), where IFN is assigned to the training data points in presence of universum data. Universum points lead to prior knowledge about data distribution and assignment of IFN to the data points reduces the effect of outliers and noise. Thus, leading to the enhanced generalization property of the model. Numerical experimental results and statistical analysis over 17 binary benchmark UCI datasets show the superiority of the proposed model over the baseline models in terms of rank and accuracy.
VAAC: V-value Attention Actor-Critic for Cooperative Multi-agent Reinforcement Learning

This paper explores value-decomposition methods in cooperative multi-agent reinforcement learning (MARL) under the paradigm of centralized training with decentralized execution. These methods decompose a global shared value into individual ones to guide the learning of decentralized policies. While Q-value decomposition methods such as QMIX show state-of-the-art performance, V-value decomposition methods are proposed to obtain a reasonable trade-off between training efficiency and algorithm performance under the A2C training paradigm. However, existing V-value decomposition methods lack theoretical analysis of the relation between the global V-value and local V-values, and do not explicitly consider the influence of individuals on the total system, which degrades their performance. To address these problems, this paper proposes a novel approach called V-value Attention Actor-Critic (VAAC) for cooperative MARL. We theoretically derive a general decomposing formulation of the global V-value in terms of local V-values of individual agents, and implement it with a multi-head attention formation to model the impact of individuals on the whole system for interpretability of decomposition. Evaluations on the challenging StarCraft II micromanagement task show that VAAC achieves a better trade-off between training efficiency and algorithm performance, and provides interpretability for its decomposition process.