Reviewerly/exHarmony huggingface dataset:
RAB: Provable Robustness Against Backdoor Attacks

Recent studies have shown that deep neural networks (DNNs) are vulnerable to adversarial attacks, including evasion and backdoor (poisoning) attacks. On the defense side, there have been intensive efforts on improving both empirical and provable robustness against evasion attacks; however, the provable robustness against backdoor attacks still remains largely unexplored. In this paper, we focus on certifying the machine learning model robustness against general threat models, especially backdoor attacks. We first provide a unified framework via randomized smoothing techniques and show how it can be instantiated to certify the robustness against both evasion and backdoor attacks. We then propose the first robust training process, RAB, to smooth the trained model and certify its robustness against backdoor attacks. We prove the robustness bound for machine learning models trained with RAB and prove that our robustness bound is tight. In addition, we theoretically show that it is possible to train the robust smoothed models efficiently for simple models such as K-nearest neighbor classifiers, and we propose an exact smooth-training algorithm that eliminates the need to sample from a noise distribution for such models. Empirically, we conduct comprehensive experiments for different machine learning (ML) models such as DNNs, support vector machines, and K-NN models on MNIST, CIFAR-10, and ImageNette datasets and provide the first benchmark for certified robustness against backdoor attacks. In addition, we evaluate K-NN models on a spambase tabular dataset to demonstrate the advantages of the proposed exact algorithm. Both the theoretic analysis and the comprehensive evaluation on diverse ML models and datasets shed light on further robust learning strategies against general training time attacks.
A Survey on Contextual Embeddings

Contextual embeddings, such as ELMo and BERT, move beyond global word representations like Word2Vec and achieve ground-breaking performance on a wide range of natural language processing tasks. Contextual embeddings assign each word a representation based on its context, thereby capturing uses of words across varied contexts and encoding knowledge that transfers across languages. In this survey, we review existing contextual embedding models, cross-lingual polyglot pre-training, the application of contextual embeddings in downstream tasks, model compression, and model analyses.
Mapping the Underground: Supervised Discovery of Cybercrime Supply Chains

Understanding the sequences of processes needed to perform a cybercrime is crucial for effective interventions. However, generating these supply chains currently requires time-consuming manual effort. We propose a method that leverages machine learning and graph-based analysis to efficiently extract supply chains from cybercrime forums. Our supply chain detection algorithm can identify 33% and 42% relevant chains within major English and Russian forums, respectively, showing improvements over the baselines of 11% and 5%, respectively. Our analysis of the supply chains demonstrates underlying connections between products and services that are potentially useful understanding and undermining the illicit activity of these forums. For example, our extracted supply chains illuminate cash out and money laundering techniques and their importance to the functioning of these forums.
Deep Residual Autoencoder for Blind Universal JPEG Restoration

In this paper we propose a deep residual autoencoder exploiting Residual-in-Residual Dense Blocks (RRDB) to remove artifacts in JPEG compressed images that is independent from the Quality Factor (QF) used. The proposed approach leverages both the learning capacity of deep residual networks and prior knowledge of the JPEG compression pipeline. The proposed model operates in the YCbCr color space and performs JPEG artifact restoration in two phases using two different autoencoders: the first one restores the luma channel exploiting 2D convolutions; the second one, using the restored luma channel as a guide, restores the chroma channels explotining 3D convolutions. Extensive experimental results on three widely used benchmark datasets (i.e. LIVE1, BDS500, and CLASSIC-5) show that our model is able to outperform the state of the art with respect to all the evaluation metrics considered (i.e. PSNR, PSNR-B, and SSIM). This results is remarkable since the approaches in the state of the art use a different set of weights for each compression quality, while the proposed model uses the same weights for all of them, making it applicable to images in the wild where the QF used for compression is unkwnown. Furthermore, the proposed model shows a greater robustness than state-of-the-art methods when applied to compression qualities not seen during training.
Computing Fuzzy Bisimulations for Fuzzy Structures Under the Gödel Semantics

Bisimulation is a well-known notion in modal logic and the theory of labeled transition systems. It is used for characterizing indiscernibility between states and has important applications in minimizing structures, separating expressive powers of modal and related logics, as well as concept learning in description logics (DLs). Fuzzy bisimulation is a counterpart of bisimulation for dealing with fuzzy structures. In this article, we present an efficient algorithm with a complexity O((m+n)n) for computing the greatest fuzzy bisimulation between two finite fuzzy interpretations in the fuzzy DL <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f</i> <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ALC</i> under the Gödel semantics, where n is the number of individuals and m is the number of nonzero instances of roles in the given fuzzy interpretations. We also adapt our algorithm for computing fuzzy bisimulations and simulations between fuzzy finite automata, as well as for dealing with other fuzzy DLs. The resulting algorithms are much more efficient than the previously known ones, as they reduce the complexity from O(n <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">5</sup> ) to O((m+n)n).
Energy estimation for differential drive mobile robots on straight and rotational trajectories

Energy autonomy is an important aspect that needs to be improved in order to increase efficiency in mobile robotic tasks. Having accurate power models allows the estimation of energy consumption along different trajectories. This article proposes a power model for two-wheel differential drive mobile robots. The proposed model takes into account the dynamic parameters of the robot and its motors, and predicts the energy consumption for trajectories with variable accelerations and variable payloads. The experimental validation of the proposed model was performed with a Nomad Super Scout II mobile robot which was driven along straight and curved trajectories, with different payloads and accelerations. The experiments using the proposed model showed accuracies of 96.67% along straight trajectories and 81.25% along curved trajectories in the estimation of energy consumption.
Spin-projection for quantum computation: A low-depth approach to strong correlation

Although spin is a core property in fermionic systems, its symmetry can be easily violated in a variational simulation, especially when strong correlation plays a vital role therein. In this study, we will demonstrate that the broken spin-symmetry can be restored exactly in a quantum computer, with little overhead in circuits, while delivering additional strong correlation energy with the desired spin quantum number. The proposed scheme permits drastic reduction of a potentially large number of measurements required to ensure spin-symmetry by employing a superposition of only a few rotated quantum states. Our implementation is universal, simple, and, most importantly, straightforwardly applicable to any ansatz proposed to date.
Learning One-Clock Timed Automata

We present an algorithm for active learning of deterministic timed automata with a single clock. The algorithm is within the framework of Angluin’s $$L^*$$ algorithm and inspired by existing work on the active learning of symbolic automata. Due to the need of guessing for each transition whether it resets the clock, the algorithm is of exponential complexity in the size of the learned automata. Before presenting this algorithm, we propose a simpler version where the teacher is assumed to be smart in the sense of being able to provide the reset information. We show that this simpler setting yields a polynomial complexity of the learning process. Both of the algorithms are implemented and evaluated on a collection of randomly generated examples. We furthermore demonstrate the simpler algorithm on the functional specification of the TCP protocol.
Named Entity Recognition in Spanish Biomedical Literature: Short Review and Bert Model

Entity Recognition (NER) is the first step for knowledge acquisition when we deal with an unknown corpus of texts. Having received these entities, we have an opportunity to form parameters space and to solve problems of text mining as concept normalization, speech recognition, etc. The recent advances in NER are related to the technology of contextualized word embeddings, which transforms text to the form being effective for Deep Learning. In the paper, we show how NER model detects pharmacological substances, compounds, and proteins in the dataset obtained from the Spanish Clinical Case Corpus (SPACCC). To achieve this goal, we train from scratch the BERT language representation model and fine-tune it for our problem. As it is expected, this model shows better results than the NER model trained over the standard word embeddings. We further conduct an error analysis showing the origins of models' errors and proposing strategies to further improve the model's quality.
Feature selection using principal component analysis and genetic algorithm

Feature engineering is the way toward utilizing domain knowledge of the records to build features that in turn assist Machine Learning (ML) algorithms to provide efficient results. It is crucial to the utilization of ML and is both difficult and costly. The next buzz word after big data is feature engineering, which involves both feature selection and feature extraction. Feature Selection (FS also called attribute selection) is a procedure of selecting a subset of pertinent features for use in model building. It is an optimization problem. In our case, we have used principal component analysis for feature transformation followed by genetic algorithm to select optimal feature set and in the last, decision tree as a classifier. The proposed approach shows that use of principal component analysis before genetic algorithms improves the accuracy of the model with less number of features.
Intuitionistic fuzzy approach for enhancement of low contrast mammogram images

Abstract Mammogram image enhancement is very much necessary in diagnosing breast cancer or tumor at an early stage. Nonuniform illumination and low contrast images are commonly encountered in mammogram images. Conventional enhancement algorithms produce either some artifacts or cannot highlight minute details present in the images, particularly when dealing with mammogram images. In this article, we propose a new mammogram image enhancement scheme using Atanassov's intuitionistic fuzzy set (IFS) theory. IFS considers two uncertainties—membership and nonmembership degree apart from membership degree as in fuzzy set theory. As mammogram images are low contrast images and many of the image definitions are vague/unclear, so IFS theory may be suitable for better image enhancement. Initially, the image is transformed to an intuitionistic fuzzy image using a novel intuitionistic fuzzy generator. Hesitation degree is computed and using the hesitation degree, two membership levels are computed to form an interval type 2 fuzzy set. These two membership functions are then combined using Zadeh's fuzzy t‐conorm to form a new membership function. Threshold of interval type 2 fuzzy image is obtained using restricted equivalence function. Using the threshold, modified fuzzy hyperbolization is carried out. Real data experiments demonstrate that the proposed algorithm has better performance on contrast and visual quality of the images both quantitatively and qualitatively when compared with different existing methods.
Fast Rates for General Unbounded Loss Functions: From ERM to Generalized Bayes

We present new excess risk bounds for general unbounded loss functions including log loss and squared loss, where the distribution of the losses may be heavy-tailed. The bounds hold for general estimators, but they are optimized when applied to η-generalized Bayesian, MDL, and empirical risk minimization estimators. In the case of log loss, the bounds imply convergence rates for generalized Bayesian inference under misspecification in terms of a generalization of the Hellinger metric as long as the learning rate η is set correctly. For general loss functions, our bounds rely on two separate conditions: the v-GRIP (generalized reversed information projection) conditions, which control the lower tail of the excess loss; and the newly introduced witness condition, which controls the upper tail. The parameter v in the v-GRIP conditions determines the achievable rate and is akin to the exponent in the Tsybakov margin condition and the Bernstein condition for bounded losses, which the v-GRIP conditions generalize; favorable v in combination with small model complexity leads to O(1/n) rates. The witness condition allows us to connect the excess risk to an “annealed” version thereof, by which we generalize several previous results connecting Hellinger and Renyi divergence to KL divergence.
Modeling Discourse Structure for Document-level Neural Machine Translation

Recently, document-level neural machine translation (NMT) has become a hot topic in the community of machine translation.Despite its success, most of existing studies ignored the discourse structure information of the input document to be translated, which has shown effective in other tasks.In this paper, we propose to improve document-level NMT with the aid of discourse structure information.Our encoder is based on a hierarchical attention network (HAN) (Miculicich et al., 2018).Specifically, we first parse the input document to obtain its discourse structure.Then, we introduce a Transformer-based path encoder to embed the discourse structure information of each word.Finally, we combine the discourse structure information with the word embedding before it is fed into the encoder.Experimental results on the Englishto-German dataset show that our model can significantly outperform both Transformer and Transformer+HAN.
Unsupervised Domain Adaptation with Dual-Scheme Fusion Network for Medical Image Segmentation

Domain adaptation aims to alleviate the problem of retraining a pre-trained model when applying it to a different domain, which requires large amount of additional training data of the target domain. Such an objective is usually achieved by establishing connections between the source domain labels and target domain data. However, this imbalanced source-to-target one way pass may not eliminate the domain gap, which limits the performance of the pre-trained model. In this paper, we propose an innovative Dual-Scheme Fusion Network (DSFN) for unsupervised domain adaptation. By building both source-to-target and target-to-source connections, this balanced joint information flow helps reduce the domain gap to further improve the network performance. The mechanism is further applied to the inference stage, where both the original input target image and the generated source images are segmented with the proposed joint network. The results are fused to obtain more robust segmentation. Extensive experiments of unsupervised cross-modality medical image segmentation are conducted on two tasks -- brain tumor segmentation and cardiac structures segmentation. The experimental results show that our method achieved significant performance improvement over other state-of-the-art domain adaptation methods.
Unsupervised Dual Paraphrasing for Two-stage Semantic Parsing

One daunting problem for semantic parsing is the scarcity of annotation.Aiming to reduce nontrivial human labor, we propose a two-stage semantic parsing framework, where the first stage utilizes an unsupervised paraphrase model to convert an unlabeled natural language utterance into the canonical utterance.The downstream naive semantic parser accepts the intermediate output and returns the target logical form.Furthermore, the entire training process is split into two phases: pre-training and cycle learning.Three tailored self-supervised tasks are introduced throughout training to activate the unsupervised paraphrase model.Experimental results on benchmarks OVERNIGHT and GE-OGRANNO demonstrate that our framework is effective and compatible with supervised training.
Cross-Modality Relevance for Reasoning on Language and Vision

This work deals with the challenge of learning and reasoning over language and vision data for the related downstream tasks such as visual question answering (VQA) and natural language for visual reasoning (NLVR).We design a novel cross-modality relevance module that is used in an end-to-end framework to learn the relevance representation between components of various input modalities under the supervision of a target task, which is more generalizable to unobserved data compared to merely reshaping the original representation space.In addition to modeling the relevance between the textual entities and visual entities, we model the higher-order relevance between entity relations in the text and object relations in the image.Our proposed approach shows competitive performance on two different language and vision tasks using public benchmarks and improves the state-of-the-art published results.The learned alignments of input spaces and their relevance representations by NLVR task boost the training efficiency of VQA task.
Educating Text Autoencoders: Latent Representation Guidance via Denoising

Generative autoencoders offer a promising approach for controllable text generation by leveraging their latent sentence representations. However, current models struggle to maintain coherent latent spaces required to perform meaningful text manipulations via latent vector operations. Specifically, we demonstrate by example that neural encoders do not necessarily map similar sentences to nearby latent vectors. A theoretical explanation for this phenomenon establishes that high capacity autoencoders can learn an arbitrary mapping between sequences and associated latent representations. To remedy this issue, we augment adversarial autoencoders with a denoising objective where original sentences are reconstructed from perturbed versions (referred to as DAAE). We prove that this simple modification guides the latent space geometry of the resulting model by encouraging the encoder to map similar texts to similar latent representations. In empirical comparisons with various types of autoencoders, our model provides the best trade-off between generation quality and reconstruction capacity. Moreover, the improved geometry of the DAAE latent space enables zero-shot text style transfer via simple latent vector arithmetic.
Dynamic Face Video Segmentation via Reinforcement Learning

For real-time semantic video segmentation, most recent works utilised a dynamic framework with a key scheduler to make online key/non-key decisions. Some works used a fixed key scheduling policy, while others proposed adaptive key scheduling methods based on heuristic strategies, both of which may lead to suboptimal global performance. To overcome this limitation, we model the online key decision process in dynamic video segmentation as a deep reinforcement learning problem and learn an efficient and effective scheduling policy from expert information about decision history and from the process of maximising global return. Moreover, we study the application of dynamic video segmentation on face videos, a field that has not been investigated before. By evaluating on the 300VW dataset, we show that the performance of our reinforcement key scheduler outperforms that of various baselines in terms of both effective key selections and running speed. Further results on the Cityscapes dataset demonstrate that our proposed method can also generalise to other scenarios. To the best of our knowledge, this is the first work to use reinforcement learning for online key-frame decision in dynamic video segmentation, and also the first work on its application on face videos.
Current Status, Issues, and Challenges of Blockchain Applications in Education

Due to the unique characteristics of decentralization and security, blockchain is believed to have considerable potential to provide a wide range of benefits for ed-ucation development. Its application in education is relatively new but increasing very quickly. This paper introduced the typical blockchain techniques and charac-teristics briefly. Then, recent applications of blockchain in education were sum-marized comprehensively, especially those regarding learning record keeper, cer-tificate issue and management, and decentralized education ecosystem. Finally, technical and non-technical challenges were discussed. It is hoped to provide an in-depth look at the perspectives of blockchain in evolving education and help to the development of new application systems.
Image Super-Resolution Based on Generative Adversarial Networks: A Brief Review

Single image super resolution (SISR) is an important research content in the field of computer vision and image processing. With the rapid development of deep neural networks, different image super-resolution models have emerged. Compared to some traditional SISR methods, deep learning-based methods can complete the superresolution tasks through a single image. In addition, compared with the SISR methods using traditional convolutional neural networks, SISR based on generative adversarial networks (GAN) has achieved the most advanced visual performance. In this review, we first explore the challenges faced by SISR and introduce some common datasets and evaluation metrics. Then, we review the improved network structures and loss functions of GAN-based perceptual SISR. Subsequently, the advantages and disadvantages of different networks are analyzed by multiple comparative experiments. Finally, we summarize the paper and look forward to the future development trends of GAN-based perceptual SISR.
A Novel Multichannel Dilated Convolution Neural Network for Human Activity Recognition

A novel multichannel dilated convolution neural network for improving the accuracy of human activity recognition is proposed. The proposed model utilizes the multichannel convolution structure with multiple kernels of various sizes to extract multiscale features of high-dimensional data of human activity during convolution operation and not to consider the use of the pooling layers that are used in the traditional convolution with dilated convolution. Its advantage is that the dilated convolution can first capture intrinsical sequence information by expanding the field of convolution kernel without increasing the parameter amount of the model. And then, the multichannel structure can be employed to extract multiscale gait features by forming multiple convolution paths. The open human activity recognition dataset is used to evaluate the effectiveness of our proposed model. The experimental results showed that our model achieves an accuracy of 95.49%, with the time to identify a single sample being approximately 0.34 ms on a low-end machine. These results demonstrate that our model is an efficient real-time HAR model, which can gain the representative features from sensor signals at low computation and is hopeful for the effective tool in practical applications.