Reviewerly/exHarmony huggingface dataset:
Hierarchical Transfer Learning Architecture for Low-Resource Neural Machine Translation

Neural Machine Translation(NMT) has achieved notable results in high-resource languages, but still works poorly on low-resource languages.As times goes on, It is widely recognized that transfer learning methods are effective for low-resource language problems.However, existing transfer learning methods are typically based on the parent-child architecture, which does not adequately take advantages of helpful languages.In this paper, inspired by human transitive inference and learning ability, we handle this issue by proposing a new hierarchical transfer learning architecture for low-resource languages.In the architecture, the NMT model is trained in the unrelated high-resource language pair, the similar intermediate language pair and the low-resource language pair in turn.Correspondingly, the parameters are transferred and fine-tuned layer by layer for initialization.In this way, our hierarchical transfer learning architecture simultaneously combines the data volume advantages of high-resource languages and the syntactic similarity advantages of cognate languages.Specially, we utilize Byte Pair Encoding(BPE) and character-level embedding for data pre-processing, which effectively solve the problem of out of vocabulary(OOV).Experimental results on Uygur-Chinese and Turkish-English translation demonstrate the superiorities of the proposed architecture over the NMT model with parent-child architecture.
PRADO: Projection Attention Networks for Document Classification On-Device

Recently, there has been a great interest in the development of small and accurate neural networks that run entirely on devices such as mobile phones, smart watches and IoT.This enables user privacy, consistent user experience and low latency.Although a wide range of applications have been targeted from wake word detection to short text classification, yet there are no on-device networks for long text classification.We propose a novel projection attention neural network PRADO that combines trainable projections with attention and convolutions.We evaluate our approach on multiple large document text classification tasks.Our results show the effectiveness of the trainable projection model in finding semantically similar phrases and reaching high performance while maintaining compact size.Using this approach, we train tiny neural networks just 200 Kilobytes in size that improve over prior CNN and LSTM models and achieve near state of the art performance on multiple long document classification tasks.We also apply our model for transfer learning, show its robustness and ability to further improve the performance in limited data scenarios.
Quantum phase transition with inhomogeneous driving in the Lechner-Hauke-Zoller model

We study the zero-temperature phase diagram of the Lechner-Hauke-Zoller model. An analytic expression for the free energy and critical coefficients for finite-size systems and in the thermodynamic limit are derived and numerically verified. With the aim to improve standard quantum annealing, we introduce an inhomogeneously driven transverse field with an additional time-dependent parameter that allows one to evade the first-order quantum phase transition and, thus, improve the efficiency of the ground-state preparation considerably.
Developing End-to-End Control Policies for Robotic Swarms Using Deep Q-learning

In this study, the use of a popular deep reinforcement learning algorithm – deep Q-learning – in developing end-to-end control policies for robotic swarms is explored. Robots only have limited local sensory capabilities; however, in a swarm, they can accomplish collective tasks beyond the capability of a single robot. Compared with most automatic design approaches proposed so far, which belong to the field of evolutionary robotics, deep reinforcement learning techniques provide two advantages: (i) they enable researchers to develop control policies in an end-to-end fashion; and (ii) they require fewer computation resources, especially when the control policy to be developed has a large parameter space. The proposed approach is evaluated in a round-trip task, where the robots are required to travel between two destinations as much as possible. Simulation results show that the proposed approach can learn control policies directly from high-dimensional raw camera pixel inputs for robotic swarms.
Phonon-number-resolving detection of multiple local phonon modes in trapped ions

We propose and demonstrate phonon-number-resolving detection of the multiple local phonon modes in a trapped-ion chain. To mitigate the effect of phonon hopping during the detection process, the probability amplitude of each local phonon mode is mapped to the auxiliary long-lived motional ground states. Sequential state-dependent fluorescence detection is then performed. In the experiment, we have successfully observed the time evolution of two local phonon modes in two ions, including the phonon-number correlation between the two modes.
Removing non-significant regions in hierarchical clustering and segmentation

We propose an efficient algorithm that removes unimportant regions from a hierarchical partition tree, while preserving the hierarchical partition structure. Various experiments demonstrate that applying this algorithm on various classification or segmentation problems does indeed improve the results by a large margin. Code is available online at https://github.com/higra/Higra. B. Perret, G. Chierchia, J. Cousty, S.J. F. Guimarães, Y. Kenmochi, L. Najman, Higra: Hierarchical Graph Analysis, SoftwareX, 10, 1--6, ISSN 2019, 2352-7110, 10.1016/j.softx.2019.100335.
Evaluation of Handwritten Urdu Text by Integration of MNIST Dataset Learning Experience

The similar nature of patterns may enhance the learning if the experience they attained during training is utilized to achieve maximum accuracy.This paper presents a novel way to exploit the transfer learning experience of similar patterns on handwritten Urdu text analysis.The MNIST pre-trained network is employed by transferring it's learning experience on Urdu Nastaliq Handwritten Dataset (UNHD) samples.The convolutional neural network is used for feature extraction.The experiments were performed using deep multidimensional long short term (MDLSTM) memory networks.The obtained result shows immaculate performance on number of experiments distinguished on the basis of handwritten complexity.The result of demonstrated experiments show that pre-trained network outperforms on subsequent target networks which enable them to focus on a particular feature learning.The conducted experiments presented astonishingly good accuracy on UNHD dataset.
Projected cooling algorithm for quantum computation

In the current era of noisy quantum devices, there is a need for quantum algorithms that are efficient and robust against noise. Towards this end, we introduce the projected cooling algorithm for quantum computation. The projected cooling algorithm is able to construct the localized ground state of any Hamiltonian with a translationally-invariant kinetic energy and interactions that vanish at large distances. The term "localized" refers to localization in position space. The method can be viewed as the quantum analog of evaporative cooling. We start with an initial state with support over a compact region of a large volume. We then drive the excited quantum states to disperse and measure the remaining portion of the wave function left behind. For the nontrivial examples we consider here, the improvement over other methods is substantial. The only additional resource required is performing the operations in a volume significantly larger than the size of the localized state. These characteristics make the projected cooling algorithm a promising tool for calculations of self-bound systems such as atomic nuclei.
Reasoning in BDI agents using Toulmin's argumentation model

The theory of argumentation pervades several fields of knowledge, and it has gained significant space in multiagent systems because it provides a way for modeling reasoning over conflicting information in intelligent agents. This work proposes the development of an argumentation-based inference mechanism for BDI agents based on Toulmin's model of argumentation. The philosopher Stephen Toulmin claimed that arguments typically consist of six parts: data, warrant, claim, backing, qualifier, and rebuttal. This argumentation structure allows arguments to be described through separated components, making it easier to define and to evaluate the inference process. By presenting and discussing some case studies, this paper shows how this mechanism supports the inference of new beliefs based on available evidence within BDI agents programmed in an agent-oriented programming language.
Game Theoretical Modelling of Network/Cybersecurity

Game theory is an established branch of mathematics that offers a rich set of mathematical tools for multi-person strategic decision making that can be used to model the interactions of decision makers in security problems who compete for limited and shared resources.This article presents a review of the literature in the area of game theoretical modelling of network/cybersecurity and identifies a number of challenges in this area that need to be addressed.
Industry-Oriented Project-Based Learning of Software Engineering

Teaching of Software Engineering is challenging, especially when dealing with large cohorts of students, as well as cohorts with the diversity in the students' backgrounds, workloads and availabilities for face-to-face study. Especially challenging it becomes when we aim not only to provide necessary theoretical knowledge, but also to improve students' employability by making the projects really industry-oriented. This paper presents our experience on the redesign of Software Engineering project course to meet the above challenges, the lessons learned and the feedback from the students. The course is provided for both Bachelor and Master students at their final year of study, so our goal is to make students work-ready and to provide opportunities for networking with local and global companies, as well as other university disciplines.
Aspect-Based Sentiment Analysis using BERT.

Sentiment analysis has become very popular in both research and business due to the increasing amount of opinionated text from Internet users. Standard sentiment analysis deals with classifying the overall sentiment of a text, but this doesn’t include other important information such as towards which entity, topic or aspect within the text the sentiment is directed. Aspect-based sentiment analysis (ABSA) is a more complex task that consists in identifying both sentiments and aspects. This paper shows the potential of using the contextual word representations from the pre-trained language model BERT, together with a fine-tuning method with additional generated text, in order to solve out-of-domain ABSA and outperform previous state-of-the-art results on SemEval-2015 Task 12 subtask 2 and SemEval-2016 Task 5. To the best of our knowledge, no other existing work has been done on out-of-domain ABSA for aspect classification.
A Novel Summarization-based Approach for Feature Reduction Enhancing Text Classification Accuracy

Automatic summarization is the process of shortening one (in single document summarization) or multiple documents (in multi-document summarization). In this paper, a new feature selection method for the nearest neighbor classifier by summarizing the original training documents based on sentence importance measure is proposed. Our approach for single document summarization uses two measures for sentence similarity: the frequency of the terms in one sentence and the similarity of that sentence to other sentences. All sentences were ranked accordingly and the sentences with top ranks (with a threshold constraint) were selected for summarization. The summary of every document in the corpus is taken into a new document used for the summarization evaluation process.
Towards Photo-Realistic Visible Watermark Removal with Conditional Generative Adversarial Networks

Visible watermark plays an important role in image copyright protection and the robustness of a visible watermark to an attack is shown to be essential. To evaluate and improve the effectiveness of watermark, watermark removal attracts increasing attention and becomes a hot research top. Current methods cast the watermark removal as an image-to-image translation problem where the encode-decode architectures with pixel-wise loss are adopted to transfer the transparent watermarked pixels into unmarked pixels. However, when a number of realistic images are presented, the watermarks are more likely to be unknown and diverse (i.e., the watermarks might be opaque or semi-transparent; the category and pattern of watermarks are unknown). When applying existing methods to the real-world scenarios, they mostly can not satisfactorily reconstruct the hidden information obscured under the complex and various watermarks (i.e., the residual watermark traces remain and the reconstructed images lack reality). To address this difficulty, in this paper, we present a new watermark processing framework using the conditional generative adversarial networks (cGANs) for visible watermark removal in the real-world application. The proposed method enables the watermark removal solution to be more closed to the photo-realistic reconstruction using a patch-based discriminator conditioned on the watermarked images, which is adversarially trained to differentiate the difference between the recovered images and original watermark-free images. Extensive experimental results on a large-scale visible watermark dataset demonstrate the effectiveness of the proposed method and clearly indicate that our proposed approach can produce more photo-realistic and convincing results compared with the state-of-the-art methods.
Accelerating SGD with momentum for over-parameterized learning

Nesterov SGD is widely used for training modern neural networks and other machine learning models. Yet, its advantages over SGD have not been theoretically clarified. Indeed, as we show in this paper, both theoretically and empirically, Nesterov SGD with any parameter selection does not in general provide acceleration over ordinary SGD. Furthermore, Nesterov SGD may diverge for step sizes that ensure convergence of ordinary SGD. This is in contrast to the classical results in the deterministic setting, where the same step size ensures accelerated convergence of the Nesterov's method over optimal gradient descent. To address the non-acceleration issue, we introduce a compensation term to Nesterov SGD. The resulting algorithm, which we call MaSS, converges for same step sizes as SGD. We prove that MaSS obtains an accelerated convergence rates over SGD for any mini-batch size in the linear setting. For full batch, the convergence rate of MaSS matches the well-known accelerated rate of the Nesterov's method. We also analyze the practically important question of the dependence of the convergence rate and optimal hyper-parameters on the mini-batch size, demonstrating three distinct regimes: linear scaling, diminishing returns and saturation. Experimental evaluation of MaSS for several standard architectures of deep networks, including ResNet and convolutional networks, shows improved performance over SGD, Nesterov SGD and Adam.
Transferable Perturbations of Deep Feature Distributions

Almost all current adversarial attacks of CNN classifiers rely on information derived from the output layer of the network. This work presents a new adversarial attack based on the modeling and exploitation of class-wise and layer-wise deep feature distributions. We achieve state-of-the-art targeted blackbox transfer-based attack results for undefended ImageNet models. Further, we place a priority on explainability and interpretability of the attacking process. Our methodology affords an analysis of how adversarial attacks change the intermediate feature distributions of CNNs, as well as a measure of layer-wise and class-wise feature distributional separability/entanglement. We also conceptualize a transition from task/data-specific to model-specific features within a CNN architecture that directly impacts the transferability of adversarial examples.
ICTs as Juju: African inspiration for understanding the compositeness of being human through digital technologies

Abstract In this article, I liken information and communication technologies (ICTs) or digital technologies to what we in West and Central Africa have the habit of referring to as Juju . I invite as scholars of the digital humanities to see in the region’s belief in incompleteness and the compositeness of being human, as well as in the capacity to be present everywhere at the same time an indication that we have much to learn from the past on how best to understand and harness current purportedly innovative advances in ICTs. The idea of digital technologies making it possible for humans and things to be present even in their absence and absent even in their presence is not that dissimilar to the belief in what is often labelled and dismissed as witchcraft and magic that lends itself to a world of infinite possibilities – a world of presence in simultaneous multiplicities and eternal powers to redefine reality. The article argues in favour of incompleteness as a normal way of being. It challenges students of humanity to envisage a relationship between humans and digital technologies that is founded less on dichotomies and binary oppositions, nor on zero-sum games of conquest and superiority. If humans are present in things and things in humans, thanks to the interconnections, the flexibility and fluidity of being that come with recognition of and provision for incompleteness, it is important to see things and humans not only as intricately entangled, but also as open-ended composites.
GSSNN: Graph Smoothing Splines Neural Networks

Graph Neural Networks (GNNs) have achieved state-of-the-art performance in many graph data analysis tasks. However, they still suffer from two limitations for graph representation learning. First, they exploit non-smoothing node features which may result in suboptimal embedding and degenerated performance for graph classification. Second, they only exploit neighbor information but ignore global topological knowledge. Aiming to overcome these limitations simultaneously, in this paper, we propose a novel, flexible, and end-to-end framework, Graph Smoothing Splines Neural Networks (GSSNN), for graph classification. By exploiting the smoothing splines, which are widely used to learn smoothing fitting function in regression, we develop an effective feature smoothing and enhancement module Scaled Smoothing Splines (S3) to learn graph embedding. To integrate global topological information, we design a novel scoring module, which exploits closeness, degree, as well as self-attention values, to select important node features as knots for smoothing splines. These knots can be potentially used for interpreting classification results. In extensive experiments on biological and social datasets, we demonstrate that our model achieves state-of-the-arts and GSSNN is superior in learning more robust graph representations. Furthermore, we show that S3 module is easily plugged into existing GNNs to improve their performance.
Histopathological Image Classification by Optimized Neural Network Using IGSA

The histopathological image classification is a vivid application for medical diagnosis and neural network has been successful in the image classification task. However, finding the optimal values of the neural network is still a challenging task. To accomplish the same, this paper considers a two-layer neural network which is optimized through intelligent gravitational search algorithm. Further, the optimized two-layer neural network is applied for the histopathological tissue classification into healthy and inflamed. The proposed method is validated on the publicly available tissue dataset, namely Animal Diagnostic Laboratory (ADL). The experimental results firm the better performance of the proposed method against state-of-the-art methods in terms of seven performance measures, namely recall, specificity, precision, false negative rate (FNR), accuracy, F1-score, and G-mean.
An Image Encryption Algorithm Based on Random Hamiltonian Path

In graph theory, Hamiltonian path refers to the path that visits each vertex exactly once. In this paper, we designed a method to generate random Hamiltonian path within digital images, which is equivalent to permutation in image encryption. By these means, building a Hamiltonian path across bit planes can shuffle the distribution of the pixel’s bits. Furthermore, a similar thought can be applied for the substitution of pixel’s grey levels. To ensure the randomness of the generated Hamiltonian path, an adjusted Bernoulli map is proposed. By adopting these novel techniques, a bit-level image encryption scheme was devised. Evaluation of simulation results proves that the proposed scheme reached fair performance. In addition, a common flaw in calculating correlation coefficients of adjacent pixels was pinpointed by us. After enhancement, correlation coefficient becomes a stricter criterion for image encryption algorithms.
Linear Program Reconstruction in Practice

We briefly report on a successful linear program reconstruction attack performed on a production statistical queries system and using a real dataset. The attack was deployed in test environment in the course of the Aircloak Challenge bug bounty program and is based on the reconstruction algorithm of Dwork, McSherry, and Talwar. We empirically evaluate the effectiveness of the algorithm and a related algorithm by Dinur and Nissim with various dataset sizes, error rates, and numbers of queries in a Gaussian noise setting.
A checklist for choosing between R packages in ecology and evolution

Abstract The open source and free programming language R is a phenomenal mechanism to address a multiplicity of challenges in ecology and evolution. It is also a complex ecosystem because of the diversity of solutions available to the analyst. Packages for R enhance and specialize the capacity to explore both niche data/experiments and more common needs. However, the paradox of choice or how we select between many seemingly similar options can be overwhelming and lead to different potential outcomes. There is extensive choice in ecology and evolution between packages for both fundamental statistics and for more specialized domain‐level analyses. Here, we provide a checklist to inform these decisions based on the principles of resilience, need, and integration with scientific workflows for evidence. It is important to explore choices in any analytical coding environment—not just R—for solutions to challenges in ecology and evolution, and document this process because it advances reproducible science, promotes a deeper understand of the scientific evidence, and ensures that the outcomes are correct, representative, and robust.