Reviewerly/exHarmony huggingface dataset:
The N-Grams Based Text Similarity Detection Approach Using Self-Organizing Maps and Similarity Measures

In the paper the word-level n-grams based approach is proposed to find similarity between texts. The approach is a combination of two separate and independent techniques: self-organizing map (SOM) and text similarity measures. SOM’s uniqueness is that the obtained results of data clustering, as well as dimensionality reduction, are presented in a visual form. The four measures have been evaluated: cosine, dice, extended Jaccard’s, and overlap. First of all, texts have to be converted to numerical expression. For that purpose, the text has been split into the word-level n-grams and after that, the bag of n-grams has been created. The n-grams’ frequencies are calculated and the frequency matrix of dataset is formed. Various filters are used to create a bag of n-grams: stemming algorithms, number and punctuation removers, stop words, etc. All experimental investigation has been made using a corpus of plagiarized short answers dataset.
Domain Adversarial Neural Networks for Large-Scale Land Cover Classification

Learning classification models require sufficiently labeled training samples, however, collecting labeled samples for every new problem is time-consuming and costly. An alternative approach is to transfer knowledge from one problem to another, which is called transfer learning. Domain adaptation (DA) is a type of transfer learning that aims to find a new latent space where the domain discrepancy between the source and the target domain is negligible. In this work, we propose an unsupervised DA technique called domain adversarial neural networks (DANNs), composed of a feature extractor, a class predictor, and domain classifier blocks, for large-scale land cover classification. Contrary to the traditional methods that perform representation and classifier learning in separate stages, DANNs combine them into a single stage, thereby learning a new representation of the input data that is both domain-invariant and discriminative. Once trained, the classifier of a DANN can be used to predict both source and target domain labels. Additionally, we also modify the domain classifier of a DANN to evaluate its suitability for multi-target domain adaptation problems. Experimental results obtained for both single and multiple target DA problems show that the proposed method provides a performance gain of up to 40%.
ABM-SpConv

Hardware accelerators for convolutional neural network (CNN) inference have been extensively studied in recent years. The reported designs tend to utilize a similar underlying architecture based on multiplier-accumulator (MAC) arrays, which has the practical consequence of limiting the FPGA-based accelerator performance by the number of available on-chip DSP blocks, while leaving other resource under-utilized. To address this problem, we consider a transformation to the convolution computation, which leads to transformation of the accelerator design space and relaxes the pressure on the required DSP resources. We demonstrate that our approach enables us to strike a judicious balance between utilization of the on-chip memory, logic, and DSP resources, due to which, our accelerator considerably outperforms state of the art. We report the effectiveness of our approach on a Stratix-V GXA7 FPGA, which shows 55% throughput improvement, while using 6.25% less DSP blocks, compared to the best reported CNN accelerator on the same device.
VIRET

Known-item search in large video collections still represents a challenging task for current video retrieval systems that have to rely both on state-of-the-art ranking models and interactive means of retrieval. We present a general overview of the current version of the VIRET tool, an interactive video retrieval system that successfully participated at several international evaluation campaigns. The system is based on multi-modal search and convenient inspection of results. Based on collected query logs of four users controlling instances of the tool at the Video Browser Showdown 2019, we highlight query modification statistics and a list of successful query formulation strategies. We conclude that the VIRET tool represents a competitive reference interactive system for effective known-item search in one thousand hours of video.
Foundations of Differentially Oblivious Algorithms

It is well-known that a program's memory access pattern can leak information about its input. To thwart such leakage, most existing works adopt the technique of oblivious RAM (ORAM) simulation. Such an obliviousness notion has stimulated much debate. Although ORAM techniques have significantly improved over the past few years, the concrete overheads are arguably still undesirable for real-world systems — part of this overhead is in fact inherent due to a well-known logarithmic ORAM lower bound by Goldreich and Ostrovsky. To make matters worse, when the program's runtime or output length depend on secret inputs, it may be necessary to perform worst-case padding to achieve full obliviousness and thus incur possibly super-linear overheads.Inspired by the elegant notion of differential privacy, we initiate the study of a new notion of access pattern privacy, which we call "(∊, δ)-differential obliviousness". We separate the notion of (∊, δ)-differential obliviousness from classical obliviousness by considering several fundamental algorithmic abstractions including sorting small-length keys, merging two sorted lists, and range query data structures (akin to binary search trees). We show that by adopting differential obliviousness with reasonable choices of ∊ and δ, not only can one circumvent several impossibilities pertaining to full obliviousness, one can also, in several cases, obtain meaningful privacy with little overhead relative to the non-private baselines (i.e., having privacy "almost for free"). On the other hand, we show that for very demanding choices of ∊ and δ, the same lower bounds for oblivious algorithms would be preserved for (∊, δ)-differential obliviousness.
Light Field Super-Resolution: A Benchmark

Lenslet-based light field imaging generally suffers from a fundamental trade-off between spatial and angular resolutions, which limits its promotion to practical applications. To this end, a substantial amount of efforts have been dedicated to light field super-resolution (SR) in recent years. Despite the demonstrated success, existing light field SR methods are often evaluated based on different degradation assumptions using different datasets, and even contradictory results are reported in literature. In this paper, we conduct the first systematic benchmark evaluation for representative light field SR methods on both synthetic and real-world datasets with various downsampling kernels and scaling factors. We then analyze and discuss the advantages and limitations of each kind of method from different perspectives. Especially, we find that CNN-based single image SR without using any angular information outperforms most light field SR methods even including learning-based ones. This benchmark evaluation, along with the comprehensive analysis and discussion, sheds light on the future researches in light field SR.
Neural Response Generation with Meta-words

We present open domain response generation with meta-words.A meta-word is a structured record that describes various attributes of a response, and thus allows us to explicitly model the one-to-many relationship within open domain dialogues and perform response generation in an explainable and controllable manner.To incorporate meta-words into generation, we enhance the sequence-to-sequence architecture with a goal tracking memory network that formalizes meta-word expression as a goal and manages the generation process to achieve the goal with a state memory panel and a state controller.Experimental results on two large-scale datasets indicate that our model can significantly outperform several state-ofthe-art generation models in terms of response relevance, response diversity, accuracy of oneto-many modeling, accuracy of meta-word expression, and human evaluation.
ADC: Advanced document clustering using contextualized representations

Document representation is central to modern natural language processing systems including document clustering. Empirical experiments in recent studies provide strong evidence that unsupervised language models can learn context-aware representations in the given documents and advance several NLP benchmark results. However, existing clustering approaches focus on the dimensionality reduction and do not exploit these informative representations. In this paper, we propose a conceptually simple but experimentally effective clustering framework called Advanced Document Clustering (ADC). In contrast to previous clustering methods, ADC is designed to leverage syntactically and semantically meaningful features through feature-extraction and clustering modules in the framework. We first extract features from pre-trained language models and initialize cluster centroids to spread out uniformly. In the clustering module of ADC, the semantic similarity can be measured using the cosine similarity and centroids update while assigning centroids to a mini-batch input. Also, we utilize cross entropy loss partially, as the self-training scheme can be biased when parameters in the model are inaccurate. As a result, ADC can take advantages of contextualized representations while mitigating the limitations introduced by high-dimensional vectors. In numerous experiments with four datasets, the proposed ADC outperforms other existing approaches. In particular, experiments on categorizing news corpus with fake news demonstrated the effectiveness of our method for contextualized representations.
A Wide Scale Classification of Class Imbalance Problem and its Solutions: A Systematic Literature Review

In today's world, most of the data (real world) is present in imbalanced form by nature.This is because of not having efficient algorithms to put this data (i.e., generated data by billion of internetconnected devices (IoTs)) in respective format.Imbalanced data poses a great challenge to (both) data mining and machine learning algorithms.The imbalanced dataset consists of a majority class and a minority class, where the majority class takes the lead over the minority class.Generally, several standard learning algorithms assume the balanced class distribution or equal misclassification costs.If prediction is performed by these learning algorithms on imbalanced data, the accuracy will be high for majority classes, i.e., resulting in poor performance.To overcome this problem (or improving accuracy of deision/prediction-making process), data mining and machine learning researchers have addressed the problem of imbalanced data using datalevel, algorithmic level and ensemble or hybrid methods.This article presents a systematic literature review and analyze the results of more than 400 research papers published between 2002-2017 (till June 2017), resulting in a broader and elaborate investigation of the literature in this area of research.Note that extension of this article/work will contain till December 2018 research articles, which will be published in June 2019 (now these more papers/articles did not include due to no. of pages/space issues).The systematic analysis of the research literature has focus on the key role of Data Intrinsic Problems in classification, handling the imbalanced data and the techniques used to overcome the skewed distribution.Furthermore, this article reveals patterns, trends and gaps in the existing literature and discusses briefly the next generation research directions in this area.
A Report on the Third

Marcos Zampieri, Shervin Malmasi, Yves Scherrer, Tanja Samardžić, Francis Tyers, Miikka Silfverberg, Natalia Klyueva, Tung-Le Pan, Chu-Ren Huang, Radu Tudor Ionescu, Andrei M. Butnaru, Tommi Jauhiainen. Proceedings of the Sixth Workshop on NLP for Similar Languages, Varieties and Dialects. 2019.
Programmable matrix operation with reconfigurable time-wavelength plane manipulation and dispersed time delay

We propose a novel optical computing architecture for massive parallel matrix manipulation based on reconfigurable time-wavelength plane manipulation and and dispersed time delay. Two linear weighting methods in either wavelength or time domain are proposed and validated. We perform the autocorrelation function of a 7-bit m-sequence with the speed at 1.18×1011 multiplications and accumulations per second (MACs/s) and a multiplication of a 4 × 4 matrix and a 4 × 1 vector at 2.69×109 MACs/s. The edge extraction of 32 × 32 binary images is also realized in simulation by optical 2D convolution at 5×108 MACs/s. The proposed optical computing unit can be a key building block to process complex computing tasks with advanced deep learning algorithms and it is promising for the future photonic neural network circuits.
Learning to Explicitate Connectives with Seq2Seq Network for Implicit Discourse Relation Classification

Implicit discourse relation classification is one of the most difficult steps in discourse parsing.The difficulty stems from the fact that the coherence relation must be inferred based on the content of the discourse relational arguments.Therefore, an effective encoding of the relational arguments is of crucial importance.We here propose a new model for implicit discourse relation classification, which consists of a classifier, and a sequence-to-sequence model which is trained to generate a representation of the discourse relational arguments by trying to predict the relational arguments including a suitable implicit connective.Training is possible because such implicit connectives have been annotated as part of the PDTB corpus.Along with a memory network, our model could generate more refined representations for the task.And on the now standard 11-way classification, our method outperforms the previous state of the art systems on the PDTB benchmark on multiple settings including cross validation.1. [I refused to pay the cobbler the full $95] Arg1 because [He did poor work.]Arg2 -Explicit, Contingency.Cause 2. [In the energy mix of the future, bio-energy will also have a key role to play in boosting rural employment and the rural economy in Europe .]Arg1 (Implicit = However) [At the same time , the promotion of bioenergy must not lead to distortions of competition.]Arg2 -
Multiattentive Recurrent Neural Network Architecture for Multilingual Readability Assessment

We present a multiattentive recurrent neural network architecture for automatic multilingual readability assessment. This architecture considers raw words as its main input, but internally captures text structure and informs its word attention process using other syntax- and morphology-related datapoints, known to be of great importance to readability. This is achieved by a multiattentive strategy that allows the neural network to focus on specific parts of a text for predicting its reading level. We conducted an exhaustive evaluation using data sets targeting multiple languages and prediction task types, to compare the proposed model with traditional, state-of-the-art, and other neural network strategies.
A novel test-cost-sensitive attribute reduction approach using the binary bat algorithm

Attribute reductions are essential pre-processing steps in such as data mining, machine learning, pattern recognition and many other fields. Moreover, test-cost-sensitive attribute reductions are often used when we have to deal with cost-sensitive data. The main result of this paper is a new meta-heuristic optimization method for finding optimal test-cost-sensitive attribute reduction that is based on binary bat algorithm that originally was designed to model the echolocation behavior of bats when they search their prey. First we provide a 0-1 integer programming algorithm that can calculate optimal reduct but is inefficient for large data sets. We will use it to evaluate other algorithms. Next, a new fitness function that utilizes the pairs of inconsistent objects and does not have any uncertain parameter is design and an efficient algorithm for counting inconsistent pairs is provided. Then, an efficient test-cost-sensitive attribute reduction technique that uses binary bat algorithm is provided. Finally, a new evaluation model with four different evaluation metrics has been proposed and used to evaluate algorithms that only provide sub-optimal solutions. Several experiments were carried out on broadly used benchmark data sets and the results have shown the superiority of our new algorithm, in terms of various metrics, computational time, and classification accuracy, especially for high-dimensional data sets.
A Graph-Based Encoding for Evolutionary Convolutional Neural Network Architecture Design

Convolutional neural networks (CNNs) have demonstrated highly effective performance in image classification across a range of data sets. The best performance can only be obtained with CNNs when the appropriate architecture is chosen, which depends on both the volume and nature of the training data available. Many of the state-of-the-art architectures in the literature have been hand-crafted by human researchers, but this requires expertise in CNNs, domain knowledge, or trial-and-error experimentation, often using expensive resources. Recent work based on evolutionary deep learning has offered an alternative, in which evolutionary computation (EC) is applied to automatic architecture search. A key component in evolutionary deep learning is the chosen encoding strategy; however, previous approaches to CNN encoding in EC typically have restrictions in the architectures that can be represented. Here, we propose an encoding strategy based on a directed acyclic graph representation, and introduce an algorithm for random generation of CNN architectures using this encoding. In contrast to previous work, our proposed encoding method is more general, enabling representation of CNNs of arbitrary connectional structure and unbounded depth. We demonstrate its effectiveness using a random search, in which 200 randomly generated CNN architectures are evaluated. To improve the computational efficiency, the 200 CNNs are trained using only 10% of the CIFAR-10 training data; the three bestperforming CNNs are then re-trained on the full training set. The results show that the proposed representation and initialisation method can achieve promising accuracy compared to manually designed architectures, despite the simplicity of the random search approach and the reduced data set. We intend that future work can improve on these results by applying evolutionary search using this encoding.
Empirical Insights Into Short Story Draft Construction

Existing cognitive models of narrative creation provide accounts for story invention that, while useful, are too high level to be directly applied to formal systems like computational models of narrative generation. Inversely, existing automatic story generation systems that try to implement cognitive models can only rely on approximations to the general concepts these models describe. In order to provide insight to fill the gap between these two approaches, we have conducted a study in which human participants would invent and write short stories while reflecting on their thoughts out loud. The sessions and the analysis of the recordings was designed so that we could observe which specific modifications the participants apply to their story drafts, with the intention to inform the process of creating computational systems based on cognitive descriptions of the narrative creation process. After running the experiments, annotating the videos and analysing the output, we have concluded that there are a number of common modifications that humans tend to apply to a newly created draft, and that this information can be used to the development of storytelling systems.
Abstract Reasoning with Distracting Features

Abstraction reasoning is a long-standing challenge in artificial intelligence. Recent studies suggest that many of the deep architectures that have triumphed over other domains failed to work well in abstract reasoning. In this paper, we first illustrate that one of the main challenges in such a reasoning task is the presence of distracting features, which requires the learning algorithm to leverage counter-evidence and to reject any of false hypothesis in order to learn the true patterns. We later show that carefully designed learning trajectory over different categories of training data can effectively boost learning performance by mitigating the impacts of distracting features. Inspired this fact, we propose feature robust abstract reasoning (FRAR) model, which consists of a reinforcement learning based teacher network to determine the sequence of training and a student network for predictions. Experimental results demonstrated strong improvements over baseline algorithms and we are able to beat the state-of-the-art models by 18.7\% in RAVEN dataset and 13.3\% in the PGM dataset.
Learning Latent Parameters without Human Response Patterns: Item Response Theory with Artificial Crowds

Incorporating Item Response Theory (IRT) into NLP tasks can provide valuable information about model performance and behavior.Traditionally, IRT models are learned using human response pattern (RP) data, presenting a significant bottleneck for large data sets like those required for training deep neural networks (DNNs).In this work we propose learning IRT models using RPs generated from artificial crowds of DNN models.We demonstrate the effectiveness of learning IRT models using DNN-generated data through quantitative and qualitative analyses for two NLP tasks.Parameters learned from human and machine RPs for natural language inference and sentiment analysis exhibit medium to large positive correlations.We demonstrate a use-case for latent difficulty item parameters, namely training set filtering, and show that using difficulty to sample training data outperforms baseline methods.Finally, we highlight cases where human expectation about item difficulty does not match difficulty as estimated from the machine RPs.
Meta Architecture Search

Neural Architecture Search (NAS) has been quite successful in constructing state-of-the-art models on a variety of tasks. Unfortunately, the computational cost can make it difficult to scale. In this paper, we make the first attempt to study Meta Architecture Search which aims at learning a task-agnostic representation that can be used to speed up the process of architecture search on a large number of tasks. We propose the Bayesian Meta Architecture SEarch (BASE) framework which takes advantage of a Bayesian formulation of the architecture search problem to learn over an entire set of tasks simultaneously. We show that on Imagenet classification, we can find a model that achieves 25.7% top-1 error and 8.1% top-5 error by adapting the architecture in less than an hour from an 8 GPU days pretrained meta-network. By learning a good prior for NAS, our method dramatically decreases the required computation cost while achieving comparable performance to current state-of-the-art methods - even finding competitive models for unseen datasets with very quick adaptation. We believe our framework will open up new possibilities for efficient and massively scalable architecture search research across multiple tasks.