Reviewerly/exHarmony huggingface dataset:
Continuous Meta-Learning without Tasks

Meta-learning is a promising strategy for learning to efficiently learn within new tasks, using data gathered from a distribution of tasks. However, the meta-learning literature thus far has focused on the task segmented setting, where at train-time, offline data is assumed to be split according to the underlying task, and at test-time, the algorithms are optimized to learn in a single task. In this work, we enable the application of generic meta-learning algorithms to settings where this task segmentation is unavailable, such as continual online learning with a time-varying task. We present meta-learning via online changepoint analysis (MOCA), an approach which augments a meta-learning algorithm with a differentiable Bayesian changepoint detection scheme. The framework allows both training and testing directly on time series data without segmenting it into discrete tasks. We demonstrate the utility of this approach on a nonlinear meta-regression benchmark as well as two meta-image-classification benchmarks.
Recurrent babbling: evaluating the acquisition of grammar from limited input data

Recurrent Neural Networks (RNNs) have been shown to capture various aspects of syntax from raw linguistic input.In most previous experiments, however, learning happens over unrealistic corpora, which do not reflect the type and amount of data a child would be exposed to.This paper remedies this state of affairs by training a Long Short-Term Memory network (LSTM) over a realistically sized subset of child-directed input.The behaviour of the network is analysed over time using a novel methodology which consists in quantifying the level of grammatical abstraction in the model's generated output (its 'babbling'), compared to the language it has been exposed to.We show that the LSTM indeed abstracts new structures as learning proceeds.
Application of Roadway Deformation Detection Method Based on Machine Vision for Underground Patrol Robot

A set of track patrol robot system is designed for the problems of inadequate monitoring of underground coal mine equipment, the high labour intensity of patrol personnel, and roadway deformation. In addition to the automatic cruise mode, the system can collect and store the environmental data in the roadway in real-time, and can intelligently identify abnormal situations, especially through machine vision to monitor the deformation of the roadway in the coal mine, which ensures the safety of underground production and personnel.
Low-Resource Text Classification via Cross-Lingual Language Model Fine-Tuning

Text classification tends to be difficult when data are inadequate considering the amount of manually labeled text corpora. For low-resource agglutinative languages including Uyghur, Kazakh, and Kyrgyz (UKK languages), in which words are manufactured via stems concatenated with several suffixes and stems are used as the representation of text content, this feature allows infinite derivatives vocabulary that leads to high uncertainty of writing forms and huge redundant features. There are major challenges of low-resource agglutinative text classification the lack of labeled data in a target domain and morphologic diversity of derivations in language structures. It is an effective solution which fine-tuning a pre-trained language model to provide meaningful and favorable-to-use feature extractors for downstream text classification tasks. To this end, we propose a low-resource agglutinative language model fine-tuning $$AgglutiFiT$$ , specifically, we build a low-noise fine-tuning dataset by morphological analysis and stem extraction, then fine-tune the cross-lingual pre-training model on this dataset. Moreover, we propose an attention-based fine-tuning strategy that better selects relevant semantic and syntactic information from the pre-trained language model and uses those features on downstream text classification tasks. We evaluate our methods on nine Uyghur, Kazakh, and Kyrgyz classification datasets, where they have significantly better performance compared with several strong baselines.
Predicting Product Review Helpfulness – A Hybrid Method

Recent years have seen a rapidly growing number of online reviews of products. As a result, it is often not possible for customers to go through each review before making purchase decisions. One way to address this problem is to build a system for automatically addressing the helpfulness of reviews and present only those reviews that are determined to be helpful by the system to an end user. The vast majority of existing approaches to the task of review helpfulness prediction are based on hand-crafted features, thus making system performance heavily dependent on the quality of these features. In light of this weakness, we propose a new model of review helpfulness prediction using a combination of Convolutional Neural Network (CNN) and TransE wherein hand-crafted features can also be incorporated to improve the output. Specifically, CNN enables us to learn the semantic information from a review and TransE is used to capture the relationship between different entities mentioned in the review. Experiments on the Amazon product review datasets demonstrate that our approach significantly outperforms the state of the art.
Sentiment Analysis of User Feedback in e-Learning Environment

Sentiment analysis (SA) is prevalent now; because it can yield useful insight from high-volume subjects and unstructured data mainly from social media networking sites and micro-blog websites or known as user-generated contents.SA in user-generated contents is difficult due to the informal nature of the communication.The informal nature introduces additional variables and properties that have to be evaluated compared to formal texts, necessitating additional resources spent on annotating the data and training the classifiers.We explore two most common methods in classifying user-generated contents called lexicon-based approach by using VADER Sentiment Analyser and Machine Learning (ML) approaches by using Naïve Bayes and Decision Tree classifiers.Our primary objective is to study the accuracy of the solutions and then apply the best solution to 126 students' feedbacks 126 student feedbacks toward an e-learning environment.The purpose is to extract the sentiment against it to acquire the initial picture of student's perception on the implementation of e-learning; so the effectiveness of its implementation can be improved.The data pre-processed and then analysed using Python as programming tool.The results show VADER outperformed two selected ML classifiers, it can achieve approximately 90% in accuracy.From the results, we conclude that VADER sentiment analyser was doing well and better than ML in SA toward user-generated content.The results on the e-learning environment also suggest further analysis should be done towards this e-learning platform to complete this initial study.
Detecting Malicious URLs Based on Machine Learning Algorithms and Word Embeddings

Relying on the appropriate features is essential in classification models for malware detection, for various important reasons, such as dealing with class imbalance, the ability to detect zero-day malware samples, or preventing attackers to successfully reverse engineer the classification process and changing nonessential feature values to avoid detection. In this paper, we propose a method that uses a combination of word embeddings together with “classical”, domain-engineered features, to obtain reliable classification models for malicious URLs detection. Additionally, we explore different traditional techniques to address class imbalance - such as synthetic oversampling or cost-sensitive learning - and several classification techniques. We find that the best overall results are obtained by using a cost-sensitive neural network - with a precision that exceeds 99% and an accuracy above 90%, while maintaining a recall rate above 89%. We have performed an analysis of the importance of the features proposed, and found that while word embeddings produce better results than bi-gram based features, domain-specific features are necessary for obtaining a high precision in detecting malicious URLs.
Functional Kernel Density Estimation: Point and Fourier Approaches to Time Series Anomaly Detection

We present an unsupervised method to detect anomalous time series among a collection of time series. To do so, we extend traditional Kernel Density Estimation for estimating probability distributions in Euclidean space to Hilbert spaces. The estimated probability densities we derive can be obtained formally through treating each series as a point in a Hilbert space, placing a kernel at those points, and summing the kernels (a “point approach”), or through using Kernel Density Estimation to approximate the distributions of Fourier mode coefficients to infer a probability density (a “Fourier approach”). We refer to these approaches as Functional Kernel Density Estimation for Anomaly Detection as they both yield functionals that can score a time series for how anomalous it is. Both methods naturally handle missing data and apply to a variety of settings, performing well when compared with an outlyingness score derived from a boxplot method for functional data, with a Principal Component Analysis approach for functional data, and with the Functional Isolation Forest method. We illustrate the use of the proposed methods with aviation safety report data from the International Air Transport Association (IATA).
eGAP: An Evolutionary Game Theoretic Approach to Random Forest Pruning

To make healthcare available and easily accessible, the Internet of Things (IoT), which paved the way to the construction of smart cities, marked the birth of many smart applications in numerous areas, including healthcare. As a result, smart healthcare applications have been and are being developed to provide, using mobile and electronic technology, higher diagnosis quality of the diseases, better treatment of the patients, and improved quality of lives. Since smart healthcare applications that are mainly concerned with the prediction of healthcare data (like diseases for example) rely on predictive healthcare data analytics, it is imperative for such predictive healthcare data analytics to be as accurate as possible. In this paper, we will exploit supervised machine learning methods in classification and regression to improve the performance of the traditional Random Forest on healthcare datasets, both in terms of accuracy and classification/regression speed, in order to produce an effective and efficient smart healthcare application, which we have termed eGAP. eGAP uses the evolutionary game theoretic approach replicator dynamics to evolve a Random Forest ensemble. Trees of high resemblance in an initial Random Forest are clustered, and then clusters grow and shrink by adding and removing trees using replicator dynamics, according to the predictive accuracy of each subforest represented by a cluster of trees. All clusters have an initial number of trees that is equal to the number of trees in the smallest cluster. Cluster growth is performed using trees that are not initially sampled. The speed and accuracy of the proposed method have been demonstrated by an experimental study on 10 classification and 10 regression medical datasets.
Design Decisions by Voice: The Next Step of Software Architecture Knowledge Management

Architectural Design Decisions (ADDs) capture the essence of relevant Architectural Knowledge (AK) and the underpinning rationale in order to produce well-designed software architectures. AK and design rationale might get lost if not captured at the same time when the architecture is discussed and modeled in early design phases. For years, this relevant knowledge has been captured using text templates and supported by a number of research tools. Nevertheless, as no commercial tool is still available combining AK capturing with UML notations to facilitate capturing the design decisions at the same time the architecture is modeled, is the major barrier to convince software architects and companies to invest in documenting the significant design decisions. As capturing AK using text templates requires an extra effort, we propose an approach to make the documentation process easier and reduce the effort thereof by using voice commands. In particular, we suggest an approach to: (i) capture ADDs using voice commands during design conversations, and (ii) link the captured ADDs to UML notations. Our approach integrates OctoUML, a modeling tool with voice commands for capturing design decisions by voice.
Deep Material Recognition in Light-Fields via Disentanglement of Spatial and Angular Information

Light-field cameras capture sub-views from multiple perspectives simultaneously, with possibly reflectance variations that can be used to augment material recognition in remote sensing, autonomous driving, etc. Existing approaches for light-field based material recognition suffer from the entanglement between angular and spatial domains, leading to inefficient training which in turn limits their performances. In this paper, we propose an approach that achieves decoupling of angular and spatial information by establishing correspondences in the angular domain, then employs regularization to enforce a rotational invariance. As opposed to relying on the Lambertian surface assumption, we align the angular domain by estimating sub-pixel displacements using the Fourier transform. The network takes sparse inputs, i.e. sub-views along particular directions, to gain structural information about the angular domain. A novel regularization technique further improves generalization by weight sharing and max-pooling among different directions. The proposed approach outperforms any previously reported method on multiple datasets. The accuracy gain over 2D images is improved by a factor of 1.5. Ablation studies are conducted to demonstrate the significance of each component.
Incremental learning strategies for credit cards fraud detection: Extended abstract

Every second, Fraud Detection Systems (FDS) check streams of thousands of credit or debit card transactions. Most of the models in production and in the literature rely on batch learning, wasting part of the most recent data. Incremental learning may be beneficial in terms of computational/architectural cost and confidentiality (e.g. avoiding the storage of sensitive data). The full paper focuses on two questions related to the use of incremental learning for FDS: (1) Can incremental learning be as competitive as batch and retraining approaches? (2) Can combining incremental, ensembles, diversity, and transfer learning lead to efficient models under concept drift? We investigate those elements and provide an experimental evaluation on a real-life case study including more than 150 days of e-commerce transactions (or 50 million transactions).
Clothing Brand Logo Prediction: From Residual Block to Dense Block

In this paper, we proposed a new clothing brand prediction method which is rooted on a dense-block based deep convolutional neural network for brand logo detection and recognition. To learn convolutional neural networks deeper and more accurately, we adopted dense blocks into deep convolutional neural networks to make connections between layers shorter. In this work, we propose several dense-block based designs to improve clothing brand logo detection and recognition accuracies. We also constructed a new large-scale clothing brand and price (CBP) dataset and its subset, called clothing brand logo (CBL) dataset with the brand attribute and logo information to carry out this task. To lower proposed framework complexity, two pixel search steps for the bounding box movement are implemented in the training procedure. In the experiment, we show our search reduced model can outperform several state-of-the-art methods and attain good performance.
Making experimental data tables in the life sciences more FAIR: a pragmatic approach

Making data compliant with the FAIR Data principles (Findable, Accessible, Interoperable, Reusable) is still a challenge for many researchers, who are not sure which criteria should be met first and how. Illustrated with experimental data tables associated with a Design of Experiments, we propose an approach that can serve as a model for research data management that allows researchers to disseminate their data by satisfying the main FAIR criteria without insurmountable efforts. More importantly, this approach aims to facilitate the FAIR compliance process by providing researchers with tools to improve their data management practices.
VICTR: Visual Information Captured Text Representation for Text-to-Vision Multimodal Tasks

Text-to-image multimodal tasks, generating/retrieving an image from a given text description, are extremely challenging tasks since raw text descriptions cover quite limited information in order to fully describe visually realistic images.We propose a new visual contextual text representation for text-to-image multimodal tasks, VICTR, which captures rich visual semantic information of objects from the text input.First, we use the text description as initial input and conduct dependency parsing to extract the syntactic structure and analyse the semantic aspect, including object quantities, to extract the scene graph.Then, we train the extracted objects, attributes, and relations in the scene graph and the corresponding geometric relation information using Graph Convolutional Networks, and it generates text representation which integrates textual and visual semantic information.The text representation is aggregated with word-level and sentencelevel embedding to generate both visual contextual word and sentence representation.For the evaluation, we attached VICTR to the state-of-the-art models in text-to-image generation.VICTR is easily added to existing models and improves across both quantitative and qualitative aspects 1 .
Syllable-based Neural Thai Word Segmentation

Word segmentation is a challenging preprocessing step for Thai Natural Language Processing due to the lack of explicit word boundaries.The previous systems rely on powerful neural network architecture alone and ignore linguistic substructures of Thai words.We utilize the linguistic observation that Thai strings can be segmented into syllables, which should narrow down the search space for the word boundaries and provide helpful features.Here, we propose a neural Thai Word Segmenter that uses syllable embeddings to capture linguistic constraints and uses dilated CNN filters to capture the environment of each character.Within this goal, we develop the first MLbased Thai orthographical syllable segmenter, which yields syllable embeddings to be used as features by the word segmenter.Our word segmentation system outperforms the previous stateoftheart system in both speed and accuracy on both indomain and outdomain datasets.
Multilingual Neural Machine Translation

The advent of neural machine translation (NMT) has opened up exciting research in building multilingual translation systems i.e. translation models that can handle more than one language pair. Many advances have been made which have enabled (1) improving translation for low-resource languages via transfer learning from high resource languages; and (2) building compact translation models spanning multiple languages. In this tutorial, we will cover the latest advances in NMT approaches that leverage multilingualism, especially to enhance low-resource translation. In particular, we will focus on the following topics: modeling parameter sharing for multi-way models, massively multilingual models, training protocols, language divergence, transfer learning, zero-shot/zero-resource learning, pivoting, multilingual pre-training and multi-source translation.
An Efficient Privacy-preserving Deep Learning Scheme for Medical Image Analysis

In recent privacy has emerged as one of the major concerns of deep learning, since it requires huge amount of personal data. Medical Image Analysis is one of the prominent areas where sensitive data are shared to a third party service provider. In this paper, a secure deep learning scheme called Metamorphosed Learning (MpLe) is proposed to protect the privacy of images in medical image analysis. An augmented convolutional layer and image morphing are two main components of MpLe scheme. Data providers morph the images without privacy information using image morphing component. The human unrecognizable image is then delivered to the service providers who then apply deep learning algorithms on morphed data using augmented convolution layer without any performance penalty. MpLe provides sturdy security and privacy with optimal computational overhead. The proposed scheme is experimented using VGG-16 network on CIFAR dataset. The performance of MpLe is compared with similar works such as GAZELLE and MiniONN and found that the MpLe attracts very less computational and data transmission overhead. MpLe is also analyzed for various adversarial attack and realized that the success rate is as low as . The efficiency of the proposed scheme is proved through experimental and performance analysis.
A Document-Level Neural Machine Translation Model with Dynamic Caching Guided by Theme-Rheme Information

Research on document-level Neural Machine Translation (NMT) models has attracted increasing attention in recent years.Although the proposed works have proved that the inter-sentence information is helpful for improving the performance of the NMT models, what information should be regarded as context remains ambiguous.To solve this problem, we proposed a novel cache-based document-level NMT model which conducts dynamic caching guided by theme-rheme information.The experiments on NIST evaluation sets demonstrate that our proposed model achieves substantial improvements over the state-of-the-art baseline NMT models.As far as we know, we are the first to introduce theme-rheme theory into the field of machine translation.
Organization of virtual interaction in the context of the coronavirus pandemic

The results of the study reflect the opinion of students of higher educational institutions about the use of virtual interaction tools. The results obtained using the Likert scale allow us to say that the majority of the respondents are active participants in virtual interaction and are ready to continue to communicate using the presented means despite the existing difficulties.
UPB at SemEval-2020 Task 6: Pretrained Language Models for Definition Extraction

This work presents our contribution in the context of the 6th task of SemEval-2020: Extracting Definitions from Free Text in Textbooks (DeftEval).This competition consists of three subtasks with different levels of granularity: (1) classification of sentences as definitional or non-definitional, (2) labeling of definitional sentences, and (3) relation classification.We use various pretrained language models (i.e., BERT, XLNet, RoBERTa, SciBERT, and ALBERT) to solve each of the three subtasks of the competition.Specifically, for each language model variant, we experiment by both freezing its weights and fine-tuning them.We also explore a multi-task architecture that was trained to jointly predict the outputs for the second and the third subtasks.Our best performing model evaluated on the DeftEval dataset obtains the 32nd place for the first subtask and the 37th place for the second subtask.