Reviewerly/exHarmony huggingface dataset:
Research on gesture image recognition method based on transfer learning

To solve the problem of low gesture image recognition rate, we propose a transfer learning based image recognition method called Mobilenet-RF. We combine the two models of MobileNet convolutional network with Random Forest to further improve image recognition accuracy. This method firstly transfers the model architecture and weight files of MobileNet to gesture images, trains the model and extracts image features, and then classifies the features extracted by convolutional network through the Random Forest model, and finally obtains the classification results. The test results on the Sign Language Digital dataset, Sign Language Gesture Image dataset and Fingers dataset showed that the recognition rate was significantly improved compared with Random Forest, Logistic Regression, Nearest Neighbor, XGBoost, VGG, Inception and MobileNet.
Improving Paraphrase Detection with the Adversarial Paraphrasing Task

If two sentences have the same meaning, it should follow that they are equivalent in their inferential properties, i.e., each sentence should textually entail the other.However, many paraphrase datasets currently in widespread use rely on a sense of paraphrase based on word overlap and syntax.Can we teach them instead to identify paraphrases in a way that draws on the inferential properties of the sentences, and is not over-reliant on lexical and syntactic similarities of a sentence pair?We apply the adversarial paradigm to this question, and introduce a new adversarial method of dataset creation for paraphrase identification: the Adversarial Paraphrasing Task (APT), which asks participants to generate semantically equivalent (in the sense of mutually implicative) but lexically and syntactically disparate paraphrases.These sentence pairs can then be used both to test paraphrase identification models (which get barely random accuracy) and then improve their performance.To accelerate dataset generation, we explore automation of APT using T5, and show that the resulting dataset also improves accuracy.We discuss implications for paraphrase detection and release our dataset in the hope of making paraphrase detection models better able to detect sentence-level meaning equivalence.
Split-and-Bridge: Adaptable Class Incremental Learning within a Single Neural Network

Continual learning has been a major problem in the deep learning community, where the main challenge is how to effectively learn a series of newly arriving tasks without forgetting the knowledge of previous tasks. Initiated by Learning without Forgetting (LwF), many of the existing works report that knowledge distillation is effective to preserve the previous knowledge, and hence they commonly use a soft label for the old task, namely a knowledge distillation (KD) loss, together with a class label for the new task, namely a cross entropy (CE) loss, to form a composite loss for a single neural network. However, this approach suffers from learning the knowledge by a CE loss as a KD loss often more strongly influences the objective function when they are in a competitive situation within a single network. This could be a critical problem particularly in a class incremental scenario, where the knowledge across tasks as well as within the new task, both of which can only be acquired by a CE loss, is essentially learned due to the existence of a unified classifier. In this paper, we propose a novel continual learning method, called Split-and-Bridge, which can successfully address the above problem by partially splitting a neural network into two partitions for training the new task separated from the old task and re-connecting them for learning the knowledge across tasks. In our thorough experimental analysis, our Split-and-Bridge method outperforms the state-of-the-art competitors in KD-based continual learning.
RealTranS: End-to-End Simultaneous Speech Translation with Convolutional Weighted-Shrinking Transformer

End-to-end simultaneous speech translation (SST), which directly translates speech in one language into text in another language in real-time, is useful in many scenarios but has not been fully investigated. In this work, we propose RealTranS, an end-to-end model for SST. To bridge the modality gap between speech and text, RealTranS gradually downsamples the input speech with interleaved convolution and unidirectional Transformer layers for acoustic modeling, and then maps speech features into text space with a weighted-shrinking operation and a semantic encoder. Besides, to improve the model performance in simultaneous scenarios, we propose a blank penalty to enhance the shrinking quality and a Wait-K-Stride-N strategy to allow local reranking during decoding. Experiments on public and widely-used datasets show that RealTranS with the Wait-K-Stride-N strategy outperforms prior end-to-end models as well as cascaded models in diverse latency settings.
Hardness Sampling for Self-Training Based Transductive Zero-Shot Learning

Transductive zero-shot learning (T-ZSL) which could alleviate the domain shift problem in existing ZSL works, has received much attention recently. However, an open problem in T-ZSL: how to effectively make use of unseen-class samples for training, still remains. Addressing this problem, we first empirically analyze the roles of unseen-class samples with different degrees of hardness in the training process based on the uneven prediction phenomenon found in many ZSL methods, resulting in three observations. Then, we propose two hardness sampling approaches for selecting a subset of diverse and hard samples from a given unseen-class dataset according to these observations. The first one identifies the samples based on the class-level frequency of the model predictions while the second enhances the former by normalizing the class frequency via an approximate class prior estimated by an explored prior estimation algorithm. Finally, we design a new Self-Training framework with Hardness Sampling for T-ZSL, called STHS, where an arbitrary inductive ZSL method could be seamlessly embedded and it is iteratively trained with unseen-class samples selected by the hardness sampling approach. We introduce two typical ZSL methods into the STHS framework and extensive experiments demonstrate that the derived T-ZSL methods outperform many state-of-the-art methods on three public benchmarks. Besides, we note that the unseen-class dataset is separately used for training in some existing transductive generalized ZSL (T-GZSL) methods, which is not strict for a GZSL task. Hence, we suggest a more strict T-GZSL data setting and establish a competitive baseline on this setting by introducing the proposed STHS framework to T-GZSL.
Investigating transportation research based on social media analysis: a systematic mapping review

Social media is a pool of users’ thoughts, opinions, surrounding environment, situation and others. This pool can be used as a real-time and feedback data source for many domains such as transportation. It can be used to get instant feedback from commuters; their opinions toward the transportation network and their complaints, in addition to the traffic situation, road conditions, events detection and many others. The problem is in how to utilize social media data to achieve one or more of these targets. A systematic review was conducted in the field of transportation-related research based on social media analysis (TRR-SMA) from the years between 2008 and 2018; 74 papers were identified from an initial set of 703 papers extracted from 4 digital libraries. This review will structure the field and give an overview based on the following grounds: activity, keywords, approaches, social media data and platforms and focus of the researches. It will show the trend in the research subjects by countries, in addition to the activity trends, platforms usage trend and others. Further analysis of the most employed approach (Lexicons) and data (text) will be also shown. Finally, challenges and future works are drawn and proposed.
A Case Study of the Shortcut Effects in Visual Commonsense Reasoning

Visual reasoning and question-answering have gathered attention in recent years. Many datasets and evaluation protocols have been proposed; some have been shown to contain bias that allows models to ``cheat'' without performing true, generalizable reasoning. A well-known bias is dependence on language priors (frequency of answers) resulting in the model not looking at the image. We discover a new type of bias in the Visual Commonsense Reasoning (VCR) dataset. In particular we show that most state-of-the-art models exploit co-occurring text between input (question) and output (answer options), and rely on only a few pieces of information in the candidate options, to make a decision. Unfortunately, relying on such superficial evidence causes models to be very fragile. To measure fragility, we propose two ways to modify the validation data, in which a few words in the answer choices are modified without significant changes in meaning. We find such insignificant changes cause models' performance to degrade significantly. To resolve the issue, we propose a curriculum-based masking approach, as a mechanism to perform more robust training. Our method improves the baseline by requiring it to pay attention to the answers as a whole, and is more effective than prior masking strategies.
Fitness landscape analysis of graph neural network architecture search spaces

Neural Architecture Search (NAS) is the name given to a set of methods designed to automatically configure the layout of neural networks. Their success on Convolutional Neural Networks inspired its use on optimizing other types of neural network architectures, including Graph Neural Networks (GNNs). GNNs have been extensively applied over several collections of real-world data, achieving state-of-the-art results in tasks such as circuit design, molecular structure generation and anomaly detection. Many GNN models have been recently proposed, and choosing the best model for each problem has become a cumbersome and error-prone task. Aiming to alleviate this problem, recent works have proposed strategies for applying NAS to GNN models. However, different search methods converge relatively fast in the search for a good architecture, which raises questions about the structure of the problem. In this work we use Fitness Landscape Analysis (FLA) measures to characterize the search space explored by NAS methods for GNNs. We sample almost 90k different architectures that cover most of the fitness range, and represent them using both a one-hot encoding and an embedding representation. Results of the fitness distance correlation and dispersion metrics show the fitness landscape is easy to be explored, and presents low neutrality.
DualGraph: A graph-based method for reasoning about label noise

Unreliable labels derived from large-scale dataset prevent neural networks from fully exploring the data. Existing methods of learning with noisy labels primarily take noise-cleaning-based and sample-selection-based methods. However, for numerous studies on account of the above two views, selected samples cannot take full advantage of all data points and cannot represent actual distribution of categories, in particular if label annotation is corrupted. In this paper, we start from a different perspective and propose a robust learning algorithm called DualGraph, which aims to capture structural relations among labels at two different levels with graph neural networks including instance-level and distribution-level relations. Specifically, the instance-level relation utilizes instance similarity characterize sample category, while the distribution-level relation describes instance similarity distribution from each sample to all other samples. Since the distribution-level relation is robust to label noise, our network propagates it as supervised signals to refine instance-level similarity. Combining two level relations, we design an end-to-end training paradigm to counteract noisy labels while generating reliable predictions. We conduct extensive experiments on the noisy CIFAR-10 dataset, CIFAR-100 dataset, and the Clothing1M dataset. The results demonstrate the advantageous performance of the proposed method in comparison to state-of-the-art baselines.
Graph coarsening: from scientific computing to machine learning

Abstract The general method of graph coarsening or graph reduction has been a remarkably useful and ubiquitous tool in scientific computing and it is now just starting to have a similar impact in machine learning. The goal of this paper is to take a broad look into coarsening techniques that have been successfully deployed in scientific computing and see how similar principles are finding their way in more recent applications related to machine learning. In scientific computing, coarsening plays a central role in algebraic multigrid methods as well as the related class of multilevel incomplete LU factorizations. In machine learning, graph coarsening goes under various names, e.g., graph downsampling or graph reduction. Its goal in most cases is to replace some original graph by one which has fewer nodes, but whose structure and characteristics are similar to those of the original graph. As will be seen, a common strategy in these methods is to rely on spectral properties to define the coarse graph.
Discriminative Reranking for Neural Machine Translation

Reranking models enable the integration of rich features to select a better output hypothesis within an n-best list or lattice.These models have a long history in NLP, and we revisit discriminative reranking for modern neural machine translation models by training a large transformer architecture.This takes as input both the source sentence as well as a list of hypotheses to output a ranked list.The reranker is trained to predict the observed distribution of a desired metric, e.g.BLEU, over the n-best list.Since such a discriminator contains hundreds of millions of parameters, we improve its generalization using pre-training and data augmentation techniques.Experiments on four WMT directions show that our discriminative reranking approach is effective and complementary to existing generative reranking approaches, yielding improvements of up to 4 BLEU over the beam search output.
Self-attention and adversary learning deep hashing network for cross-modal retrieval

Multi-modal information retrieval is among the prevailing forms of daily human–computer interaction. The recent deep cross-modal hashing methods have received increasing attention because of their superior search performance and efficiency capability. However, effectively exploring the high-ranking semantic correlation and preserving representation consistency are still challengeable due to the heterogeneity property of different modalities. In this paper, a Self-Attention and Adversary Learning Hashing Network (SAALDH) is designed for large scale cross-modal retrieval. Specifically, the hash representations across different layers of the deep network are integrated and then the significance of each position in the integrated hash representation is enhanced by employing a novel self-attention mechanism. Meanwhile, an adversarial learning mechanism is adopted to further preserve the consistency of hash representations during hash functions learning. Moreover, a novel batch semi-hard selection is designed for triplet loss to solve the issue of local optimum during the optimization of SAALDH. Experimental results evaluated on two large scale image-text modality datasets show the effectiveness and efficiency of the proposed SAALDH. And SAALDH achieves better performances by comparing with several state-of-the-art methods. The source code URL of our SAALDH is: http://github.com/SWU-CS-MediaLab/SAALDH.
A topological solution to object segmentation and tracking

The world is composed of objects, the ground, and the sky. Visual perception of objects requires solving two fundamental challenges: 1) segmenting visual input into discrete units and 2) tracking identities of these units despite appearance changes due to object deformation, changing perspective, and dynamic occlusion. Current computer vision approaches to segmentation and tracking that approach human performance all require learning, raising the question, Can objects be segmented and tracked without learning? Here, we show that the mathematical structure of light rays reflected from environment surfaces yields a natural representation of persistent surfaces, and this surface representation provides a solution to both the segmentation and tracking problems. We describe how to generate this surface representation from continuous visual input and demonstrate that our approach can segment and invariantly track objects in cluttered synthetic video despite severe appearance changes, without requiring learning.
The light: science and applications

“The light: science and application” is a project originally applied to fourteen to sixteen-year-old students of Laimos Junior and Senior High School the Municipality of Prespes, in the North Western Greece, although it was originally designed for slightly older students. The project aims to increase students’ motivation and interest in science by renewing the corresponding educational material, implementing modern teaching methods and bringing them into contact with the work of university scientific laboratories. The project is based on the collaboration among schools, universities and scientific laboratories. It encourages the use of new teaching methods and applications in the classroom and the contextualization of STEM learning.
Efficient Machine Learning-Enhanced Channel Estimation for OFDM Systems

Recently much research work has focused on employing deep learning (DL) algorithms to perform channel estimation in the upcoming 6G communication systems.However, these DL algorithms are usually computationally demanding and require a large number of training samples.Hence, this work investigates the feasibility of designing efficient machine learning (ML) algorithms that can effectively estimate and track time-varying, frequency-selective channels.The proposed algorithm is integrated with orthogonal frequency-division multiplexing (OFDM) to eliminate intersymbol interference (ISI) induced by the frequency-selective multipath channel and compared with the well-known least square (LS) and linear minimum mean square error (LMMSE) channel estimation algorithms.The obtained results have demonstrated that even when a small number of pilot samples, N P , is inserted before the N subcarriers OFDM symbol, the introduced ML-based channel estimation is superior to the LS and LMMSE algorithms.This dominance is reflected in the bit-error-rate (BER) performance of the proposed algorithm, which attains a gain of 2.5 dB and 5.5 dB over the LMMSE and LS algorithms, respectively, when N P = N 8 .Furthermore, the BER performance of the proposed algorithm is shown to degrade by only 0.2 dB when the maximum Doppler frequency is randomly varied.Finally, the number of iterations required by the proposed algorithm to converge to the smallest achievable mean-squared error (MSE) are thoroughly examined for various signalto-noise ratio (SNR) levels.
A Trusted Approach for Decentralised and Privacy-Preserving Identity Management

Identity Management (IdM) systems have traditionally relied on a centralized model prone to privacy, trust, and security problems, like potential massive data breaches or identity spoofing. Identity providers accumulate excessive power that might allow them to become a big brother, analyzing and storing as much data as possible. Users should be able to trust identity providers and manage their personal information straightforwardly without compromising their privacy. The European OLYMPUS project introduces a distributed approach for IdM based on enhanced Attribute-Based Credentials (ABC) that splits the role of Identity Provider to limit their influence and chances to become a unique point of failure. However, the trust relationship between service providers, users, and identity providers is still a gap in those kinds of privacy-preserving ABC systems. Decentralized technologies are an opportunity to break away from the centralized model and propose systems that respect privacy while increasing users' trust. This paper presents an evolution of the OLYMPUS architecture, maintaining all the privacy features and incorporating distributed ledger technologies to enhance trust and security in online transactions and IdM systems. The proposed system has been implemented, tested, and validated, showing its performance and feasibility to manage user's identity in a fully privacy-preserving, distributed and reliable way.
Smart Education 4.0: Balancing Dual-Distance and Reskilling Revolution

On January 22, 2020, in Davos, the World Economic Forum launched Reskilling Revolution, a multistakeholder initiative aimed at providing better education, new skills, and better jobs for a billion people worldwide by 2030, so questions about improving the quality of education by introducing dual distance education and retraining of personnel at enterprises are becoming more and more relevant. In the course of the study, the authors analyzed the current problems of universities and enterprises, which allowed them to propose the creation of a smart cluster. The authors demonstrated the interaction of cluster participants using the example of education. The study showed that it is with such a tool that it is possible to solve the current problems of the participants, and in general, to improve the quality of education, the well-being of the region and, as a result, when scaling up, the well-being of the country and its image and culture on the world stage. The study showed that only through joint efforts and cooperation between universities, employers, research institutes and the state, it is possible not only to improve the quality of education but also to provide enterprises with a qualified workforce on a permanent rotation basis than to increase the competitiveness of business. This will allow research institutes and universities to raise funding, which can be directed not only to maintaining the current state but also to the development and introduction of resource-saving technologies, which we see as further research methods.
A benchmark for multivariate probabilistic solar irradiance forecasts

It is well-known that decision-making processes benefit from the inclusion of uncertainty. Such optimization problems typically extend over a control horizon and could span multiple locations or regions. In addition to uncertainty, these optimization problems require as input a trajectory of scalar values that exhibits the correct spatial and temporal dependencies. Probabilistic forecasts quantify the uncertainty by means of quantiles, predictive distributions or ensembles for a forecast horizon and a site or a region separately, and therefore generally lack spatial and temporal dependencies. One solution is to use a copula to model the spatial or temporal dependencies, which, in combination with the probabilistic forecasts, can be used to issue correlated trajectory forecasts. However, there is currently no benchmark model available to compare multivariate probabilistic solar forecasts with. This paper proposes a multivariate probabilistic ensemble (MuPEn) benchmark model and shows that it generalizes the complete-history persistence ensemble (CH-PeEn) to the multivariate case. The proposed benchmark model requires a forecast issue time and a forecast horizon to construct a multivariate empirical distribution of historical clear-sky index measurements from which a multivariate ensemble forecast can be sampled. Similar to CH-PeEn, the proposed benchmark model generates forecasts that are generally calibrated and consistent in terms of energy score and variogram score.