Reviewerly/exHarmony huggingface dataset:
Language Variety Prediction using Word Embeddings and Machine Leaning Algorithms

Abstract: Author Profiling is a technique of predicting demographic characteristics like gender, age, location, nativity language, educational background etc., of an author by analysing their written texts. Author profiling is used in several text processing applications like forensics analysis, marketing, security. The author profiling techniques identify the stylistic differences among the author writing styles to identify the demographics of authors. Researchers experimented with various stylistic features like lexical features, content-based features, syntactic features, semantic features, domain specific features, structural features, readability features etc., to identify the stylistic differences among different author’s texts. The dataset plays an important role to analyse the stylistic differences of authors. PAN is one competition organizes different types of tasks in every year to encourage the participants around the globe for providing solutions to different types of text classification problems like plagiarism detection, authorship attribution, authorship verification, authorship profiling, celebrity profiling, style change detection, fake news spreaders detection, hate speech spreaders detection etc. The author profiling task was introduced in 2013 by the organizers of PAN competition. The organizers carefully gather the datasets and make available to the researchers for providing solutions to the problems. Every year the organizers conduct competitions on different sub-tasks of author profiling and provides datasets in different languages and in different genres. In 2017 competition, PAN introduces a task of predicting the language variety of an author. They release the dataset in four languages. In this work, we proposed an approach for English language dataset of language variety prediction. The proposed approach used the word embeddings generated by the Word2Vec model and BERT (Bidirectional Encoder Representations from Transformers) model. The word embeddings are used for generating the document vectors by combining the word embeddings of words those contain in documents. The document vectors are trained with two machine learning algorithms such as support vector machine and random forest. The Random Forest attained best accuracy of 96.87 for language variety prediction when experiment conducted with BERT embeddings
PVGCRA: Prediction Variance Guided Cross Region Domain Adaptation

Semantic segmentation has become a very important task in computer vision in recent years, however, it usually requires a large amount of labeled data matching the considered scene to obtain reliable performance. Collecting and labeling large datasets for each new task and domain is very expensive, time-consuming and error-prone. To cope with this, unsupervised domain adaptation methods have been attempted for semantic segmentation tasks. Existing methods still suffer from poor class-level feature alignment and pseudo labels contain much noise. The latest method Cross Region Domain Adaptation (CRA) applies adversarial training to align the feature distribution of trusted and untrusted regions of the target domain image. In this paper, we re-model the uncertainty estimation module and propose to use the prediction variance as an uncertainty estimation method to align the feature distribution in the new trusted and untrusted regions. This approach is simply called Prediction Variance Guided Cross Region Domain Adaptation (PVGCRA). Experiments on the typical unsupervised domain-adaptive semantic segmentation task scenario GTA5 → Cityscapes show that this method improves the performance of the segmentation model and possesses better performance.
Catastrophic forgetting avoidance method for a Classification Model by Model Synthesis and Introduction of Background Data

Animals including humans, continuously acquire knowledge and skills throughout their lives. However, many machine learning models cannot learn new tasks without forgetting past knowledge. In neural networks, it is common to use one neural network for each training task, and successive training will reduce the accuracy of the previous task. This problem is called catastrophic forgetting, and research on continual learning is being conducted to solve it. In this paper, we proposed a method to reducing catastrophic forgetting, where new tasks are trained without retaining previously trained data. Our method assumes that tasks are classification. Our method adds random data to the training data in order to combine models trained on different tasks to avoid exceed generalization in the domain where train data do not exist combines models separately trained for each tasks. In the evaluation experiments, we confirmed that our method reduced forgetting for the original two-dimensional dataset and MNIST dataset.
Response Sentence Modification Using a Sentence Vector for a Flexible Response Generation of Retrieval-based Dialogue Systems

We proposed a sentence modification method that makes retrieval-based dialogue response generation more flexible. A retrieval-based dialogue model is simple and robust, but it cannot output a response sentence that is not prepared in the example-response database. On the other hand, a generation-based dialogue model can flexibly generate the response, but the developer cannot control how the system responds to the input. The proposed model is based on the retrieval-based model, and the idea is that the model modifies the response sentence according to the difference between the example sentence and the input sentence. To realize the proposed method, we developed a new Transformer-based autoencoder model to calculate sentence vectors that contain enough information to reconstruct the input sentence. The proposed autoencoder, m-Query, can recover sentences much more accurately than the conventional model. Next, we trained the sentence modification model using a dialogue corpus and conducted an experiment to generate response sentences. As a result, it was shown that the proposed model generated modified responses reflecting the difference between the input and example sentences, such as subject or tense.
Improving co-SVD for cold-start recommendations using sparsity reduction

Recommender systems are highly dependent on the users' or items' historical data. The completeness of the data determines the performance of the models, especially for models based on the collaborative filtering (CF) technique. Under cold-start situations, where there are limited relevant historical data on the users' or items' information, producing accurate recommendations are challenging. We propose the use of the implicit Alternating Least Square (iALS) method to predict users' preferences and impute it into the matrix co-factorization algorithm, co-SVD. The proposed approach aims to alleviate the cold-start problem that is most evident in large datasets that have high sparsity. In addition, we included the results for two cold-start situations, cold-start user and cold-start item (long-tail), using our hybrid co-SVD with artificial ratings imputation. The F1 score of the top-5 recommendations generated by the proposed approach improved from 25.08% to 30.09% under the cold-start user situation. With the long-tail item situation, the proposed approach improved from 20.8% to 23.19%. The proposed approach is method-agnostic, and other CF-based models can benefit from this imputation method.
Neural Network Based Watermarking Trained with Quantized Activation Function

We propose a watermarking method that incor-porates a quantized activation function to provide robustness against quantization. Zhu et al. showed that the introduction of a noise layer between the encoder and decoder can increase the robustness against attacks. Although there are various attacks on stego-images, these images are often JPEG-compressed. As the process of JPEG compression includes quantization, the watermark decoder must be able to estimate watermarks from compressed images. Hence, we propose a quantization layer that introduces a quantized activation function consisting of the hy-perbolic tangent function. The proposed neural network is based on that proposed by Hamamoto and Kawamura. By simulating the quantization of JPEG compression, the quantization layer is expected to improve the robustness against JPEG compression. The robustness was evaluated by the bit error rate (BER), and the stego-image quality was evaluated by the peak signal-to-noise ratio (PSNR). The proposed network achieved a high image quality of more than 35 dB, and it could extract watermarks with a BER of less than 0.1 for Q-values of 30 or higher in JPEG compression. It was thus more robust against JPEG compression than Hamamoto and Kawamura's model.
Unified Angle Adjustment Network for Image Composition Enhancement

We propose an angle adjustment algorithm for the composition enhancement of digital photographs. The proposed algorithm jointly learns the scene type, composition, and semantic line information of an image to improve the accuracy of angle adjustment. To this end, we design a unified angle adjustment network (UAAN), which consists of a unified encoder and four task-specific refinement modules and estimators. First, we generate shared features using the unified encoder. Then, we refine those features using the refinement modules to perform the four tasks of angle regression, scene type classification, composition classification, and semantic line detection. Experimental results demonstrate the effectiveness of the proposed UAAN algorithm.
Enhanced Bidirectional Motion Estimation Using Feature Refinement for HDR Imaging

We propose a high dynamic range (HDR) image synthesis algorithm based on enhanced bidirectional motion estimation using feature refinement. First, we extract multiscale features from input low dynamic range (LDR) images and then estimate accurate motion vector fields between them in a coarse-to-fine manner via progressive refinement. Then, we estimate adaptive local kernels to merge only valid information in the spatio-exposed neighboring pixels for synthesis. Finally, we refine the initially merged image by exploiting global information to further improve synthesis performance. Experimental results show that the proposed algorithm outperforms state-of-the-art algorithms in quantitative and qualitative comparisons.
A Fair Model is not Fair in a Biased Environment

Facial images contain sensitive attributes such as skin color, and the elimination of them from the input in the face recognition is not easy. In addition, the input data includes the influence of the environment in which the system is actually used, so the interaction between sensitive attributes and the environment may make it inherently difficult for the facial feature extractor to extract facial features. Therefore, studies on the fairness of face recognition should consider the fairness of environmental factors. Common datasets used to evaluate the fairness of face recognition includes a variety of environmental factors, and the fairness evaluated by these datasets are usually the fairness in a typical shooting environment. However, a dataset that includes only extremely biased environmental factors potentially results in less equity among attributes. We construct a dataset with pseudo-biased environmental factors by dynamically changing environmental factors such as brightness in the test data. The results also show that the biased environmental factors deteriorate the fairness inter-attribute. Also, we showed that the distinguished attributes in terms of fairness in a biased environment vary based on the architecture of the model and the training dataset.
How Should We Evaluate Synthesized Environmental Sounds

Although several methods of environmental sound synthesis have been proposed, there has been no discussion on how synthesized environmental sounds should be evaluated. Only either subjective or objective evaluations have been conducted in conventional evaluations, and it is not clear what type of evaluation should be carried out. In this paper, we investigate how to evaluate synthesized environmental sounds. We also propose a subjective evaluation methodology to evaluate whether the synthesized sound appropriately represents the information input to the environmental sound synthesis system. In our experiments, we compare the proposed and conventional evaluation methods and show that the results of subjective evaluations tended to differ from those of objective evaluations. From these results, we conclude that it is necessary to conduct not only objective evaluation but also subjective evaluation.
Flow-Based Variational Sequence Autoencoder

Posterior collapse, also known as the Kullback-Leibler (KL) vanishing, is a long-standing problem in variational recurrent autoencoder (VRAE) which is essentially developed for sequence generation. To alleviate the vanishing problem, a complicated latent variable is required instead of assuming it as standard Gaussian. Normalizing flow was proposed to build the bijective neural network which converts a simple distribution into a complex distribution. The resulting approximate posterior is closer to real posterior for better sequence generation. The KL divergence in learning objective is accordingly preserved to enrich the capability of generating the diverse sequences. This paper presents the flow-based VRAE to build the disentangled latent representation for sequence generation. KL preserving flows are exploited for conditional VRAE and evaluated for text representation as well as dialogue generation. In the im-plementation, the schemes of amortized regularization and skip connection are further imposed to strengthen the embedding and prediction. Experiments on different tasks show the merit of this latent variable representation for language modeling, sentiment classification and dialogue generation.
Source Tracing: Detecting Voice Spoofing

Recent anti-spoofing systems focus on spoofing detection, where the task is only to determine whether the test audio is fake. However, there are few studies putting attention to identifying the methods of generating fake speech. Common spoofing attack algorithms in the logical access (LA) scenario, such as voice conversion and speech synthesis, can be divided into several stages: input processing, conversion, waveform generation, etc. In this work, we propose a system for classifying different spoofing attributes, representing characteristics of different modules in the whole pipeline. Classifying attributes for the spoofing attack other than determining the whole spoofing pipeline can make the system more robust when encountering complex combinations of different modules at different stages. In addition, our system can also be used as an auxiliary system for anti-spoofing against unseen spoofing methods. The experiments are conducted on ASVspoof 2019 LA data set and the proposed method achieved a 20% relative improvement against conventional binary spoof detection methods.
A Unified Compression and Watermarking Scheme for MT-BTC Images

As most multimedia is compressed for optimal storage or transmission, the watermarking during compression is more appropriate and demanding. Multi-tone block truncation coding image (MT-BTC) is a latest and superior version of halftone based block truncation coding images. In this paper, a novel watermarking strategy is proposed for MT-BTC images using the adaptive dither array selection (ADAS) and inter-tone shifting. ADAS method utilize dither array constructed using two different gaussian filters based on the Human Visual System (HVS) model to embed the watermark information. Further, various configuration of dither array such as actual, conjugate, transpose and transpose conjugate is used to embed more data. Moreover, a new approach termed inter-tone shifting is also proposed to improve the decoding rate and security. The decoding scheme is performed using the pattern similarity on the dithered watermarked image. For result validation, a 2,4,6 and 8-Tone watermark image is embedded in the 4-Tone MTBTC image. From extensive analysis on decoding rate and robustness, it has been validated that the proposed scheme outperforms many halftone watermarking and consistent with conventional BTC methods.
Multi-class Vehicle Counting System for Multi-view Traffic Videos

This paper presents a vehicle counting system using multi-class vehicle detection using YOLOX and multi-object tracking using ByteTrack. Counting is performed for each class of the vehicle including bus, car, motorcycle, pickup, truck, and van in a predefined region of interest (ROI). Our proposed system is designed to handle noisy and low contrast traffic videos of top and side view of the vehicles. In particular, side view videos show the occlusion of the two directional lanes which lead to vehicle occlusion problems. For object detection and classification, YOLOX shows promising mean average precision (mAP) approximately at 77.2 and 58.8 percent for top and side views, respectively, which outperforms YOLOv3 for both top and side view datasets. The counting results show that ByteTrack with YOLOX can handle vehicle occlusion which occurs in the side view videos. Counting the vehicles within an ROI can reduce fault detections for both top view and side view videos.
Selection of Supplementary Acoustic Data for Meta-Learning in Under-Resourced Speech Recognition

Automatic speech recognition (ASR) for under-resourced languages has been a challenging task during the past decade. In this paper, regarding Taiwanese as the under resourced language, the speech data of the high-resourced languages which have most phonemes in common with Taiwanese are selected as the supplementary resources for meta-training the acoustic models for Taiwanese ASR. Mandarin, English, Japanese, Cantonese and Thai as the high-resourced languages are selected as the supplementary languages based on the designed selection criteria. Model-agnostic meta-learning (MAML) is then used as the meta-training strategy. For evaluation, when 4000 utterances were selected from each supplementary language, we obtained the WER of 20.89% and the SER of 8.86% for Taiwanese ASR. The results were better than the baseline model (26.18% and 13.99%) using only the Taiwanese corpus and traditional method.
A Method for Adversarial Example Generation by Perturbing Selected Pixels

Recent research has shown that deep neural networks can intentionally change their output by adding perturbation to the input. Such images are called adversarial examples. An attack method that uses sparse perturbations is Jacobian-based Saliency Map Attack(JSMA), which finds the pixel to perturb by generating a saliency map from the gradient of the output. It deceives the neural network by changing the pixel value to the maximum or minimum value. However, changing the value of a pixel to a maximum or minimum value is not optimal for generating adversarial examples because the perturbation is perceived by human eyes. In this study, we propose a new method to reduce perturbations and generate adversarial examples in which perturbations are not easily recognized by human eyes. Our method generates adversarial examples with smaller perturbations by improving the extraction conditions of pixels in JSMA to be perturbed and the method of adding perturbations. Experimental results show that the method can generate smaller perturbations with a misclassification rate comparable to that of JSMA. This makes the perturbations less recognizable to human eyes.
Pre-trained multimodal end-to-end network for spoken language assessment incorporating prompts

Conventional systems for spoken language assessment rely on intermediate text features extracted from recognized transcripts by an automatic speech recognition (ASR) system, and/or additional handcrafted delivery features (e.g., pronunciation and fluency). It is challenging for less restricted tasks due to deteriorating performance of ASR and difficulty to design sufficient features. In this paper, we propose a multimodal end-to-end network with audio and text functioning as two modalities, avoiding utilizing intermediate features. We design a novel pre-training scheme to guide and tie the audio and text representations. Specifically, we utilize paired audio-transcript and premise-hypothesis data, where the former is commonly used for ASR training and the latter for natural language inference (NLI). The network is optimized by a multi-task learning (MTL) method combining a word matching task and an NLI task. The pre-trained network is then fine-tuned using a small number of paired audio-prompt and scoring data. Experimental results based on data from spoken English tests demonstrate the superior performance of the proposed model in Pearson correlation coefficient (PCC) and accuracy to the baselines.
Automated Audio Captioning with Epochal Difficult Captions for curriculum learning

In this paper, we propose an algorithm, Epochal Difficult Captions, to supplement the training of any model for the Automated Audio Captioning task. Epochal Difficult Captions is an elegant evolution to the keyword estimation task that previous work have used to train the encoder of the AAC model. Epochal Difficult Captions modifies the target captions based on a curriculum and a difficulty level determined as a function of current epoch. Epochal Difficult Captions can be used with any model architecture and is a lightweight function that does not increase training time. We test our results on three systems and show that using Epochal Difficult Captions consistently improves performance by up to 0.013 SPIDEr score.
Multi-Stage Superpixel-Based Segmentation Algorithm Using Fully Convolutional Networks and Discriminative Features

Image segmentation is a process to partition an image into multiple segments. Pixels within the same segments share certain characteristics. Therefore, in this paper, we simplify the segmentation task to a superpixel-merging decision problem by considering discriminative features. Characteristics, including border, color, brightness, area size, sharpness, saliency, texton, and etc. are taken into account. We propose a multi-stage superpixel-clustering algorithm to merge superpixels of similar characteristics. In the first stage, the fully convolutional network is applied to decide whether the boundary of two adjacent superpixels should be kept or not. The model is trained basing on color, superpixel boundary, and edge detection result of the image. In the second stage, scoring method and SVM classification model are used to further decide whether the rest superpixels should be merged or not. We consider up to 14 factors to further improve the performance. Overall, simulations and evaluation metrics show that our algorithm has highly accurate segmentation results.