Reviewerly/exHarmony huggingface dataset:
Pose-driven attention-guided image generation for person re-Identification

Person re-identification (re-ID) concerns the matching of subject images across different camera views in a multi camera surveillance system. One of the major challenges in person re-ID is pose variations across the camera network, which significantly affects the appearance of a person. Existing development data lack adequate pose variations to carry out effective training of person re-ID systems. To solve this issue, in this paper we propose an end-to-end pose-driven attention-guided generative adversarial network, to generate multiple poses of a person. We propose to attentively learn and transfer the subject pose through an attention mechanism. A semantic-consistency loss is proposed to preserve the semantic information of the person during pose transfer. To ensure fine image details are realistic after pose translation, an appearance discriminator is used while a pose discriminator is used to ensure the pose of the transferred images will exactly be the same as the target pose. We show that by incorporating the proposed approach in a person re-identification framework, realistic pose transferred images and state-of-the-art re-identification results can be achieved.
Copyright Protection Based on Zero Watermarking and Blockchain for Vector Maps

Zero watermarking does not alter the original information contained in vector map data and provides perfect imperceptibility. The use of zero watermarking for data copyright protection has become a significant trend in digital watermarking research. However, zero watermarking encounters tremendous obstacles to its development and application because of its requirement to store copyright information with a third party and its difficulty in confirming copyright ownership. Aiming at the shortcomings of the existing zero watermarking technology, this paper proposes a new zero watermarking construction method based on the angular features of vector data that store the zero watermarking and copyright information on the blockchain after an XOR operation. When the watermark is being extracted, the copyright information can be extracted with the XOR operation to obtain the information stored on the blockchain. Experimental results show that the combination of zero watermarking and blockchain proposed in this paper gives full play to the advantages of the two technologies and protects the copyright of data in a lossless fashion. Compared with the traditional zero watermarking algorithms, the proposed zero watermarking algorithm exhibits stronger robustness. Moreover, the proposed data copyright protection framework with a combination of zero watermarking and blockchain can also be applied to other data types, such as images, audio, video, and remote sensing images.
An Efficient Self-Organizing Deep Fuzzy Neural Network for Nonlinear System Modeling

A fuzzy neural network (FNN) is an effective learning system that combines neural network and fuzzy logic, which has achieved great success in nonlinear system modeling. However, when the input is practical complex data with external disturbance, the existing FNN cannot extract effective input features sufficiently, leading to unsatisfactory performances in learning speed and accuracy. It also fails to achieve a better generalization capability because of its fixed structure size (the number of rule neurons). In this article, an efficient self-organizing FNN (SOFNN) with incremental deep pretraining (IDPT), called IDPT-SOFNN, is developed to overcome these shortcomings. First, IDPT is designed to extract effective features and consider them as the input of the SOFNN. Different from the existing pretraining, the self-growing structure of IDPT improves pretraining efficiency with a more compact structure. Second, the SOFNN can dynamically add and delete neurons according to the current error and error-reduction rate. In this case, it can obtain better modeling performance with a more compact structure as well. Third, as a novel hybrid model with the cascade dual-self-organizing algorithm, the IDPT-SOFNN combines the advantage of IDPT and SOFNN. Moreover, the convergence and stability are analyzed. Finally, simulation studies and comparisons demonstrate that the proposed IDPT-SOFNN has better performances than its peers in learning speed, accuracy, and generalization capability.
Dance Movement Recognition Based on Feature Expression and Attribute Mining

There are complex posture changes in dance movements, which lead to the low accuracy of dance movement recognition. And none of the current motion recognition uses the dancer’s attributes. The attribute feature of dancer is the important high-level semantic information in the action recognition. Therefore, a dance movement recognition algorithm based on feature expression and attribute mining is designed to learn the complicated and changeable dancer movements. Firstly, the original image information is compressed by the time-domain fusion module, and the information of action and attitude can be expressed completely. Then, a two-way feature extraction network is designed, which extracts the details of the actions along the way and takes the sequence image as the input of the network. Then, in order to enhance the expression ability of attribute features, a multibranch spatial channel attention integration module (MBSC) based on an attention mechanism is designed to extract the features of each attribute. Finally, using the semantic inference and information transfer function of the graph convolution network, the relationship between attribute features and dancer features can be mined and deduced, and more expressive action features can be obtained; thus, high-performance dance motion recognition is realized. The test and analysis results on the data set show that the algorithm can recognize the dance movement and improve the accuracy of the dance movement recognition effectively, thus realizing the movement correction function of the dancer.
Persian handwritten digit, character and word recognition using deep learning

In spite of various applications of digit, letter and word recognition, only a few studies have dealt with Persian scripts. In this paper, deep neural networks are utilized through different DenseNet and Xception architectures, being further boosted by means of data augmentation and test time augmentation. Dividing the datasets to training, validation and test sets, and utilizing k-fold cross-validation, the comparison of the proposed method with various state-of-the-art alternatives is performed. Three datasets: HODA, Sadri and Iranshahr are used, which offer the most comprehensive collections of samples in terms of handwriting styles and the forms each letter may take depending on its position within a word. On the HODA dataset, we achieve recognition rates of 99.49% and 98.10% for digits and characters, being 99.72%, 89.99% and 98.82% for digits, characters and words from the Sadri dataset, respectively, as well as 98.99% for words from the Iranshahr dataset, each of which outperforms the performances achieved by the most advanced alternative networks, namely ResNet50 and VGG16. An additional contribution of the paper arises from its capability of words recognition as a holistic image classification. This improves the resulting speed and versatility significantly, as it does not require explicit character models, unlike earlier alternatives such as hidden Markov models and convolutional recursive neural networks. In addition, computation times have been compared with alternative state-of-the-art models and better performance has been observed.
Adversarial Graph Convolutional Network for Cross-Modal Retrieval

The completeness of semantic expression plays an important role in cross-modal retrieval tasks, which contributes to align the cross-modal data and thus narrow the modality gap. But due to the abstractness of semantics, the same topic may have different aspects to be well described so it may be incomplete to express semantics with only one sample. In order to obtain semantic complementary information and strengthen similar information for samples with the same semantics, we utilize a graph convolutional network (GCN) to reconstruct the sample representation based on the adjacency relationship between the sample itself and its neighborhoods. We construct a local graph for each instance, and propose a novel Graph Feature Generator based on GCN and a fully-connected network to reconstruct node features based on local graph and map the features of two modalities into a common space. The Graph Feature Generator and Graph Feature Discriminator adopt a minimax game strategy to generate modality-invariant graph feature representations. Experiments on three benchmark datasets demonstrate the superiority of our proposed model compared with several state-of-the-art methods.
To Ameliorate Classification Accuracy using Ensemble Distributed Decision Tree (DDT) Vote Approach: An Empirical discourse of Geographical Data Mining

Weather data of Kashmir province has 6 attributes recorded at three different substations. This paper proposes a distributed decision tree algorithm and its implementation on Historical Geographical data of Kashmir province. The machine learning Decision tree algorithm applied on the Kashmir province dataset generates the accuracy of 81.54%. The distributed decision tree generates multiple trees based on the partitions of the original dataset in which the data is segregated according to the substations (42026, 42027 and 42044). The ratio between generated data sets was distributed in 32.38%, 34.19% and 33.42% respectively which is appropriate for the parallelism. Its distributed implementation, i.e. Distributed Decision Tree produces a specified number of sub-trees (depending upon number of partitions of input dataset) and at the end collects votes or averages the prediction or classification. In this paper, we have implemented the hard- voting approach to calculate the overall performance of the n-number of trees in distributed environment. The empirical results demonstrate that distributed decision trees approach has not improved the overall accuracy as compared to the original dataset without partitioning.
Transformer-Transducers for Code-Switched Speech Recognition

We live in a world where 60% of the population can speak two or more languages fluently. Members of these communities constantly switch between languages when having a conversation. As automatic speech recognition (ASR) systems are being deployed to the real-world, there is a need for practical systems that can handle multiple languages both within an utterance or across utterances. In this paper, we present an end-to-end ASR system using a transformer-transducer model architecture for code-switched speech recognition. We propose three modifications over the vanilla model in order to handle various aspects of code-switching. First, we introduce two auxiliary loss functions to handle the low-resource scenario of code-switching. Second, we propose a novel mask-based training strategy with language ID information to improve the label encoder training towards intra-sentential code-switching. Finally, we propose a multi-label/multi-audio encoder structure to leverage the vast monolingual speech corpora towards code-switching. We demonstrate the efficacy of our proposed approaches on the SEAME dataset, a public Mandarin-English code-switching corpus, achieving a mixed error rate of 18.5% and 26.3% on test <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">man</inf> and test <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">sge</inf> sets respectively.
Deep Learning in Image Classification: A Survey Report

Recently, deep learning is emerging as a powerful tool and has become a leading machine learning tool in computer vision and image analysis. In this survey paper, we provide a snapshot of this fast-growing field, image classification, specifically. We briefly introduce several popular neutral networks and summarize their applications in image classification. In addition, we also discuss the challenge of deep learning in image classification.
AUTOTRAINER: An Automatic DNN Training Problem Detection and Repair System

With machine learning models especially Deep Neural Network (DNN) models becoming an integral part of the new intelligent software, new tools to support their engineering process are in high demand. Existing DNN debugging tools are either post-training which wastes a lot of time training a buggy model and requires expertises, or limited on collecting training logs without analyzing the problem not even fixing them. In this paper, we propose AUTOTRAINER, a DNN training monitoring and automatic repairing tool which supports detecting and auto repairing five commonly seen training problems. During training, it periodically checks the training status and detects potential problems. Once a problem is found, AUTOTRAINER tries to fix it by using built-in state-of-the-art solutions. It supports various model structures and input data types, such as Convolutional Neural Networks (CNNs) for image and Recurrent Neural Networks (RNNs) for texts. Our evaluation on 6 datasets, 495 models show that AUTOTRAINER can effectively detect all potential problems with 100% detection rate and no false positives. Among all models with problems, it can fix 97.33% of them, increasing the accuracy by 47.08% on average.
Experimental evaluation of ensemble classifiers for imbalance in Big Data

Datasets are growing in size and complexity at a pace never seen before, forming ever larger datasets known as Big Data. A common problem for classification, especially in Big Data, is that the numerous examples of the different classes might not be balanced. Some decades ago, imbalanced classification was therefore introduced, to correct the tendency of classifiers that show bias in favor of the majority class and that ignore the minority one. To date, although the number of imbalanced classification methods have increased, they continue to focus on normal-sized datasets and not on the new reality of Big Data. In this paper, in-depth experimentation with ensemble classifiers is conducted in the context of imbalanced Big Data classification, using two popular ensemble families (Bagging and Boosting) and different resampling methods. All the experimentation was launched in Spark clusters, comparing ensemble performance and execution times with statistical test results, including the newest ones based on the Bayesian approach. One very interesting conclusion from the study was that simpler methods applied to unbalanced datasets in the context of Big Data provided better results than complex methods. The additional complexity of some of the sophisticated methods, which appear necessary to process and to reduce imbalance in normal-sized datasets were not effective for imbalanced Big Data.
Reading in VR: The Effect of Text Presentation Type and Location

Reading is a fundamental activity to obtain information both in the real and the digital world. Virtual reality (VR) allows novel approaches for users to view, read, and interact with a text. However, for efficient reading, it is necessary to understand how a text should be displayed in VR without impairing the VR experience. Therefore, we conducted a study with 18 participants to investigate text presentation type and location in VR. We compared world-fixed, edge-fixed, and head-fixed text locations. Texts were displayed using Rapid Serial Visual Presentation (RSVP) or as a paragraph. We found that RSVP is a promising presentation type for reading short texts displayed in edge-fixed or head-fixed location in VR. The paragraph presentation type using world-fixed or edge-fixed location is promising for reading long text if movement in the virtual environment is not required. Insights from our study inform the design of reading interfaces for VR applications.
Non-Autoregressive Transformer ASR with CTC-Enhanced Decoder Input

Non-autoregressive (NAR) transformer models have achieved significantly inference speedup but at the cost of inferior accuracy compared to autoregressive (AR) models in automatic speech recognition (ASR). Most of the NAR transformers take a fixed-length sequence filled with MASK tokens or a redundant sequence copied from encoder states as decoder input, they cannot provide efficient target-side information thus leading to accuracy degradation. To address this problem, we propose a CTC-enhanced NAR transformer, which generates target sequence by refining predictions of the CTC module. Experimental results show that our method outperforms all previous NAR counterparts and achieves 50x faster decoding speed than a strong AR baseline with only 0.0∼ 0.3 absolute CER degradation on Aishell-1 and Aishell-2 datasets.
Data Animator: Authoring Expressive Animated Data Graphics

Animation helps viewers follow transitions in data graphics. When authoring animations that incorporate data, designers must carefully coordinate the behaviors of visual objects such as entering, exiting, merging and splitting, and specify the temporal rhythms of transition through staging and staggering. We present Data Animator, a system for authoring animated data graphics without programming. Data Animator leverages the Data Illustrator framework to analyze and match objects between two static visualizations, and generates automated transitions by default. Designers have the flexibility to interpret and adjust the matching results through a visual interface. Data Animator also supports the division of a complex animation into stages through hierarchical keyframes, and uses data attributes to stagger the start time and vary the speed of animating objects through a novel timeline interface. We validate Data Animator’s expressiveness via a gallery of examples, and evaluate its usability in a re-creation study with designers.
nnenum: Verification of ReLU Neural Networks with Optimized Abstraction Refinement

The surge of interest in applications of deep neural networks has led to a surge of interest in verification methods for such architectures. In summer 2020, the first international competition on neural network verification was held. This paper presents and evaluates the main optimizations used in the nnenum tool, which outperformed all other tools in the ACAS Xu benchmark category, sometimes by orders of magnitude. The method uses fast abstractions for speed, combined with refinement through ReLU splitting to increase accuracy when properties cannot be proven. Although the abstraction refinement process is a classic approach in formal methods, directly applying it to the neural network verification problem actually reduces performance, due to a cascade of overapproxmation error when using abstraction. This makes optimizations and their systematic evaluation essential for high performance.
Cross-Domain Sentiment Classification with Contrastive Learning and Mutual Information Maximization

Existing language models usually require large amount of labeled data and are severely challenged by domain shift. In this work we propose a novel model for cross-domain sentiment classification - CLIM - Contrastive Learning with mutual Information Maximization, to explore the potential of contrastive learning for learning domain-invariant and task-discriminative features. To the best of our knowledge, CLIM is the first to investigate contrastive learning for cross-domain sentiment classification. Due to the scarcity of labels on the target domain, we introduce mutual information maximization (MIM) to explore the features that best support the final prediction. Furthermore, MIM is able to maintain a relatively balanced distribution of the model’s prediction, and enlarge the margin between classes on the target, which increases the model robustness and enables the same classifier to be optimal across domains. Consequently, we achieve new state-of-the-art results on the Amazon-review dataset as well as the Airlines dataset, demonstrating the efficacy of our methods.
Designing Interactive Transfer Learning Tools for ML Non-Experts

Interactive machine learning (iML) tools help to make ML accessible to users with limited ML expertise. However, gathering necessary training data and expertise for model-building remains challenging. Transfer learning, a process where learned representations from a model trained on potentially terabytes of data can be transferred to a new, related task, offers the possibility of providing "building blocks" for non-expert users to quickly and effectively apply ML in their work. However, transfer learning largely remains an expert tool due to its high complexity. In this paper, we design a prototype to understand non-expert user behavior in an interactive environment that supports transfer learning. Our findings reveal a series of data- and perception-driven decision-making strategies non-expert users employ, to (in)effectively transfer elements using their domain expertise. Finally, we synthesize design implications which might inform future interactive transfer learning environments.
Semi-automatic liver tumor segmentation with adaptive region growing and graph cuts

Segmenting liver tumors from computed tomography (CT) images plays a very important role in computer-aided diagnosis, surgical planning, and treatment monitoring. However, accurate and robust segmentation of the tumors remains a challenging issue, due to low contrast and vague boundaries between the tumors and surrounding tissues as well as the wide variations of the tumors in intensity, shape, and location across patients. In this paper, we developed an effective method for liver tumor segmentation with adaptive region growing and graph cuts. First, initial segmentation results for liver tumors and the regions of interest (ROIs) that contain the tumors are extracted by adaptive region growing with a manual selected seed specified for each tumor region. Then, the ROIs are enhanced by Gaussian fitting based nonlinear mapping according to the intensity distributions of the initially segmented tumor regions. Finally, the enhanced information combined with gradient information is integrated into graph cuts to extract the tumors from the ROIs effectively and accurately. The method is non-sensitive to noise and does not involve a pre-segmentation of liver or a complicated and tedious procedure of training. Results on 3Dircadb dataset demonstrate that the method achieves much better comprehensive performance on liver tumor segmentation compared with many art-of-state methods and has a huge advantage in segmenting the tumors with low contrast, small size, and weak boundaries.
Relation-based Discriminative Cooperation Network for Zero-Shot Classification

Zero-shot learning (ZSL) aims to assign the category corresponding to the relevant semantic as the label of the unseen sample based on the relationship between the learned visual and semantic features. However, most typical ZSL models faced with the domain bias problem, which leads to unseen or test samples being easily misclassified into seen or training categories. To handle this problem, we propose a relation-based discriminative cooperation network (RDCN) model for ZSL in this work. The proposed model effectively utilize the robust metric space spanned by the cooperated semantics with the help of a set of relations. On the other hand, we devise the relation network to measure the relationship between the visual features and embedded semantics, and the validation information will guide the embedding module to learn more discriminative information. At last, the proposed RDCN model is validated on six benchmarks, and extensive experiments demonstrate the superiority of proposed method over most existing ZSL models on the traditional zero-shot setting and the more realistic generalized zero-shot setting.