Reviewerly/exHarmony huggingface dataset:
Data Aggregation and Privacy Preserving Using Computational Intelligence

In today's smart world, the privacy protection of data is an important issue. Data is distributed, reproduced, and disclosed with extensive use of communication technologies. Many non-traditional challenges arise with the rapid increase of IoT devices for system design and implementation. However, security and privacy are the main issues in IoT. With advanced technologies, an illegal copy of the content can easily be generated and shared. Therefore, it is crucial for users to protect and secure their data. In the said perspective, an efficient third-generation watermarking technique is proposed, which works on the computational intelligence model to insert a large amount of robust watermark and make an extra effort to hide more information than first and second-generation techniques. The Advanced Encryption Standard (AES) encryption algorithm is employed to guarantee secure communication, which has a significantly less computational cost. The proposed technique evaluated parameters including security, robustness, imperceptibility, and capacity. The results of the proposed technique are compared to existing text watermarking methods, which illustrates it is secure, robust, imperceptible, and inserts a large amount of watermark information through computational intelligence.
Automated Video Description for Blind and Low Vision Users

Video accessibility is crucial for blind and low vision users for equitable engagements in education, employment, and entertainment. Despite the availability of professional description services and tools for amateur description, most human-generated descriptions are expensive and time consuming, and the rate of human-generated descriptions simply cannot match the speed of video production. To overcome the increasing gaps in video accessibility, we developed a system to automatically generate descriptions for videos and answer blind and low vision users’ queries on the videos. Results from a pilot study with eight blind video aficionados indicate the promise of this system for meeting needs for immediate access to videos and validate our efforts in developing tools in partnership with the individuals we aim to benefit. Though the results must be interpreted with caution due to the small sample size, participants overall reported high levels of satisfaction with the system, and all preferred use of the system over no support at all.
Gabor-Deep CNN based Masked Face Recognition for Fraud Prevention

With the new overall COVID-19 pandemic, utilizing face veils have become a significant piece of our lives. To evade the spread of contamination Individuals are urged to cover their faces using a mask when in open territory. The utilization of these face veils makes the orthodox facial recognition system inadequate for authentication in school/office participation, face access control, facial participation, facial security checks at airport and train stations, etc. The recent progressed deep learning-based face acknowledgment approaches rely upon countless training samples and significant features of the entire face. This paper addresses the issue of recognition of face and veil using Gabor wavelet and deep transfer learning. Here the Gabor wavelet features are extracted from the non-masked region of the face and integrated with deep learned CNN features to generate a more robust feature vector for optimizing recognition. The proposed method is experimented on 4 benchmark datasets and a manually prepared dates and achieved averaged 97% recognition accuracy.
Improvement of k-nearest neighbor algorithm based on double filtering

Aiming at the shortcoming that the efficiency of k-nearest neighbor algorithm (kNN) greatly decreases with the increase of the number of samples, a two-screen k-nearest neighbor algorithm is proposed. Its purpose is to increase the calculation speed of the algorithm while ensuring that the accuracy rate does not decrease, so that the improved algorithm can handle big data problems. Firstly, large-scale spectral clustering based on landmark representation (LSC algorithm) is used to cluster the data to find the cluster closest to the point to be measured; secondly, triangle inequality is used to process the data set close to the point to be measured and far from the nearest cluster center. The data after these two screenings is then used as the final training sample of the k-nearest neighbor algorithm. It can improve the diversity and effectiveness of data on the basis of reducing the number of training samples. Finally, it was verified on a public data set and compared with the traditional k nearest neighbor algorithm. The results show that the speed of the algorithm has been improved, the calculation amount has been reduced by 49%, and the accuracy has been maintained at about 95%.
Object Detection in Urban Aerial Image Based on Advanced YOLO v3 Algorithm

Aiming at the characteristics of large changes in object scale and complex background in urban aerial image, we propose an advanced YOLOv3 detection algorithm to solve it. First, we analyze the aerial data through a clustering algorithm and calculate optimal size of the prior anchors. Then, a relatively lightweight and easily extensible backbone network-deep residual network is used for feature extraction. Meanwhile, in order to obtain a better receptive field and further reduce the information loss in the process of convolution, we add deformable convolution to the 8 times down-sampling feature map. And it improves the network's learning ability for geometrically deformed objects and attention mechanism before the 32 times down-sampling feature map to adaptively learn the weight relationship between different channels. Finally, IoU (Intersection over Union) is added to the regression loss function to improve the accuracy of evaluating the bounding box. The experimental results show that the inference speed of the optimized algorithm we proposed can reach 50 FPS, and the avera ge accuracy is 20% higher than the original network, which can complete the detection more effectively.
Item based recommendation using matrix-factorization-like embeddings from deep networks

In this paper we describe a method for computing item based recommendations using matrix-factorization-like embeddings of the items computed using a neural network. Matrix factorizations (MF) compute near optimal item embeddings by minimizing a loss that measures the discrepancy between the predicted and known values of a sparse user-item rating matrix. Though useful for recommendation tasks, they are computationally intensive and hard to compute for large sets of users and items. Hence there is need to compute MF-like embeddings using other less computationally intensive methods, which can be substituted for the actual ones. In this work we explore the possibility of doing the same using a deep neural network (DNN). Our network is trained to learn matrix-factorization-like embeddings from easy to compute natural language processing (NLP) based semantic embeddings. The resulting MF-like embeddings are used to compute recommendations using an anonymized user product engagement dataset from the online retail company Overstock.com. We present the results of using our embeddings for computing recommendations with the Overstock.com production dataset consisting of ~3.5 million items and ~6 million users. Recommendations from Overstock.com's own recommendation system is compared against those obtained by using our MF-like embeddings, by comparing the results from both to the ground truth, which in our case is actual user co-clicks data. Our results show that it is possible to use DNNs for efficiently computing MF-like embeddings which can then be used in conjunction with the NLP based embeddings to improve the recommendations obtained from the NLP based embeddings.
Spatio-Temporal 3D Action Recognition with Hierarchical Self-Attention Mechanism

3D action recognition is a long-standing problem in the field of computer vision. Given the 3D coordinate set of body joints, it is desired to recognize what activity is performed. The problem can be approached using a time-series model. Recent advancements in the field of recurrent neural networks have enabled the use of sophisticated memory cells that can predict time series using the information from earlier elements of a sequence. In this article, we proposed a hierarchical architecture that attends to its own signature through time, which can put more weight on time frames of the sequence that are more specific to the performed action. Accordingly, using memory cells, a self-attention mechanism is implemented. In addition, spatial attention is also considered by sub-grouping and then regrouping body parts down the architecture hierarchy. We evaluate the proposed model on NTU and MSR 3D action datasets. An accuracy of 79.8% and 97.8% on NTU and MSR datasets indicated that the proposed method outperforms the previous methods tested in this paper.
Cross Scene Video Foreground Segmentation Via Co-Occurrence Probability Oriented Supervised and Unsupervised Model Interaction

Using only one deep model for cross scene video foreground segmentation is still very challenging because existing methods are scene-dependent, which restricts the consistent segmentation. In this paper, we propose a cross scene video foreground segmentation framework to extend the generalization capability of those supervised model depending on scene-specific training. The proposed framework flexibly utilizes three well-trained supervised models as guidance to yield a coarse segmentation mask. The co-occurrence probability-based unsupervised background subtraction model is introduced to achieve scene adaptation in the plug and play style without any fine-tuning and labels. Experimental results on LIMU and CDNet2014 datasets validate our framework outperforms the state-of-the-art supervised/unsupervised approaches that participate in the comparison. Experiments also show the training efficiency-related improvements – when introducing the guidance models, the demand for quantity and quality of training samples to train the unsupervised model is reduced. Codes https://github.com/MeteoorLiu/Venus/tree/MeteoorLiu-SUMC
A Perspective on Building Ethical Datasets for Children's Conversational Agents

Artificial intelligence (AI)-powered technologies are becoming an integral part of youth's environments, impacting how they socialize and learn. Children (12 years of age and younger) often interact with AI through conversational agents (e.g., Siri and Alexa) that they speak with to receive information about the world. Conversational agents can mimic human social interactions, and it is important to develop socially intelligent agents appropriate for younger populations. Yet it is often unclear what data are curated to power many of these systems. This article applies a sociocultural developmental approach to examine child-centric intelligent conversational agents, including an overview of how children's development influences their social learning in the world and how that relates to AI. Examples are presented that reflect potential data types available for training AI models to generate children's conversational agents' speech. The ethical implications for building different datasets and training models using them are discussed as well as future directions for the use of social AI-driven technology for children.
HMFlow: Hybrid Matching Optical Flow Network for Small and Fast-Moving Objects

In optical flow estimation task, coarse-to-fine warping strategy is widely used to deal with the large displacement problem and provides efficiency and speed. However, limited by the small search range between the first images and warped second images, current coarse-to-fine optical flow networks fail to capture small and fast-moving objects which has disappeared at coarse resolution levels. To address this problem, we introduce a lightweight but effective Global Matching Component (GMC) to grab global matching features. We propose a new Hybrid Matching Optical Flow Network (HMFlow) by integrating GMC into existing coarse-to-fine networks seamlessly. Besides keeping in high accuracy and small model size, our proposed HMFlow can apply global matching features to guide the network to discover the small and fast-moving objects mismatched by local matching features. We also build a new dataset, named SFChairs, for evaluation. The experimental results show that our proposed network achieves considerable performance, especially at regions with small and fast-moving objects.
Human Activity Recognition for Indoor Robotics: A Deep Learning Based Approach Using a Human Detection Stage

In this paper, it is proposed a Deep-Learning (DL) based Human Activity Recognition (HAR) framework which uses as input RGB video data. It consists on a two-stage cascade approach: 1) a human detection stage where a bounding box, containing the person performing the activity, is extracted from each frame; 2) a DL-based feature extraction and classification stage. The main focus is on daily activities that are useful for social, service and assistive robots operating in indoor environments. Three different frameworks, including the one proposed, are evaluated and compared. Seven different DL architectures for the feature extraction and classification step are trained and tested. Training and testing is performed on the large scale NTU RGB+D 120 [1] action recognition dataset. The runtime performance of each framework is also evaluated by calculating the number of samples that each processes per second. Obtained results show that the proposed framework improves the classification accuracy up to 22.6 percentage points (depending on the DL architecture used for the feature extraction and classification module). The reported real-time performances indicate that a real-world scenario application for the evaluated frameworks is possible, given the current technology available in laptop units.
Case Study on Data-Driven Deployment of Program Analysis on an Open Tools Stack

Program analysis can assist software development but its use islimited by a number of usability issues, including false positivesand integration issues. Previous systems have show significantprogress addressing these by using a data-driven approach totrack the 'not useful' results, and integrating the results into codereview. However, these systems were very context dependent. Inthis paper, we motivate the application of the same data-drivenprinciples and code-review based integration to the design of theMEAN system, an analysis infrastructure running on an opentools stack. We then evaluate its implementation within a localsoftware company, and present implications for the design offuture program analysis systems.
Hessian-regularized spectral clustering for behavior recognition

In recent years, human behavior recognition has become a hot research issue in computer vision, pattern recognition and other fields. There are a large number of image or video samples with unknown labels in real life, and labeling unknown samples is a time-consuming and laborious task. Therefore, this paper adopts unsupervised learning method to study human behavior recognition, that is, this research propose a Hessian-regularized spectral clustering algorithm and apply it to human behavior recognition. This method uses the Hessian matrix to construct the spectral clustering graph, which can make better use of a large amount of unlabeled information. In order to verify the effectiveness of the improved spectral clustering algorithm, a large number of experiments are conducted on UCF-iphone data set, which is a human behavior data set. The experimental results show that the Hessian-regularized spectral clustering algorithm can effectively improve the accuracy of behavior recognition.
Compressing Local Descriptor Models for Mobile Applications

Feature-based image matching has been significantly improved through the use of deep learning and new large datasets. However, there has been little work addressing the computational cost, model size, and matching accuracy tradeoffs for the state of the art models. In this paper, we consider these practical aspects and improve the state-of-the-art HardNet model through the use of depthwise separable layers and an efficient tensor decomposition. We propose the Convolution-Depthwise-Pointwise (CDP) layer, which partitions the weights into a low and full rank decomposition to exploit the naturally emergent structure in the convolutional weights. We can achieve an 8× reduction in the number of parameters on the HardNet model, 13× reduction in the computational complexity, while sacrificing less than 1% on the overall accuracy across the HPatches benchmarks. To further demonstrate the generalisation of this approach, we apply it to other state-of-the-art descriptor models, where we are able to a significant performance improvement.
Sentiment Analysis of China's Education Policy Online Opinion Based on Text Mining

With the in-depth development of China's education reform, the combination of educational public opinion and emotion analysis is conducive to the discovery of the evolution law of public sentiment and related factors, and provides ideas for the guidance of public opinion of relevant government departments. This paper proposes an analysis model of public opinion on the education policy network based on text mining, obtains network comment contents through Octopus crawler, preprocesses the data, and introduces the calculation formula of emotion heat, on which the evolution process of public opinion is divided into incubation period, outbreak period, spreading period and decline period. Combined with visualization and emotional heat maps of provinces and ages are constructed, so as to grasp the spatial-temporal evolution law of educational public opinions. Taking the postponement of the college entrance examination in 2020 as an example, the empirical analysis proves that the online public opinion analysis model of education policy can reflect the evolution law.
Towards Data Selection on TTS Data for Children’s Speech Recognition

Although great progress has been made on automatic speech recognition (ASR) systems, children’s speech recognition still remains a challenging task. General ASR systems for children’s speech suffer from the lack of corpora and mismatch between children’s and adults’ speech. Efforts have been made to reduce such mismatch by applying normalization methods to generate modified adults’ speech for ASR training. However, modified adults’ data can reflect the characteristics of children’s speech to a very limited extent. In this work, we adopt text-to-speech data augmentation to improve the performance of children’s speech recognition system. We find that the children’s TTS model generates speech with inconsistent quality due to children’s substandard pronunciations of phonemes, and the ASR system suffers when trained with these additional synthesized data. To solve this problem, we propose data selection strategies on the TTS augmented data, and the effectiveness of the synthesized data can be substantially boosted for children’s ASR modeling. We show that the speaker embedding similarity based data selection strategy can obtain the best position: relative 14.0% and 14.7% CER reduction for child conversation and child reading test set respectively compared to the baseline model trained on real data.
A Novel CP-ABE Based Sidechain Protocol for Distributed Power System Data Storage Management with the Blockchain

Due to technical bottlenecks such as systems vulnerable to attacks and data are easily tampered with, and it is difficult for the distributed power system data storage management to achieve multi-agent integration and development. Known as the "trust machine," the Blockchain has an essential role in promoting the energy Internet construction with its distributed storage, tamper-proof, traceable, and multi-node sharing technical characteristics. Its technical advantages have great potential application value in the power industry. Most of the existing blockchain schemes directly combine the CP-ABE algorithm with the Blockchain. However, due to the lack of permission inheritance, these schemes have permission to redundancy and waste computing resources. In this paper, a new user-controlled on-chain permission management scheme is proposed. The original CP-ABE model is extended in this scheme, and the concept of role assignment and attribute tree is proposed. The attribute tree uses the multi-tree structure to realize the attribute set's specification description. The connectivity between roles in the attribute tree represents the inheritance relationship of permission so that the permission inheritance and permission granting can be realized. In this paper, a sidechain prototype system is developed for evaluating the scheme, and its performance is tested and analyzed. The experimental results show that the prototype system can realize the user-controlled permission assignment and permission inheritance function of data owner on the Blockchain.
Adaptable Ensemble Distillation

Online knowledge distillation (OKD), which simultaneously trains several peer networks to construct a powerful teacher on on-the-fly, has drawn much attention in recent years. OKD is designed to simplify the training procedure of conventional offline distillation. However, the ensemble strategy of existing OKD methods is inflexible and highly relies on random initializations. In this paper, we propose Adaptable Ensemble Distillation (AED) that inherits the merits of existing OKD methods while overcoming their major drawbacks. The novelty of our AED lies in three aspects: (1) an individual-regulated mechanism is proposed to flexibly regulate individual model and further generates an online ensemble with strong adaptability; (2) a diversity-aroused loss is designed to explicitly diversify individual models, which enhances the robustness of the ensemble; (3) an empirical distillation technique is adopted to directly promote knowledge transfer in OKD framework. Extensive experiments show that our proposed AED consistently outperforms the existing state-of-the-art OKD methods on various datasets.
Drawing Order Recovery from Trajectory Components

In spite of widely discussed, drawing order recovery (DOR) from static images is still a great challenge task. Based on the idea that drawing trajectories are able to be recovered by connecting their trajectory components in correct orders, this work proposes a novel DOR method from static images. The method contains two steps: firstly, we adopt a convolution neural network (CNN) to predict the next possible drawing components, which is able to covert the components in images to their reasonable sequences. We denote this architecture as Im2Seq-CNN; secondly, considering possible errors exist in the reasonable sequences generated by the first step, we construct a sequence to order structure (Seq2Order) to adjust the sequences to the correct orders. The main contributions include: (1) the Img2Seq-CNN step considers DOR from components instead of traditional pixels one by one along trajectories, which contributes to static images to component sequences; (2) the Seq2Order step adopts image position codes instead of traditional points’ coordinates in its encoder-decoder gated recurrent neural network (GRU-RNN). The proposed method is experienced on two well-known open handwriting databases, and yields robust and competitive results on handwriting DOR tasks compared to the state-of-arts.