Reviewerly/exHarmony huggingface dataset:
Recurrent Attention Network with Reinforced Generator for Visual Dialog

In Visual Dialog, an agent has to parse temporal context in the dialog history and spatial context in the image to hold a meaningful dialog with humans. For example, to answer “what is the man on her left wearing?” the agent needs to (1) analyze the temporal context in the dialog history to infer who is being referred to as “her,” (2) parse the image to attend “her,” and (3) uncover the spatial context to shift the attention to “her left” and check the apparel of the man. In this article, we use a dialog network to memorize the temporal context and an attention processor to parse the spatial context. Since the question and the image are usually very complex, which makes it difficult for the question to be grounded with a single glimpse, the attention processor attends to the image multiple times to better collect visual information. In the Visual Dialog task, the generative decoder (G) is trained under the word-by-word paradigm, which suffers from the lack of sentence-level training. We propose to reinforce G at the sentence level using the discriminative model (D), which aims to select the right answer from a few candidates, to ameliorate the problem. Experimental results on the VisDial dataset demonstrate the effectiveness of our approach.
Constructing multilayer locality-constrained matrix regression framework for noise robust face super-resolution

Representation learning methods have attracted considerable attention for learning-based face super-resolution in recent years. Conventional methods perform local models learning on low-resolution (LR) manifold and face reconstruction on high-resolution (HR) manifold respectively, leading to unsatisfactory reconstruction performance when the acquired LR face images are severely degraded (e.g., noisy, blurred). To tackle this issue, this paper proposes an efficient multilayer locality-constrained matrix regression (MLCMR) framework to learn the representation of the input LR patch and meanwhile preserve the manifold of the original HR space. Particularly, MLCMR uses nuclear norm regularization to capture the structural characteristic of the representation residual and applies an adaptive neighborhood selection scheme to find the HR patches that are compatible with its neighbors. Also, MLCMR iteratively applies the manifold structure of the desired HR space to induce the representation weights learning in the LR space, aims at reducing the inconsistency gap between different manifolds. Experimental results on widely used FEI database and real-world faces have demonstrated that compared with several state-of-the-art face super-resolution approaches, our proposed approach has the capability of obtaining better results both in objective metrics and visual quality.
Trustworthy Edge Storage Orchestration in Intelligent Transportation Systems Using Reinforcement Learning

A large scale fast-growing data generated in intelligent transportation systems (ITS) has become a ponderous burden on the coordination of heterogeneous transportation networks, which makes the traditional cloud-centric storage architecture no longer satisfy new data analytics requirements. Meanwhile, the lack of storage trust between ITS devices and edge servers could lead to security risks in the data storage process. However, a unified data distributed storage architecture for ITS with intelligent management and trustworthiness is absent in the previous works. To address these challenges, this paper proposes a distributed trustworthy storage architecture with reinforcement learning in ITS, which also promotes edge services. We adopt an intelligent storage scheme to store data dynamically with reinforcement learning based on trustworthiness and popularity, which improves resource scheduling and storage space allocation. Besides, trapdoor hashing based identity authentication protocol is proposed to secure transportation network access. Due to the interaction between cooperative devices, our proposed trust evaluation mechanism is provided with extensibility in the various ITS. Simulation results demonstrate that our proposed distributed trustworthy storage architecture outperforms the compared ones in terms of trustworthiness and efficiency.
Supervised Pixel-Wise GAN for Face Super-Resolution

For many face-related multimedia applications, low-resolution face images may greatly degrade the face recognition performance and necessitate face super-resolution (SR). Among the current SR methods, MSE-oriented SR methods often produce over-smoothed outputs and could miss some texture details while GAN-oriented SR methods may generate artifacts which are harmful to face recognition. To resolve the above issues, this paper presents a supervised pixel-wise Generative Adversarial Network (SPGAN) that can resolve a very low-resolution face image of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$16\times 16$</tex-math></inline-formula> or smaller pixel-size to its larger version of multiple scaling factors ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$2\times$</tex-math></inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$4\times$</tex-math></inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$8\times$</tex-math></inline-formula> and even <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$16\times$</tex-math></inline-formula> ) in a unified framework. Being different from traditional unsupervised discriminators which generate a single number to represent the likelihood whether the input image is real or fake, the proposed supervised pixel-wise discriminator mainly focus on whether each pixel of the generated SR face image is as photo-realistic as its corresponding pixel in the ground-truth HR (high-resolution) face image. To further improve the face recognition performance of SPGAN, we take advantage of the face identity prior by sending two inputs to the discriminator, including an input face image (either a real HR face image or its corresponding SR face image) and its face features which are extracted from a pre-trained face recognition model. Due to the introduced face identity prior, the identity-based discriminator can pay more attention to texture details which are closely related to face recognition. Extensive experiments demonstrate that the proposed SPGAN can achieve more photo-realistic SR images and higher face recognition accuracy than some state-of-the-art methods.
GRVMP: A Greedy Randomized Algorithm for Virtual Machine Placement in Cloud Data Centers

Cloud computing efficiency greatly depends on the efficiency of the virtual machines (VMs) placement strategy used. However, VM placement has remained one of the major challenging issues in cloud computing mainly because of the heterogeneity in both virtual and physical machines (PMs), the multidimensionality of the resources, and the increasing scale of the cloud data centers (CDCs). An inefficiency in VM placement strategy has a significant influence on the quality of service provided, the amount of energy consumed, and the running costs of the CDCs. To address these issues, in this article, we propose a greedy randomized VM placement (GRVMP) algorithm in a large-scale CDC with heterogeneous and multidimensional resources. GRVMP inspires the “power of two choices” model and places VMs on the more power-efficient PMs to jointly optimize CDC energy usage and resource utilization. The performance of GRVMP is evaluated using synthetic and real-world production scenarios (Amazon EC2) with several performance matrices. The results of the experiment confirm that GRVMP jointly optimizes power usage and the overall wastage of resource utilization. The results also show that GRVMP significantly outperforms the baseline schemes in terms of the performance metrics used.
Real-time 3D unstructured environment reconstruction utilizing VR and Kinect-based immersive teleoperation for agricultural field robots

Moving and operating autonomously in a field or orchard environment is challenging for agricultural robots due to the complex task requirements and highly unstructured conditions. The human–computer interaction-based remote control can provide the robots with an alternative solution to assist decision making and motion planning. In this study, a virtual reality (VR) and Kinect-based immersive teleoperation system were proposed to connect the physical and virtual world by utilizing real-time large-scale unstructured agricultural environment reconstruction and simultaneous virtual environment creation. The proposed system, with a relatively large server, can convert the scene into a realistic model by combing the depth and color image streams received from Kinect, and project them back into 3D (three dimensions) space in such a manner that the real 3D scene inside the camera’s field of view is recreated virtually. To create a VR environment for a VR headset, an optimized Bundlefusion-based algorithm was developed for real-time 3D reconstruction of the unstructured agricultural scene in the natural environment. Additionally, the performance of the proposed real-time 3D reconstruction algorithm was evaluated and compared with Bundlefusion and voxel hashing algorithms in different large-scale unstructured agricultural environments. Performing optimizing pose on our optimized algorithm leverages a large number of processing cores available to minimize the delay between data capture and rendering, and it reduces the average acquired time to process each frame no more than 0.9 ms. The experimental results including, less computer storage occupied, fast frame processing time, and high-quality 3D realistic model indicate that our proposed system and algorithm have the potential applicability of immersive teleoperation in an unstructured agricultural environment.
Comparative Study of Table Tennis Forehand Strokes Classification Using Deep Learning and SVM

Object sensors are widely used for motion capture, particularly in sport motion analysis for classification of strokes. In this paper, a comparative study was performed to examine the Forehand strokes classification of three Machine Learning (ML) models in Table Tennis. Support Vector Machine (SVM) with Radial Basis Function (RBF) kernel function, Long Short-Term Memory (LSTM) and 2-Dimensional Convolutional Neural Network (2D-CNN) are considered. Tuning the models' parameters and examining sensitivities of the models to the number of training datasets and type of data are studied. All models were trained and tested on Table Tennis Forehand strokes' signals, which were collected from professional and novice Table Tennis players'. A BNO055 sensor was attached to the center of the standard racket as an Object sensor to measure the strokes' signals. To provide a robustness assessment of all models to type of dataset, the study employs 80% of novice players' strokes samples for testing classification accuracy. The empirical results suggest that the LSTM and the 2D-CNN classification outperforms with a substantial performance increase of approximately 7% more than the RBF-SVM. In addition, in the case of models sensitivity to the number of training samples it can be said that, the LSTM model performance on the self-collected dataset is not significantly sensitive to the number of training samples. However, the number of the developed RBF-SVM parameters is significantly less than other two deep models and are easy to set. Study shows that the modified LSTM model performed better than other two models.
Early Prediction of Response to Neoadjuvant Chemotherapy in Breast Cancer Sonography Using Siamese Convolutional Neural Networks

Early prediction of response to neoadjuvant chemotherapy (NAC) in breast cancer is crucial for guiding therapy decisions. In this work, we propose a deep learning based approach for the early NAC response prediction in ultrasound (US) imaging. We used transfer learning with deep convolutional neural networks (CNNs) to develop the response prediction models. The usefulness of two transfer learning techniques was examined. First, a CNN pre-trained on the ImageNet dataset was utilized. Second, we applied double transfer learning, the CNN pre-trained on the ImageNet dataset was additionally fine-tuned with breast mass US images to differentiate malignant and benign lesions. Two prediction tasks were investigated. First, a L1 regularized logistic regression prediction model was developed based on generic neural features extracted from US images collected before the chemotherapy (a priori prediction). Second, Siamese CNNs were used to quantify differences between US images collected before the treatment and after the first and second course of NAC. The proposed methods were evaluated using US data collected from 39 tumors. The better performing deep learning models achieved areas under the receiver operating characteristic curve of 0.797 and 0.847 in the case of the a priori prediction and the Siamese model, respectively. The proposed approach was compared with a method based on handcrafted morphological features. Our study presents the feasibility of using transfer learning with CNNs for the NAC response prediction in US imaging.
Emotional Analysis using Twitter Data during Pandemic Situation: COVID-19

During the crisis situation caused due to COVID-19 disease, managing mental health and psychological well -being is as important as physical health of people.As web based life is broadly utilized by individuals to communicate their feeling and supposition, our framework utilizes Twitter information posted by individuals during this emergency circumstance to dissect the feelings of individuals.For processing the cleaned data NRC Word-Emotion Association Lexicon (have aka EmoLex) is used.NRC Word-Emotion Association Lexicon is a list of English with real-valued scores of intensity for eight basic emotion words ns (anger, anticipation, disgust, fear, joy, sadness, surprise, and trust).The text content of tweeter dataset created by fetching tweets across the world have classified into basic emotions like anger, anticipation, disgust, fear, joy, sadness, surprise and trust.This analysis can be used by authorities to understand the mental health of the people and can take necessary measures to decide on policies to fight against coronavirus which is affecting the social well-being as well as economy of the whole world.
Review of research on lightweight convolutional neural networks

The application of convolutional neural networks is becoming more and more mature, and the use of mobile terminals is becoming more and more widespread. In order to deploy convolutional neural networks to mobile terminals, it is necessary to carry out lightweight design of the network, mainly from the aspects of parameter quantity and calculation quantity. At the same time, the accuracy of the network must be guaranteed. The main idea of lightweight model design is to design a more efficient “network computing method”, mainly for the convolution method. Firstly, several convolution methods are introduced, then six lightweight convolutional neural networks that have been excellent in recent years are described, and the innovations of the model are discussed. Next, the accuracy and parameters of each model on the ImageNet data set are analyzed, and the lightweight techniques of each model are further compared. The importance of group convolution, 1 × 1 convolution, residual unit, and channel information circulation is found. Also combining lightweight design with reinforcement learning and neural network architecture search methods will have better results. Finally, the lightweight convolutional neural network is summarized and prospected.
Discriminative Ridge Machine: A Classifier for High-Dimensional Data or Imbalanced Data

In this article, we introduce a discriminative ridge regression approach to supervised classification. It estimates a representation model while accounting for discriminativeness between classes, thereby enabling accurate derivation of categorical information. This new type of regression model extends the existing models, such as ridge, lasso, and group lasso, by explicitly incorporating discriminative information. As a special case, we focus on a quadratic model that admits a closed-form analytical solution. The corresponding classifier is called the discriminative ridge machine (DRM). Three iterative algorithms are further established for the DRM to enhance the efficiency and scalability for real applications. Our approach and the algorithms are applicable to general types of data including images, high-dimensional data, and imbalanced data. We compare the DRM with current state-of-the-art classifiers. Our extensive experimental results show the superior performance of the DRM and confirm the effectiveness of the proposed approach.
Soft-Weighted-Average Ensemble Vehicle Detection Method Based on Single-Stage and Two-Stage Deep Learning Models

The deep learning object detection algorithms have become one of the powerful tools for road vehicle detection in autonomous driving. However, the limitation of the number of high-quality labeled training samples makes the single-object detection algorithms unable to achieve satisfactory accuracy in road vehicle detection. In this paper, by comparing the pros and cons of various object detection algorithms, two different algorithms with a different emphasis are selected for a weighted ensemble. Besides, a new ensemble method named the Soft-Weighted-Average method is proposed. The proposed method is attenuated by the confidence, and it “punishes” the detection result of the corresponding relationship by the confidence attenuation, instead of by deleting the output of a certain model. The proposed method can further reduce the vehicle misdetection of the target detection algorithm, obtaining a better detection result. Lastly, the ensemble method can achieve an average accuracy of 94.75% for simple targets, which makes it the third-ranked method in the KITTI evaluation system.
Contact and Human Dynamics from Monocular Video

Existing deep models predict 2D and 3D kinematic poses from video that are approximately accurate, but contain visible errors that violate physical constraints, such as feet penetrating the ground and bodies leaning at extreme angles. In this paper, we present a physics-based method for inferring 3D human motion from video sequences that takes initial 2D and 3D pose estimates as input. We first estimate ground contact timings with a novel prediction network which is trained without hand-labeled data. A physics-based trajectory optimization then solves for a physically-plausible motion, based on the inputs. We show this process produces motions that are significantly more realistic than those from purely kinematic methods, substantially improving quantitative measures of both kinematic and dynamic plausibility. We demonstrate our method on character animation and pose estimation tasks on dynamic motions of dancing and sports with complex contact patterns.
Network education recommendation and teaching resource sharing based on improved neural network

Intelligent network teaching system provides learners with abundant teaching resources and convenient, excellent and efficient learning environment. However, network teaching resources are widely distributed and difficult to centralize. Resource sharing has become a key problem to be solved in the network environment. The current research on online education resource recommendation mainly focuses on offline education, and there are few studies on online education resources. Based on this, this study studies the link prediction methods in online education and establishes appropriate models for online education. In the research, through improved analysis of traditional algorithms, an improved neural network path sorting algorithm based on path sorting method is proposed. At the same time, we use the path sorting algorithm based on random walk model and neural network-path sorting algorithm to realize the link prediction problem in the online learning knowledge base. In addition, the performance analysis of the algorithm is carried out by contrast method, and the performance comparison analysis is carried out by combining various common traditional recommendation algorithms with the research algorithm of this study.
Privacy-preserving point-of-interest recommendation based on geographical and social influence

We investigate a privacy-preserving problem for point-of-interest (POI) recommendation system for rapidly growing location-based social networks (LBSNs). The LBSN-based recommendation algorithms usually consider three factors: user similarity, social influence between friends and geographical influence in. The LBSN-based recommendation system first needs to collect relevant information of users and then provide them with potentially interesting contents. However, sensitive information of users may be leaked when the recommendation is provided. In this article, we focus on preventing user’s privacy from disclosure upon geographical location and friend relationship factors. We propose a geographical location privacy-preserving algorithm (GLP) that achieves 〈r,h〉-privacy and present a friend relationship privacy-preserving algorithm (FRP) through adding Laplacian distributed noise for fusing the user trusts. Subsequently, we integrate the GLP and FRP algorithms into a general recommendation system and build a privacy-preserving recommendation system. The novel system enjoys the privacy guarantee under the metric differential entropy through theoretical analysis. Experimental results demonstrate a good trade-off between privacy and accuracy of the proposed recommendation system.
Source fingerprinting loess deposits in Central Asia using elemental geochemistry with Bayesian and GLUE models

The provenance of loess deposits in Central Asia is largely unexplored. Accordingly, the goals of this research were to test and compare the performance of two different models (generalized likelihood uncertainty estimation - GLUE and a Bayesian model) for quantifying the uncertainty in source apportionment estimated for 46 target loess samples collected in the Ili basin, in eastern Central Asia. Model performance was evaluated using goodness-of-fit (GOF), mean absolute fit (MAF) and virtual mixtures (VM) in combination with root mean square error (RMSE) and mean absolute error (MAE). Our dataset comprised 132 surficial samples collected from three potential sources comprising river alluvium (n = 29), sand dunes (n = 35) and topsoils (n = 68). All samples were analysed for elemental geochemistry. Six geochemical properties (Co, Er, Y, Ga, Dy and Pb) were selected in a composite fingerprint which classified 83% of the samples from the three source categories correctly. Based on both models, source contributions to the loess samples were in the following order: topsoils > river alluvium > sand dunes. Based on the GOF and MAF tests, both models were accurate in predicting measured tracer concentrations in the loess samples. The Bayesian model was slightly more accurate (mean RMSE 1.6%, mean MAE 1.8%) than the GLUE (mean RMSE 5.0%, mean MAE 4.7%) model in predicting known source contributions. Overall, our results provide confirmation that application of source fingerprinting with elemental geochemistry and uncertainty modelling techniques is useful for identifying the provenance of loess sediments in arid and desert environments.