Reviewerly/exHarmony huggingface dataset:
ScatterNet: A Deep Subjective Similarity Model for Visual Analysis of Scatterplots

Similarity measuring methods are widely adopted in a broad range of visualization applications. In this work, we address the challenge of representing human perception in the visual analysis of scatterplots by introducing a novel deep-learning-based approach, ScatterNet, captures perception-driven similarities of such plots. The approach exploits deep neural networks to extract semantic features of scatterplot images for similarity calculation. We create a large labeled dataset consisting of similar and dissimilar images of scatterplots to train the deep neural network. We conduct a set of evaluations including performance experiments and a user study to demonstrate the effectiveness and efficiency of our approach. The evaluations confirm that the learned features capture the human perception of scatterplot similarity effectively. We describe two scenarios to show how ScatterNet can be applied in visual analysis applications.
Multi-view approaches for software and system modelling: a systematic literature review

Over the years, a number of approaches have been proposed on the description of systems and software in terms of multiple views represented by models. This modelling branch, so-called multi-view software and system modelling, praises a differentiated and complex scientific body of knowledge. With this study, we aimed at identifying, classifying, and evaluating existing solutions for multi-view modelling of software and systems. To this end, we conducted a systematic literature review of the existing state of the art related to the topic. More specifically, we selected and analysed 40 research studies among over 8600 entries. We defined a taxonomy for characterising solutions for multi-view modelling and applied it to the selected studies. Lastly, we analysed and discussed the data extracted from the studies. From the analysed data, we made several observations, among which: (i) there is no uniformity nor agreement in the terminology when it comes to multi-view artefact types, (ii) multi-view approaches have not been evaluated in industrial settings and (iii) there is a lack of support for semantic consistency management and the community does not appear to consider this as a priority. The study results provide an exhaustive overview of the state of the art for multi-view software and systems modelling useful for both researchers and practitioners.
Intelligent Edge-Assisted Crowdcast with Deep Reinforcement Learning for Personalized QoE

Recent years have seen booming development and great success in interactive crowdsourced livecast (i.e., crowdcast). Different from traditional livecast services, crowdcast is featured with tremendous video contents at the broadcaster side, highly diverse viewer side content watching environments/preferences as well as viewers' personalized quality of experience (QoE) demands (e.g., individual preferences for streaming delays, channel switching latencies and bitrates). This imposes unprecedented key challenges on how to flexibly and cost-effectively accommodate the heterogeneous and personalized QoE demands for the mass of viewers. In this paper, we propose DeepCast, an edge-assisted crowdcast framework, which makes intelligent decisions at edges based on the massive amount of real-time information from the network and viewers to accommodate personalized QoE with minimized system cost. Given the excessive computation complexity in this context, we propose a data-driven deep reinforcement learning (DRL) based solution that can automatically learn the best suitable strategies for viewer scheduling and transcoding selection. To our best knowledge, DeepCast is the first edge-assisted framework that applies the advance of DRL to explicitly accommodate personalized QoE optimization for crowdcast services. We collect multiple real-world datasets and evaluate the performance of DeepCast using trace-driven experiments. The results demonstrate the superiority of our DeepCast framework and its DRL-based solution.
Research on Auto-Scaling of Web Applications in Cloud: Survey, Trends and Future Directions

Cloud computing emerging environment attracts many applications providers to deploy web applications on cloud data centers. The primary area of attraction is elasticity, which allows to auto-scale the resources on-demand. However, web applications usually have dynamic workload and hard to predict. Cloud service providers and researchers are working to reduce the cost while maintaining the Quality of Service (QoS). One of the key challenges for web application in cloud computing is auto-scaling. The auto-scaling in cloud computing is still in infancy and required detail investigation of taxonomy, approach and types of resources mapped to the current research. In this article, we presented the literature survey for auto-scaling techniques of web applications in cloud computing. This survey supports the research community to find the requirements in auto-scaling techniques. We present a taxonomy of reviewed articles with parameters such as auto-scaling techniques, approach, resources, monitoring tool, experiment, workload, and metric, etc. Based on the analysis, we proposed the new areas of research in this direction.
Enhancing Collaborative Filtering with Generative Augmentation

Collaborative filtering (CF) has become one of the most popular and widely used methods in recommender systems, but its performance degrades sharply for users with rare interaction data. Most existing hybrid CF methods try to incorporate side information such as review texts to alleviate the data sparsity problem. However, the process of exploiting and integrating side information is computationally expensive. Existing hybrid recommendation methods treat each user equally and ignore that the pure CF methods have already achieved both effective and efficient recommendation performance for active users with sufficient interaction records and the little improvement brought by side information to these active users is ignorable. Therefore, they are not cost-effective solutions. One cost-effective idea to bypass this dilemma is to generate sufficient "real" interaction data for the inactive users with the help of side information, and then a pure CF method could be performed on this augmented dataset effectively. However, there are three major challenges to implement this idea. Firstly, how to ensure the correctness of the generated interaction data. Secondly, how to combine the data augmentation process and recommendation process into a unified model and train the model end-to-end. Thirdly, how to make the solution generalizable for various side information and recommendation tasks. In light of these challenges, we propose a generic and effective CF model called AugCF that supports a wide variety of recommendation tasks. AugCF is based on Conditional Generative Adversarial Nets that additionally consider the class (like or dislike) as a feature to generate new interaction data, which can be a sufficiently real augmentation to the original dataset. Also, AugCF adopts a novel discriminator loss and Gumbel-Softmax approximation to enable end-to-end training. Finally, extensive experiments are conducted on two large-scale recommendation datasets, and the experimental results show the superiority of our proposed model.
Object Detection from Scratch with Deep Supervision

In this paper, we propose Deeply Supervised Object Detectors (DSOD), an object detection framework that can be trained from scratch. Recent advances in object detection heavily depend on the off-the-shelf models pre-trained on large-scale classification datasets like ImageNet and OpenImage. However, one problem is that adopting pre-trained models from classification to detection task may incur learning bias due to the different objective function and diverse distributions of object categories. Techniques like fine-tuning on detection task could alleviate this issue to some extent but are still not fundamental. Furthermore, transferring these pre-trained models across discrepant domains will be more difficult (e.g., from RGB to depth images). Thus, a better solution to handle these critical problems is to train object detectors from scratch, which motivates our proposed method. Previous efforts on this direction mainly failed by reasons of the limited training data and naive backbone network structures for object detection. In DSOD, we contribute a set of design principles for learning object detectors from scratch. One of the key principles is the deep supervision, enabled by layer-wise dense connections in both backbone networks and prediction layers, plays a critical role in learning good detectors from scratch. After involving several other principles, we build our DSOD based on the single-shot detection framework (SSD). We evaluate our method on PASCAL VOC 2007, 2012 and COCO datasets. DSOD achieves consistently better results than the state-of-the-art methods with much more compact models. Specifically, DSOD outperforms baseline method SSD on all three benchmarks, while requiring only 1/2 parameters. We also observe that DSOD can achieve comparable/slightly better results than Mask RCNN [1] + FPN [2] (under similar input size) with only 1/3 parameters, using no extra data or pre-trained models.
Human Motion Prediction via Learning Local Structure Representations and Temporal Dependencies

Human motion prediction from motion capture data is a classical problem in the computer vision, and conventional methods take the holistic human body as input. These methods ignore the fact that, in various human activities, different body components (limbs and the torso) have distinctive characteristics in terms of the moving pattern. In this paper, we argue local representations on different body components should be learned separately and, based on such idea, propose a network, Skeleton Network (SkelNet), for long-term human motion prediction. Specifically, at each time-step, local structure representations of input (human body) are obtained via SkelNet’s branches of component-specific layers, then the shared layer uses local spatial representations to predict the future human pose. Our SkelNet is the first to use local structure representations for predicting the human motion. Then, for short-term human motion prediction, we propose the second network, named as Skeleton Temporal Network (Skel-TNet). Skel-TNet consists of three components: SkelNet and a Recurrent Neural Network, they have advantages in learning spatial and temporal dependencies for predicting human motion, respectively; a feed-forward network that outputs the final estimation. Our methods achieve promising results on the Human3.6M dataset and the CMU motion capture dataset, and the code is publicly available 1.
The Impact of Mislabeled Changes by SZZ on Just-in-Time Defect Prediction

Just-in-Time (JIT) defect prediction-a technique which aims to predict bugs at change level-has been paid more attention. JIT defect prediction leverages the SZZ approach to identify bug-introducing changes. Recently, researchers found that the performance of SZZ (including its variants) is impacted by a large amount of noise. SZZ may considerably mislabel changes that are used to train a JIT defect prediction model, and thus impact the prediction accuracy. In this paper, we investigate the impact of the mislabeled changes by different SZZ variants on the performance and interpretation of JIT defect prediction models. We analyze four SZZ variants (i.e., B-SZZ, AG-SZZ, MA-SZZ, and RA-SZZ) that are proposed by prior studies. We build the prediction models using the labeled data by these four SZZ variants. Among the four SZZ variants, RA-SZZ is least likely to generate mislabeled changes, and we construct the testing set by using RA-SZZ. All of the four prediction models are then evaluated on the same testing set. We choose the prediction model built on the labeled data by RA-SZZ as the baseline model, and we compare the performance and metric importance of the models trained using the labeled data by the other three SZZ variants with the baseline model. Through a large-scale empirical study on a total of 126,526 changes from ten Apache open source projects, we find that in terms of various performance measures (AUC, F1-score, G-mean and Recall@20%), the mislabeled changes by B-SZZ and MA-SZZ are not likely to cause a considerable performance reduction, while the mislabeled changes by AG-SZZ cause a statistically significant performance reduction with an average difference of 1-5 percent. When considering developers' inspection effort (measured by LOC) in practice, the changes mislabeled B-SZZ and AG-SZZ lead to 9-10 and 1-15 percent more wasted inspection effort, respectively. And the mislabeled changes by B-SZZ lead to significantly more wasted effort. The mislabeled changes by MA-SZZ do not cause considerably more wasted effort. We also find that the top-most important metric for identifying bug-introducing changes (i.e., number of files modified in a change) is robust to the mislabeling noise generated by SZZ. But the second- and third-most important metrics are more likely to be impacted by the mislabeling noise, unless random forest is used as the underlying classifier.
DiffChaser: Detecting Disagreements for Deep Neural Networks

The platform migration and customization have become an indispensable process of deep neural network (DNN) development lifecycle. A high-precision but complex DNN trained in the cloud on massive data and powerful GPUs often goes through an optimization phase (e.g, quantization, compression) before deployment to a target device (e.g, mobile device). A test set that effectively uncovers the disagreements of a DNN and its optimized variant provides certain feedback to debug and further enhance the optimization procedure. However, the minor inconsistency between a DNN and its optimized version is often hard to detect and easily bypasses the original test set. This paper proposes DiffChaser, an automated black-box testing framework to detect untargeted/targeted disagreements between version variants of a DNN. We demonstrate 1) its effectiveness by comparing with the state-of-the-art techniques, and 2) its usefulness in real-world DNN product deployment involved with quantization and optimization.
Deep Residual Squeeze and Excitation Network for Remote Sensing Image Super-Resolution

Recently, deep convolutional neural networks (DCNN) have obtained promising results in single image super-resolution (SISR) of remote sensing images. Due to the high complexity of remote sensing image distribution, most of the existing methods are not good enough for remote sensing image super-resolution. Enhancing the representation ability of the network is one of the critical factors to improve remote sensing image super-resolution performance. To address this problem, we propose a new SISR algorithm called a Deep Residual Squeeze and Excitation Network (DRSEN). Specifically, we propose a residual squeeze and excitation block (RSEB) as a building block in DRSEN. The RSEB fuses the input and its internal features of current block, and models the interdependencies and relationships between channels to enhance the representation power. At the same time, we improve the up-sampling module and the global residual pathway in the network to reduce the parameters of the network. Experiments on two public remote sensing datasets (UC Merced and NWPU-RESISC45) show that our DRSEN achieves better accuracy and visual improvements against most state-of-the-art methods. The DRSEN is beneficial for the progress in the remote sensing images super-resolution field.
Flexible Multi-View Representation Learning for Subspace Clustering

In recent years, numerous multi-view subspace clustering methods have been proposed to exploit the complementary information from multiple views. Most of them perform data reconstruction within each single view, which makes the subspace representation unpromising and thus can not well identify the underlying relationships among data. In this paper, we propose to conduct subspace clustering based on Flexible Multi-view Representation (FMR) learning, which avoids using partial information for data reconstruction. The latent representation is flexibly constructed by enforcing it to be close to different views, which implicitly makes it more comprehensive and well-adapted to subspace clustering. With the introduction of kernel dependence measure, the latent representation can flexibly encode complementary information from different views and explore nonlinear, high-order correlations among these views. We employ the Alternating Direction Minimization (ADM) method to solve our problem. Empirical studies on real-world datasets show that our method achieves superior clustering performance over other state-of-the-art methods.
Image compression-encryption scheme based on fractional order hyper-chaotic systems combined with 2D compressed sensing and DNA encoding

An image compression-encryption scheme is proposed based on fractional order hyper-chaotic systems combined with 2D compressed sensing (CS) and DNA encoding. This scheme adopts the CS technique to cut down the size of the original image proportionally in the encryption process and the construction parameters of the CS measurement matrix are controlled by fractional order hyper-chaotic systems. The chaotic sequences generated by fractional order hyper-chaotic systems are used to dynamically control DNA encoding and DNA operation mode to improve the complexity of encryption process. The fractional order and initial value of the fractional hyper-chaotic system are used as the key of the encryption scheme, which greatly expands the key space of the proposed scheme. The proposed scheme maximizes the utilization of chaotic sequences generated by fractional order hyper-chaotic systems, and the chaotic sequences fit perfectly with the whole process of the scheme. The experimental results and security analysis show the security of the scheme against multiple attacks.
Improving Back-Translation with Uncertainty-based Confidence Estimation

While back-translation is simple and effective in exploiting abundant monolingual corpora to improve low-resource neural machine translation (NMT), the synthetic bilingual corpora generated by NMT models trained on limited authentic bilingual data are inevitably noisy.In this work, we propose to quantify the confidence of NMT model predictions based on model uncertainty.With word-and sentence-level confidence measures based on uncertainty, it is possible for back-translation to better cope with noise in synthetic bilingual corpora.Experiments on Chinese-English and English-German translation tasks show that uncertainty-based confidence estimation significantly improves the performance of backtranslation. 1
Multi-View Convolutional Neural Networks for Mammographic Image Classification

In recent years, deep learning has been widely applied for mammographic image classification. However, most of the existing methods are based on a single mammography view and cannot sufficiently extract discriminative features, thereby resulting in an unsatisfactory classification accuracy. To solve this problem and improve the mammographic image classification performance, we propose a novel multi-view convolutional neural network (CNN) based on multiple mammography views in this paper. Considering that the images acquired from different perspectives contain different and complementary breast mass information, we modify the CNN architecture to exploit the complementary information from the various views of mammography. The new architecture can extract discriminative features from the mediolateral oblique (MLO) and craniocaudal (CC) views of the mammographic images and can effectively incorporate these features for mammographic images. The dilated convolutional layers enable the feature maps extracted from the multiple breast mass views to capture information from a large “field of vision”. Moreover, multi-scale features are obtained by using the convolutional and dilated convolutions. In addition, we incorporate a penalty term into the cross entropy loss function, which enables the model evolution to reduce the misclassification rate by enhancing the contributions of the samples misclassified in the training process. The proposed method was evaluated and compared with several state-of-the-art methods on the open Digital Database for Screening Mammography (DDSM) and Mammographic Image Analysis Society (MIAS) datasets. The experimental results show that the proposed method exhibits a better performance than those of the state-of-the-art methods.
Deep Fuzzy K-Means With Adaptive Loss and Entropy Regularization

Neural network based clustering methods usually have better performance compared to the conventional approaches due to more efficient feature extraction. Most of existing deep clustering techniques either exploit graph information as prior to extract pivotal deep structure from the raw data and simply utilizes stochastic gradient descent (SGD). However, they often suffer from separating the learning steps regarding dimensionality reduction and clustering. To address these issues, a novel deep model named as deep fuzzy k-means (DFKM) with adaptive loss function and entropy regularization is proposed. DFKM performs deep feature extraction and fuzzy clustering simultaneously to generate a more appropriate nonlinear feature map. Additionally, DFKM incorporates FKM so that fuzzy information is utilized to represent a clear structure of deep clusters. To further promote the robustness of the model, a robust loss function is applied to the objective with adaptive weights. Moreover, an entropy regularization is employed for affinity to provide confidence of each assignment and the corresponding membership and centroid matrices are updated by close-form solutions rather than SGD. Extensive experiments show that DFKM has better performance compared to the state-of-the-art fuzzy clustering techniques under three clustering metrics.
<i>Retracted:</i> Multiscale fast correlation filtering tracking algorithm based on a feature fusion model

Summary Retraction: Multiscale fast correlation filtering tracking algorithm based on a feature fusion model Yuantao Chen, Jin Wang, Songjie Liu, Xi Chen, Jie Xiong, Jingbo Xie, Kai Yang, 2021, 33 (15), ( https://doi.org/10.1002/cpe.5533
Interestingness elements for explainable reinforcement learning: Understanding agents' capabilities and limitations

We propose an explainable reinforcement learning (XRL) framework that analyzes an agent's history of interaction with the environment to extract interestingness elements that help explain its behavior. The framework relies on data readily available from standard RL algorithms, augmented with data that can easily be collected by the agent while learning. We describe how to create visual summaries of an agent's behavior in the form of short video-clips highlighting key interaction moments, based on the proposed elements. We also report on a user study where we evaluated the ability of humans to correctly perceive the aptitude of agents with different characteristics, including their capabilities and limitations, given visual summaries automatically generated by our framework. The results show that the diversity of aspects captured by the different interestingness elements is crucial to help humans correctly understand an agent's strengths and limitations in performing a task, and determine when it might need adjustments to improve its performance.