Reviewerly/exHarmony huggingface dataset:
On Learning Meaningful Code Changes Via Neural Machine Translation

Recent years have seen the rise of Deep Learning (DL) techniques applied to source code. Researchers have exploited DL to automate several development and maintenance tasks, such as writing commit messages, generating comments and detecting vulnerabilities among others. One of the long lasting dreams of applying DL to source code is the possibility to automate non-trivial coding activities. While some steps in this direction have been taken (e.g., learning how to fix bugs), there is still a glaring lack of empirical evidence on the types of code changes that can be learned and automatically applied by DL. Our goal is to make this first important step by quantitatively and qualitatively investigating the ability of a Neural Machine Translation (NMT) model to learn how to automatically apply code changes implemented by developers during pull requests. We train and experiment with the NMT model on a set of 236k pairs of code components before and after the implementation of the changes provided in the pull requests. We show that, when applied in a narrow enough context (i.e., small/medium-sized pairs of methods before/after the pull request changes), NMT can automatically replicate the changes implemented by developers during pull requests in up to 36% of the cases. Moreover, our qualitative analysis shows that the model is capable of learning and replicating a wide variety of meaningful code changes, especially refactorings and bug-fixing activities. Our results pave the way for novel research in the area of DL on code, such as the automatic learning and applications of refactoring.
A Novel Hybrid Harris Hawks Optimization for Color Image Multilevel Thresholding Segmentation

Multilevel thresholding has got more attention in recent years with various successful applications. However, the implementation becomes more and more complex and time-consuming when the number of thresholds is high, and color images which contain more information are even worse. Therefore, this paper proposes an alternative hybrid algorithm for color image segmentation, the advantages of which lie in extracting the best features from the high performance of two algorithms and overcoming the limitations of each algorithm to some extent. Two techniques, Otsu's method, and Kapur's entropy, are used as fitness function to determine the segmentation threshold values. Harris hawks optimization (HHO) is a novel and general-purpose algorithm, and the hybridization of HHO is fulfilled by adding another powerful algorithm-differential evolution (DE), which is known as HHO-DE. More specifically, the whole population is divided into two equal subpopulations which will be assigned to HHO and DE algorithms, respectively. Then both algorithms operate in parallel to update the positions of each subpopulation during the iterative process. In order to fully demonstrate the superior performance of HHO-DE, the proposed method is compared with the seven state-of-the-art algorithms by an array of experiments on ten benchmark images. Meanwhile, five measures, including the average fitness values, standard deviation (STD), peak signal to noise ratio (PSNR), structure similarity index (SSIM), and feature similarity index (FSIM), are used to evaluate the performance of each algorithm. In addition, Wilcoxon's rank sum test for statistical analysis and the comparison with the super-pixel method are also conducted to verify the superiority of HHO-DE. The experimental results reveal that the proposed method significantly outperforms other algorithms. Hence, the HHO-DE algorithm is a remarkable and promising tool for multilevel thresholding color image segmentation.
FPGA/DNN Co-Design

While embedded FPGAs are attractive platforms for DNN acceleration on edge-devices due to their low latency and high energy efficiency, the scarcity of resources of edge-scale FPGA devices also makes it challenging for DNN deployment. In this paper, we propose a simultaneous FPGA/DNN co-design methodology with both bottom-up and top-down approaches: a bottom-up hardware-oriented DNN model search for high accuracy, and a top-down FPGA accelerator design considering DNN-specific characteristics. We also build an automatic co-design flow, including an Auto-DNN engine to perform hardware-oriented DNN model search, as well as an Auto-HLS engine to generate synthesizable C code of the FPGA accelerator for explored DNNs. We demonstrate our co-design approach on an object detection task using PYNQ-Z1 FPGA. Results show that our proposed DNN model and accelerator outperform the state-of-the-art FPGA designs in all aspects including Intersection-over-Union (IoU) (6.2% higher), frames per second (FPS) (2.48× higher), power consumption (40% lower), and energy efficiency (2.5× higher). Compared to GPU-based solutions, our designs deliver similar accuracy but consume far less energy.
Mask-Guided Attention Network for Occluded Pedestrian Detection

Pedestrian detection relying on deep convolution neural networks has made significant progress. Though promising results have been achieved on standard pedestrians, the performance on heavily occluded pedestrians remains far from satisfactory. The main culprits are intra-class occlusions involving other pedestrians and inter-class occlusions caused by other objects, such as cars and bicycles. These results in a multitude of occlusion patterns. We propose an approach for occluded pedestrian detection with the following contributions. First, we introduce a novel mask-guided attention network that fits naturally into popular pedestrian detection pipelines. Our attention network emphasizes on visible pedestrian regions while suppressing the occluded ones by modulating full body features. Second, we empirically demonstrate that coarse-level segmentation annotations provide reasonable approximation to their dense pixel-wise counterparts. Experiments are performed on CityPersons and Caltech datasets. Our approach sets a new state-of-the-art on both datasets. Our approach obtains an absolute gain of 9.5% in log-average miss rate, compared to the best reported results [32] on the heavily occluded HO pedestrian set of CityPersons test set. Further, on the HO pedestrian set of Caltech dataset, our method achieves an absolute gain of 5.0% in log-average miss rate, compared to the best reported results [13]. Code and models are available at: https://github.com/Leotju/MGAN.
The Jester Dataset: A Large-Scale Video Dataset of Human Gestures

Gesture recognition and its application in human-computer interfaces have been growing increasingly popular in recent years. Although many gestures can be recognized from a single image frame, to build a responsive, accurate system, that can recognize complex gestures with subtle differences between them we need large-scale real-world video datasets. In this work, we introduce the largest collection of short clips of videos of humans performing gestures in front of the camera. The dataset has been collected with the help of over 1300 different actors in their unconstrained environments. Additionally, we present an on-going gesture recognition challenge based on our dataset and the current results. We also describe how a baseline achieving over 93% recognition accuracy can be obtained with a simple 3D convolutional neural network.
Color image compression and encryption scheme based on compressive sensing and double random encryption strategy

Based on compressive sensing and double random encryption strategy, a novel color image compression and encryption scheme is proposed in this paper. The architecture of compression, confusion and diffusion is adopted. Firstly, the red, green and blue components of color plain image are converted to three sparse coefficient matrices by discrete wavelet transform (DWT), and then a double random position permutation (DRPP) is introduced to confuse the coefficient matrices. Subsequently, Logistic-Tent system is utilized to generate the asymptotic deterministic random measurement matrix based on chaotic system and plain image (ADMMCP), which is used to measure the coefficient matrices to obtain measurement value matrices. Moreover, simultaneous double random pixel diffusion between inter-intra components (SDRDIC) is presented to modify the elements of measurement value matrices to obtain the final cipher image. A 4-D hyperchaotic system is applied to produce chaotic sequences for confusion and diffusion, the initial conditions of the used chaotic systems are controlled by the SHA 512 hash value of plain image and external keys, such that the proposed image cryptosystem may withstand known-plaintext and chosen-plaintext attacks. Experimental results and security analyses verify the effectiveness of the proposed cipher.
Ship Detection in Large-Scale SAR Images Via Spatial Shuffle-Group Enhance Attention

Ship target detection using large-scale synthetic aperture radar (SAR) images has important application in military and civilian fields. However, ship targets are difficult to distinguish from the surrounding background and many false alarms can occur due to the influence of land area. False alarms always occur with ship target detection because most of the area in large-scale SAR images are treated as background and clutter, and the ship targets are considered unevenly distributing small targets. To address these issues, a ship detection method in large-scale SAR images via CenterNet is proposed in this article. As an anchor-free method, CenterNet defines the target as a point, and the center point of the target is located through key point estimation, which can effectively avoid the missing detection of small targets. At the same time, the spatial shuffle-group enhance (SSE) attention module is introduced into CenterNet. Through SSE, the stronger semantic features are extracted while suppressing some noise to reduce false positives caused by inshore and inland interferences. The experiments on the public SAR-ship-data set show that the proposed method can detect all targets without missed detection with dense-docking targets. For the ship targets in large-scale SAR images from Sentinel 1, the proposed method can also detect targets near the shore and in the sea with high accuracy, which outperforms the methods like faster R-convolutional neural network (CNN), single-shot multibox detector (SSD), you only look once (YOLO), feature pyramid network (FPN), and their variations.
On relationships between the Pearson and the distance correlation coefficients

In this paper we show that for any fixed Pearson correlation coefficient strictly between −1 and 1, the distance correlation coefficient can take any value in the open unit interval (0,1).
Combating Copycats in the Supply Chain with Permissioned Blockchain Technology

The phenomenon of copycats is common in a wide range of industries. Recently, to indicate product authenticity and combat copycats, many brand name companies (BNCs) have started selling products through retailers. These BNCs deploy a scalable protocol that is integrated into a permissioned blockchain technology (PBT) platform. We examine how PBT combats copycats in the supply chain and how it benefits BNCs. Although PBT implementation helps novice customers identify product authenticity and the real quality of products, that is, to take advantage of a quality disclosure effect, we show that, if and only if the number of novice customers is large enough, then selling through a PBT retailer can effectively combat copycats. Thus, PBT increases the profit of the BNC, consumer surplus, social welfare, and reduces the profit of a copycat. Moreover, conventional wisdom tells us that PBT ensures supply chain transparency and motivates a firm to improve its product quality. However, the BNC reduces the quality of its products when using PBT, because an improvement in product quality is not profitable if consumers can distinguish between genuine and imitation products. Furthermore, we extend the model by considering the case where the BNC itself implements PBT. Without the double marginalization effect, even if the number of novice customers is small, blockchain technology may exist in the market (the BNC self‐implements). In addition, if the unit production cost of a genuine product is large enough, social welfare increases when production cost increases.
Semantic Communication Systems for Speech Transmission

Semantic communications could improve the transmission efficiency significantly by exploring the semantic information. In this paper, we make an effort to recover the transmitted speech signals in the semantic communication systems, which minimizes the error at the semantic level rather than the bit or symbol level. Particularly, we design a deep learning (DL)-enabled semantic communication system for speech signals, named DeepSC-S. In order to improve the recovery accuracy of speech signals, especially for the essential information, DeepSC-S is developed based on an attention mechanism by utilizing a squeeze-and-excitation (SE) network. The motivation behind the attention mechanism is to identify the essential speech information by providing higher weights to them when training the neural network. Moreover, in order to facilitate the proposed DeepSC-S for dynamic channel environments, we find a general model to cope with various channel conditions without retraining. Furthermore, we investigate DeepSC-S in telephone systems as well as multimedia transmission systems to verify the model adaptation in practice. The simulation results demonstrate that our proposed DeepSC-S outperforms the traditional communications in both cases in terms of the speech signals metrics, such as signal-to-distortion ration and perceptual evaluation of speech distortion. Besides, DeepSC-S is more robust to channel variations, especially in the low signal-to-noise (SNR) regime.
Dense Label Encoding for Boundary Discontinuity Free Rotation Detection

Rotation detection serves as a fundamental building block in many visual applications involving aerial image, scene text, and face etc. Differing from the dominant regression-based approaches for orientation estimation, this paper explores a relatively less-studied methodology based on classification. The hope is to inherently dismiss the boundary discontinuity issue as encountered by the regression-based detectors. We propose new techniques to push its frontier in two aspects: i) new encoding mechanism: the design of two Densely Coded Labels (DCL) for angle classification, to replace the Sparsely Coded Label (SCL) in existing classification-based detectors, leading to three times training speed increase as empirically observed across benchmarks, further with notable improvement in detection accuracy; ii) loss re-weighting: we propose Angle Distance and Aspect Ratio Sensitive Weighting (ADARSW), which improves the detection accuracy especially for square-like objects, by making DCL-based detectors sensitive to angular distance and object’s aspect ratio. Extensive experiments and visual analysis on large-scale public datasets for aerial images i.e. DOTA, UCAS-AOD, HRSC2016, as well as scene text dataset ICDAR2015 and MLT, show the effectiveness of our approach. The source code is available at $\color{Red}{\text{DCL}}$ and is also integrated in our open source rotation detection benchmark:$\color{Red}{\text{RotationDetection}}$.
A global inventory of photovoltaic solar energy generating units

Photovoltaic (PV) solar energy generating capacity has grown by 41 per cent per year since 20091. Energy system projections that mitigate climate change and aid universal energy access show a nearly ten-fold increase in PV solar energy generating capacity by 20402,3. Geospatial data describing the energy system are required to manage generation intermittency, mitigate climate change risks, and identify trade-offs with biodiversity, conservation and land protection priorities caused by the land-use and land-cover change necessary for PV deployment. Currently available inventories of solar generating capacity cannot fully address these needs1-9. Here we provide a global inventory of commercial-, industrial- and utility-scale PV installations (that is, PV generating stations in excess of 10 kilowatts nameplate capacity) by using a longitudinal corpus of remote sensing imagery, machine learning and a large cloud computation infrastructure. We locate and verify 68,661 facilities, an increase of 432 per cent (in number of facilities) on previously available asset-level data. With the help of a hand-labelled test set, we estimate global installed generating capacity to be 423 gigawatts (-75/+77 gigawatts) at the end of 2018. Enrichment of our dataset with estimates of facility installation date, historic land-cover classification and proximity to vulnerable areas allows us to show that most of the PV solar energy facilities are sited on cropland, followed by aridlands and grassland. Our inventory could aid PV delivery aligned with the Sustainable Development Goals.
Learning Multi-Task Correlation Particle Filters for Visual Tracking

In this paper, we propose a multi-task correlation particle filter (MCPF) for robust visual tracking. We first present the multi-task correlation filter (MCF) that takes the interdependencies among different object parts and features into account to learn the correlation filters jointly. Next, the proposed MCPF is introduced to exploit and complement the strength of a MCF and a particle filter. Compared with existing tracking methods based on correlation filters and particle filters, the proposed MCPF enjoys several merits. First, it exploits the interdependencies among different features to derive the correlation filters jointly, and makes the learned filters complement and enhance each other to obtain consistent responses. Second, it handles partial occlusion via a part-based representation, and exploits the intrinsic relationship among local parts via spatial constraints to preserve object structure and learn the correlation filters jointly. Third, it effectively handles large scale variation via a sampling scheme by drawing particles at different scales for target object state estimation. Fourth, it shepherds the sampled particles toward the modes of the target state distribution via the MCF, and effectively covers object states well using fewer particles than conventional particle filters, thereby resulting in robust tracking performance and low computational cost. Extensive experimental results on four challenging benchmark datasets demonstrate that the proposed MCPF tracking algorithm performs favorably against the state-of-the-art methods.
Separable and Reversible Data Hiding in Encrypted Images Using Parametric Binary Tree Labeling

This paper first introduces a parametric binary tree labeling scheme (PBTL) to label image pixels in two different categories. Using PBTL, a data embedding method (PBTL-DE) is proposed to embed secret data to an image by exploiting spatial redundancy within small image blocks. We then apply PBTL-DE into the encrypted domain and propose a PBTL-based reversible data hiding method in encrypted images (PBTL-RDHEI). PBTL- RDHEI is a separable and reversible method that both the original image and secret data can be recovered and extracted losslessly and independently. Experiment results and analysis show that PBTL- RDHEI is able to achieve an average embedding rate as large as 1.752 bpp and 2.003 bpp when block size is set to 2 × 2 and 3 × 3, respectively.
Survey of State-of-the-Art Mixed Data Clustering Algorithms

Mixed data comprises both numeric and categorical features, and mixed datasets occur frequently in many domains, such as health, finance, and marketing. Clustering is often applied to mixed datasets to find structures and to group similar objects for further analysis. However, clustering mixed data is challenging because it is difficult to directly apply mathematical operations, such as summation or averaging, to the feature values of these datasets. In this paper, we present a taxonomy for the study of mixed data clustering algorithms by identifying five major research themes. We then present a state-of-the-art review of the research works within each research theme. We analyze the strengths and weaknesses of these methods with pointers for future research directions. Lastly, we present an in-depth analysis of the overall challenges in this field, highlight open research questions and discuss guidelines to make progress in the field.
Using neuroscience to develop artificial intelligence

Combining deep learning with brain-like innate structures may guide network models toward human-like learning
Cross-Modality Bridging and Knowledge Transferring for Image Understanding

The understanding of web images has been a hot research topic in both artificial intelligence and multimedia content analysis domains. The web images are composed of various complex foregrounds and backgrounds, which makes the design of an accurate and robust learning algorithm a challenging task. To solve the above significant problem, first, we learn a cross-modality bridging dictionary for the deep and complete understanding of a vast quantity of web images. The proposed algorithm leverages the visual features into the semantic concept probability distribution, which can construct a global semantic description for images while preserving the local geometric structure. To discover and model the occurrence patterns between intra- and inter-categories, multi-task learning is introduced for formulating the objective formulation with Capped-ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> penalty, which can obtain the optimal solution with a higher probability and outperform the traditional convex function-based methods. Second, we propose a knowledge-based concept transferring algorithm to discover the underlying relations of different categories. This distribution probability transferring among categories can bring the more robust global feature representation, and enable the image semantic representation to generalize better as the scenario becomes larger. Experimental comparisons and performance discussion with classical methods on the ImageNet, Caltech-256, SUN397, and Scene15 datasets show the effectiveness of our proposed method at three traditional image understanding tasks.