Reviewerly/exHarmony huggingface dataset:
Mining and Restoration Monitoring of Rare Earth Element (REE) Exploitation by New Remote Sensing Indicators in Southern Jiangxi, China

Rare earth elements (REEs) are widely used in various industries. The open-pit mining and chemical extraction of REEs in the weathered crust in southern Jiangxi, China, since the 1970s have provoked severe damages to the environment. After 2010, different restorations have been implemented by various enterprises, which seem to have a spatial variability in both management techniques and efficiency from one mine to another. A number of vegetation indices, e.g., normalized difference vegetation index (NDVI), soil-adjusted vegetation index (SAVI), enhanced vegetation index (EVI) and atmospherically resistant vegetation index (ARVI), can be used for this kind of monitoring and assessment but lack sensitivity to subtle differences. For this reason, the main objective of this study was to explore the possibility to develop new, mining-tailored remote sensing indicators to monitor the impacts of REE mining on the environment and to assess the effectiveness of its related restoration using multitemporal Landsat data from 1988 to 2019. The new indicators, termed mining and restoration assessment indicators (MRAIs), were developed based on the strong contrast of spectral reflectance, albedo, land surface temperature (LST) and tasseled cap brightness (TCB) of REE mines between mining and postmining restoration management. These indicators were tested against vegetation indices such as NDVI, EVI, SAVI and generalized difference vegetation index (GDVI), and found to be more sensitive. Of similar sensitivity to each other, one of the new indicators was employed to conduct the restoration assessment of the mined areas. Six typically managed mines with different restoration degrees and management approaches were selected as hotspots for a comparative analysis to highlight their temporal trajectories using the selected MRAI. The results show that REE mining had experienced a rapid expansion in 1988–2010 with a total mined area of about 66.29 km2 in the observed counties. With implementation of the post-2010 restoration measures, an improvement of varying degrees in vegetation cover in most mines was distinguished and quantified. Hence, this study with the newly developed indicators provides a relevant approach for assessing the sustainable exploitation and management of REE resources in the study area.
Automatic Classification of Semantic Content of Classroom Dialogue

Due to benefits for teaching and learning, an increasing number of studies have focused on classroom dialogue and how to make it productive. Coding, in which the transcribed conversation is allocated to a set of features, is commonly employed to deal with the textual data arising from this dialogue. This is generally done manually and cannot provide timely feedback to the participants. To address this issue, we explored the possibility of automatically classifying the semantic content of classroom dialogue. Seven categories (prior-known knowledge, analysis, coordination, speculation, uptake, agreement and querying) were distinguished automatically using an artificial neural network-based model. The model achieved acceptable performance and was comparable to human coding. Information about quality of dialogue can be identified in a timely manner. With this knowledge, classroom dialogue can be managed more skilfully, and a more productive form of dialogue is likely to be achieved by teachers and students.
A Blockchain-Driven IIoT Traffic Classification Service for Edge Computing

Nowadays, more and more sensors, devices and applications are connected in Industrial Internet of Things (IIoT), producing massive real-time flows which need to be scheduled for Quality-of-Service provision. To realize application-aware and adaptive flow scheduling, the problem of traffic classification must be addressed at first. When edge computing paradigm is introduced into IIoT, the traffic classification service can be deployed on edge node in the near-end. Recently, deep-learning-based IIoT traffic classification methods show better performance, but the computational cost of deep learning model is too high to be deployed on edge node. Moreover, increasingly unknown flows generated by new devices and emerging industrial APPs lead to frequent training of traffic classifiers. It is difficult to migrate the complex process of classifier training from cloud server to edge nodes with limited resources. To address these issues, we take the benefits of hash mechanism and consensus mechanism in blockchain to design a lightweight IIoT traffic classification service, which is more applicable for edge computing paradigm. First, inspired by the hash mechanism in blockchain and the learning to hash for big data, we propose a new learning-to-hash method named extension hashing. By this method, we can build the set of binary coding tress (BCT set), then generating hash table for more efficient k-nearest neighbor-based classification without complex classifier training. Then, we design a new voting-based consensus algorithm to synchronize the BCT sets and the hash tables across edge nodes, thereby providing the traffic classification service. Finally, we conduct data-driven simulations to evaluate the proposed service. By comparing traffic classification results on public data set, we can see that the proposed service achieves the highest classification accuracy with the minimal time cost and memory usage.
Hand Gesture Recognition for Smart Devices by Classifying Deterministic Doppler Signals

Personal devices such as smartphones and tablets are rapidly becoming personal communication, information, and control centers. Apart from multitouch screens, human gestures are considered as a new interactive human-smart device interface. In this work, we propose a noncontact solution to implement hand gesture recognitions for smart devices. It is based on a continuous wave, time-division-multiplexing (TDM), single-input multiple-output (SIMO) Doppler radar sensor that can be realized by slightly modifying existing RF front ends of smart devices, and a machine-learning algorithm to recognize predefined gestures by classifying deterministic Doppler signals. An experimental setup emulating a smartphone-based radar sensor was implemented, and the experimental results verified the robustness and the accuracy of the proposed approach.
An Agglomerative Greedy Brain Storm Optimization Algorithm for Solving the TSP

The brain storm optimization algorithm(BSO) is a population based metaheuristic algorithm inspried by the human conferring process that was proposed in 2010. Since its first implementation, BSO has been widely used in various fields. In this article, we propose an agglomerative greedy brain storm optimization algorithm (AG-BSO) to solve classical traveling salesman problem(TSP). Due to the low accuracy and slow convergence speed of current heuristic algorithms when solving TSP, this article consider four improvement strategies for basic BSO. First, a greedy algorithm is introduced to ensure the diversity of the population. Second, hierarchical clustering is used in place of the k-means clustering algorithm in standard BSO to eliminate the noise sensitivity of the original BSO algorithm when solving TSP. Exchange rules for the individuals in the population individuals were introduced to improve the efficiency of the algorithm. Finally, a heuristic crossover operator is used to update the individuals. In addition, the AG-BSO algorithm is compared with the genetic algorithm (GA), particle swarm optimization (PSO), the simulated annealing(SA) and the ant colony optimization (ACO) on standard TSP data sets for performance testing. We also compare it with a recently improved version of the BSO algorithm. The simulations show the encouraging results that AG-BSO greatly improved the solution accuracy, optimization speed and robustness.
Exploring Collection of Sign Language Datasets: Privacy, Participation, and Model Performance

As machine learning algorithms continue to improve, collecting training data becomes increasingly valuable. At the same time, increased focus on data collection may introduce compounding privacy concerns. Accessibility projects in particular may put vulnerable populations at risk, as disability status is sensitive, and collecting data from small populations limits anonymity. To help address privacy concerns while maintaining algorithmic performance on machine learning tasks, we propose privacy-enhancing distortions of training datasets. We explore this idea through the lens of sign language video collection, which is crucial for advancing sign language recognition and translation. We present a web study exploring signers’ concerns in contributing to video corpora and their attitudes about using filters, and a computer vision experiment exploring sign language recognition performance with filtered data. Our results suggest that privacy concerns may exist in contributing to sign language corpora, that filters (especially expressive avatars and blurred faces) may impact willingness to participate, and that training on more filtered data may boost recognition accuracy in some cases.
Boost image captioning with knowledge reasoning

Automatically generating a human-like description for a given image is a potential research in artificial intelligence, which has attracted a great of attention recently. Most of the existing attention methods explore the mapping relationships between words in sentence and regions in image, such unpredictable matching manner sometimes causes inharmonious alignments that may reduce the quality of generated captions. In this paper, we make our efforts to reason about more accurate and meaningful captions. We first propose word attention to improve the correctness of visual attention when generating sequential descriptions word-by-word. The special word attention emphasizes on word importance when focusing on different regions of the input image, and makes full use of the internal annotation knowledge to assist the calculation of visual attention. Then, in order to reveal those incomprehensible intentions that cannot be expressed straightforwardly by machines, we introduce a new strategy to inject external knowledge extracted from knowledge graph into the encoder-decoder framework to facilitate meaningful captioning. Finally, we validate our model on two freely available captioning benchmarks: Microsoft COCO dataset and Flickr30k dataset. The results demonstrate that our approach achieves state-of-the-art performance and outperforms many of the existing approaches.
Graph-based Multi-view Binary Learning for image clustering

Hashing techniques, also known as binary code learning, have recently attracted increasing attention in large-scale data analysis and storage. Generally, most existing hash clustering methods are single-view ones, which lack complete structure or complementary information from multiple views. For clustering tasks, most hashing research mainly mapped the original data into the Hamming space while heavily ignoring the original feature space structure. To solve these problems, we propose a novel binary code algorithm for clustering, which adopts graph embedding to preserve the original data structure, called Graph-based Multi-view Binary Learning (GMBL). The binary code learning combines the graph structure of original data and the complementary information of different views for clustering. Specifically, GMBL preserves the local linear relationship utilizing the Laplacian matrix aim to maintain the graph-based structure of the original data in Hamming space. Moreover, by automatically assigning weights for each view to improve clustering performance, which takes distinctive contributions of multiple views into considerations. Besides, an alternating iterative optimization method is designed to solve the resulting optimization problems. Extensive experimental results on five public datasets are provided to reveal the effectiveness of the algorithm and its superior performance over other state-of-the-art alternatives.
Image superpixel segmentation based on hierarchical multi-level LI-SLIC

In computer vision, superpixel segmentation has been widely used as a very important preprocessing to reduce the number of image primitives for subsequent image processing tasks. To improve the segmentation accuracy and the robustness to noise, a hierarchical multi-level segmentation framework is developed in this paper. First, original image is initially segmented by a local information based simple liner iterative clustering (LI-SLIC) method. Then, the initial generated superpixels are further segmented hierarchically by LI-SLIC to ensure that all pixels contained within each superpixel belong to a same object. Finally, to eliminate over-segmentation and reduce the number of superpixels, adjacent superpixels belonging to a same object are merged based on the probability distribution similarity. The proposed method does not require setting the seeds or number of the superpixels to be generated in advance, which can segment image into an appropriate number of superpixels without under- or over- segmentation automatically according to its content. Experiments are conducted on two public datasets Berkeley and 3Dircadb, and the results demonstrate that our method is more effective and accurate than many existing superpixel methods and shows a great advantage in dealing with images corrupted by various noises.
Deep Learning-Based Chroma Prediction for Intra Versatile Video Coding

Color images always exhibit a high correlation between luma and chroma components. Cross component linear model (CCLM) has been introduced to exploit such correlation for removing redundancy in the on-going video coding standard, i.e., versatile video coding (VVC). To further improve the coding performance, this paper presents a deep learning based intra chroma prediction method, termed as convolutional neural network based chroma prediction (CNNCP). More specifically, the process of chroma prediction is formulated to produce the colorful version from available information input. CNNCP includes two sub-networks for luma down-sampling and chroma prediction, which are jointly optimized to fully exploit spatial and cross component information. In addition, the outputs of CCLM are adopted as chroma initialization for performance enhancement, and the coding distortion level characterized by quantization parameter is fed into the network to release the negative affect from compression artifacts. To further improve the coding performance, the competition is performed between the conventional chroma prediction and CNNCP in terms of rate-distortion cost with a binary flag signalled. The learned CNNCP is incorporated into both video encoder and decoder. Extensive experimental results demonstrate that the proposed scheme can achieve 4.283%, 3.343%, and 4.634% bit rate savings for luma and two chroma components, compared with the VVC test model version 4.0 (VTM 4.0).
Learning Disentangled Feature Representation for Hybrid-Distorted Image Restoration

Hybrid-distorted image restoration (HD-IR) is dedicated to restore real distorted image that is degraded by multiple distortions. Existing HD-IR approaches usually ignore the inherent interference among hybrid distortions which compromises the restoration performance. To decompose such interference, we introduce the concept of Disentangled Feature Learning to achieve the feature-level divide-and-conquer of hybrid distortions. Specifically, we propose the feature disentanglement module (FDM) to distribute feature representations of different distortions into different channels by revising gain-control-based normalization. We also propose a feature aggregation module (FAM) with channel-wise attention to adaptively filter out the distortion representations and aggregate useful content information from different channels for the construction of raw image. The effectiveness of the proposed scheme is verified by visualizing the correlation matrix of features and channel responses of different distortions. Extensive experimental results also prove superior performance of our approach compared with the latest HD-IR schemes.
2D to 3D Evolutionary Deep Convolutional Neural Networks for Medical Image Segmentation

Developing a Deep Convolutional Neural Network (DCNN) is a challenging task that involves deep learning with significant effort required to configure the network topology. The design of a 3D DCNN not only requires a good complicated structure but also a considerable number of appropriate parameters to run effectively. Evolutionary computation is an effective approach that can find an optimum network structure and/or its parameters automatically. Note that the Neuroevolution approach is computationally costly, even for developing 2D networks. As it is expected that it will require even more massive computation to develop 3D Neuroevolutionary networks, this research topic has not been investigated until now. In this article, in addition to developing 3D networks, we investigate the possibility of using 2D images and 2D Neuroevolutionary networks to develop 3D networks for 3D volume segmentation. In doing so, we propose to first establish new evolutionary 2D deep networks for medical image segmentation and then convert the 2D networks to 3D networks in order to obtain optimal evolutionary 3D deep convolutional neural networks. The proposed approach results in a massive saving in computational and processing time to develop 3D networks, while achieved high accuracy for 3D medical image segmentation of nine various datasets.
DeepWiPHY: Deep Learning-Based Receiver Design and Dataset for IEEE 802.11ax Systems

In this work, we develop DeepWiPHY, a deep learning-based architecture to replace the channel estimation, common phase error (CPE) correction, sampling rate offset (SRO) correction, and equalization modules of IEEE 802.11ax based orthogonal frequency division multiplexing (OFDM) receivers. We first train DeepWiPHY with a synthetic dataset, which is generated using representative indoor channel models and includes typical radio frequency (RF) impairments that are the source of nonlinearity in wireless systems. To further train and evaluate DeepWiPHY with real-world data, we develop a passive sniffing-based data collection testbed composed of Universal Software Radio Peripherals (USRPs) and commercially available IEEE 802.11ax products. The comprehensive evaluation of DeepWiPHY with synthetic and real-world datasets (110 million synthetic OFDM symbols and 14 million real-world OFDM symbols) confirms that, even without fine-tuning the neural network's architecture parameters, DeepWiPHY achieves comparable performance to or outperforms the conventional WLAN receivers, in terms of both bit error rate (BER) and packet error rate (PER), under a wide range of channel models, signal-to-noise (SNR) levels, and modulation schemes.
Diagnosing acute promyelocytic leukemia by using convolutional neural network

To evaluate the efficacy of diagnosis systems based upon instance segmentation with convolutional neural networks (CNNs) for diagnosing acute promyelocytic leukemia (APL) in bone marrow smear images. A self-established dataset was used in this study that was exempted from review by the institution review board, which consisted of 13,504 bone marrow smear images. One subset of the dataset with 12,215 labeled images was split into training (80%) and validation (20%), another with 1289 labeled images was used to test, in which each test entry consists of about 130 images. An instance segmentation method named Mask R-CNN was used to detect and classify the nucleated cells. Here, we train a trained neural network from scratch; for comparison, we also use a network pre-trained on MS COCO (common objects in context, a data set provided by Microsoft which can be used for image recognition, the images in MS coco dataset are divided into training, validation and test sets) and fine-tuned with our dataset and both were trained with same data augmentation scheme. Diagnosis systems based on trained models and "FAB Classification" (French–American–British classification systems, a series of diagnostic criteria for acute leukemia, which was first proposed in 1976) were developed for diagnosing the test entry as APL or as not. Average precision (AP) and average recall (AR) were used to evaluate model performance. The best-performing model had an average precision of 62.5%, which was the augmented pre-trained Mask R-CNN with average recall 84.1%. The average precision of the pre-trained model was greater than that of the model trained from scratch (P < 0.05). Augmenting the dataset further increased accuracy (P < 0 0.03). Deep learning technology such as instance segmentation with Mask R-CNN may accurately diagnose APL in bone marrow smear images with an average precision of 62.5% when 0.5 as IoU thresholds. A data augmentation and pre-trained approach further improved accuracy.
SEN: A Novel Feature Normalization Dissimilarity Measure for Prototypical Few-Shot Learning Networks

In this paper, we equip Prototypical Networks (PNs) with a novel dissimilarity measure to enable discriminative feature normalization for few-shot learning. The embedding onto the hypersphere requires no direct normalization and is easy to optimize. Our theoretical analysis shows that the proposed dissimilarity measure, denoted the Squared root of the Euclidean distance and the Norm distance (SEN), forces embedding points to be attracted to its correct prototype, while being repelled from all other prototypes, keeping the norm of all points the same. The resulting SEN PN outperforms the regular PN with a considerable margin, with no additional parameters as well as with negligible computational overhead.
EasiEdge: A Novel Global Deep Neural Networks Pruning Method for Efficient Edge Computing

Deep neural networks (DNNs) have shown tremendous success in many areas, such as signal processing, computer vision, and artificial intelligence. However, the DNNs require intensive computation resources, hindering their practical applications on the edge devices with limited storage and computation resources. Filter pruning has been recognized as a useful technique to compress and accelerate the DNNs, but most existing works tend to prune filters in a layerwise manner, facing some significant drawbacks. First, the layerwise pruning methods require prohibitive computation for per-layer sensitivity analysis. Second, layerwise pruning suffers from the accumulation of pruning errors, leading to performance degradation of pruned networks. To address these challenges, we propose a novel global pruning method, namely, EasiEdge, to compress and accelerate the DNNs for efficient edge computing. More specifically, we introduce an alternating direction method of multipliers (ADMMs) to formulate the pruning problem as a performance improving subproblem and a global pruning subproblem. In the global pruning subproblem, we propose to use information gain (IG) to quantify the impact of filters removal on the class probability distributions of network output. Besides, we propose a Taylor-based approximate algorithm (TBAA) to efficiently calculate the IG of filters. Extensive experiments on three data sets and two edge computing platforms verify that our proposed EasiEdge can efficiently accelerate DNNs on edge computing platforms with nearly negligible accuracy loss. For example, when EasiEdge prunes 80% filters in VGG-16, the accuracy drops by 0.22%, but inference latency on CPU of Jetson TX2 decreases from 76.85 to 8.01 ms.