Reviewerly/exHarmony huggingface dataset:
An SSA-LC-DAE Method for Extracting Network Security Elements

In the current network environment, the original network data present the characteristics of multiple features and high dimensions. Furthermore, in the existing element extraction methods based on deep neural networks, the loss of feature information layer by layer increases as the data dimensions decrease, which greatly affects the retention of network data information elements and brings a huge challenge to effective network security protection. This paper refers to a residual neural network to improve the deep autoencoder (DAE) and then utilizes them to propose a novel element extraction method named the layer-by-layer loss compensation deep autoencoder (LC-DAE) based on the sparrow search algorithm (SSA). In the proposed method, a loss compensation module is added to each encoding layer of the DAE. Specifically, this module first restores the data by using the decoding layer corresponding to the encoding layer. Then, the loss value of the calculated characteristic information is compensated using the output of the corresponding encoding layer. Subsequently, the SSA is used to optimize the LC-DAE in the training process. Finally, the experimental results show that compared with the existing methods, this method retains more sufficient element information, and significantly improves the classification performance of data using neural networks.
A Novel Adaptive Kalman Filter Based on Credibility Measure

It is quite often that the theoretic model used in the Kalman filtering may not be sufficiently accurate for practical applications, due to the fact that the covariances of noises are not exactly known. Our previous work reveals that in such scenario the filter calculated mean square errors (FMSE) and the true mean square errors (TMSE) become inconsistent, while FMSE and TMSE are consistent in the Kalman filter with accurate models. This can lead to low credibility of state estimation regardless of using Kalman filters or adaptive Kalman filters. Obviously, it is important to study the inconsistency issue since it is vital to understand the quantitative influence induced by the inaccurate models. Aiming at this, the concept of credibility is adopted to discuss the inconsistency problem in this paper. In order to formulate the degree of the credibility, a trust factor is constructed based on the FMSE and the TMSE. However, the trust factor can not be directly computed since the TMSE cannot be found for practical applications. Based on the definition of trust factor, the estimation of the trust factor is successfully modifled to online estimation of the TMSE. More importantly, a necessary and sufficient condition is found, which turns out to be the basis for better design of Kalman filters with high performance. Accordingly, beyond trust factor estimation with Sage-Husa technique (TFE-SHT), three novel trust factor estimation methods, which are directly numerical solving method (TFE-DNS), the particle swarm optimization method (PSO) and expectation max-imization-particle swarm optimization method (EM-PSO) are proposed. The analysis and simulation results both show that the proposed TFE-DNS is better than the TFE-SHT for the case of single unknown noise covariance. Meanwhile, the proposed EM-PSO performs completely better than the EM and PSO on the estimation of the credibility degree and state when both noise covariances should be estimated online.
SUNet++: A Deep Network with Channel Attention for Small-Scale Object Segmentation on 3D Medical Images

As a deep learning network with an encoder-decoder architecture, UNet and its series of improved versions have been widely used in medical image segmentation with great applications. However, when used to segment targets in 3D medical images such as magnetic resonance imaging (MRI), computed tomography (CT), these models do not model the relevance of images in vertical space, resulting in poor accurate analysis of consecutive slices of the same patient. On the other hand, the large amount of detail lost during the encoding process makes these models incapable of segmenting small-scale tumor targets. Aiming at the scene of small-scale target segmentation in 3D medical images, a fully new neural network model SUNet++ is proposed on the basis of UNet and UNet++. SUNet++ improves the existing models mainly in three aspects: 1) the modeling strategy of slice superposition is used to thoroughly excavate the three dimensional information of the data; 2) by adding an attention mechanism during the decoding process, small scale targets in the picture are retained and amplified; 3) in the up-sampling process, the transposed convolution operation is used to further enhance the effect of the model. In order to verify the effect of the model, we collected and produced a dataset of hyperintensity MRI liver-stage images containing over 400 cases of liver nodules. Experimental results on both public and proprietary datasets demonstrate the superiority of SUNet++ in small-scale target segmentation of three-dimensional medical images.
MFF-Net: Towards Efficient Monocular Depth Completion With Multi-Modal Feature Fusion

Remarkable progress has been achieved by current depth completion approaches, which produce dense depth maps from sparse depth maps and corresponding color images. However, the performances of these approaches are limited due to the insufficient feature extractions and fusions. In this work, we propose an efficient multi-modal feature fusion based depth completion framework (MFF-Net), which can efficiently extract and fuse features with different modals in both encoding and decoding processes, thus more depth details with better performance can be obtained. In specific, the encoding process contains three branches where different modals of features from both color and sparse depth input can be extracted, and a multi-feature channel shuffle is utilized to enhance these features thus features with better representation abilities can be obtained. Meanwhile, the decoding process contains two branches to sufficiently fuse the extracted multi-modal features, and a multi-level weighted combination is employed to further enhance and fuse features with different modals, thus leading to more accurate and better refined depth maps. Extensive experiments on different benchmarks demonstrate that we achieve state-of-the-art among online methods. Meanwhile, we further evaluate the predicted dense depth by RGB-D SLAM, which is a commonly used downstream robotic perception task, and higher accuracy on vehicle's trajectory can be obtained in KITTI odometry dataset, which demonstrates the high quality of our depth prediction and the potential of improving the related downstream tasks with depth completion results.
Sparse random neural networks for online anomaly detection on sensor nodes

Whether it is used for predictive maintenance, intrusion detection or surveillance, on-device anomaly detection is a very valuable functionality in sensor and Internet-of-things (IoT) systems. In this paper, we introduce a novel anomaly detection technique based on sparse, random neural networks. The sparsity in the model allows for a very efficient implementation on embedded or resource constrained hardware. Our approach supports continuous online learning where the model is deployed to the sensor device without any prior training. As new data becomes available, the model is updated and becomes better at detecting anomalies. We experimentally validate our approach on several default benchmark data sets in the visual domain as well as on industrial quality inspection and predictive maintenance tasks. We show that our approach achieves a very favorable trade-off between computational cost and anomaly detection accuracy.
MapChain-D: A Distributed Blockchain for IIoT Data Storage and Communications

With the rapid growth of Industrial Internet of Things (IIoT) devices, managing an extensive volume of IIoT data becomes a significant challenge. While the conventional cloud storage approaches with centralized data centers suffer from high latency for large-scale IIoT data storage due to increased communication and latency overheads, distributed storage frameworks, such as blockchains, have become promising solutions. In this article, we design and analyze a dual-blockchain framework for secure and scalable distributed data management in large-scale IIoT networks. The proposed framework, named <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MapChain-D</i> , consists of a data chain that is mapped to an index chain to provide efficient data storage and lookup. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MapChain-D</i> is designed for practical IIoT applications with storage, latency, and communication constraints. Detailed data exchange protocols are presented for data insertion and retrieval operations in <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MapChain-D</i> . Based on these, theoretical analyses are provided on the space, time, and communication complexities of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MapChain-D</i> compared with conventional single-chain frameworks with local and distributed data storage. We implement our <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MapChain-D</i> prototype using open-source LoRaWAN communications with multiple Raspberry Pi and Arduino devices, Kademlia-based distributed hash table, and Ethereum-based blockchain with proof-of-authority consensus. Experimental results from our prototype show that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MapChain-D</i> is more suitable to be deployed on resource-constrained IIoT devices. We also highlight the scalability and flexibility of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MapChain-D</i> with different number of edge nodes in the system.
Co-Salient Object Detection with Co-Representation Purification

Co-salient object detection (Co-SOD) aims at discovering the common objects in a group of relevant images. Mining a co-representation is essential for locating co-salient objects. Unfortunately, the current Co-SOD method does not pay enough attention that the information not related to the co-salient object is included in the co-representation. Such irrelevant information in the co-representation interferes with its locating of co-salient objects. In this paper, we propose a Co-Representation Purification (CoRP) method aiming at searching noise-free co-representation. We search a few pixel-wise embeddings probably belonging to co-salient regions. These embeddings constitute our co-representation and guide our prediction. For obtaining purer co-representation, we use the prediction to iteratively reduce irrelevant embeddings in our co-representation. Experiments on three datasets demonstrate that our CoRP achieves state-of-the-art performances on the benchmark datasets. Our source code is available at https://github.com/ZZY816/CoRP.
Fast semi-supervised self-training algorithm based on data editing

Self-training is a commonly semi-supervised learning Algorithm framework. How to select the high-confidence samples is a crucial step for algorithms based on self-training framework. To alleviate the impact of noise data, researchers have proposed many data editing methods to improve the selection quality of high-confidence samples. However, the state-of-the-art data editing methods have high time complexity, which is not less than O(n2), where n denotes the number of samples. To improve the training speed while ensuring the quality of the selected high-confidence samples, inspired by Ball-k-means algorithm, we propose a fast semi-supervised self-training Algorithm based on data editing (EBSA), which defines ball-cluster partition and editing to improve the quality of high-confidence samples. The time complexity of the proposed EBSA is O(t2kn+nlogn+n+k2), where k denotes the number of centers, t denotes the number of iterates. k is far less than n, EBSA has linear time complexity with respect to n. A large number of experiments on 20 benchmark data sets have been carried out and the experimental results show that the proposed Algorithm not only ran faster, but also obtained better classification performance compared with the comparison algorithms.
Adaptive Manipulability-Based Path Planning Strategy for Industrial Robot Manipulators

In this article, a novel manipulability-based optimal rapidly exploring random tree (RRT*) path planning strategy is proposed for industrial robot manipulators. When sampling in the search space, two constraints, namely, path length and manipulability measure, are imposed to find a minimal-cost path connecting the start and goal points. By tracking the generated path, a robot manipulator's end-effector can traverse the workspace with a shorter length and, meanwhile, avoid configuration singularities. A constrained closed-loop inverse kinematics technique is utilized to exploit the kinematic redundancy to assign a higher manipulability to an end-effector position. Additionally, the metrics of path length and manipulability measure are used to determine the adaptive step size for the RRT* planner. This helps the space-filling tree to grow efficiently toward unsearched areas and find an optimal path. Simulation analysis and experimental results of a six-degree-of-freedom FANUC-M-20iA industrial robot illustrate the efficiency of the proposed path planning methods.
An optimized GAN method based on the Que-Attn and contrastive learning for underwater image enhancement

Research on underwater image processing has increased significantly in the past decade due to the precious resources that exist underwater. However, it is still a challenging problem to restore degraded underwater images. Existing prior-based methods show limited performance in many cases due to their reliance on hand-crafted features. Therefore, in this paper, we propose an effective unsupervised generative adversarial network(GAN) for underwater image restoration. Specifically, we embed the idea of contrastive learning into the model. The method encourages two elements (corresponding patches) to map the similar points in the learned feature space relative to other elements (other patches) in the data set, and maximizes the mutual information between input and output through PatchNCE loss. We design a query attention (Que-Attn) module, which compares feature distances in the source domain, and gives an attention matrix and probability distribution for each row. We then select queries based on their importance measure calculated from the distribution. We also verify its generalization performance on several benchmark datasets. Experiments and comparison with the state-of-the-art methods show that our model outperforms others.
Security analysis of lightweight IoT encryption algorithms: SIMON and SIMECK

This paper presents the differential cryptanalysis of two lightweight ciphers, namely, SIMON and SIMECK. The first cipher SIMON was presented by the U.S. National Security Agency in 2013 and the second one SIMECK was presented by researchers from the University of Waterloo, Canada in 2015. Both block ciphers belong to the lightweight encryption family that relies on the Feistel structure. The primary goal of making a differential attack is finding a high probability differential characteristic (path) for the cipher. However, finding a differential characteristic in a limited amount of time is the most challenging and take several hours, which can possibly be reduced by a heuristic approach. This paper used nested tree search-based methods to find differential paths and presented a state of art results but in reduced time and simpler framework.
Deep convolutional cross-connected kernel mapping support vector machine based on SelectDropout

Deep neural mapping support vector machine (DNMSVM) has achieved good results in numerous tasks by mapping the input from a low-dimensional space to a high-dimensional space and then using support vector machine for classification. However, it did not consider the connection of different spaces and increased the model parameters. To improve the classification performance while reducing the number of model parameters, we propose a deep Convolutional Cross-connected Kernel Mapping Support Vector Machine framework based on SelectDropout (CCKMSVM-SD). It consists of a feature extraction module and a classification module. The feature extraction module maps the data from low-dimensional to high-dimensional space by fusing the representations of different dimensional spaces through convolutional layers with cross-connections. For some convolutional layers, we use the depthwise separable convolution to replace the original convolution to reduce the number of parameters. Besides, we use SelectDropout to improve its generalization capability. The classification module uses a soft margin support vector machine for classification. The results on three tasks with ten different datasets indicate that CCKMSVM-SD obtains higher classification accuracy than other models with fewer parameters, demonstrating its effectiveness.
Diff attention: A novel attention scheme for person re-identification

The Person Re-ID has made great progress benefiting from the advance of deep neural networks and the attention mechanism in recent years. However, existing models and attentions for person Re-ID only focus on learning the robust features while neglecting the difference between features of pairs, which is the core of feature similarity matching tasks. To address this problem, we propose the Diff Attention idea and design Diff Attention Module to guide the network to learn a more discriminative attention map based on the difference of feature vectors. Taking the Diff Attention Module, we develop Diff Attention Framework to match various backbone Re-ID models and train them. To efficiently train Diff Attention Framework, we also propose two training strategies: Joint Training and Separate Training. Our framework and strategies have achieved excellent performance on Market-1501, CUHK03, and MSMT17 datasets. Our code will be made publicly available at https://github.com/linxin98/ReID-DiffAttention.
Projected patterns of climate change impact on photovoltaic energy potential: A case study of Iraq

Ambitious plans for the decarbonization of the global energy system necessitate the scaling up of renewable energy exploitation. This could render the energy supply system more susceptible to the effects of climate change as most renewables are climate-dependent by nature. Therefore, understanding the consequences of climate change on renewable energy systems at a regional level plays an important role in the financial management, process optimization, and energy yield assessment of these systems. In this work, the key objective is to examine the projected impacts of surface insolation and temperature fluctuations on the future photovoltaic (PV) energy potentials in Iraq. Projections are quantified across two climate scenarios, RCP4.5 and RCP8.5, which represent the intermediate and worst-case scenarios in which insolation will rise by 4.5 and 8.5 W/m2 by the beginning of the next century. The results show that the average temperature across the country is anticipated to rise by 1.5 °C and 2.4 °C under RCP4.5 and RCP8.5, respectively. The results also show that the change in solar PV potential relative to current climatic conditions, would be −0.3 to 8.1% under RCP4.5 while it is −5.1 to 6.3% under RCP8.5 by the beginning of the next century. The highest potentials are predicted in the western parts, and the greatest drops are in the southeastern parts. Temporal stability of PV potentials also appears to be little affected by future climatic scenarios, with some southern parts even indicating a little positive rise. So, even though climate change is expected to cause small drops in PV output in some areas, it is unlikely to pose a danger under RCP4.5 to PV productivity in Iraq.
A cascaded deep learning framework for photovoltaic power forecasting with multi-fidelity inputs

Accurate forecasts of photovoltaic power (PVP) are essential to the production, transmission, and distribution of electricity in power systems. However, PVP output is strongly weather-dependent, and the forecasting of PVP is highly dependent on the quality of numerical weather prediction (NWP) data. In recent years, a huge volume of numerical weather observation (NWO) data which are strongly associated with PVP output have been collected on-site by widely-installed smart meters and sensors. Appropriately utilizing high-fidelity NWO, in addition to low-fidelity NWP, has great potential in promoting the forecasting capability of deep learning (DL) models. Therefore, this paper proposes a cascaded multi-fidelity deep learning (CMF-DL) framework, which is coordinately driven by the data of both NWO and NWP, to deal with the day-ahead PVP forecasting problem. The proposed CMF-DL framework possesses great compatibility, and thus it can be incorporated with various DL models, such as the long short-term memory (LSTM) model and the gated recurrent unit (GRU) model. Subsequently, incorporated with CMF-DL, two newly-developed forecasting models, i.e., CMF-LSTM and CMF-GRU, are proposed, and datasets from a real-life PV plant are utilized, to evaluate the feasibility and effectiveness of the proposed approaches. From the results, the proposed CMF-LSTM and CMF-GRU show greater forecasting capability and anti-noise ability than the basic LSTM and GRU. Both CMF-LSTM and CMF-GRU can accept noisy NWP data with up to 35% errors. Additionally, compared to the persistence model, the forecasting skills of CMF-LSTM and CMF-GRU can be significantly promoted by 39.87% and 44.02%, respectively. The proposed CMF-LSTM and CMF-GRU also achieve better day-ahead PVP forecasting performance than the widely-used reference models in previous works.
Progressive cross-domain knowledge distillation for efficient unsupervised domain adaptive object detection

Unsupervised domain adaptation (UDA) is a technique for relieving domain shifts via transferring relevant domain knowledge from the full-labeled source domain to an unlabeled target domain. While tremendous advances have been witnessed recently, the adoption of deep CNN-based UDA methods in real-world scenarios is still constrained by low-resource computers. Most prior strategies either handle domain shift problems via UDA or compress CNNs using knowledge distillation (KD), we seek to implement the model on constrained-resource devices to learn domain adaptive knowledge without sacrificing accuracy. In this paper, we proposed a three-step Progressive Cross-domain Knowledge Distillation (PCdKD) paradigm for efficient unsupervised adaptive object detection, since directly alleviating the significant discrepancy across domains could result in unstable training procedures and suboptimal performance. First, we apply pixel-level alignment via image-to-image translation to reduce the appearance discrepancy between different domains. Then, a focal multi-domain discriminator is utilized to train the teacher–student peer networks for gradually distilling domain adaptive knowledge in a cooperative manner. Finally, reliable pseudo labels obtained by the adapted teacher detector are further utilized to retrain the teacher–student models. Our proposed method can boost the transferability of the teacher model as well as enhance the student model to meet the demand of real-time applications. Comprehensive experiments on four different cross-domain datasets show that our PCdKD outperforms most existing state-of-the-art approaches.