Reviewerly/exHarmony huggingface dataset:
An FPGA-Based Energy-Efficient Reconfigurable Convolutional Neural Network Accelerator for Object Recognition Applications

The computational efficiency is the prime concern of a computation-intensive deep convolutional neural network (CNN). In this Brief, we report an FPGA-based computation-efficient reconfigurable CNN accelerator. It innovates in the utilization of a kernel partition technique to substantially reduce the repeated access to the input feature maps and the kernels. As a result, it balances the ability for parallel computing while consuming less system power. Experimental results prove that the proposed CNN accelerator achieves a peak throughput of 220.0 GOP/s with an energy efficiency of 22.9 GOPs/W at 151.4 frames/s for the AlexNet. It is also reconfigurable to process VGG-16 befitting complex object recognition.
Rethinking Graph Neural Architecture Search from Message-passing

Graph neural networks (GNNs) emerged recently as a standard toolkit for learning from data on graphs. Current GNN designing works depend on immense human expertise to explore different message-passing mechanisms, and require manual enumeration to determine the proper message-passing depth. Inspired by the strong searching capability of neural architecture search (NAS) in CNN, this paper proposes Graph Neural Architecture Search (GNAS) with novel-designed search space. The GNAS can automatically learn better architecture with the optimal depth of message passing on the graph. Specifically, we design Graph Neural Architecture Paradigm (GAP) with tree-topology computation procedure and two types of fine-grained atomic operations (feature filtering & neighbor aggregation) from message-passing mechanism to construct powerful graph network search space. Feature filtering performs adaptive feature selection, and neighbor aggregation captures structural information and calculates neighbors’ statistics. Experiments show that our GNAS can search for better GNNs with multiple message-passing mechanisms and optimal message-passing depth. The searched network achieves remarkable improvement over state-of-the-art manual designed and search-based GNNs on five large-scale datasets at three classical graph tasks. Codes can be found at https://github.com/phython96/GNAS-MP.
Novel Deep Learning Based on Data Fusion Integrating Correlation Analysis for Soft Sensor Modeling

Accurate soft sensing modeling of complex industrial processes can provide operation guidance for improving the product quality. However, most modeling methods cannot mine the process data sufficiently, which leads to low prediction accuracy and generalization performance. Therefore, a novel soft sensing method based on dilated convolution neural network (DCNN) combining data fusion and correlation analysis is proposed. The fused data can be obtained by the sliding window approach, with window sizes of 1 day, to eliminate noise caused by uncertain working conditions. Then, the correlation analysis method is used to analyze the relevance of the fused data to reduce redundant variables. Moreover, the processed variables and target values are taken as inputs and outputs of the DCNN to build the soft sensing model. Finally, the proposed method is applied in a polypropylene production system to predict the melt index. Compared with the extreme learning machine, the convolution neural network, the DCNN, the DCNN based on data fusion, and the DCNN based on correlation analysis, the proposed soft sensing method has achieved state-of-the-art prediction accuracy and generalization ability, which can improve the production efficiency.
AlphaMatch: Improving Consistency for Semi-supervised Learning with Alpha-divergence

Semi-supervised learning (SSL) is a key approach toward more data-efficient machine learning by jointly leverage both labeled and unlabeled data. We propose AlphaMatch, an efficient SSL method that leverages data augmentations, by efficiently enforcing the label consistency between the data points and the augmented data derived from them. Our key technical contribution lies on: 1) using alpha-divergence to prioritize the regularization on data with high confidence, achieving similar effect as FixMatch [32] but in a more flexible fashion, and 2) proposing an optimization-based, EM-like algorithm to enforce the consistency, which enjoys better convergence than iterative regularization procedures used in recent SSL methods such as FixMatch, UDA, and MixMatch. AlphaMatch is simple and easy to implement, and consistently outperforms prior arts on standard benchmarks, e.g. CIFAR-10, SVHN, CIFAR-100, STL-10. Specifically, we achieve 91.3% test accuracy on CIFAR-10 with just 4 labelled data per class, substantially improving over the previously best 88.7% accuracy achieved by FixMatch.
ISCL: Interdependent Self-Cooperative Learning for Unpaired Image Denoising

With the advent of advances in self-supervised learning, paired clean-noisy data are no longer required in deep learning-based image denoising. However, existing blind denoising methods still require the assumption with regard to noise characteristics, such as zero-mean noise distribution and pixel-wise noise-signal independence; this hinders wide adaptation of the method in the medical domain. On the other hand, unpaired learning can overcome limitations related to the assumption on noise characteristics, which makes it more feasible for collecting the training data in real-world scenarios. In this paper, we propose a novel image denoising scheme, Interdependent Self-Cooperative Learning (ISCL), that leverages unpaired learning by combining cyclic adversarial learning with self-supervised residual learning. Unlike the existing unpaired image denoising methods relying on matching data distributions in different domains, the two architectures in ISCL, designed for different tasks, complement each other and boost the learning process. To assess the performance of the proposed method, we conducted extensive experiments in various biomedical image degradation scenarios, such as noise caused by physical characteristics of electron microscopy (EM) devices (film and charging noise), and structural noise found in low-dose computer tomography (CT). We demonstrate that the image quality of our method is superior to conventional and current state-of-the-art deep learning-based unpaired image denoising methods.
A secure and efficient Internet of Things cloud encryption scheme with forensics investigation compatibility based on identity-based encryption

Data security is a challenge for end-users of cloud services as the users have no control over their data once it is transmitted to the cloud. A potentially corrupt cloud service provider can obtain the end-users’ data. Conventional PKI-based solutions are insufficient for large-scale cloud systems, considering efficiency, scalability, and security. In large-scale cloud systems, the key management requirements include scalable encryption, authentication, and non-repudiation services, as well as the ability to share files with different users and data recovery when the user keys of encrypted data are not accessible. Further requirements in cloud systems include the ability to provide the means for digital forensic investigations on encrypted data. Once data on the cloud is encrypted with a user’s key it becomes impossible to access by forensic investigation teams. In this regard, distributing the trust of key management into multiple authorities is desirable. In the literature, there is no available secure cloud storage system with secure and efficient Type-3 pairings, supporting Encryption-as-a-Service (EaaS) and multiple Public Key Generators (PKGs). This paper proposes an efficient Identity-based cryptography (IBC) architecture for secure cloud storage, named Secure Cloud Storage System (SCSS), which supports distributed key management and encryption mechanisms and support for multiple PKGs. During forensic investigations, the legal authorities will be able to use the multiple PKG mechanism for data access, while an account locking mechanism prevents a single authority to access user data due to trust distribution. We also demonstrate that, the IBC scheme used in SCSS has better performance compared to similar schemes in the literature. For the security levels of 128-bits and above, SCSS has better scalability compared to existing schemes, with respect to encryption and decryption operations. Since the decryption operation is frequently needed for forensic analysis, the improved scalability results in a streamlined forensic investigation process on the encrypted data in the cloud. • A secure cloud storage system based on Identity Based Encryption and Type-3 pairings. • An efficient IBC-based architecture with Multiple PKGs. • Proposed scheme has better performance than similar schemes in the literature.
Learning Scene Structure Guidance via Cross-Task Knowledge Transfer for Single Depth Super-Resolution

Existing color-guided depth super-resolution (DSR) approaches require paired RGB-D data as training samples where the RGB image is used as structural guidance to recover the degraded depth map due to their geometrical similarity. However, the paired data may be limited or expensive to be collected in actual testing environment. Therefore, we explore for the first time to learn the cross-modality knowledge at training stage, where both RGB and depth modalities are available, but test on the target dataset, where only single depth modality exists. Our key idea is to distill the knowledge of scene structural guidance from RGB modality to the single DSR task without changing its network architecture. Specifically, we construct an auxiliary depth estimation (DE) task that takes an RGB image as input to estimate a depth map, and train both DSR task and DE task collaboratively to boost the performance of DSR. Upon this, a cross-task interaction module is proposed to realize bilateral cross-task knowledge transfer. First, we design a cross-task distillation scheme that encourages DSR and DE networks to learn from each other in a teacher-student role-exchanging fashion. Then, we advance a structure prediction (SP) task that provides extra structure regularization to help both DSR and DE networks learn more informative structure representations for depth recovery. Extensive experiments demonstrate that our scheme achieves superior performance in comparison with other DSR methods.
Gradual back-projection residual attention network for magnetic resonance image super-resolution

Magnetic Resonance Image (MRI) analysis can provide anatomical examination of internal organs, which is helpful for diagnosis of the disease. Aiming at the problems of insufficient feature information mining in the process of MRI super-resolution (SR) reconstruction, the difficulty of determining the interdependence between the channels of the feature map, and the reconstruction error when reconstructing high-resolution (HR) images, we propose a SR method to solve these problems. In this work, we propose a gradual back-projection residual attention network for MRI super-resolution (GRAN), which outperforms most of the state-of-the-art methods. Firstly, we use the gradual upsampling method to gradually scale the low-resolution (LR) image to a given magnification to alleviate the high-frequency information loss caused by the upsampling process. Secondly, we merge the idea of iterative back-projection at each stage of gradual upsampling, learn the mapping relationship between HR and LR feature maps and reduce the noise introduced during the upsampling process. Finally, we use the attention mechanism to dynamically allocate attention resources to the feature maps generated at different stages of the gradual back-projection network, so that the network model can learn the interdependence between each feature map. For the 2 × and 4 × enlargement, the proposed GRAN method shows the superiority over the state-of-the-art methods on the Set5, Set14, and Urban100 benchmark datasets, extensive benchmark experiment and analysis show that the superiority of the GRAN algorithm in terms of peak signal-to-noise ratio and structural similarity index indicators. The MRI results reconstructed by gradual back-projection residual attention network on the public dataset IDI have good image sharpness, rich texture details and good visual experience. In addition, the reconstructed image is the closest to the real image, enabling the medical expert to see the biological tissue structure and its early pathological changes more clearly, providing assistance and support to the medical expert in the diagnosis and treatment of the disease.
Multivariate hierarchical multiscale fluctuation dispersion entropy: Applications to fault diagnosis of rotating machinery

With a view to completing the fault diagnosis of rotating machinery efficiently and accurately, this paper presents a novel fault diagnosis model that combines multivariate hierarchical multiscale fluctuation dispersion entropy (MHMFDE), multi-cluster feature selection (MCFS) and gray wolf optimization-based kernel extreme learning machine(GWO-KELM). Firstly, MHMFDE is presented to capture the high-dimensional fault features hidden in the attained multichannel vibration signals. Integrating the multiscale entropy-based method and the hierarchical entropy-based method that are currently popular in the domain of fault identification, MHMFDE can simultaneously extract affluent fault features from multivariate vibration signals in-depth as well as overcoming the problem of information loss in the existing single-channel data analysis methods. Afterward, MCFS is used to pick sensitive features from the attained raw fault features to form the sensitive feature vectors, thereby reducing the impact of redundant features. Finally, GWO-KELM is adopted to quantitatively analyze the diagnostic effect. Three examples reveal that the presented approach enjoys excellent performance in the fault diagnosis domain. Especially for the identification of compound faults of rotating machinery, the performance of the presented method is significantly superior to that of existing methods.
Evidential reasoning based ensemble classifier for uncertain imbalanced data

Various studies have focused on the classification of uncertain or imbalanced data. However, previous studies rarely consider the classification for uncertain imbalanced data. To address this research gap, this study proposes an evidential reasoning (ER) based ensemble classifier (EREC). In the proposed method, an affinity propagation based oversampling method is developed to obtain the balanced class distributions of the training datasets for individual classifiers. Using the balanced training datasets, ER-based classifiers are constructed as individual classifiers to handle data uncertainty, in which attribute weights are learned from the similarity between the values of attributes and labels. With trained individual classifiers, final results are generated by combining the results of individual classifiers using the ER algorithm, in which the weights of individual classifiers are determined according to the classification performance on out-of-bag data. The proposed EREC is applied to the diagnosis of thyroid nodules using the datasets of five radiologists, obtained from a tertiary hospital located in Hefei, Anhui, China. Using real datasets and UCI datasets, the EREC is compared with 12 representative ensemble classifiers and other oversampling methods based ensemble classifiers to highlight its high performance.
A holistic approach to interpretability in financial lending: Models, visualizations, and summary-explanations

Lending decisions are usually made with proprietary models that provide minimally acceptable explanations to users. In a future world without such secrecy, what decision support tools would one want to use for justified lending decisions? This question is timely, since the economy has dramatically shifted due to a pandemic, and a massive number of new loans will be necessary in the short term. We propose a framework for such decisions, including a globally interpretable machine learning model, an interactive visualization of it, and several types of summaries and explanations for any given decision. The machine learning model is a two-layer additive risk model, which resembles a two-layer neural network, but is decomposable into subscales. In this model, each node in the first (hidden) layer represents a meaningful subscale model, and all of the nonlinearities are transparent. Our online visualization tool allows exploration of this model, showing precisely how it came to its conclusion. We provide three types of explanations that are simpler than, but consistent with, the global model: case-based reasoning explanations that use neighboring past cases, a set of features that were the most important for the model's prediction, and summary-explanations that provide a customized sparse explanation for any particular lending decision made by the model. Our framework earned the FICO recognition award for the Explainable Machine Learning Challenge, which was the first public challenge in the domain of explainable machine learning.1
Potentials of Augmented Reality in a BIM based building submission process

The digital transformation in the construction industry, which is also referred to as Construction 4.0, is gaining impact in recent years. BIM as the central point for model-based planning and communication is at the centre of this development. This paves the way for the implementation of further technologies, such as augmented reality (AR) or artificial intelligence (AI) in the construction process. However, the submission process currently receives little attention in the BIM project cycle, even though it has a significant impact on a building project. There is a huge potential for the application of BIM and AR in the building submission process. BIM allows partly automatic compliance checks to be performed. The BIM model and the results of these checks can be used as basis for the AR model. This paper investigates an application for AR in a BIM submission process. Here we show that especially the plan check process and hearings in the approval process can be improved and sped up by combining BIM and the visualization technology of AR. In this process, AR can support the building authority in checking the compliance with building regulations. Additionally, non-experts involved often have problems grasping the planned building project just based on the technical 2D plans. In this case, AR helps them to visualise the impact of the planned project and allows for better judgement. Use-cases are presented, showing strategies to improve the plan check process and assist the building authority as well as the persons involved, for both experts as well as non-experts. Finally, the requirements, technical implementations, and effects of an AR application for the plan check process are described. An AR-supported plan checking process can vastly improve the presentation of technical content. This leads to better understanding and more transparency for non-experts. Because AR is relatively easy to use, it might become ubiquitous not only to experts, but also to the general public.
A Fusion Crossover Mutation Sparrow Search Algorithm

Aiming at the inherent problems of swarm intelligence algorithm, such as falling into local extremum in early stage and low precision in later stage, this paper proposes an improved sparrow search algorithm (ISSA). Firstly, we introduce the idea of flight behavior in the bird swarm algorithm into SSA to keep the diversity of the population and reduce the probability of falling into local optimum; Secondly, we creatively introduce the idea of crossover and mutation in genetic algorithm into SSA to get better next-generation population. These two improvements not only keep the diversity of the population at all times but also make up for the defect that the sparrow search algorithm is easy to fall into local optimum at the end of the iteration. The optimization ability of the improved SSA is greatly improved.
Learning a Self-Expressive Network for Subspace Clustering

State-of-the-art subspace clustering methods are based on the self-expressive model, which represents each data point as a linear combination of other data points. However, such methods are designed for a finite sample dataset and lack the ability to generalize to out-of-sample data. Moreover, since the number of self-expressive coefficients grows quadratically with the number of data points, their ability to handle large-scale datasets is often limited. In this paper, we propose a novel framework for subspace clustering, termed Self-Expressive Network (SENet), which employs a properly designed neural network to learn a self-expressive representation of the data. We show that our SENet can not only learn the self-expressive coefficients with desired properties on the training data, but also handle out-of-sample data. Besides, we show that SENet can also be leveraged to perform subspace clustering on large-scale datasets. Extensive experiments conducted on synthetic data and real world benchmark data validate the effectiveness of the proposed method. In particular, SENet yields highly competitive performance on MNIST, Fashion MNIST and Extended MNIST and state-of-the-art performance on CIFAR-10.
Relay-Assisted Cooperative Federated Learning

Federated learning (FL) has recently emerged as a promising technology to enable artificial intelligence (AI) at the network edge, where distributed mobile devices collaboratively train a shared AI model under the coordination of an edge server. To significantly improve the communication efficiency of FL, over-the-air computation allows a large number of mobile devices to concurrently upload their local models by exploiting the superposition property of wireless multi-access channels. Due to wireless channel fading, the model aggregation error at the edge server is dominated by the weakest channel among all devices, causing severe straggler issues. In this paper, we propose a relay-assisted cooperative FL scheme to effectively address the straggler issue. In particular, we deploy multiple half-duplex relays to cooperatively assist the devices in uploading the local model updates to the edge server. The nature of the over-the-air computation poses system objectives and constraints that are distinct from those in traditional relay communication systems. Moreover, the strong coupling between the design variables renders the optimization of such a system challenging. To tackle the issue, we propose an alternating-optimization-based algorithm to optimize the transceiver and relay operation with low complexity. Then, we analyze the model aggregation error in a single-relay case and show that our relay-assisted scheme achieves a smaller error than the one without relays provided that the relay transmit power and the relay channel gains are sufficiently large. The analysis provides critical insights on relay deployment in the implementation of cooperative FL. Extensive numerical results show that our design achieves faster convergence compared with state-of-the-art schemes.