Reviewerly/exHarmony huggingface dataset:
Subjective and Objective De-Raining Quality Assessment Towards Authentic Rain Image

Images acquired by outdoor vision systems easily suffer poor visibility and annoying interference due to the rainy weather, which brings great challenge for accurately understanding and describing the visual contents. Recent researches have devoted great efforts on the task of rain removal for improving the image visibility. However, there is very few exploration about the quality assessment of de-rained image, even it is crucial for accurately measuring the performance of various de-raining algorithms. In this paper, we first create a de-raining quality assessment (DQA) database that collects 206 authentic rain images and their de-rained versions produced by 6 representative single image rain removal algorithms. Then, a subjective study is conducted on our DQA database, which collects the subject-rated scores of all de-rained images. To quantitatively measure the quality of de-rained image with non-uniform artifacts, we propose a bi-directional feature embedding network (B-FEN) which integrates the features of global perception and local difference together. Experiments confirm that the proposed method significantly outperforms many existing universal blind image quality assessment models. To help the research towards perceptually preferred de-raining algorithm, we will publicly release our DQA database and B-FEN source code on https://github.com/wqb-uestc.
Visually meaningful encryption for color images by using Qi hyper-chaotic system and singular value decomposition in YCbCr color space

In traditional image encryption algorithms, the plain-text image is encrypted into a noise-like image, which exhibits an existence of the encryption to an attacker. To solve the problem, the concept of visually meaningful image encryption (VMIE) is introduced, which can avoid human eyes' detection and improve the security of encryption. In this paper, we propose an improved VMIE scheme for color images. In the present scheme, the original image in RGB color space is transformed into YCbCr color space. Discrete wavelet transform (DWT) and discrete cosine transform (DCT) are used in Cb and Cr regions. Singular value decomposition (SVD) is performed on each DCT block and the singular value is embedded into the cover image. Qi hyper-chaotic mapping is used in the pre-encryption to enhance the security. Experimental results show that the proposed scheme has better security and performance than existing VMIE schemes and can also effectively resist common attacks.
Benchmarks for Aerial Manipulation

This letter is devoted to benchmarks for aerial manipulation robots (drones equipped with robotic arms), which are demonstrating their potential to conduct tasks involving physical interactions with objects or the environment in high altitude workspaces, being a cost effective solution for example in inspection and maintenance operations. Thus, the letter deals with different methods and criteria to evaluate and compare the performance of aerial manipulators. This is not an easy task, taking into account the wide variety of designs, morphologies and implementations that can be found in recent works. In order to cope with this problem, this letter analyzes the capabilities and functionalities of several aerial manipulation prototypes (aerial platform + manipulator), identifying a set of relevant metrics and criteria. A number of benchmarks are defined to evaluate the performance of the aerial manipulator in terms of accuracy, execution time, manipulation capability, or impact response. Experimental results carried out with a compliant joint aerial manipulator in test-bench and in indoor-outdoor testbeds illustrate some of the benchmarks.
Highly-efficient fog-based deep learning AAL fall detection system

Falls is one of most concerning accidents in aged population due to its high frequency and serious repercussion; thus, quick assistance is critical to avoid serious health consequences. There are several Ambient Assisted Living (AAL) solutions that rely on the technologies of the Internet of Things (IoT), Cloud Computing and Machine Learning (ML). Recently, Deep Learning (DL) have been included for its high potential to improve accuracy on fall detection. Also, the use of fog devices for the ML inference (detecting falls) spares cloud drawback of high network latency, non-appropriate for delay-sensitive applications such as fall detectors. Though, current fall detection systems lack DL inference on the fog, and there is no evidence of it in real environments, nor documentation regarding the complex challenge of the deployment. Since DL requires considerable resources and fog nodes are resource-limited, a very efficient deployment and resource usage is critical. We present an innovative highly-efficient intelligent system based on a fog-cloud computing architecture to timely detect falls using DL technics deployed on resource-constrained devices (fog nodes). We employ a wearable tri-axial accelerometer to collect patient monitoring data. In the fog, we propose a smart-IoT-Gateway architecture to support the remote deployment and management of DL models. We deploy two DL models (LSTM/GRU) employing virtualization to optimize resources and evaluate their performance and inference time. The results prove the effectiveness of our fall system, that provides a more timely and accurate response than traditional fall detector systems, higher efficiency, 98.75% accuracy, lower delay, and service improvement.
Upper semi-continuous convergence of attractors for a Hopfield-type lattice model

To investigate dynamical behavior of the Hopfield neural network model when its dimension becomes increasingly large, a Hopfield-type lattice system is developed as the infinite dimensional extension of the classical Hopfield model. The existence of global attractors is established for both the lattice system and its finite dimensional approximations. Moreover, the global attractors for the finite dimensional approximations are shown to converge to the attractor for the infinite dimensional lattice system upper semi-continuously.
State-of-the-Art Speech Recognition Using Multi-Stream Self-Attention with Dilated 1D Convolutions

Self-attention has been a huge success for many downstream tasks in NLP, which led to exploration of applying self-attention to speech problems as well. The efficacy of self-attention in speech applications, however, seems not fully blown yet since it is challenging to handle highly correlated speech frames in the context of self-attention. In this paper we propose a new neural network model architecture, namely multi-stream self-attention, to address the issue thus make the self-attention mechanism more effective for speech recognition. The proposed model architecture consists of parallel streams of self-attention encoders, and each stream has layers of 1D convolutions with dilated kernels whose dilation rates are unique given stream, followed by a self-attention layer. The self-attention mechanism in each stream pays attention to only one resolution of input speech frames and the attentive computation can be more efficient. In a later stage, outputs from all the streams are concatenated then linearly projected to the final embedding. By stacking the proposed multi-stream self-attention encoder blocks and rescoring the resultant lattices with neural network language models, we achieve the word error rate of 2.2% on the test-clean dataset of the LibriSpeech corpus, the best number reported thus far on the dataset.
Bi-objective web service composition problem in multi-cloud environment: a bi-objective time-varying particle swarm optimisation algorithm

Cloud computing became an inevitable information technology industry. Despite its several plus points such as economy of scale and rapid elasticity, it suffers from vendor lock-in, resource limitation and cybersecurity attacks in which it leads business discontinuity or even business failure. Multi-cloud, on the other hand, can be trustable paradigm to obviate obstacles such as aforesaid unpleasant features of a single cloud. One of the biggest challenges is to know which cloud is commensurate with user’s business process with regards to security objectives. To this end, the new method is presented to quantify the amount of cloud security risk (CSR) in regards to user’s business process. Therefore, in this paper, the web service composition problem is formulated to bi-objective optimisation problem with service cost and multi-cloud risk viewpoints in ever-increasing multi-cloud environment (MCE) in which each provider has its variable pricing policy and different security level. It is obviously an NP-Hard problem. To solve the combinatorial problem, we develop a bi-objective time-varying particle swarm optimisation (BOTV-PSO) algorithm. The parameters are tuned based on elapsed time so a good balance between exploration and exploitation is achieved. To illustrate the effectiveness of proposed algorithm, we defined several scenarios and compared the performance of proposed algorithm with multi-objective GA-based (MOGA) optimiser, a single objective genetic algorithm (SOGA) that only optimises cost function and neglects CSR, and multi-objective simulated annealing algorithm (MOSA). The experimental results showed the superiority of proposed BOTV-PSO against other approaches in terms of convergence, diversity, fitness, performance, and even scalability.
Streaming End-to-End Speech Recognition with Joint CTC-Attention Based Models

In this paper, we present a one-pass decoding algorithm for streaming recognition with joint connectionist temporal classification (CTC) and attention-based end-to-end automatic speech recognition (ASR) models. The decoding scheme is based on a frame-synchronous CTC prefix beam search algorithm and the recently proposed triggered attention concept. To achieve a fully streaming end-to-end ASR system, the CTC-triggered attention decoder is combined with a unidirectional encoder neural network based on parallel time-delayed long short-term memory (PTDLSTM) streams, which has demonstrated superior performance compared to various other streaming encoder architectures in earlier work. A new type of pre-training method is studied to further improve our streaming ASR models by adding residual connections to the encoder neural network and layer-wise removing them during the training process. The proposed joint CTC-triggered attention decoding algorithm, which enables streaming recognition of attention-based ASR systems, achieves similar ASR results compared to offline CTC-attention decoding and significantly better results compared to CTC prefix beam search decoding alone.
Transformer ASR with Contextual Block Processing

The Transformer self-attention network has recently shown promising performance as an alternative to recurrent neural networks (RNNs) in end-to-end (E2E) automatic speech recognition (ASR) systems. However, the Transformer has a drawback in that the entire input sequence is required to compute self-attention. In this paper, we propose a new block processing method for the Transformer encoder by introducing a context-aware inheritance mechanism. An additional context embedding vector handed over from the previously processed block helps to encode not only local acoustic information but also global linguistic, channel, and speaker attributes. We introduce a novel mask technique to implement the context inheritance to train the model efficiently. Evaluations of the Wall Street Journal (WSJ), Librispeech, VoxForge Italian, and AISHELL-1 Mandarin speech recognition datasets show that our proposed contextual block processing method outperforms naive block processing consistently. Furthermore, the attention weight tendency of each layer is analyzed to clarify how the added contextual inheritance mechanism models the global information.
SGNN: A Graph Neural Network Based Federated Learning Approach by Hiding Structure

Networks are general tools for modeling numerous information with features and complex relations. Network Embedding aims to learn low-dimension representations for vertexes in the network with rich information including content information and structural information. In recent years, many models based on neural network have been proposed to map the network representations into embedding space whose dimension is much lower than that in original space. However, most of existing methods have the following limitations: 1) they are based on content of nodes in network, failing to measure the structure similarity of nodes; 2) they cannot do well in protecting the privacy of users including the original content information and the structural information. In this paper, we propose a similarity-based graph neural network model, SGNN, which captures the structure information of nodes precisely in node classification tasks. It also takes advantage of the thought of federated learning to hide the original information from different data sources to protect users' privacy. We use deep graph neural network with convolutional layers and dense layers to classify the nodes based on their structures and features. The node classification experiment results on public data sets including Aminer coauthor network, Brazil and Europe flight networks indicate that our proposed model outperforms state-of-the-art models with a higher accuracy.
Bug severity prediction using question-and-answer pairs from Stack Overflow

Nowadays, bugs have been common in most software systems. For large-scale software projects, developers usually conduct software maintenance tasks by utilizing software artifacts (e.g., bug reports). The severity of bug reports describes the impact of the bugs and determines how quickly it needs to be fixed. Bug triagers often pay close attention to some features such as severity to determine the importance of bug reports and assign them to the correct developers. However, a large number of bug reports submitted every day increase the workload of developers who have to spend more time on fixing bugs. In this paper, we collect question-and-answer pairs from Stack Overflow and use logical regression to predict the severity of bug reports. In detail, we extract all the posts related to bug repositories from Stack Overflow and combine them with bug reports to obtain enhanced versions of bug reports. We achieve severity prediction on three popular open source projects (e,g., Mozilla, Ecplise, and GCC) with Naïve Bayesian, k-Nearest Neighbor algorithm (KNN), and Long Short-Term Memory (LSTM). The results of our experiments show that our model is more accurate than the previous studies for predicting the severity. Our approach improves by 23.03%, 21.86%, and 20.59% of the average F-measure for Mozilla, Eclipse, and GCC by comparing with the Naïve Bayesian based approach which performs the best among all baseline approaches.
Drynx: Decentralized, Secure, Verifiable System for Statistical Queries and Machine Learning on Distributed Datasets

Data sharing has become of primary importance in many domains such as big-data analytics, economics and medical research, but remains difficult to achieve when the data are sensitive. In fact, sharing personal information requires individuals’ unconditional consent or is often simply forbidden for privacy and security reasons. In this paper, we propose Drynx, a decentralized system for privacy-conscious statistical analysis on distributed datasets. Drynx relies on a set of computing nodes to enable the computation of statistics such as standard deviation or extrema, and the training and evaluation of machine-learning models on sensitive and distributed data. To ensure data confidentiality and the privacy of the data providers, Drynx combines interactive protocols, homomorphic encryption, zero-knowledge proofs of correctness, and differential privacy. It enables an efficient and decentralized verification of the input data and of all the system’s computations thus provides auditability in a strong adversarial model in which no entity has to be individually trusted. Drynx is highly modular, dynamic and parallelizable. Our evaluation shows that it enables the training of a logistic regression model on a dataset (12 features and 600,000 records) distributed among 12 data providers in less than 2 seconds. The computations are distributed among 6 computing nodes, and Drynx enables the verification of the query execution’s correctness in less than 22 seconds.
Planning Jerk-Optimized Trajectory With Discrete Time Constraints for Redundant Robots

We present a method for effectively planning the motion trajectory of robots in manufacturing tasks, the toolpaths of which are usually complex and have a large number of discrete time constraints as waypoints.Kinematic redundancy also exists in these robotic systems.The jerk of motion is optimized in our trajectory planning method at the meanwhile of fabrication process to improve the quality of fabrication.Our method is based on a sampling strategy and consists of two major parts.After determining an initial path by graph-search, a greedy algorithm is adopted to optimize a path by locally applying adaptive filers in the regions with large jerks.The filtered result is obtained by numerical optimization.In order to achieve efficient computation, an adaptive sampling method is developed for learning a collision-indication function that is represented as a support-vector machine.Applications in robotassisted 3D printing are given in this paper to demonstrate the functionality of our approach.Note to Practitioner Abstract-In robot-assisted manufacturing applications, robotic arms are employed to realize the motion of workpieces (or machining tools) specified as a sequence of waypoints with the positions of tool tip and the tool orientations constrained.The required degree-of-freedom (DOF) is often less than the robotic hardware system (e.g., a robotic arm has 6-DOF).Specifically, rotations of the workpiece around the axis of a tool can be arbitrary (see Fig. 1 for an example).By using this redundancy -i.e., there are many possible poses of a robotic arm to realize a given waypoint, the trajectory of robots can be optimized to consider the performance of motion in velocity, acceleration and jerk in the joint space.In addition, when fabricating complex models each tool-path can have a large amount of waypoints.It is crucial for a motion planning algorithm to compute a smooth and collision-free trajectory of robot to improve fabrication quality.The time taken by the planning algorithm should not significantly lengthen the total manufacturing time; ideally it would remain hidden as computing motions for a layer can be done while the previous layer is printing.The method presented in this paper provides an efficient framework to tackle this problem.The framework has been well tested on our robot-assisted additive manufacturing system to demonstrate its effectiveness and can be generally applied to other robot-assisted manufacturing systems.
A Fraud-Resilient Blockchain-Based Solution for Invoice Financing

Invoice financing has been a steadily growing component of the financing market as a whole for the last few years, and, in 2016, it became the third largest financing market. Nonetheless, the risk of frauds is still very high, and most solutions proposed so far are based on private, proprietary platforms that cannot match the global nature of such a market. Even the most recent proposals based on blockchain are mainly adopting a private, permissioned blockchain due to the lack of confidentiality in public blockchain. In this article, we propose an Invoice financing platform based on a public blockchain supporting both fully open and group-restricted auctioning of invoices. We addressed the confidentiality issue by storing the confidential data encrypted in IPFS and the corresponding hash in the smart contract hosted on Ethereum blockchain. Our blockchain-based solution ensures data confidentiality and benefits from the main properties of the public blockchain required in Invoice financing systems, such as transparency, immutability, trustworthiness, and security. Furthermore, our platform introduces a reputation system based on the past behavior of entities, computed using the blockchain global ledger. Such a reputation system allows insurance companies to modulate the cost of the insurance contracts they offer. This combination guarantees the complete transparency and tamperproofness of a public blockchain, while it allows reducing insurance costs and fraud possibilities.
A Novel Active Contour Model Guided by Global and Local Signed Energy-Based Pressure Force

Active contour models (ACMs) have been widely applied in the field of image segmentation. However, it is still very challenging to construct an efficient ACM to segment images with intensity inhomogeneity. In this paper, a novel ACM guided by global and local signed energy-based pressure force (GLSEPF) is proposed. First, by computing the energy difference between the inner and outer energies of the evolution curve, a global signed energy-based pressure force (GSEPF) is designed, which can improve the robustness to initial curves. Second, a local signed energy-based pressure force (LSEPF) is introduced by computing the pixel-by-pixel energy difference within local neighborhood region, which can handle images with intensity inhomogeneity and noise. Finally, the global image information and the local energy information are used for the global and local force propagation functions, respectively. The global and local variances are used to automatically balance the weights of the GSEPF and the LSEPF, which can solve the problem of setting parameters. Meanwhile, a regularization term and a penalty term are applied to avoid the re-initialization process during iterations and smooth the level set function. Experimental results on different types of images demonstrate that the proposed model is more robust than the popular region-based and mixed ACMs for segmenting images with intensity inhomogeneity and noise. The code is available at: https://github.com/HuaxiangLiu/GLSEPF/.
A comparison of AMQP and MQTT protocols for Internet of Things

In the industrial revolution 4.0, the Internet of Things (IoT) system is being seen everywhere. The data is constantly exchanged between divices in the IoT system and Edge or Cloud server with different protocols. Data, that is exchanged between IoT gateway and IoT server, usually uses 2 protocols: MQTT and AMQP. The selection of a good protocol is foundation for the entire IoT system works effectively. In this paper, we present the results of research and evaluation of the effectiveness in using the above two protocols based on experiments, simulating data transmission environment with different latency and packet loss rate. The condition is simulated with the real condition in IoT system, which is using in the Internet Radio system. Finally, a protocol is recommended for different environment's conditions.
A parallel computing method based on zeroing neural networks for time-varying complex-valued matrix Moore-Penrose inversion

This paper analyzes the existing zeroing neural network (ZNN) models from the perspective of control theory. It proposes an exclusive ZNN model for solving the dynamic complex-valued matrix Moore-Penrose inverse problem: the complex-valued zeroing neural network (CVZNN). Then, a method of constructing a special type of saturation-allowed activation function is defined, which relaxes the convex constraint on the activation function when constructing the ZNN model. The convergence of the CVZNN model activated by proposed saturation-allowed functions is analyzed. Besides, the robustness of the CVZNN model under different types of noise interference is investigated based on the perspective of the control theory. Finally, the effectiveness and superiority of the CVZNN model are illustrated by simulation experiments.