Reviewerly/exHarmony huggingface dataset:
Global Visual Feature and Linguistic State Guided Attention for Remote Sensing Image Captioning

The encoder–decoder framework is prevalent in existing remote-sensing image captioning (RSIC) models. The appearance of attention mechanisms brings significant results. However, current attention-based caption models only build up the relationships between the local features without introducing the global visual feature and removing redundant feature components. It will cause caption models to generate descriptive sentences that are weakly related to the scene of images. To solve the problems, this article proposed a global visual feature-guided attention (GVFGA) mechanism. First, GVFGA introduces the global visual feature and fuses them with local visual features to build up their relationships between them. Second, an attention gate utilizing the global visual feature is proposed in GVFGA to filter out redundant feature components in the fused image features and provide more salient image features. In addition, to relieve the hidden state’s burden, a linguistic state (LS) is proposed to specifically provide textual features, making the hidden state only guiding visual–textual attention process. What’s more, to further refine the fusion of visual features and textual features, a LS-Guided Attention (LSGA) mechanism is proposed. It can also filter out the irrelevant information in the fused visual–textual feature with the help of an attention gate. The experimental results show that this proposed image captioning model can achieve better results on three RSIC datasets, UCM-Captions, Sydney-Captions, and RSICD datasets.
Soft Error Tolerant Convolutional Neural Networks on FPGAs With Ensemble Learning

Convolutional neural networks (CNNs) are widely used in computer vision and natural language processing. Field-programmable gate arrays (FPGAs) are popular accelerators for CNNs. However, if used in critical applications, the reliability of FPGA-based CNNs becomes a priority because FPGAs are prone to suffer soft errors. Traditional protection schemes, such as triple modular redundancy (TMR), introduce a large overhead, which is not acceptable in resource-limited platforms. This article proposes to use an ensemble of weak CNNs to build a robust classifier with low cost. To have a group of base CNNs with low complexity and balanced similarity and diversity, residual neural networks (ResNets) with different layers (20/32/44/56) are combined in the ensemble system to replace a single strong ResNet 110. In addition, a robust combiner is designed based on the reliability evaluation of a single ResNet. Single ResNets with different layers and different ensemble schemes are implemented on the FPGA accelerator based on Xilinx Zynq 7000 SoC. The reliability of the ensemble systems is evaluated based on a large-scale fault injection platform and compared with that of the TMR-protected ResNet 110 and ResNet 20. Experiment results show that the proposed ensembles could effectively improve the system reliability when suffering soft errors with an overhead much lower than TMR.
Functional Bayesian Filter

We present a general nonlinear Bayesian filter for high-dimensional state estimation using the theory of reproducing kernel Hilbert space (RKHS). By applying the kernel method and the representer theorem to perform linear quadratic estimation in a functional space, we derive a Bayesian recursive state estimator for a general nonlinear dynamical system in the original input space. Unlike existing nonlinear extensions of the Kalman filter where the system dynamics are assumed known, the state-space representation for the Functional Bayesian Filter (FBF) is completely learned online from measurement data in the form of an infinite impulse response (IIR) filter or recurrent network in the RKHS, with universal approximation property. Using a positive definite kernel function satisfying Mercer’s conditions to compute and evolve information quantities, the FBF exploits both the statistical and time-domain information about the signal, extracts higher-order moments, and preserves the properties of covariances without the ill effects due to conventional arithmetic operations. We apply this novel kernel adaptive filtering (KAF) to recurrent network training, chaotic time-series estimation and cooperative filtering using Gaussian and non-Gaussian noises, and inverse kinematics modeling. Simulation results show FBF outperforms existing Kalman-based algorithms.
Using artificial intelligence (AI) and deepfakes to deceive victims: the need to rethink current romance fraud prevention messaging

Romance fraud uses the guise of a genuine relationship to deceive a victim for a financial gain. Each year, millions of individuals globally lose money to these approaches. Current prevention messaging focuses heavily on promoting the use of internet searches (specifically reverse image searches) to verify or refute the identity/scenario that one is being presented with. For those who choose to do this, it can be successful and avoid initial financial losses or reduce the overall amount of money lost to an offender. However, as technology evolves, it is likely offenders will alter their methods to deceive victims. This is already evident through the rapid progression and improvements of artificial intelligence and deepfakes to create unique images. This article argues that the adoption of these new techniques requires a need to rethink current prevention messaging, as the utility of conducting reverse image searches may become somewhat redundant into the future.
Autonomous and cooperative control of UAV cluster with multi-agent reinforcement learning

Abstract In this paper, we expolore Multi-Agent Reinforcement Learning (MARL) methods for unmanned aerial vehicle (UAV) cluster. Considering that the current UAV cluster is still in the program control stage, the fully autonomous and intelligent cooperative combat has not been realised. In order to realise the autonomous planning of the UAV cluster according to the changing environment and cooperate with each other to complete the combat goal, we propose a new MARL framework. It adopts the policy of centralised training with decentralised execution, and uses Actor-Critic network to select the execution action and then to make the corresponding evaluation. The new algorithm makes three key improvements on the basis of Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm. The first is to improve learning framework; it makes the calculated Q value more accurate. The second is to add collision avoidance setting, which can increase the operational safety factor. And the third is to adjust reward mechanism; it can effectively improve the cluster’s cooperative ability. Then the improved MADDPG algorithm is tested by performing two conventional combat missions. The simulation results show that the learning efficiency is obviously improved, and the operational safety factor is further increased compared with the previous algorithm.
Pancreas segmentation by two-view feature learning and multi-scale supervision

Automatic organ segmentation systems can accelerate the development of computer-aided diagnosis (CAD) in clinical applications. In this paper, we focus on the challenging pancreas segmentation task. The tiny size, poor contrast, and blurred boundaries of the pancreas make it hard to detect. Current approaches emphasize decomposing this task into subtasks (localization and segmentation) and using the same network to solve different tasks. However, they overestimate the generalization ability of their models. In addition, current methods rely too much on the result of localization. To address these challenges, we propose a novel network by two-view feature learning based on attention mechanism and multi-scale supervision, which we term TVMS-Net. For localization, we adopt Attention Gate (AG) to distinguish appearance features of the pancreas in shallow layers. For segmentation, a simple and effective Residual Multi-Scale Dilated Attention (RMSA) module is designed to extract comprehensive inter-channel relationships and multi-scale spatial information. TVMS-Net is supervised in multi-scale to learn specific-level semantic information. Experimental results on two pancreas datasets show that TVMS-Net obtains remarkable performance. Importantly, TVMS-Net also achieves excellent segmentation accuracy on another tiny organ dataset, i.e., the spleen, which justifies the reliability and robustness of our method.
X-vector anonymization using autoencoders and adversarial training for preserving speech privacy

The rapid increase in web services and mobile apps, which collect personal data from users, has also increased the risk that their privacy may be severely compromised. In particular, the increasing variety of spoken language interfaces and voice assistants empowered by the vertiginous breakthroughs in deep learning have prompted important concerns in the European Union in terms of preserving the privacy of speech data. For instance, an attacker can record speech from users and impersonate them to obtain access to systems that require voice identification. By extracting speaker, linguistic (e.g., dialect), and paralinguistic features (e.g., age) from a speech signal, the speaker profiles can also be hacked from users through existing technology. To mitigate these weaknesses, in this study, we present a speech anonymization method based on autoencoders and adversarial training. Given an utterance, we first extract an x-vector, which is a powerful utterance-level embedding used in state-of-the-art speaker recognition. This original x-vector is transformed by an autoencoder producing a new x-vector, where speaker, gender, and accent information are suppressed through adversarial training. The anonymized speech is finally generated through a neural speech synthesizer driven by the anonymized x-vector, fundamental frequency, and phoneme information extracted from the input speech. For the evaluation, we followed the VoicePrivacy Challenge framework, where anonymization or privacy is measured using automatic speaker verification and the preservation of the intelligibility is evaluated through automatic speech recognition. Our experimental results show that the proposed method achieves higher privacy than the VoicePrivacy baseline (i.e., a higher speaker verification error) while preserving a similar intelligibility for the spoken content (i.e., a similar word error rate).
Open Problems Related to Quantum Query Complexity

I offer a case that quantum query complexity still has loads of enticing and fundamental open problems—from relativized QMA versus QCMA and BQP versus IP , to time/space tradeoffs for collision and element distinctness, to polynomial degree versus quantum query complexity for partial functions, to the Unitary Synthesis Problem and more.
SmartCon: A Blockchain-Based Framework for Smart Contracts and Transaction Management

A smart contract is known to be useful for automating business processes triggered by specific events caused by IoT sensors, data feeds, or other applications. A blockchain-based smart contract management system is an innovative technology that is foreseen to automate future business-to-business (B2B) processes. Blockchain is well-known to play a central role in business process re-engineering by optimizing business workflow operations, especially in multi-party arrangements. This paper presents a multi-organizational smart contract management system in which a user can create, deploy, and execute smart contracts. This paper consists of two parts; in the first part, we have compared existing smart contract management systems based on different characteristics that can play a vital role in selecting a particular system for a specific business need. In the second part, while utilizing and building upon the state-of-the-art techniques, we have built a framework for a blockchain-based smart contract and transaction management system. It is a unified architecture supporting DAO (Decentralized Autonomous Organizations) and organizational level blockchain-based smart contract execution. There are two types of separate blockchains utilized in the proposed framework, i.e., SBlockchain and TBlockchain. SBlockchain is used to store smart contracts, whereas all the data generated by the smart contracts is stored inside the TBlockchain. In addition, each smart contract has some terms and clauses necessary for some event execution. Various components of the framework and their implementation have been described in detail with the help of relevant use-cases.
Blockchain-based secure delivery of medical supplies using drones

The advantages provided by the drones with regards to three dimensional mobility and ease of deployment makes them a viable candidate for 5G and beyond (B5G) networks. Significant amount of research has been conducted on the aspect of networking for using drones as base stations to provide different services. In this work, we deviate from the traditional use of drones to provide connectivity and explore the delivery of products through drones in the context of maintaining social distancing. However, drone delivery process for critical applications such as delivering medical supplies is vulnerable to attacks such as impersonation attacks. The security of drone operation is important to save the users from any breaches that can lead to financial and physical losses. To cope with these security issues and to make the delivery process transparent, we propose a blockchain-based drone delivery system that registers and authenticates the participating entities including products (medical supplies), warehouse (medical centers) and drones. To this end, we utilize Ethereum platform for implementation of blockchain and smart contract and we present an analysis of different factors that influence the authentication process in terms of time and the number of transactions. Furthermore, to make the communication of a drone with command and control center more secure and robust, we use machine learning (ML)-based intrusion detection system.
Semantic Segmentation of High-Resolution Airborne Images with Dual-Stream DeepLabV3+

In geospatial applications such as urban planning and land use management, automatic detection and classification of earth objects are essential and primary subjects. When the significant semantic segmentation algorithms are considered, DeepLabV3+ stands out as a state-of-the-art CNN. Although the DeepLabV3+ model is capable of extracting multi-scale contextual information, there is still a need for multi-stream architectural approaches and different training approaches of the model that can leverage multi-modal geographic datasets. In this study, a new end-to-end dual-stream architecture that considers geospatial imagery was developed based on the DeepLabV3+ architecture. As a result, the spectral datasets other than RGB provided increments in semantic segmentation accuracies when they were used as additional channels to height information. Furthermore, both the given data augmentation and Tversky loss function which is sensitive to imbalanced data accomplished better overall accuracies. Also, it has been shown that the new dual-stream architecture using Potsdam and Vaihingen datasets produced 88.87% and 87.39% overall semantic segmentation accuracies, respectively. Eventually, it was seen that enhancement of the traditional significant semantic segmentation networks has a great potential to provide higher model performances, whereas the contribution of geospatial data as the second stream to RGB to segmentation was explicitly shown.
Attention based automated radiology report generation using CNN and LSTM

The automated generation of radiology reports provides X-rays and has tremendous potential to enhance the clinical diagnosis of diseases in patients. A new research direction is gaining increasing attention that involves the use of hybrid approaches based on natural language processing and computer vision techniques to create auto medical report generation systems. The auto report generator, producing radiology reports, will significantly reduce the burden on doctors and assist them in writing manual reports. Because the sensitivity of chest X-ray (CXR) findings provided by existing techniques not adequately accurate, producing comprehensive explanations for medical photographs remains a difficult task. A novel approach to address this issue was proposed, based on the continuous integration of convolutional neural networks and long short-term memory for detecting diseases, followed by the attention mechanism for sequence generation based on these diseases. Experimental results obtained by using the Indiana University CXR and MIMIC-CXR datasets showed that the proposed model attained the current state-of-the-art efficiency as opposed to other solutions of the baseline. BLEU-1, BLEU-2, BLEU-3, and BLEU-4 were used as the evaluation metrics.
Subwavelength-Structure-Assisted Ultracompact Polarization-Handling Components on Silicon

Silicon photonic integrated circuits with the abilities of large-scale integration and low-cost fabrication are one of the most promising technologies for various emerging applications. For silicon photonic integrated circuits, which are usually polarization-sensitive, polarization-handling is of critical importance as a key manipulation of light. Basically, ultracompact polarization-handling devices with high performances in a broad band are highly desired. In this paper, a review is given for recent progresses of high-performance silicon polarization-handling devices assisted with subwavelength structures (SWSs), which introduce more degrees of freedom (DoFs) in design. In Section II, the fundamental of both regular and disordered SWSs with different DoFs is introduced and discussed. Different kinds of polarization-handling devices are presented in Section III, including polarizers, polarization beam splitters, polarization rotators/polarization-splitter-rotators, dual-polarization devices. Finally, the challenges and the opportunities of silicon polarization-handling devices are discussed in Section IV.
On the Use of Proof-of-Work in Permissioned Blockchains: Security and Fairness

In permissioned blockchains, a set of identifiable miners validates transactions and creates new blocks. In scholarship, the proposed solution for the consensus protocol is usually inspired by the Byzantine fault tolerance (BFT) based on voting rather than the proof-of-work (PoW). The advantage of PoW with respect to BFT is that it allows the final user to evaluate the cost required to change a confirmed transaction without the need to trust the consortium of miners. In this paper, we analyse the problems that arise from the application of PoW in permissioned blockchains. In standard PoW, it may be easy for colluded miners to temporarily reach 50% of the total hash power (HP). Moreover, since mining rewards are not usually expected in permissioned contexts, the problem of balancing the computational efforts among the miners becomes crucial. We propose a solution based on a sliding window algorithm to address these problems and analyse its effectiveness in terms of fairness and security. Furthermore, we present a quantitative, analytical model in order to assess its capacity to balance the hash power provided by heterogeneous miners. Our study considers the trade-off between the need to trust the entire consortium of miners guaranteed by the global HP invested by the mining process and the need to prevent collusion among malicious miners aimed at reaching 50% of the total HP. As a result, the model can be used to find the optimal parameters for the sliding window protocol.
The digital asset value and currency supervision under deep learning and blockchain technology

With the rapid development of digital currencies such as Bitcoin, it is difficult to extract the effective information from massive data and quantify the value of digital assets using current methods. As a key underlying technology, blockchain technology can no longer meet most of the needs of digital currency transactions. Based on this, the digital asset is taken as the research object and an analysis model for digital asset value is established with the deep learning technology in this study. Then, the authorization mechanism in the distributed position and orientation system (DPOS) algorithm is extracted and applied to the precise backward error tracking (PBET) algorithm based on the existing consensus algorithm in blockchain technology. Thus, a dynamic delegated practical byzantine fault tolerance (DDPBFT) algorithm that can be applied to the blockchain is proposed. Finally, a supervisable digital currency system is constructed based on the improved blockchain technology. After specific analysis, it is found that the analysis model for digital asset value based on the deep learning proposed in this study shows good stability and accuracy, and can help enterprises to analyze the value of digital assets. Compared with the existing consensus mechanism algorithms, the proposed DDPBFT algorithm shows better results in terms of throughput and delay. Finally, the supervisable digital currency model based on the improved blockchain technology can unite the public chains, alliance chains, and user wallets, and realize the traceability of transaction information. In short, the quantitative analysis of the value of digital assets has been realized and the supervision of digital currency transactions has been achieved by using the improved blockchain technology.
Continuous Delivery of Blockchain Distributed Applications

Ensuring a production-ready state of the application under development is the imminent feature of the Continuous Delivery (CD) approach. In a blockchain network, nodes communicate and store data in a distributed manner. Each node executes the same business application but operates in a distinct execution environment. The literature lacks research focusing on continuous practices for blockchain and Distributed Ledger Technology (DLT). Specifically, it lacks such works with support for both design and deployment. The author has proposed a solution that takes into account the continuous delivery of a business application to diverse deployment environments in the DLT network. As a result, two continuous delivery pipelines have been implemented using the Jenkins automation server. The first pipeline prepares a business application whereas the second one generates complete node deployment packages. As a result, the framework ensures the deployment package in the actual version of the business application with the node-specific up-to-date version of deployment configuration files. The Smart Contract Design Pattern has been used when building a business application. The modeling aspect of blockchain network installation has required using Unified Modeling Language (UML) and the UML Profile for Distributed Ledger Deployment. The refined model-to-code transformation generates deployment configurations for nodes. Both the business application and deployment configurations are stored in the GitHub repositories. For the sake of verification, tests have been conducted for the electricity consumption and supply management system designed for prosumers of renewable energy.