Reviewerly/exHarmony huggingface dataset:
Segmenting Objects From Relational Visual Data

In this article, we model a set of pixelwise object segmentation tasks - automatic video segmentation (AVS), image co-segmentation (ICS) and few-shot semantic segmentation (FSS) - in a unified view of segmenting objects from relational visual data. To this end, we propose an attentive graph neural network (AGNN) that addresses these tasks in a holistic fashion, by formulating them as a process of iterative information fusion over data graphs. It builds a fully-connected graph to efficiently represent visual data as nodes and relations between data instances as edges. The underlying relations are described by a differentiable attention mechanism, which thoroughly examines fine-grained semantic similarities between all the possible location pairs in two data instances. Through parametric message passing, AGNN is able to capture knowledge from the relational visual data, enabling more accurate object discovery and segmentation. Experiments show that AGNN can automatically highlight primary foreground objects from video sequences (i.e., automatic video segmentation), and extract common objects from noisy collections of semantically related images (i.e., image co-segmentation). AGNN can even generalize segment new categories with little annotated data (i.e., few-shot semantic segmentation). Taken together, our results demonstrate that AGNN provides a powerful tool that is applicable to a wide range of pixel-wise object pattern understanding tasks with relational visual data. Our algorithm implementations have been made publicly available at https://github.com/carrierlxk/AGNN.
RRNet: Relational Reasoning Network With Parallel Multiscale Attention for Salient Object Detection in Optical Remote Sensing Images

Salient object detection (SOD) for optical remote sensing images (RSIs) aims at locating and extracting visually distinctive objects/regions from the optical RSIs. Despite some saliency models were proposed to solve the intrinsic problem of optical RSIs (such as complex background and scale-variant objects), the accuracy and completeness are still unsatisfactory. To this end, we propose a relational reasoning network with parallel multi-scale attention for SOD in optical RSIs in this paper. The relational reasoning module that integrates the spatial and the channel dimensions is designed to infer the semantic relationship by utilizing high-level encoder features, thereby promoting the generation of more complete detection results. The parallel multi-scale attention module is proposed to effectively restore the detail information and address the scale variation of salient objects by using the low-level features refined by multi-scale attention. Extensive experiments on two datasets demonstrate that our proposed RRNet outperforms the existing state-of-the-art SOD competitors both qualitatively and quantitatively.
Incremental Learning in Deep Convolutional Neural Networks Using Partial Network Sharing

Deep convolutional neural network (DCNN) based supervised learning is a widely practiced approach for large-scale image classification. However, retraining these large networks to accommodate new, previously unseen data demands high computational time and energy requirements. Also, previously seen training samples may not be available at the time of retraining. We propose an efficient training methodology and incrementally growing DCNN to learn new tasks while sharing part of the base network. Our proposed methodology is inspired by transfer learning techniques, although it does not forget previously learned tasks. An updated network for learning new set of classes is formed using previously learned convolutional layers (shared from initial part of base network) with addition of few newly added convolutional kernels included in the later layers of the network. We employed a `clone-and-branch' technique which allows the network to learn new tasks one after another without any performance loss in old tasks. We evaluated the proposed scheme on several recognition applications. The classification accuracy achieved by our approach is comparable to the regular incremental learning approach (where networks are updated with new training samples only, without any network sharing), while achieving energy efficiency, reduction in storage requirements, memory access and training time.
Associative Deep Clustering: Training a Classification Network with No Labels

We propose a novel end-to-end clustering training schedule for neural networks that is direct, i.e. the output is a probability distribution over cluster memberships. A neural network maps images to embeddings. We introduce centroid variables that have the same shape as image embeddings. These variables are jointly optimized with the network's parameters. This is achieved by a cost function that associates the centroid variables with embeddings of input images. Finally, an additional layer maps embeddings to logits, allowing for the direct estimation of the respective cluster membership. Unlike other methods, this does not require any additional classifier to be trained on the embeddings in a separate step. The proposed approach achieves state-of-the-art results in unsupervised classification and we provide an extensive ablation study to demonstrate its capabilities.
Quantum Implementation Circuits of Quantum Signal Representation and Type Conversion

Quantum signal processing offers a possible solution to store and process massive signals efficiently. As the foundation of quantum signal processing, quantum signal representation (QSR) that stores color information using basis states, is convenient to retrieve accurately signals from quantum systems. But the existing QSRs are only focus on integer signals (i.e., images) and are difficult to be directly operated by quantum Fourier transform (QFT) and quantum wavelet transform (QWT). Thus, we propose four models of QSR for integer, real, and complex signals. Furthermore, we design quantum implementation circuits of type conversions of QSR with complexity <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$O$ </tex-math></inline-formula> (1). Type conversions of QSR are illustrated by general processes of QFT and Haar QWT, and demonstrate that the proposed QSRs are suitable for QFT and QWT.
Green supplier selection in straw biomass industry based on cloud model and possibility degree

Selecting the supplier is a very critical part to the comprehensive performance of modern enterprises. With the large-scale development of straw biomass industry, green supplier selection has become a key decision-making task which needs to collect and process mass of information, it is necessary to make the supplier to go green. Nevertheless, it is safe to say, in fact, that so far there are a few researches on supplier selection in straw biomass industry. At the same time, some crucial problems urgently needed to be solved exist in present research, as shown below:①decision-making methods are frequently used, even though they have some serious problems, e.g. information loss.②some key index factors are ignored, green supplier selection hasn't got attracted sufficient attention.③little work has been performed on the fuzziness or the uncertainty of index weight. To solve the problems mentioned above, this paper tries to find out some ways to make improvements:①Cloud model is proposed to handle the fuzziness and randomness of evaluation information.②Index system is more comprehensive. Important issues, such as on-time delivery, pollutant emissions per unit fuel(t) and green certification, are paid more attention to.③Fuzzy AHP is applied to determine the index weight, the uncertainty of criteria and sub-criteria are considered, the vagueness of human thought is dealt with.④A decision framework based on Cloud model and possibility degree is put forward to guide the optimal selection of green supplier selection.⑤A Chinese case is carried out and related sensitivity analysis is performed. The results show that the proposed novel model not only can find the more suitable green supplier, but also reveal the big gap between alternatives clearly. The strength of the proposed new decision framework is proved.
Blockchain-based searchable symmetric encryption scheme

The mechanism for traditional searchable symmetric encryption (SSE) is pay-then-use. This situation is not fair to user because the cloud server may return wrong results. Besides, the user needs to verify these results locally. In order to ensure fairness and reduce user’s calculations, we combined blockchain with SSE, and proposed a fair SSE scheme based on blockchain. Our scheme can guarantee fairness for both parties. That is, if the user is not honest, he cannot get right results from the server, and at the same time the server cannot get any information related to the plaintexts during this search process. If the server is not honest, except for the service charge, it will be punished automatically. Moreover, the user in our scheme does not need to verify the results locally. The security and performance analyses showed our scheme was semantic secure and feasible.
Attentive Long Short-Term Preference Modeling for Personalized Product Search

E-commerce users may expect different products even for the same query, due to their diverse personal preferences. It is well known that there are two types of preferences: long-term ones and short-term ones. The former refers to users’ inherent purchasing bias and evolves slowly. By contrast, the latter reflects users’ purchasing inclination in a relatively short period. They both affect users’ current purchasing intentions. However, few research efforts have been dedicated to jointly model them for the personalized product search. To this end, we propose a novel Attentive Long Short-Term Preference model, dubbed as ALSTP, for personalized product search. Our model adopts the neural networks approach to learn and integrate the long- and short-term user preferences with the current query for the personalized product search. In particular, two attention networks are designed to distinguish which factors in the short-term as well as long-term user preferences are more relevant to the current query. This unique design enables our model to capture users’ current search intentions more accurately. Our work is the first to apply attention mechanisms to integrate both long- and short-term user preferences with the given query for the personalized search. Extensive experiments over four Amazon product datasets show that our model significantly outperforms several state-of-the-art product search methods in terms of different evaluation metrics.
Observer Design of Discrete-Time Fuzzy Systems Based on an Alterable Weights Method

This paper proposes an improvement on observer design of discrete-time fuzzy systems based on an alterable weights method. Different from the recent result, a more effective ranking-based switching mechanism is developed by introducing a bank of alterable weights for the sake of making use of the size difference information of the normalized fuzzy weighting functions more freely than before. Therefore, a positive result can be provided in this paper, that is, less conservative conditions of designing feasible fuzzy observers can be obtained than those existing results, while the computational cost of designing feasible fuzzy observers is even less than the up-to-date one. Finally, two numerical examples are given to show the progressiveness of the proposed method.
Summarizing Videos with Attention

In this work we propose a novel method for supervised, keyshots based video summarization by applying a conceptually simple and computationally efficient soft, self-attention mechanism. Current state of the art methods leverage bi-directional recurrent networks such as BiLSTM combined with attention. These networks are complex to implement and computationally demanding compared to fully connected networks. To that end we propose a simple, self-attention based network for video summarization which performs the entire sequence to sequence transformation in a single feed forward pass and single backward pass during training. Our method sets a new state of the art results on two benchmarks TvSum and SumMe, commonly used in this domain.
Effective Repair Strategy Against Advanced Persistent Threat: A Differential Game Approach

Advanced persistent threat (APT) is a new kind of cyberattack that poses a serious threat to modern society. When an APT campaign on an organization has been identified, the available repair resources must be reasonably allocated to the potentially insecure hosts to mitigate the potential loss of the organization. We refer to the feasible repair resource allocation strategies as repair strategies. This paper focuses on the APT repair problem, i.e., the problem of developing effective repair strategies for organizations. First, for an organization with time-varying communication relationship, we establish an evolution model of the organization's expected state, in which the impact of lateral movement of APT is accommodated. On this basis, we model the APT repair problem as a differential Nash game problem (the APT repair game) in which the attacker attempts to maximize his potential benefit, and the organization manages to minimize its potential loss. Second, we derive a system (the potential system) for calculating a potential Nash equilibrium of an APT repair game, and we examine the structure of the potential attack and repair strategies in a potential Nash equilibrium. Next, we solve some potential systems to get the corresponding potential Nash equilibria. Finally, by comparison with a large number of randomly generated attack and repair strategies, we conclude that the potential Nash equilibrium of each APT repair game is a Nash equilibrium of the game. Therefore, we recommend to organizations their respective potential repair strategies. Our findings help to better understand and effectively defend against APT.
An automatic traffic density estimation using Single Shot Detection (SSD) and MobileNet-SSD

Traffic density estimation is a very important component of an automated traffic monitoring system. Traffic density estimation can be used in a number of traffic applications – from congestion identification to macroscopic traffic control in urban environment. In this paper, we implemented Single Shot Detection (SSD) and MobileNet-SSD to estimate traffic density. SSD is capable of handling different shape, size and view angle of the objects. MobileNet-SSD is a cross-trained model from SSD to MobileNet architecture, which is faster than SSD. In this study, we show a key application area for the SSD and MobileNet-SSD framework. The advantages and shortcomings of the SSD and MobileNet-SSD framework were analyzed using fifty-nine individual traffic cameras. In addition, we compared the two algorithms with manually estimated density. The SSD framework shows significant potential in the field of traffic density estimation. SSD achieved 92.97% average detection accuracy in our experiment. On the other hand, the MobileNet-SSD achieved 79.30% average detection accuracy.
Is Blockchain Technology Suitable for Managing Personal Health Records? Mixed-Methods Study to Test Feasibility

There are many perspectives on the advantages of introducing blockchain in the medical field, but there are no published feasibility studies regarding the storage, propagation, and management of personal health records (PHRs) using blockchain technology.The purpose of this study was to investigate the usefulness of blockchains in the medical field in relation to transactions with and propagation of PHRs in a private blockchain.We constructed a private blockchain network using Ethereum version 1.8.4 and conducted verification using the de-identified PHRs of 300 patients. The private blockchain network consisted of one hospital node and 300 patient nodes. In order to verify the effectiveness of blockchain-based PHR management, PHRs at a time were loaded in a transaction between the hospital and patient nodes and propagated to the whole network. We obtained and analyzed the time and gas required for data transaction and propagation on the blockchain network. For reproducibility, these processes were repeated 100 times.Of 300 patient records, 74 (24.7%) were not loaded in the private blockchain due to the data block size of the transaction block. The remaining 226 individual health records were classified into groups A (80 patients with outpatient visit data less than 1 year old), B (84 patients with outpatient data from between 1 and 3 years before data collection), and C (62 patients with outpatient data 3 to 5 years old). With respect to mean transaction time in the blockchain, C (128.7 seconds) had the shortest time, followed by A (132.2 seconds) and then B (159.0 seconds). The mean propagation times for groups A, B, and C were 1494.2 seconds, 2138.9 seconds, and 4111.4 seconds, respectively; mean file sizes were 5.6 KB, 18.6 KB, and 45.38 KB, respectively. The mean gas consumption values were 1,900,767; 4,224,341; and 4,112,784 for groups A, B, and C, respectively.This study confirms that it is possible to exchange PHR data in a private blockchain network. However, to develop a blockchain-based PHR platform that can be used in practice, many improvements are required, including reductions in data size, improved personal information protection, and reduced operating costs.
End-to-End Automatic Image Annotation Based on Deep CNN and Multi-Label Data Augmentation

Automatic image annotation is a key step in image retrieval and image understanding. In this paper, we present an end-to-end automatic image annotation method based on a deep convolutional neural network (CNN) and multi-label data augmentation. Different from traditional annotation models that usually perform feature extraction and annotation as two independent tasks, we propose an end-to-end automatic image annotation model based on deep CNN (E2E-DCNN). E2E-DCNN transforms the image annotation problem into a multi-label learning problem. It uses a deep CNN structure to carry out the adaptive feature learning before constructing the end-to-end annotation structure using multiple cross-entropy loss functions for training. It is difficult to train a deep CNN model using small-scale datasets or scale up multi-label datasets using traditional data augmentation methods; hence, we propose a multi-label data augmentation method based on Wasserstein generative adversarial networks (ML-WGAN). The ML-WGAN generator can approximate the data distribution of a single multi-label image. The images generated by ML-WGAN can assist in the reduction of the over-fitting problem of training a deep CNN model and enhance the generalization ability of the trained CNN model. We optimize the network structure by using deformable convolution and spatial pyramid pooling. We experiment the proposed E2E-DCNN model with data augmentation by the proposed ML-WGAN on several public datasets. The experimental results demonstrate that the proposed model outperforms the state-of-the-art automatic image annotation models.
Causal Interactive Links Between Presence and Fear in Virtual Reality Height Exposure

Virtual reality plays an increasingly important role in research and therapy of pathological fear. However, the mechanisms how virtual environments elicit and modify fear responses are not yet fully understood. Presence, a psychological construct referring to the 'sense of being there' in a virtual environment, is widely assumed to crucially influence the strength of the elicited fear responses, however, causality is still under debate. The present study is the first that experimentally manipulated both variables to unravel the causal link between presence and fear responses. Height-fearful participants (N = 49) were immersed into a virtual height situation and a neutral control situation (fear manipulation) with either high versus low sensory realism (presence manipulation). Ratings of presence and verbal and physiological (skin conductance, heart rate) fear responses were recorded. Results revealed an effect of the fear manipulation on presence, i.e., higher presence ratings in the height situation compared to the neutral control situation, but no effect of the presence manipulation on fear responses. However, the presence ratings during the first exposure to the high quality neutral environment were predictive of later fear responses in the height situation. Our findings support the hypothesis that experiencing emotional responses in a virtual environment leads to a stronger feeling of being there, i.e., increase presence. In contrast, the effects of presence on fear seem to be more complex: on the one hand, increased presence due to the quality of the virtual environment did not influence fear; on the other hand, presence variability that likely stemmed from differences in user characteristics did predict later fear responses. These findings underscore the importance of user characteristics in the emergence of presence.
Bidirectional Convolutional LSTM for the Detection of Violence in Videos

The field of action recognition has gained tremendous traction in recent years. A subset of this, detection of violent activity in videos, is of great importance, particularly in unmanned surveillance or crowd footage videos. In this work, we explore this problem on three standard benchmarks widely used for violence detection: the Hockey Fights, Movies, and Violent Flows datasets. To this end, we introduce a Spatiotemporal Encoder, built on the Bidirectional Convolutional LSTM (BiConvLSTM) architecture. The addition of bidirectional temporal encodings and an elementwise max pooling of these encodings in the Spatiotemporal Encoder is novel in the field of violence detection. This addition is motivated by a desire to derive better video representations via leveraging long-range information in both temporal directions of the video. We find that the Spatiotemporal network is comparable in performance with existing methods for all of the above datasets. A simplified version of this network, the Spatial Encoder is sufficient to match state-of-the-art performance on the Hockey Fights and Movies datasets. However, on the Violent Flows dataset, the Spatiotemporal Encoder outperforms the Spatial Encoder.
Multiple classifiers fusion and CNN feature extraction for handwritten digits recognition

Handwritten digits recognition has been treated as a multi-class classification problem in the machine learning context, where each of the ten digits (0–9) is viewed as a class and the machine learning task is essentially to train a classifier that can effectively discriminate the ten classes. In practice, it is very usual that the performance of a single classifier trained using a standard learning algorithm is varied on different datasets, which indicates that the same learning algorithm may train strong classifiers on some datasets but weak classifiers may be trained on other datasets. It is also possible that the same classifier shows different performance on different test sets, especially when considering the case that image instances can be highly diverse due to the different handwriting styles of different people on the same digits. To address the above issue, development of ensemble learning approaches have been very necessary to improve the overall performance and make the performance more stable on different datasets. In this paper, we propose a framework that involves CNN-based feature extraction from the MINST dataset and algebraic fusion of multiple classifiers trained on different feature sets, which are prepared through feature selection applied to the original feature set extracted using CNN. The experimental results show that the classifiers fusion can achieve the classification accuracy of ≥ 98%.