Reviewerly/exHarmony huggingface dataset:
Extracting Relational Explanations From Deep Neural Networks: A Survey From a Neural-Symbolic Perspective

The term “explainable AI” refers to the goal of producing artificially intelligent agents that are capable of providing explanations for their decisions. Some models (e.g., rule-based systems) are designed to be explainable, while others are less explicit “black boxes” for which their reasoning remains a mystery. One example of the latter is the neural network, and over the past few decades, researchers in the field of neural-symbolic integration (NSI) have sought to extract relational knowledge from such networks. Extraction from deep neural networks, however, has remained a challenge until recent years in which many methods of extracting distinct, salient features from input or hidden feature spaces of deep neural networks have been proposed. Furthermore, methods of identifying relationships between these features have also emerged. This article presents examples of old and new developments in extracting relational explanations in order to argue that the latter have analogies in the former and, as such, can be described in terms of long-established taxonomies and frameworks presented in early neural-symbolic literature. We also outline potential future research directions that come to light from this refreshed perspective.
A systematic literature review on requirement prioritization techniques and their empirical evaluation

[Context and Motivation] Many requirements prioritization approaches have been proposed, however not all of them have been investigated empirically in real-life settings. As a result, our knowledge of their applicability and actual use is incomplete. [Question/problem] A 2007 systematic review on requirements prioritization mapped out the landscape of proposed prioritization approaches and their prioritization criteria. To understand how this sub-field of requirements engineering has developed since 2007 and what evidence has been accumulated through empirical evaluations, we carried out a literature review that takes as input publications published between 2007 and 2019. [Principle ideas/results] We evaluated 102 papers that proposed and/or evaluated requirements prioritization methods. Our results show that the newly proposed requirements prioritization methods tend to use as basis fuzzy logic and machine learning algorithms. We also concluded that the Analytical Hierarchy Process is the most accurate and extensively used requirement prioritization method in industry. However, scalability is still its major limitation when requirements are large in number. We have found that machine learning has shown potential to deal with this limitation. Last, we found that experiments were the most used research method to evaluate the various aspects of the proposed prioritization approaches. [Contribution] This paper identified and evaluated requirements prioritization techniques proposed between 2007 and 2019, and derived some trends. Limitations of the proposals and implications for research and practice are identified as well.
Detecting Visual Relationships Using Box Attention

We propose a new model for detecting visual relationships, such as "person riding motorcycle" or "bottle on table". This task is an important step towards comprehensive structured mage understanding, going beyond detecting individual objects. Our main novelty is a Box Attention mechanism that allows to model pairwise interactions between objects using standard object detection pipelines. The resulting model is conceptually clean, expressive and relies on well-justified training and prediction procedures. Moreover, unlike previously proposed approaches, our model does not introduce any additional complex components or hyperparameters on top of those already required by the underlying detection model. We conduct an experimental evaluation on two datasets, V-COCO and Open Images, demonstrating strong quantitative and qualitative results.
SRHandNet: Real-Time 2D Hand Pose Estimation With Simultaneous Region Localization

This paper introduces a novel method for real-time 2D hand pose estimation from monocular color images, which is named as SRHandNet. Existing methods can not time efficiently obtain appropriate results for small hand. Our key idea is to simultaneously regress the hand region of interests (RoIs) and hand keypoints for a given color image, and iteratively take the hand RoIs as feedback information for boosting the performance of hand keypoints estimation with a single encoder-decoder network architecture. Different from previous region proposal network (RPN), a new lightweight bounding box representation, which is called region map, is proposed. The proposed bounding box representation map together with hand keypoints heatmaps are combined into the unified multi-channel feature maps, which can be easily acquired with only one forward network inference and thus improve the runtime efficiency of the network. Our proposed SRHandNet can run at 40fps for hand bounding box detection and up to 30fps accurate hand keypoints estimation under the desktop environment without implementation optimization. Experiments demonstrate the effectiveness of the proposed method. State-of-the-art results are also achieved out competing all recent methods.
Bidirectional Attention-Recognition Model for Fine-Grained Object Classification

Fine-grained object classification (FGOC) is a challenging research topic in multimedia computing with machine learning, which faces two pivotal conundrums: focusing attention on the discriminate part regions, and then processing recognition with the part-based features. Existing approaches generally adopt a unidirectional two-step structure, that first locate the discriminate parts and then recognize the part-based features. However, they neglect the truth that part localization and feature recognition can be reinforced in a bidirectional process. In this paper, we propose a novel bidirectional attention-recognition model (BARM) to actualize the bidirectional reinforcement for FGOC. The proposed BARM consists of one attention agent for discriminate part regions proposing and one recognition agent for feature extraction and recognition. Meanwhile, a feedback flow is creatively established to optimize the attention agent directly by recognition agent. Therefore, in BARM the attention agent and the recognition agent can reinforce each other in a bidirectional way and the overall framework can be trained end-to-end without neither object nor parts annotations. Moreover, a novel Multiple Random Erasing data augmentation is proposed, and it exhibits impressive pertinency and superiority for FGOC. Conducted on several extensive FGOC benchmarks, BARM outperforms the present state-of-the-art methods in classification accuracy. Furthermore, BARM exhibits a clear interpretability and keeps consistent with the human perception in visualization experiments.
SIFT-aided path-independent digital image correlation accelerated by parallel computing

Current iterative digital image correlation (DIC) algorithms can efficiently converge at the deformation vector with high accuracy when they are fed with reliable initial guess. Thus, the adaptability of DIC method is dominated to a large extent by the estimation of initial guess. In recent years, image feature-based technique, especially the scale-invariant feature transform (SIFT), was introduced to DIC for the estimation of initial guess in the case of large and complex deformation, due to its robustness in handling the images with translation, rotation, scaling, and localized distortion. However, feature extraction and matching in SIFT are very time consuming, which limits the applications of the SIFT-aided DIC. In this study, we developed a SIFT-aided path-independent DIC method and accelerated it by introducing the parallel computing on graphics processing unit (GPU) or multi-core CPU. In our method, SIFT features are used to estimate the initial guess for the inverse compositional Gauss-Newton (IC-GN) algorithm at each point of interest (POI). The experimental study shows that the developed method can deal with large and inhomogeneous deformation with high accuracy. Parallel computing (especially on GPU) accelerates significantly the proposed DIC method. The achieved computation speed satisfies the need for real-time processing with high resolution for the images of normal sizes.
Implementation and practical problems of chaos-based cryptography revisited

Chaos-based cryptography, since its inception, has become a widely published subject. Despite the vast amount of contributions in the area, its applications in real-world scenarios are minimal as compared to conventional cryptography. Chaotic maps have been used in the design of cryptosystems because they depict desirable characteristics such as pseudorandomness, complexity, and sensitivity to parameter changes. Despite these characteristics being analogous to cryptographic requirements, the resulting chaos-based cryptosystems are usually difficult to analyze, inefficient, and have reproducibility issues. In this paper, we highlight some of the problems which deter the practical application of chaos-based cryptosystems. We show that recently published work in reputable journals still do not address these problems and remain only of academic interest. We also perform experiments to depict some of the implementation issues of digital chaos that need to be taken into consideration when designing chaos-based algorithms. We then discuss a number of possible solutions that can be explored to overcome these problems.
A new binary grasshopper optimization algorithm for feature selection problem

The grasshopper optimization algorithm is one of the recently population-based optimization techniques inspired by the behaviours of grasshoppers in nature. It is an efficient optimization algorithm and since demonstrates excellent performance in solving continuous problems, but cannot resolve directly binary optimization problems. Many optimization problems have been modelled as binary problems since their decision variables varied in binary space such as feature selection in data classification. The main goal of feature selection is to find a small size subset of feature from a sizeable original set of features that optimize the classification accuracy. In this paper, a new binary variant of the grasshopper optimization algorithm is proposed and used for the feature subset selection problem. This proposed new binary grasshopper optimization algorithm is tested and compared to five well-known swarm-based algorithms used in feature selection problem. All these algorithms are implemented and experimented assessed on twenty data sets with various sizes. The results demonstrated that the proposed approach could outperform the other tested methods.
Object-Aware Instance Labeling for Weakly Supervised Object Detection

Weakly supervised object detection (WSOD), where a detector is trained with only image-level annotations, is attracting more and more attention. As a method to obtain a well-performing detector, the detector and the instance labels are updated iteratively. In this study, for more efficient iterative updating, we focus on the instance labeling problem, a problem of which label should be annotated to each region based on the last localization result. Instead of simply labeling the top-scoring region and its highly overlapping regions as positive and others as negative, we propose more effective instance labeling methods as follows. First, to solve the problem that regions covering only some parts of the object tend to be labeled as positive, we find regions covering the whole object focusing on the context classification loss. Second, considering the situation where the other objects contained in the image can be labeled as negative, we impose a spatial restriction on regions labeled as negative. Using these instance labeling methods, we train the detector on the PASCAL VOC 2007 and 2012 and obtain significantly improved results compared with other state-of-the-art approaches.
Spatial Residual Layer and Dense Connection Block Enhanced Spatial Temporal Graph Convolutional Network for Skeleton-Based Action Recognition

Recent research has shown that modeling the dynamic joint features of the human body by a graph convolutional network (GCN) is a groundbreaking approach for skeleton-based action recognition, especially for the recognition of the body-motion, human-object and human-human interactions. Nevertheless, how to model and utilize coherent skeleton information comprehensively is still an open problem. In order to capture the rich spatiotemporal information and utilize features more effectively, we introduce a spatial residual layer and a dense connection block enhanced spatial temporal graph convolutional network. More specifically, our work introduces three aspects. Firstly, we extend spatial graph convolution to spatial temporal graph convolution of cross-domain residual to extract more precise and informative spatiotemporal feature, and reduce the training complexity by feature fusion in the, so-called, spatial residual layer. Secondly, instead of simply superimposing multiple similar layers, we use dense connection to take full advantage of the global information. Thirdly, we combine the above mentioned two components to create a spatial temporal graph convolutional network (ST-GCN), referred to as SDGCN. The proposed graph representation has a new structure. We perform extensive experiments on two large datasets: Kinetics and NTU-RGB+D. Our method achieves a great improvement in performance compared to the mainstream methods. We evaluate our method quantitatively and qualitatively, thus proving its effectiveness.
MMAct: A Large-Scale Dataset for Cross Modal Human Action Understanding

Unlike vision modalities, body-worn sensors or passive sensing can avoid the failure of action understanding in vision related challenges, e.g. occlusion and appearance variation. However, a standard large-scale dataset does not exist, in which different types of modalities across vision and sensors are integrated. To address the disadvantage of vision-based modalities and push towards multi/cross modal action understanding, this paper introduces a new large-scale dataset recorded from 20 distinct subjects with seven different types of modalities: RGB videos, keypoints, acceleration, gyroscope, orientation, Wi-Fi and pressure signal. The dataset consists of more than 36k video clips for 37 action classes covering a wide range of daily life activities such as desktop-related and check-in-based ones in four different distinct scenarios. On the basis of our dataset, we propose a novel multi modality distillation model with attention mechanism to realize an adaptive knowledge transfer from sensor-based modalities to vision-based modalities. The proposed model significantly improves performance of action recognition compared to models trained with only RGB information. The experimental results confirm the effectiveness of our model on cross-subject, -view, -scene and -session evaluation criteria. We believe that this new large-scale multimodal dataset will contribute the community of multimodal based action understanding.
FDA: Feature Disruptive Attack

Though Deep Neural Networks (DNN) show excellent performance across various computer vision tasks, several works show their vulnerability to adversarial samples, i.e., image samples with imperceptible noise engineered to manipulate the network's prediction. Adversarial sample generation methods range from simple to complex optimization techniques. Majority of these methods generate adversaries through optimization objectives that are tied to the pre-softmax or softmax output of the network. In this work we, (i) show the drawbacks of such attacks, (ii) propose two new evaluation metrics: Old Label New Rank (OLNR) and New Label Old Rank (NLOR) in order to quantify the extent of damage made by an attack, and (iii) propose a new attack FDA: Feature Disruptive attack, to address the drawbacks of existing attacks. FDA works by generating image perturbation that disrupts features at each layer of the network and causes deep-features to be highly corrupt. This allows FDA adversaries to severely reduce the performance of deep networks. We experimentally validate that FDA generates stronger adversaries than other state-of-the-art methods for Image classification, even in the presence of various defense measures. More importantly, we show that FDA disrupts feature-representation based tasks even without access to the task-specific network or methodology.
Deanonymizing Tor hidden service users through Bitcoin transactions analysis

With the rapid increase of threats on the Internet, people are continuously seeking privacy and anonymity. Services such as Bitcoin and Tor were introduced to provide anonymity for online transactions and Web browsing. Due to its pseudonymity model, Bitcoin lacks retroactive operational security, which means historical pieces of information could be used to identify a certain user. By exploiting publicly available information, we show how relying on Bitcoin for payments on Tor hidden services could lead to deanonymization of these services’ users. Such linking is possible by finding at least one past transaction in the Blockchain that involves their publicly declared Bitcoin addresses. To demonstrate the consequences of this deanonymization approach, we carried out a real-world experiment simulating a passive, limited adversary. We crawled 1.5K hidden services and collected 88 unique and active Bitcoin addresses. We then crawled 5B tweets and 1M BitcoinTalk forum pages and collected 4.2K and 41K unique Bitcoin addresses, respectively. Each user address was associated with an online identity along with its public profile information. By analyzing the transactions in the Blockchain, we were able to link 125 unique users to 20 hidden services, including sensitive ones, such as The Pirate Bay and Silk Road. We also analyzed two case studies in detail to demonstrate the implications of the information leakage on users anonymity. In particular, we confirm that Bitcoin addresses should be considered exploitable, as they can be used to deanonymize users retroactively. This is especially important for Tor hidden service users who actively seek and expect privacy and anonymity.
Virtual reality experiences, attachment and experiential outcomes in tourism

Purpose The purpose of this study is to examine the structural relationships between the dimensions of virtual reality (VR) experiences (immersion, interaction, usability and illusion), the dimensions of VR attachment (VR dependence, VR identity, VR affect and VR familiarity) and the dimensions of VR experiential outcomes (VR experiential satisfaction, VR experiential loyalty and VR experiential advocacy). Design/methodology/approach A total of 490 visitors who tried 360 VR travel video games in the TripMoment VR were surveyed. Findings In this study, there are positive relationships between VR experiential satisfaction and immersion, interaction, illusion, as well as VR identity, VR affect and VR familiarity. In addition, VR experiential satisfaction has a positive relationship with VR experiential loyalty. In addition, VR experiential advocacy has a positive relationship with both VR experiential satisfaction and VR experiential loyalty. Practical implications The results show that tourism operators should focus on increasing VR experiential satisfaction and experiential loyalty to enable visitors to have intentions to advocate their VR experiences. Originality/value This paper provides the data that lead to a better understanding of the relationship between the dimensions of VR experiences, VR attachment and VR experiential outcomes in the tourism industry.
A Survey of Exploitation and Detection Methods of XSS Vulnerabilities

As web applications become more prevalent, web security becomes more and more important.Cross-site scripting vulnerability abbreviated as XSS is a kind of common injection web vulnerability.The exploitation of XSS vulnerabilities can hijack users' sessions, modify, read and delete business data of web applications, place malicious codes in web applications, and control victims to attack other targeted servers.This paper discusses classification of XSS, and designs a demo website to demonstrate attack processes of common XSS exploitation scenarios.The paper also compares and analyzes recent research results on XSS detection, divides them into three categories according to different mechanisms.The three categories are static analysis methods, dynamic analysis methods and hybrid analysis methods.The paper classifies 30 detection methods into above three categories, makes overall comparative analysis among them, lists their strengths and weaknesses and detected XSS vulnerability types.In the end, the paper explores some ways to prevent XSS vulnerabilities from being exploited.
Echo state network optimization using binary grey wolf algorithm

The echo state network (ESN) is a powerful recurrent neural network for time series modelling. ESN inherits the simplified structure and relatively straightforward training process of conventional neural networks, and shows strong computational capabilities to solve nonlinear problems. It is able to map low-dimensional input signals to high-dimensional space for information extraction, but it is found that not every dimension of the reservoir output directly contributes to the model generalization. This work aims to improve the generalization capabilities of the ESN model by reducing the redundant reservoir output features. A novel hybrid model, namely binary grey wolf echo state network (BGWO-ESN), is proposed which optimises the ESN output connection by the feature selection scheme. Specially, the feature selection scheme of BGWO is developed to improve the ESN output connection structure. The proposed method is evaluated using synthetic and financial data sets. Experimental results demonstrate that the proposed BGWO-ESN model is more effective than other benchmarks, and obtains the lowest generalization error.
Emotion AI-Driven Sentiment Analysis: A Survey, Future Research Directions, and Open Issues

The essential use of natural language processing is to analyze the sentiment of the author via the context. This sentiment analysis (SA) is said to determine the exactness of the underlying emotion in the context. It has been used in several subject areas such as stock market prediction, social media data on product reviews, psychology, judiciary, forecasting, disease prediction, agriculture, etc. Many researchers have worked on these areas and have produced significant results. These outcomes are beneficial in their respective fields, as they help to understand the overall summary in a short time. Furthermore, SA helps in understanding actual feedback shared across different platforms such as Amazon, TripAdvisor, etc. The main objective of this thorough survey was to analyze some of the essential studies done so far and to provide an overview of SA models in the area of emotion AI-driven SA. In addition, this paper offers a review of ontology-based SA and lexicon-based SA along with machine learning models that are used to analyze the sentiment of the given context. Furthermore, this work also discusses different neural network-based approaches for analyzing sentiment. Finally, these different approaches were also analyzed with sample data collected from Twitter. Among the four approaches considered in each domain, the aspect-based ontology method produced 83% accuracy among the ontology-based SAs, the term frequency approach produced 85% accuracy in the lexicon-based analysis, and the support vector machine-based approach achieved 90% accuracy among the other machine learning-based approaches.