Reviewerly/exHarmony huggingface dataset:
Deep learning-based classification of earthquake-impacted buildings using textual damage descriptions

To make informed decisions and have a well-coordinated response to a major hazard event, a rapid assessment of the spatial distribution and severity of building damage is needed. To this end, the long short-term memory (LSTM) deep learning method is applied to classify building damage based on textual descriptions of damage. The damaged state of an individual building is classified using the ATC-20 tags (red, yellow and green). The application of the LSTM approach is demonstrated using building damage descriptions recorded following the 2014 South Napa, California earthquake. The dataset, which consists of 3423 buildings (1552 green tagged, 1674 yellow tagged, 197 red tagged), is randomly divided into training and testing subsets. A predictive model is established using the training set and the performance of this model is evaluated with the test set. An overall accuracy of 86% is achieved when the deep learning model is used to identify the ATC-20 tags for the test set. Despite some noted limitations, the study highlights the overall potential of using the LSTM method to rapidly assess building cluster damage using textual information, which can be generated by building professionals, stakeholders or social media platforms.
Multi-features descriptors for Human Activity Tracking and Recognition in Indoor-Outdoor Environments

Tracking human activities and analyzing their effect in real life settings has become a task of high interest within the computer vision field as it enable many industrial and commercial applications. Sensors and communication technologies are being used for capturing human movements and providing interactive interfaces for man-machine collaboration. However, introducing intelligent computing for better scene understanding is still an unexplored domain for the researchers. In this paper, we have made an effort to allow machines understand the behaviors in outer environment by proposing a novel methodology to recognize human interactions. The objective of this research is to embed cognitive processes in information technologies for exploring new directions of intelligent media. Our proposed human activity recognition (HAR) system recognize eight complex human activities taken from BIT- Interaction dataset i.e. bow, boxing, handshake, high-five, hug, kick, pat and push. We have designed multiple features algorithms along with convolutional neural network (CNN) to evaluate the performance of our system compared with other state of the art classifiers. While, experimental results show that the proposed methodology is reliable at complex realistic settings and applicable in security systems, e-learning, smart surveillance, violence detection, child abuse protection, elderly care, virtual games, intelligent video retrievals and human computer interaction.
Lightweight Privacy-Preserving Ensemble Classification for Face Recognition

The development of machine learning technology and visual sensors is promoting the wider applications of face recognition into our daily life. However, if the face features in the servers are abused by the adversary, our privacy and wealth can be faced with great threat. Many security experts have pointed out that, by 3-D-printing technology, the adversary can utilize the leaked face feature data to masquerade others and break the E-bank accounts. Therefore, in this paper, we propose a lightweight privacy-preserving adaptive boosting (AdaBoost) classification framework for face recognition (POR) based on the additive secret sharing and edge computing. First, we improve the current additive secret sharing-based exponentiation and logarithm functions by expanding the effective input range. Then, by utilizing the protocols, two edge servers are deployed to cooperatively complete the ensemble classification of AdaBoost for face recognition. The application of edge computing ensures the efficiency and robustness of POR. Furthermore, we prove the correctness and security of our protocols by theoretic analysis. And experiment results show that, POR can reduce about 58% computation error compared with the existing differential privacy-based framework.
The Case For In-Network Computing On Demand

Programmable network hardware can run services traditionally deployed on servers, resulting in orders-of-magnitude improvements in performance. Yet, despite these performance improvements, network operators remain skeptical of in-network computing. The conventional wisdom is that the operational costs from increased power consumption outweigh any performance benefits. Unless in-network computing can justify its costs, it will be disregarded as yet another academic exercise.
RNN based online handwritten word recognition in Devanagari and Bengali scripts using horizontal zoning

Devanagari and Bengali scripts are two of the most popular scripts in India. Most of the existing word recognition studies in these two scripts have relied upon the widely used Hidden Markov Model (HMM), in spite of its familiar shortcomings. The existing works were evaluated against and performed well in their chosen metrics. But, the existing word recognition systems in these two scripts could not achieve more than 90% recognition accuracy. This article proposes a novel approach for online handwritten cursive and non-cursive word recognition in Devanagari and Bengali scripts based on two recently developed models of Recurrent Neural Network (RNN)—Long–Short Term Memory (LSTM) and Bidirectional Long–Short Term Memory (BLSTM). The proposed approach divides each word horizontally into three zones—upper, middle, and lower, to reduce the variations in basic stroke order within a word. Next, the word portions from middle zone are re-segmented into its basic strokes. Various structural and directional features are then extracted from each basic stroke of the word separately for each zone. These zone wise basic stroke features are then studied using both LSTM and BLSTM versions of RNN. Most of the existing word recognition systems in these two scripts have followed word based class labelling approach, whereas proposed system has followed the basic stroke based class labelling approach. An exhaustive experiment on large datasets has been performed to evaluate the performance of the proposed approach using both RNN and HMM to make a comparative performance analysis. Experimental results show that the proposed RNN based system is superior over HMM achieving 99.50% and 95.24% accuracies in Devanagari and Bengali scripts respectively and outperforms existing HMM based systems in the literature as well.
Deep Reinforcement Learning Using Genetic Algorithm for Parameter Optimization

Reinforcement learning (RL) enables agents to take decision based on a reward function. However, in the process of learning, the choice of values for learning algorithm parameters can significantly impact the overall learning process. In this paper, we use a genetic algorithm (GA) to find the values of parameters used in Deep Deterministic Policy Gradient (DDPG) combined with Hindsight Experience Replay (HER), to help speed up the learning agent. We used this method on fetch-reach, slide, push, pick and place, and door opening in robotic manipulation tasks. Our experimental evaluation shows that our method leads to better performance, faster than the original algorithm.
Medical Image Retrieval Based on Convolutional Neural Network and Supervised Hashing

In recent years, with extensive application in image retrieval and other tasks, a convolutional neural network (CNN) has achieved outstanding performance. In this paper, a new content-based medical image retrieval (CBMIR) framework using CNN and hash coding is proposed. The new framework adopts a Siamese network in which pairs of images are used as inputs, and a model is learned to make images belonging to the same class have similar features by using weight sharing and a contrastive loss function. In each branch of the network, CNN is adapted to extract features, followed by hash mapping, which is used to reduce the dimensionality of feature vectors. In the training process, a new loss function is designed to make the feature vectors more distinguishable, and a regularization term is added to encourage the real value outputs to approximate the desired binary values. In the retrieval phase, the compact binary hash code of the query image is achieved from the trained network and is subsequently compared with the hash codes of the database images. We experimented on two medical image datasets: the cancer imaging archive-computed tomography (TCIA-CT) and the vision and image analysis group/international early lung cancer action program (VIA/I-ELCAP). The results indicate that our method is superior to existing hash methods and CNN methods. Compared with the traditional hashing method, feature extraction based on CNN has advantages. The proposed algorithm combining a Siamese network with the hash method is superior to the classical CNN-based methods. The application of a new loss function can effectively improve retrieval accuracy.
PriRadar: A Privacy-Preserving Framework for Spatial Crowdsourcing

Privacy leakage is a serious issue in spatial crowdsourcing in various scenarios. In this paper, we study privacy protection in spatial crowdsourcing. The main challenge is to efficiently assign tasks to nearby workers without needing to know the exact locations of tasks and workers. To address this problem, we propose a privacy-preserving framework without online trusted third parties. We devise a grid-based location protection method, which can protect the locations of workers and tasks while keeping the distance-aware information on the protected locations such that we can quantify the distance between tasks and workers. We propose an efficient task assignment algorithm, which can instantly assign tasks to nearby workers on encrypted data. To protect the task content, we leverage both attribute-based encryption and symmetric-key encryption to establish secure channels through servers, which ensures that the task is delivered securely and accurately by any untrusted server. Moreover, we analyze the security properties of our method. We have conducted real experiments on real-world datasets. Experimental results show that our method outperforms existing approaches.
Making Sense of Human-Food Interaction

Activity in Human-Food Interaction (HFI) research is skyrocketing across a broad range of disciplinary interests and concerns. The dynamic and heterogeneous nature of this emerging field presents a challenge to scholars wishing to critically engage with prior work, identify gaps and ensure impact. It also challenges the formation of community. We present a Systematic Mapping Study of HFI research and an online data visualisation tool developed to respond to these issues. The tool allows researchers to engage in new ways with the HFI literature, propose modifications and additions to the review, and thereby actively engage in community-making. Our contribution is threefold: (1) we characterize the state of HFI, reporting trends, challenges and opportunities; (2) we provide a taxonomy and tool for diffractive reading of the literature; and (3) we offer our approach for adaptation by research fields facing similar challenges, positing value of the tool and approach beyond HFI.
English teaching practice based on artificial intelligence technology

The automatic scoring of English composition is an inevitable trend of the rapid development of computer technology and artificial intelligence technology. This makes the research on the automatic scoring system of the composition more theoretical and feasible. The integration of information techn ology and English curriculum has deepened the application of artificial intelligence in the field of teaching, creating a new opportunity for the optimization of English teaching process, and opening up a new space for the creation of intelligent and personalized English teaching environment. Based on the integration of information technology and curriculum, aiming at improving the quality and effect of English teaching in middle schools, this paper explores the application of artificial intelligence in English teaching in middle schools by using relevant theories of curriculum theory, literature analysis and field investigation. An implementation plan of College English assisted instruction system based on AI technology is proposed. Combined with English teaching, some functions of the English teaching system are improved and humanized. The application of artificial intelligence technology in English teaching is explored in order to improve the quality and effect of English teaching.
Detecting Mid-Air Gestures for Digit Writing With Radio Sensors and a CNN

In this paper, we classify digits written in mid-air using hand gestures. Impulse radio ultrawideband (IR-UWB) radar sensors are used for data acquisition, with three radar sensors placed in a triangular geometry. Conventional radar-based gesture recognition methods use whole raw data matrices or a group of features for gesture classification using convolutional neural networks (CNNs) or other machine learning algorithms. However, if the training and testing data differ in distance, orientation, hand shape, hand size, or even gesture speed or the radar setup environment, these methods become less accurate. To develop a more robust gesture recognition method, we propose not using raw data for the CNN classifier, but instead employing the hand's mid-air trajectory for classification. The hand trajectory has a stereotypical shape for a given digit, regardless of the hand's orientation or speed, making its classification easy and robust. Our proposed method consists of three stages: signal preprocessing, hand motion localization, and tracking and transforming the trajectory data into an image to classify it using a CNN. Our proposed method outperforms conventional approaches because it is robust to changes in orientation, distance, and hand shape and size. Moreover, this method does not require building a huge training database of digits drawn by different users in different orientations; rather, we can use training databases already available in the image processing field. Overall, the proposed mid-air handwritten digit recognition system provides a user-friendly and accurate mid-air handwriting modality that does not place restrictions on users.
Fast task allocation for heterogeneous unmanned aerial vehicles through reinforcement learning

A task allocation problem for heterogeneous unmanned aerial vehicles (UAVs) in the presence of environment uncertainty is studied in this paper. Generally, the process of finding efficient allocation scheme can be computationally prohibitive. This work presents a Q-learning based fast task allocation (FTA) algorithm through neural network approximation and prioritized experience replay, which effectively offloads the online computation to an offline learning procedure. Specifically, the proposed approach develops a Q network that encodes the allocation rules. The Q network not only considers the effect of environment uncertainty, but also is capable of handling total different tasks. Comparison simulations are provided to show the efficiency of the proposed algorithm.
Cross-Modal Video Moment Retrieval with Spatial and Language-Temporal Attention

Given an untrimmed video and a description query, temporal moment retrieval aims to localize the temporal segment within the video that best describes the textual query. Existing studies predominantly employ coarse frame-level features as the visual representation, obfuscating the specific details which may provide critical cues for localizing the desired moment. We propose a SLTA (short for "Spatial and Language-Temporal Attention") method to address the detail missing issue. Specifically, the SLTA method takes advantage of object-level local features and attends to the most relevant local features (e.g., the local features "girl", "cup") by spatial attention. Then we encode the sequence of local features on consecutive frames to capture the interaction information among these objects (e.g., the interaction "pour" involving these two objects). Meanwhile, a language-temporal attention is utilized to emphasize the keywords based on moment context information. Therefore, our proposed two attention sub-networks can recognize the most relevant objects and interactions in the video, and simultaneously highlight the keywords in the query. Extensive experiments on TACOS, Charades-STA and DiDeMo datasets demonstrate the effectiveness of our model as compared to state-of-the-art methods.
Seml: A Semantic LSTM Model for Software Defect Prediction

Software defect prediction can assist developers in finding potential bugs and reducing maintenance cost. Traditional approaches usually utilize software metrics (Lines of Code, Cyclomatic Complexity, etc.) as features to build classifiers and identify defective software modules. However, software metrics often fail to capture programs' syntax and semantic information. In this paper, we propose Seml, a novel framework that combines word embedding and deep learning methods for defect prediction. Specifically, for each program source file, we first extract a token sequence from its abstract syntax tree. Then, we map each token in the sequence to a real-valued vector using a mapping table, which is trained with an unsupervised word embedding model. Finally, we use the vector sequences and their labels (defective or non-defective) to build a Long Short Term Memory (LSTM) network. The LSTM model can automatically learn the semantic information of programs and perform defect prediction. The evaluation results on eight open source projects show that Seml outperforms three state-of-the-art defect prediction approaches on most of the datasets for both within-project defect prediction and cross-project defect prediction.
Gigahorse: Thorough, Declarative Decompilation of Smart Contracts

The rise of smart contracts - autonomous applications running on blockchains - has led to a growing number of threats, necessitating sophisticated program analysis. However, smart contracts, which transact valuable tokens and cryptocurrencies, are compiled to very low-level bytecode. This bytecode is the ultimate semantics and means of enforcement of the contract. We present the Gigahorse toolchain. At its core is a reverse compiler (i.e., a decompiler) that decompiles smart contracts from Ethereum Virtual Machine (EVM) bytecode into a highlevel 3-address code representation. The new intermediate representation of smart contracts makes implicit data- and control-flow dependencies of the EVM bytecode explicit. Decompilation obviates the need for a contract's source and allows the analysis of both new and deployed contracts. Gigahorse advances the state of the art on several fronts. It gives the highest analysis precision and completeness among decompilers for Ethereum smart contracts - e.g., Gigahorse can decompile over 99.98% of deployed contracts, compared to 88% for the recently-published Vandal decompiler and under 50% for the state-of-the-practice Porosity decompiler. Importantly, Gigahorse offers a full-featured toolchain for further analyses (and a “batteries included” approach, with multiple clients already implemented), together with the highest performance and scalability. Key to these improvements is Gigahorse's use of a declarative, logic-based specification, which allows high-level insights to inform low-level decompilation.
TED

Artificial intelligence systems are being increasingly deployed due to their potential to increase the efficiency, scale, consistency, fairness, and accuracy of decisions. However, as many of these systems are opaque in their operation, there is a growing demand for such systems to provide explanations for their decisions. Conventional approaches to this problem attempt to expose or discover the inner workings of a machine learning model with the hope that the resulting explanations will be meaningful to the consumer. In contrast, this paper suggests a new approach to this problem. It introduces a simple, practical framework, called Teaching Explanations for Decisions (TED), that provides meaningful explanations that match the mental model of the consumer. We illustrate the generality and effectiveness of this approach with two different examples, resulting in highly accurate explanations with no loss of prediction accuracy for these two examples.
CrossInfoNet: Multi-Task Information Sharing Based Hand Pose Estimation

This paper focuses on the topic of vision based hand pose estimation from single depth map using convolutional neural network (CNN). Our main contributions lie in designing a new pose regression network architecture named CrossInfoNet. The proposed CrossInfoNet decomposes hand pose estimation task into palm pose estimation sub-task and finger pose estimation sub-task, and adopts two-branch cross-connection structure to share the beneficial complementary information between the sub-tasks. Our work is inspired by multi-task information sharing mechanism, which has been few discussed in hand pose estimation using depth data in previous publications. In addition, we propose a heat-map guided feature extraction structure to get better feature maps, and train the complete network end-to-end. The effectiveness of the proposed CrossInfoNet is evaluated with extensively self-comparative experiments and in comparison with state-of-the-art methods on four public hand pose datasets. The code is available.
Hybrid intelligent phishing website prediction using deep neural networks with genetic algorithm‐based feature selection and weighting

In recent years, the web phishing attack has become one of the most serious web security problems, in which the phishers can steal significant financial information about the internet users to carry out financial thefts. Several blacklist-based conventional phishing website detection methods are used to predict the phishing websites. However, numerous phishing websites are not predicted precisely by these blacklist-based conventional methods since many new phishing websites are constantly developed and launched on the Web over time. In this study, hybrid intelligent phishing website prediction using deep neural networks (DNNs) with evolutionary algorithm-based feature selection and weighting methods are suggested to enhance the phishing website prediction. In the proposed hybrid intelligent phishing website prediction approaches, the most influential features and the optimal weights of website features are heuristically identified with the genetic algorithm (GA) to help in increasing the accuracy of phishing website prediction. Accordingly, the website features selected and weighted by the GA are utilised to train DNNs to accurately predict the phishing websites. The experimental results demonstrated that the proposed hybrid intelligent phishing website prediction approaches achieved significantly higher classification accuracy, sensitivity, specificity, and geometric mean in phishing website prediction compared to those proposed in other studies.
Development of UAV-Based Target Tracking and Recognition Systems

Unmanned aerial vehicles (UAVs) are advantageous in their high maneuverability for long-range outdoor target tracking. In this paper, we develop a UAV-based target tracking and recognition system using an intelligent gimbal system with the capabilities of accurate camera positioning, fast image processing, and multi-modality information fusion. Algorithms of consensus-based target tracking, moving background processing, and neural network-based target detection/recognition are optimized for embedded implementation. A geographic information system (GIS) is utilized to provide geo-location, environmental, and contextual information. The experimental results demonstrate the robust performance of the proposed UAV-based target tracking and recognition framework.