Reviewerly/exHarmony huggingface dataset:
Dynamic Graph CNN for Event-Camera Based Gesture Recognition

Event camera is a kind of bio-inspired sensor which is able to capture the motion in asynchronous events stream. An event is triggered when the pixel has a brightness change. In spatio-temporal space, those events will form an event cloud, which has specific 3D geometry to capture the dynamic scene. To analyze the event cloud, previous works usually convert event streams into frame-based images which did not fully utilize its 3D geometry in the spatio-temporal event space. In this work, we propose to recognize the spatio-temporal 3D event clouds for gesture recognition using Dynamic Graph CNN (DGCNN) which directly takes 3D points as input and is successfully used for 3D object recognition. We adapt DGCNN to perform action recognition by recognizing 3D geometry features in spatio-temporal space of the event data. We achieve state-of-the-art accuracy of 98.56% on the IBM DVS128 Gesture dataset and 95.94% on the DHP19 dataset.
Towards a seamful ethics of Covid-19 contact tracing apps?

In the early months of 2020, the deadly Covid-19 disease spread rapidly around the world. In response, national and regional governments implemented a range of emergency lockdown measures, curtailing citizens' movements and greatly limiting economic activity. More recently, as restrictions begin to be loosened or lifted entirely, the use of so-called contact tracing apps has figured prominently in many jurisdictions' plans to reopen society. Critics have questioned the utility of such technologies on a number of fronts, both practical and ethical. However, little has been said about the ways in which the normative design choices of app developers, and the products that result therefrom, might contribute to ethical reflection and wider political debate. Drawing from scholarship in critical design and human-computer interaction, this paper examines the development of a QR code-based tracking app called Zwaai ('Wave' in Dutch), where its designers explicitly positioned the app as an alternative to the predominant Bluetooth and GPS-based approaches. Through analyzing these designers' choices, this paper argues that QR code infrastructures can work to surface a set of ethical-political seams, two of which are discussed here-responsibilization and networked (im)permanence-that more 'seamless' protocols like Bluetooth actively aim to bypass, and which may go otherwise unnoticed by existing ethical frameworks.
Triplet Distillation For Deep Face Recognition

Convolutional neural networks (CNNs) have achieved great successes in face recognition, which unfortunately comes at the cost of massive computation and storage consumption. Many compact face recognition networks are thus proposed to resolve this problem, and triplet loss is effective to further improve the performance of these compact models. However, it normally employs a fixed margin to all the samples, which neglects the informative similarity structures between different identities. In this paper, we borrow the idea of knowledge distillation and define the informative similarity as the transferred knowledge. Then, we propose an enhanced version of triplet loss, named triplet distillation, which exploits the capability of a teacher model to transfer the similarity information to a student model by adaptively varying the margin between positive and negative pairs. Experiments on the LFW, AgeDB and CPLFW datasets show the merits of our method compared to the original triplet loss.
Deep Neural Networks for Low-Cost Eye Tracking

The paper presents a detailed analysis of modern techniques that can be used to track gaze with a webcam. We present a practical implementation of the most popular methods for tracking gaze. Various models of deep neural networks that can be involved in the process of online gaze monitoring are reviewed. We introduce a new eye-tracking approach where the effectiveness of using a deep learning method is significantly increased. Implementation is in Python where its application is demonstrated by controlling interaction with the computer. Specifically, a dual coordinate system is given for controlling the computer with the help of a gaze. The first set of coordinates-the position of the face relative to the computer, is implemented by detecting color from the infrared LED via the OpenCV library. The second set of coordinates-giving gaze position-is obtained via the YOLO (v3) package. A method of labeling the eyes is given, in which 3 objects are used to track gaze (to the left, to the right, and in the center).
Intelligent Transport System for a Smart City

The term smart city was coined in the early 1990s. This term includes urban development with new improved developments in the technology, innovation, and globalization. The major contribution is towards adapting the smart growth movement of the late 1990s. This has advocated improved urban planning and utilization of improved Wi-Fi enabled gadgets. It is needed for growth in new global knowledge economy. It also integrates the operation of urban infrastructure and services used in the buildings, transportation, electrical and water distribution and public safety. The smart city is part of urban development which has information and communication technology (ICT) to facilitate improved insight into as well as control over the various systems that affect the lives of residents.
On-board Deep-learning-based Unmanned Aerial Vehicle Fault Cause Detection and Identification

With the increase in use of Unmanned Aerial Vehicles (UAVs)/drones, it is important to detect and identify causes of failure in real time for proper recovery from a potential crash-like scenario or post incident forensics analysis. The cause of crash could be either a fault in the sensor/actuator system, a physical damage/attack, or a cyber attack on the drone's software. In this paper, we propose novel architectures based on deep Convolutional and Long Short-Term Memory Neural Networks (CNNs and LSTMs) to detect (via Autoencoder) and classify drone mis-operations based on real-time sensor data. The proposed architectures are able to learn high-level features automatically from the raw sensor data and learn the spatial and temporal dynamics in the sensor data. We validate the proposed deep-learning architectures via simulations and realworld experiments on a drone. Empirical results show that our solution is able to detect (with over 90% accuracy) and classify various types of drone mis-operations (with about 99% accuracy (simulation data) and upto 85% accuracy (experimental data)).
NOHAR - NOvelty discrete data stream for Human Activity Recognition based on smartphones with inertial sensors

Smartphone sensing capabilities have enabled human activity recognition (HAR) solutions to better understand human behavior through computational techniques. However, such solutions have suffered from scalability problems due to the high consumption of computational resources (e.g. memory and processing) and the difficulty of acting in real time due to not observing data evolution over time. These problems occur because the HAR solutions for smartphones have been solved through offline learning with models limited by a data history. The disadvantage of this approach is that human activities constantly change over time and are strongly influenced by the physical environment and user profile. To overcome such problem, this paper proposes a novel low-cost learning algorithm called NOHAR (NOvelty discrete data stream for Human Activity Recognition), focused on continuous flow of data analysis. NOHAR is an online classification algorithm based on symbolic data generated by a discretization process using algorithms as SAX and SFA. The advantages of these algorithms are their abilities to compress and reduce the dimensionality of data. In addition, this paper proposes a new framework called DISTAR (DIscrete STream learning for Activity Recognition) focused on the data streaming analysis. Its goals include standardizing the development of online algorithms for symbolic data. Experimental results using three databases show that NOHAR is on average 33 times faster than the state of the art and can reduce memory consumption by an average of 99.97%.
The impact of knowledge management capabilities on innovation in entrepreneurial companies in Jordan

Purpose The purpose of this research is to identify and clarify the important variables in knowledge management capabilities that affect product/service innovation in entrepreneurial companies in Jordan. Design/methodology/approach A five-point Likert scale questionnaire was designed for measuring knowledge-based (infrastructural and dynamic) capabilities and product/service innovation in entrepreneurial companies in Jordan that specialized in information and communication technology. A questionnaire was distributed to 297 individuals in 40 companies, and only 202 questionnaires were completely filled and returned by the entrepreneurs at King Hussein Business Park and Al Hassan Business Park. Structural equation modeling was applied to statistically test the research hypotheses. Findings The research findings showed that knowledge-based infrastructural capabilities are a prerequisite for knowledge-based dynamic capabilities. On the other hand, knowledge combination capabilities have a statistically significant mediating effect between knowledge based infrastructural capabilities and product/service innovation. Originality/value This research contributes to the literature by studying and investigating the key variables in knowledge management that affect product/service innovation and by providing a comprehensive understanding of the knowledge management capabilities and innovation in entrepreneurial companies. This research provides academicians with a framework for better understanding of the different factors of knowledge management capabilities and how they affect product/service innovation.
Context-Dependent Implicit Authentication for Wearable Device Users

As market wearables are becoming popular with a range of services, including making financial transactions, accessing cars, etc. that they provide based on various private information of a user, security of this information is becoming very important. However, users are often flooded with PINs and passwords in this internet of things (IoT) world. Additionally, hard-biometric, such as facial or finger recognition, based authentications are not adaptable for market wearables due to their limited sensing and computation capabilities. Therefore, it is a time demand to develop a burden-free implicit authentication mechanism for wearables using the less-informative soft-biometric data that are easily obtainable from the market wearables. In this work, we present a context-dependent soft-biometric-based wearable authentication system utilizing the heart rate, gait, and breathing audio signals. From our detailed analysis, we find that a binary support vector machine (SVM) with radial basis function (RBF) kernel can achieve an average accuracy of 0.94 ± 0.07, F <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> score of 0.93 ± 0.08, an equal error rate (EER) of about 0.06 at a lower confidence threshold of 0.52, which shows the promise of this work.
A survey for trust-aware recommender systems: A deep learning perspective

A significant remaining challenge for existing recommender systems is that users may not trust recommender systems for either inaccurate recommendation or lack of explanation. Thus, it becomes critical to embrace a trustworthy recommender system. This survey provides a systematic summary of three categories of trust issues in recommender systems: social-aware recommender systems, which leverage users’ social trust relationships; robust recommender systems, which filter untruthful information, noises and enhance attack resistance; and explainable recommender systems, which provide explanations of the recommended items. We focus on the work based on deep learning techniques, which is an emerging area in the recommendation research.
An augmented reality learning system for Japanese compound verbs: study of learning performance and cognitive load

Abstract To address the difficulties related to acquiring Japanese compound verbs, which lack the clarity of verb combinations and the opacity of compound verb meanings, we designed and developed an augmented reality (AR) learning system based on image schema and AR animations. We investigated the effects of the AR-based language learning system developed in this study on the learning performance and cognitive load of an AR learning system and paper-based image schema materials. This study also examined the correlation between learning performance and cognitive load. Learners of these two learning methods had significantly improved performance on post-tests. Especially, regarding the retention of knowledge, the AR learning system was more effective. However, there was no significant difference in the perceived cognitive loads between the two learning methods. It is also found that the learning performance of the two learning methods was related to the perceived different types of cognitive load.
Balancing Efficiency and Flexibility for DNN Acceleration via Temporal GPU-Systolic Array Integration

The research interest in specialized hardware accelerators for deep neural networks (DNN) spikes recently owing to their superior performance and efficiency. However, today’s DNN accelerators primarily focus on accelerating specific "kernels" such as convolution and matrix multiplication, which are vital but only part of an end-to-end DNN-enabled application. Meaningful speedups over the entire application often require supporting computations that are, while massively parallel, ill-suited to DNN accelerators. Integrating a general-purpose processor such as a CPU or a GPU incurs significant data movement overhead and leads to resource under-utilization on the DNN accelerators.We propose Simultaneous Multi-mode Architecture (SMA), a novel architecture design and execution model that offers general-purpose programmability on DNN accelerators in order to accelerate end-to-end applications. The key to SMA is the temporal integration of the systolic execution model with the GPU-like SIMD execution model. The SMA exploits the common components shared between the systolic-array accelerator and the GPU, and provides lightweight reconfiguration capability to switch between the two modes in-situ. The SMA achieves up to 63% performance improvement while consuming 23% less energy than the baseline Volta architecture with TensorCore.
Knowledge hypergraph-based approach for data integration and querying: Application to Earth Observation

According to the world economic forum report, around 70% of generated data are not used. This limitation of usage is mainly due to the lack of interoperability and linking of data that resides in isolated silos. Indeed, the overwhelming amount of data has worsened heterogeneity problems, as have the types of sources generating data in heterogeneous formats and different semantics. Those data related problematics are frequent in the domain of Earth Observation (EO). Earth observed data use different terminologies, which are difficult to reconcile because they reflect overlapped disciplines. These issues lead to misunderstandings and inefficient exchange and management of data in terms of access, pricing, and data rights, which can hamper environmental phenomena understanding. Virtual Knowledge Graph (VKG), allows semantic integration of existing data sources into a wide Knowledge Graph. In this work we propose a knowledge hypergraph-based approach for data integration and querying, with an application to Earth Observation data. Our proposal takes place in two phases (1) a knowledge hypergraph-based virtual data integration and (2) a hypergraph-based query processing. The first phase allows to generate a virtual knowledge hypergraph consisting of RML mappings between an ontology and the data. The second phase consists of enhancing the user's query by extracting and consolidating a global view of data from different sources based on the generated knowledge hypergraph. The proposed approach is implemented in the Onto-KIT tool (Ontology-based Knowledge hypergraph data Integration and querying Tool) and evaluated through real use case studies. The obtained results show that our proposal enhances query processing in terms of accuracy, completeness, and semantic richness of response.
Momentum Batch Normalization for Deep Learning with Small Batch Size

Normalization layers play an important role in deep network training. As one of the most popular normalization techniques, batch normalization (BN) has shown its effectiveness in accelerating the model training speed and improving model generalization capability. The success of BN has been explained from different views, such as reducing internal covariate shift, allowing the use of large learning rate, smoothing optimization landscape, etc. To make a deeper understanding of BN, in this work we prove that BN actually introduces a certain level of noise into the sample mean and variance during the training process, while the noise level depends only on the batch size. Such a noise generation mechanism of BN regularizes the training process, and we present an explicit regularizer formulation of BN. Since the regularization strength of BN is determined by the batch size, a small batch size may cause the under-fitting problem, resulting in a less effective model. To reduce the dependency of BN on batch size, we propose a momentum BN (MBN) scheme by averaging the mean and variance of current mini-batch with the historical means and variances. With a dynamic momentum parameter, we can automatically control the noise level in the training process. As a result, MBN works very well even when the batch size is very small (e.g., 2), which is hard to achieve by traditional BN.
Echo State Networks and Long Short-Term Memory for Continuous Gesture Recognition: a Comparative Study

Abstract Recent developments of sensors that allow tracking of human movements and gestures enable rapid progress of applications in domains like medical rehabilitation or robotic control. Especially the inertial measurement unit (IMU) is an excellent device for real-time scenarios as it rapidly delivers data input. Therefore, a computational model must be able to learn gesture sequences in a fast yet robust way. We recently introduced an echo state network (ESN) framework for continuous gesture recognition (Tietz et al., 2019) including novel approaches for gesture spotting, i.e., the automatic detection of the start and end phase of a gesture. Although our results showed good classification performance, we identified significant factors which also negatively impact the performance like subgestures and gesture variability. To address these issues, we include experiments with Long Short-Term Memory (LSTM) networks, which is a state-of-the-art model for sequence processing, to compare the obtained results with our framework and to evaluate their robustness regarding pitfalls in the recognition process. In this study, we analyze the two conceptually different approaches processing continuous, variable-length gesture sequences, which shows interesting results comparing the distinct gesture accomplishments. In addition, our results demonstrate that our ESN framework achieves comparably good performance as the LSTM network but has significantly lower training times. We conclude from the present work that ESNs are viable models for continuous gesture recognition delivering reasonable performance for applications requiring real-time performance as in robotic or rehabilitation tasks. From our discussion of this comparative study, we suggest prospective improvements on both the experimental and network architecture level.
Deep NeuralNet For Violence Detection Using Motion Features From Dynamic Images

Violent Action Recognition has been a challenging topic in monitoring human activities, especially in public premises. Deep neural nets and Transfer learning have proven highly successful in the detection of violent activities. In this paper, a novel deep NeuralNet system is proposed for the task of Detecting Violence by extraction of motion features from RGB Dynamic Images (DI). Motion feature extraction and prediction of violent content using a stream of RGB DI is done effectively by fine-tuning the pre-trained Inception-Resnet-V2 model. Improved and highly effective methods for intelligent analysis are highly demanded. For performance validation of the proposed novel model, tests are performed on three popular and publically available benchmarks - Hockey Fight dataset, Real Life Violence Dataset, and movie dataset.
Lossless robust image watermarking by using polar harmonic transform

In this paper, we propose a lossless robust watermarking method by using the polar harmonic transform with reversible and robust watermarking techniques. Firstly, a new quantization watermarking strategy is designed to embed bits into the polar harmonic transform moments. Secondly, the compensation information for recovery of the original image is computed by using the quantized error, the watermarked error and the rounded error to represent the difference between the original image and the robust watermarked image. Finally, the final watermarked image is achieved by reversibly inserting the compensation information into the the robust watermarked image. The proposed method is able to recover the original image lossless in case of no attacks. In case of attacks, the watermark can be directly extracted for copyright authentication. Experimental results show that the proposed method is reversible and provides strong robustness to those content-preserving manipulations. Compared with previous state of the art methods, the proposed one not only provides stronger robustness to those noise-like image processing operations (e.g., JPEG compression with quality factor 10), but has an ability to resist geometric deformations like rotation and scaling.
Deep Feature-Based Three-Stage Detection of Banknotes and Coins for Assisting Visually Impaired People

Owing to the rapid advancements in smartphone technology, there is an emerging need for a technology that can detect banknotes and coins to assist visually impaired people using the cameras embedded in smartphones. Previous studies have mostly used handcrafted feature-based methods, such as scale-invariant feature transform or speeded-up robust features, which cannot produce robust detection results for banknotes or coins captured in various backgrounds and environments. With the recent advancement in deep learning technology, some studies have been conducted on banknote and coin detection using a deep convolutional neural network (CNN). However, these studies also showed degraded performance depending on the changes in background and environment. To overcome these drawbacks, this paper proposes a three-stage detection technology for new banknotes and coins by applying faster region-based CNN, geometric constraints, and the residual network (ResNet). In the experiment performed using the open database of Jordanian dinar (JOD) and 6,400 images of eight types of Korean won banknotes and coins obtained using our smartphones, the proposed method exhibited a better detection performance than the state-of-the-art methods based on handcrafted features and deep features.