Reviewerly/exHarmony huggingface dataset:
LoRMIkA: Local rule-based model interpretability with k-optimal associations

As we rely more and more on machine learning models for real-life decision-making, being able to understand and trust the predictions becomes ever more important. Local explainer models have recently been introduced to explain the predictions of complex machine learning models at the instance level. In this paper, we propose Local Rule-based Model Interpretability with k-optimal Associations (LoRMIkA), a novel model-agnostic approach that obtains k-optimal association rules from a neighbourhood of the instance to be explained. Compared with other rule-based approaches in the literature, we argue that the most predictive rules are not necessarily the rules that provide the best explanations. Consequently, the LoRMIkA framework provides a flexible way to obtain predictive and interesting rules. It uses an efficient search algorithm guaranteed to find the k-optimal rules with respect to objectives such as confidence, lift, leverage, coverage, and support. It also provides multiple rules which explain the decision and counterfactual rules, which give indications for potential changes to obtain different outputs for given instances. We compare our approach to other state-of-the-art approaches in local model interpretability on three different datasets and achieve competitive results in terms of local accuracy and interpretability.
The AI Effect: Working at the Intersection of AI and SE

This special issue explores the intersection of artificial intelligence (AI) and software engineering (SE), that is, what can AI do for SE, and how can we as software engineers design and build better AI systems?
A Flexible and Scalable NTT Hardware : Applications from Homomorphically Encrypted Deep Learning to Post-Quantum Cryptography

The Number Theoretic Transform (NTT) enables faster polynomial multiplication and is becoming a fundamental component of next-generation cryptographic systems. NTT hardware designs have two prevalent problems related to design-time flexibility. First, algorithms have different arithmetic structures causing the hardware designs to be manually tuned for each setting. Second, applications have diverse throughput/area needs but the hardware have been designed for a fixed, pre-defined number of processing elements. This paper proposes a parametric NTT hardware generator that takes arithmetic configurations and the number of processing elements as inputs to produce an efficient hardware with the desired parameters and throughput. We illustrate the employment of the proposed design in two applications with different needs: A homomorphically encrypted deep neural network inference (CryptoNets) and a post-quantum digital signature scheme (qTESLA). We propose the first NTT hardware acceleration for both applications on FPGAs. Compared to prior software and high-level synthesis solutions, the results show that our hardware can accelerate NTT up to 28× and 48×, respectively. Therefore, our work paves the way for high-level, automated, and modular design of next-generation cryptographic hardware solutions.
CA-GAN: Class-Condition Attention GAN for Underwater Image Enhancement

Underwater images suffer from serious color distortion and detail loss because of the wavelength-dependent light absorption and scattering, which seriously influences the subsequent underwater object detection and recognition.The latest methods for underwater image enhancement are based on deep models, which focus on finding a mapping function from the underwater image subspace to a ground-truth image subspace.They neglect the diversity of underwater conditions which leads to different background colors of underwater images.In this paper, we propose a Class-condition Attention Generative Adversarial Network (CA-GAN) to enhance an underwater image.We build an underwater image dataset which contains ten categories generated by the simulator with different water attenuation coefficient and depth.Relying on the underwater image classes, CA-GAN creates a many-to-one mapping function for an underwater image.Moreover, in order to generate the realistic image, attention mechanism is utilized.In the channel attention block, the feature maps in the front-end layers and the back-end layers are fused along channels, and in the spatial attention block, feature maps are pixel-wise fused.Extensive experiments are conducted on synthetic and real underwater images.The experimental results demonstrate that CA-GAN can effectively recover color and detail of various scenes of underwater images and is superior to the state-of-the-art methods.
New Hybrid Quantum Annealing Algorithms for Solving Vehicle Routing Problem

We introduce new hybrid algorithms, DBSCAN Solver and Solution Partitioning Solver, which use quantum annealing for solving Vehicle Routing Problem (VRP) and its practical variant: Capacitated Vehicle Routing Problem (CVRP). Both algorithms contain important classical components, but we also present two other algorithms, Full QUBO Solver and Average Partitioning Solver, which can be run only on a quantum processing unit (without CPU) and were prototypes which helped us develop better hybrid approaches. In order to validate our methods, we run comprehensive tests using D-Wave’s Leap framework on well-established benchmark test cases as well as on our own test scenarios built based on realistic road networks. We also compared our new quantum and hybrid methods with classical algorithms - well-known metaheuristics for solving VRP and CVRP. The experiments indicate that our hybrid methods give promising results and are able to find solutions of similar or even better quality than the tested classical algorithms.
Self-aware Cyber-Physical Systems

In this article, we make the case for the new class of Self-aware Cyber-physical Systems. By bringing together the two established fields of cyber-physical systems and self-aware computing, we aim at creating systems with strongly increased yet managed autonomy, which is a main requirement for many emerging and future applications and technologies. Self-aware cyber-physical systems are situated in a physical environment and constrained in their resources, and they understand their own state and environment and, based on that understanding, are able to make decisions autonomously at runtime in a self-explanatory way. In an attempt to lay out a research agenda, we bring up and elaborate on five key challenges for future self-aware cyber-physical systems: (i) How can we build resource-sensitive yet self-aware systems? (ii) How to acknowledge situatedness and subjectivity? (iii) What are effective infrastructures for implementing self-awareness processes? (iv) How can we verify self-aware cyber-physical systems and, in particular, which guarantees can we give? (v) What novel development processes will be required to engineer self-aware cyber-physical systems? We review each of these challenges in some detail and emphasize that addressing all of them requires the system to make a comprehensive assessment of the situation and a continual introspection of its own state to sensibly balance diverse requirements, constraints, short-term and long-term objectives. Throughout, we draw on three examples of cyber-physical systems that may benefit from self-awareness: a multi-processor system-on-chip, a Mars rover, and an implanted insulin pump. These three very different systems nevertheless have similar characteristics: limited resources, complex unforeseeable environmental dynamics, high expectations on their reliability, and substantial levels of risk associated with malfunctioning. Using these examples, we discuss the potential role of self-awareness in both highly complex and rather more simple systems, and as a main conclusion we highlight the need for research on above listed topics.
Design and Construction of a Smart Wheelchair

The real objection people have an emergency in walking due to illness or injury. The suggested system is easy and efficient to solve the problem of the disabled person, and it is unique and fully automatic. A wheelchair provides the umbrella, Foot mat, Head mat, and obstacle detection, which does not depend on the ability to participate in society. The humidity sensor checks the weather (rainy/hot/cold), and the head mat works automatically. Also, the ultrasonic sensor is used to detect the obstacle according to the location tracking by the GPS. A prototype system implementation shows that the proposed day/season based system is efficient in interacting with a living environment.
FiFTy: Large-Scale File Fragment Type Identification Using Convolutional Neural Networks

We present FiFTy, a modern file-type identification tool for memory forensics and data carving. In contrast to previous approaches based on hand-crafted features, we design a compact neural network architecture, which uses a trainable embedding space. Our approach dispenses with the explicit feature extraction which has been a bottleneck in legacy systems. We evaluate the proposed method on a novel dataset with 75 filetypes - the most diverse and balanced dataset reported to date. FiFTy consistently outperforms all baselines in terms of speed, accuracy and individual misclassification rates. We achieved an average accuracy of 77.5% with processing speed of ≈38 sec/GB, which is better and more than an order of magnitude faster than the previous state-of-the-art tool - Sceadan (69% at 9 min/GB). Our tool and the corresponding dataset is open-source.
Regularising LSTM classifier by transfer learning for detecting misogynistic tweets with small training set

Supervised machine learning methods depend highly on the quality of the training dataset and the underlying model. In particular, neural network models, that have shown great success in dealing with natural language problems, require a large dataset to learn a vast number of parameters. However, it is not always easy to build a large (labelled) dataset. For example, due to the complex nature of tweets and the manual labour involved, it is hard to create a large Twitter data set with the misogynistic label. In this paper, we propose to regularise a long short-term memory (LSTM) classifier using a pretrained LSTM-based language model (LM) to build an accurate classification model with a small training set. We explain transfer learning (TL) with a Bayesian interpretation and show that TL can be viewed as an uncertainty regularisation technique in Bayesian inference. We show that a LM pre-trained on a sequence of general to task-specific domain datasets can be used to regularise a LSTM classifier effectively when a small training dataset is available. Empirical analysis with two small Twitter datasets reveals that an LSTM model trained in this way can outperform the state-of-the-art classification models.
Deep multi-task learning with relational attention for business success prediction

Multi-task learning is a promising machine learning branch, which aims to improve the generalization of the prediction models by sharing knowledge among tasks. Most of the existing multi-task learning methods rely on predefined task relationships and guide the learning process of models by linear regularization terms. On the one hand, improper setting of task relationships may result in negative knowledge transfer; on the other hand, these methods also suffer from the insufficiency of representation ability. To overcome these problems, this paper focuses on attention-based deep multi-task learning method, and provides a novel deep multi-task learning method, namely, Deep Multi-task Learning with Relational Attention (DMLRA). In particular, we first provide a task-specific attention module to specify features for different learning tasks, because different prediction tasks may rely on different parts of the shared feature set. Then, we design a relational attention module to learn relationships among multiple tasks automatically, and transfer positive and negative knowledge among multiple tasks accordingly. Moreover, we provide a joint deep multi-task learning framework to combine task-specific module and relational attention module. Finally, we apply our method on a multi-criteria business success assessment problem, both classical and the state-of-the-art multi-task learning methods are employed to provide baseline performance. The experiments are conducted on real-world datasets, results demonstrate the superiority of our method over the existing methods.
A study on IMU-Based Human Activity Recognition Using Deep Learning and Traditional Machine Learning

Human Activity Recognition (HAR) has been an increasingly popular range to do researches which stems from the ubiquitous computing. And lately, identifying activities during daily life has become one of more and more challenges. Subsequently, more and more methods can be used in the recognition of human activities such as Support Vector Machine (SVM), Random Forests (RF) which are the representatives of Traditional Machine Learning (TML) and also some Deep Learning (DL) methods like Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN). However, neither TML nor DL is suitable for all kinds of situations and various datasets. As a result, we would like to explore more about such consequences. In this paper, we discover a discrepancy and phenomenon that different sizes of collected HAR datasets may produce influences on the effectiveness of traditional machine learning methods as well as the deep learning architectures. We conduct experiments on two kinds of different datasets USC-HAD and WISDM with the best accuracy nearly 90% in DL and 87% in TML. Due to the consequences of the experiments we give a conclusion on the individual heterogeneity problems of the HAR datasets-when dealing with the HAR datasets of small scales, the TML structures are more suitable. However, conversely, when the datasets have large amount of datasets. Specifically, DL approaches such as CNN and LSTM are more sensible choices.
Design science research for learning software engineering and computational thinking: Four cases

Abstract This study proposes design science research (DSR) as a relevant framework for computational thinking, software engineering, and science, technology, engineering, arts and mathematics integration in higher education. We present four concrete case examples, where DSR has been applied in projects that consist of interdisciplinary teams, which learn by designing solutions for pressing real‐world problems. Our results show that DSR is a potent framework for learning to tackle pressing and wicked problems, and connecting technology projects with research methodology and knowledge from other fields. Our results also show multiple learning gains, including creativity, communication, critical thinking and collaboration. Based on our case analysis, we recommend more extensive use of DSR in future technology curricula.
SMAT: Smart Multiple Affinity Metrics for Multiple Object Tracking

This research introduces a novel multiple object tracking algorithm called SMAT (Smart Multiple Affinity Metric Tracking) that works as an online tracking-by-detection approach. The use of various characteristics from observation is established as a critical factor for improving tracking performance. By using the position, motion, appearance, and a correction component, our approach achieves an accuracy comparable to state of the art trackers. We use the optical flow to track the motion of the objects, we show that tracking accuracy can be improved by using a neural network to select key points to be tracked by the optical flow. The proposed algorithm is evaluated by using the KITTI Tracking Benchmark for the class CAR.
PPCensor: Architecture for real-time pornography detection in video streaming

Convolutional neural network (CNN) models are typically composed of several gigabytes of data, requiring dedicated hardware and significant processing capabilities for proper handling. In addition, video-detection tasks are typically performed offline, and each video frame is analyzed individually, meaning that the video's categorization (class assignment) as normal or pornographic is only complete after all the video frames have been evaluated. This paper proposes the Private Parts Censor (PPCensor), a CNN-based architecture for transparent and near real-time detection and obfuscation of pornographic video frame regions. Our contribution is two-fold. First, the proposed architecture is the first that addresses the detection of pornographic content as an object detection problem. The objective is to apply user-friendly content filtering such that an inevitable false positive will obfuscate only regions (objects) within the video frames instead of blocking the entire video. Second, the PPCensor architecture is deployed on dedicated hardware, and real-time detection is deployed using a video-oriented streaming proxy. If a pornographic video frame is identified in the video, the system can hide pornographic content (private parts) in real time without user interaction or additional processing on the user's device. Based on more than 50,000 objects labeled manually, the evaluation results show that the PPCensor is capable of detecting private parts in near real time for video streaming. Compared to cutting-edge CNN architectures for image classification, PPCensor achieved similar results, but operated in real time. In addition, when deployed on a desktop computer, PPCensor handled up to 35 simultaneous connections without the need for additional processing on the end-user device.
Efficacy of Imbalanced Data Handling Methods on Deep Learning for Smart Homes Environments

Abstract Human activity recognition as an engineering tool as well as an active research field has become fundamental to many applications in various fields such as health care, smart home monitoring and surveillance. However, delivering sufficiently robust activity recognition systems from sensor data recorded in a smart home setting is a challenging task. Moreover, human activity datasets are typically highly imbalanced because generally certain activities occur more frequently than others. Consequently, it is challenging to train classifiers from imbalanced human activity datasets. Deep learning algorithms perform well on balanced datasets, yet their performance cannot be promised on imbalanced datasets. Therefore, we aim to address the problem of class imbalance in deep learning for smart home data. We assess it with Activities of Daily Living recognition using binary sensors dataset. This paper proposes a data level perspective combined with a temporal window technique to handle imbalanced human activities from smart homes in order to make the learning algorithms more sensitive to the minority class. The experimental results indicate that handling imbalanced human activities from the data-level outperforms algorithms level and improved the classification performance.
Augmenting Dementia Cognitive Assessment With Instruction-Less Eye-Tracking Tests

Eye-tracking technology is an innovative tool that holds promise for enhancing dementia screening. In this work, we introduce a novel way of extracting salient features directly from the raw eye-tracking data of a mixed sample of dementia patients during a novel instruction-less cognitive test. Our approach is based on self-supervised representation learning where, by training initially a deep neural network to solve a pretext task using well-defined available labels (e.g. recognising distinct cognitive activities in healthy individuals), the network encodes high-level semantic information which is useful for solving other problems of interest (e.g. dementia classification). Inspired by previous work in explainable AI, we use the Layer-wise Relevance Propagation (LRP) technique to describe our network's decisions in differentiating between the distinct cognitive activities. The extent to which eye-tracking features of dementia patients deviate from healthy behaviour is then explored, followed by a comparison between self-supervised and handcrafted representations on discriminating between participants with and without dementia. Our findings not only reveal novel self-supervised learning features that are more sensitive than handcrafted features in detecting performance differences between participants with and without dementia across a variety of tasks, but also validate that instruction-less eye-tracking tests can detect oculomotor biomarkers of dementia-related cognitive dysfunction. This work highlights the contribution of self-supervised representation learning techniques in biomedical applications where the small number of patients, the non-homogenous presentations of the disease and the complexity of the setting can be a challenge using state-of-the-art feature extraction methods.
Linguistic Steganalysis via Densely Connected LSTM with Feature Pyramid

With the growing attention on multimedia security and rapid development of natural language processing technologies, various linguistic steganographic algorithms based on automatic text generation technology have been proposed increasingly, which brings great challenges in maintaining security of cyberspace. The prevailing linguistic steganalysis methods based on neural networks only conduct linguistic steganalysis with feature vectors from last layer of neural network, which may be insufficient for neural linguistic steganalysis. In this paper, we propose a neural linguistic steganalysis scheme based on densely connected Long short-term memory networks (LSTM) with feature pyramids which can incorporate more low level features to detect generative text steganographic algorithms. In the proposed framework, words in text are firstly mapped into semantic space with a hidden representation for better exploitation of the semantic features. Then, stacked bidirectional Long short-term memory networks are ultilized to extract different levels of semantic features. In order to incorporate more low level features from neural networks, we introduced two components: dense connections and feature pyramids to enhance the low level features in feature vectors. Finally, the semantic features from all levels are fused and we use a sigmoid layer to categorize the input text as cover or stego. Experiments showed that the proposed scheme can achieve the state-of-the-art results in detecting recently proposed linguistic steganographic algorithms.