Reviewerly/exHarmony huggingface dataset:
RNN-Test: Towards Adversarial Testing for Recurrent Neural Network Systems

While massive efforts have been investigated in adversarial testing of convolutional neural networks (CNN), testing for recurrent neural networks (RNN) is still limited and leaves threats for vast sequential application domains. In this paper, we propose an adversarial testing framework RNN-Test for RNN systems, focusing on sequence-to-sequence (seq2seq) tasks of widespread deployments, not only classification domains. First, we design a novel search methodology customized for RNN models by maximizing the inconsistency of RNN states against their inner dependencies to produce adversarial inputs. Next, we introduce two state-based coverage metrics according to the distinctive structure of RNNs to exercise more system behaviors. Finally, RNN-Test solves the joint optimization problem to maximize state inconsistency and state coverage, and crafts adversarial inputs for various tasks of different kinds of inputs. For evaluations, we apply RNN-Test on four RNN models of common structures. On the tested models, the RNN-Test approach is demonstrated to be competitive in generating adversarial inputs, outperforming FGSM-based and DLFuzz-based methods to reduce the model performance more sharply with 2.78% to 37.94% higher success (or generation) rate. RNN-Test could also achieve 52.65% to 66.45% higher adversary rate than testRNN on MNIST LSTM model, as well as 53.76% to 58.02% more perplexity with 16% higher generation rate than DeepStellar on PTB language model.Compared with the traditional neuron coverage, the proposed state coverage metrics as guidance excel with 4.17% to 97.22% higher success (or generation) rate.
On misbehaviour and fault tolerance in machine learning systems

Machine learning (ML) provides us with numerous opportunities, allowing ML systems to adapt to new situations and contexts. At the same time, this adaptability raises uncertainties concerning the run-time product quality or dependability, such as reliability and security, of these systems. Systems can be tested and monitored, but this does not provide protection against faults and failures in adapted ML systems themselves. We studied software designs that aim at introducing fault tolerance in ML systems so that possible problems in ML components of the systems can be avoided. The research was conducted as a case study, and its data was collected through five semi-structured interviews with experienced software architects. We present a conceptualisation of the misbehaviour of ML systems, the perceived role of fault tolerance, and the designs used. Common patterns to incorporating ML components in design in a fault tolerant fashion have started to emerge. ML models are, for example, guarded by monitoring the inputs and their distribution, and enforcing business rules on acceptable outputs. Multiple, specialised ML models are used to adapt to the variations and changes in the surrounding world, and simpler fall-over techniques like default outputs are put in place to have systems up and running in the face of problems. However, the general role of these patterns is not widely acknowledged. This is mainly due to the relative immaturity of using ML as part of a complete software system: the field still lacks established frameworks and practices beyond training to implement, operate, and maintain the software that utilises ML. ML software engineering needs further analysis and development on all fronts.
Optimization based ECG watermarking in RDWT-SVD domain

With the increase in point of care services, communication of digital patient records through open network has multi-folded. This digital data is used to obtain the remote medical assistance from the smart healthcare centres. Protecting this data during transmission is a very big challenge. One of the most important medical data is electrocardiogram (ECG) signal which detects the cardiovascular diseases and any alteration in the signal may affect the diagnosis. In this work, an ECG watermarking based on redundant discrete wavelet transform (RDWT) and singular value decomposition (SVD) is developed. First, the ECG signal is converted into 2-D matrix using pan-tompkins algorithm. Then, we use the hybrid of RDWT and SVD to conceal the patient data and logo image into the 2-D ECG image. We also use hybrid of optimization scheme to improve the robustness of the watermark. Preliminary experimental results indicate the optimal invisibility and robustness result is more effective up to 97.89% than the traditional schemes respectively, which makes it suitable for ownership authentication of ECG signal.
Improving Variational Autoencoder based Out-of-Distribution Detection for Embedded Real-time Applications

Uncertainties in machine learning are a significant roadblock for its application in safety-critical cyber-physical systems (CPS). One source of uncertainty arises from distribution shifts in the input data between training and test scenarios. Detecting such distribution shifts in real-time is an emerging approach to address the challenge. The high dimensional input space in CPS applications involving imaging adds extra difficulty to the task. Generative learning models are widely adopted for the task, namely out-of-distribution (OoD) detection. To improve the state-of-the-art, we studied existing proposals from both machine learning and CPS fields. In the latter, safety monitoring in real-time for autonomous driving agents has been a focus. Exploiting the spatiotemporal correlation of motion in videos, we can robustly detect hazardous motion around autonomous driving agents. Inspired by the latest advances in the Variational Autoencoder (VAE) theory and practice, we tapped into the prior knowledge in data to further boost OoD detection’s robustness. Comparison studies over nuScenes and Synthia data sets show our methods significantly improve detection capabilities of OoD factors unique to driving scenarios, 42% better than state-of-the-art approaches. Our model also generalized near-perfectly, 97% better than the state-of-the-art across the real-world and simulation driving data sets experimented. Finally, we customized one proposed method into a twin-encoder model that can be deployed to resource limited embedded devices for real-time OoD detection. Its execution time was reduced over four times in low-precision 8-bit integer inference, while detection capability is comparable to its corresponding floating-point model.
Keyed Parallel Hash Algorithm Based on Multiple Chaotic Maps (KPHA-MCM)

Hash functions are considered as the core of cryptography, which are the basic technique used for data security. Cryptographic hash functions also can be used to achieve the integrity of large data such as the data stored in a hard disk and set of financial data. So, in the era of large data and increasing capacity of data in applications, fast hash schemes with parallel operations are extremely desirable which effectively increases the computational speed. Functions of random behavior like chaotic maps are used in hash functions to generate the fixed length message digest from the original message. This paper proposes a parallel hash algorithm based on multiple chaotic functions by mixing logistic map, tent map, and sine function. In the proposed scheme the structure of coupling lattice is changed and using diamond lattice as new structure. This algorithm is flexible to generate 128, 256 or longer hash value. The simulation analysis such as hash distribution, key sensitivity, confusion and diffusion statistical properties, and collision resistance are executed. The results demonstrated that the proposed hash is an efficient, simple and fast algorithm comparing with some recent hash algorithms based on chaotic maps.
Analysis of User Responses to the Application of Web-Based Assessment on Character Assessment

There are currently several obstacles in the assessment using conventional media, including excessive use of paper, which causes environmental damage, and increases in school fees to conduct an assessment. In addition, it is less effective in conducting assessments because it takes a long time to assess the results of student answers manually. . Conventional media allows fraud and results in a lack of confidence in the assessment results. This study aims to analyze user responses using web-based assessment as a character assessment medium. The population of this study amounted to 91 students. This study uses a mixed-method explanatory method with descriptive analysis and qualitative methods. The results of the study show that the user response obtained an average value of 3.52, which includes the SB category (excellent) because the use of the website can save time, display real-time data, and is very easy to use anywhere, so with a good response from users make this web-based assessment suitable for use in student character assessment. This research implies that teachers can use web-based assessment in conducting assessments.
Pedestrian Trajectory Prediction Based on Transfer Learning for Human-Following Mobile Robots

Recent developments in the field of service robots have led to a renewed interest in humanrobot coexistence environments such as at home and office.In this regard, this study focuses on one of such service robots, a human-following mobile robot.In particular, we consider predicting the future trajectory of pedestrians using a machine learning algorithm to improve the accuracy of tracking people.Massive trajectory data is required in existing methods to train the prediction model; however, collecting a sufficient amount of data in general public places before providing services is challenging.Therefore, in this study, we propose a trajectory prediction method based on extracting similar datasets from a large-size dataset and generating a pre-trained prediction model using the extracting datasets.We express the data features in the source and target environments as probability distributions and evaluate the divergence between them.Specifically, the dataset features are expressed as a multidimensional Gaussian distribution and discrete distribution of samples.Then, similarities using the Kullback-Leibler divergence are compared.To verify the effectiveness of the proposed method, we compare the prediction results of the LSTM-based algorithm with those obtained by extracting multiple source datasets from a large dataset and training prediction models using these datasets.The result shows that the proposed method makes it possible to construct an appropriate prediction model with high accuracy in trajectory prediction.
Residual Adapters for Parameter-Efficient ASR Adaptation to Atypical and Accented Speech

Automatic Speech Recognition (ASR) systems are often optimized to work best for speakers with canonical speech patterns.Unfortunately, these systems perform poorly when tested on atypical speech and heavily accented speech.It has previously been shown that personalization through model fine-tuning substantially improves performance.However, maintaining such large models per speaker is costly and difficult to scale.We show that by adding a relatively small number of extra parameters to the encoder layers via socalled residual adapter, we can achieve similar adaptation gains compared to model finetuning, while only updating a tiny fraction (less than 0.5%) of the model parameters.We demonstrate this on two speech adaptation tasks (atypical and accented speech) and for two state-of-the-art ASR architectures.
Accented Speech Recognition Based on End-to-End Domain Adversarial Training of Neural Networks

The performance of automatic speech recognition (ASR) may be degraded when accented speech is recognized because the speech has some linguistic differences from standard speech. Conventional accented speech recognition studies have utilized the accent embedding method, in which the accent embedding features are directly fed into the ASR network. Although the method improves the performance of accented speech recognition, it has some restrictions, such as increasing the computational costs. This study proposes an efficient method of training the ASR model for accented speech in a domain adversarial way based on the Domain Adversarial Neural Network (DANN). The DANN plays a role as a domain adaptation in which the training data and test data have different distributions. Thus, our approach is expected to construct a reliable ASR model for accented speech by reducing the distribution differences between accented speech and standard speech. DANN has three sub-networks: the feature extractor, the domain classifier, and the label predictor. To adjust the DANN for accented speech recognition, we constructed these three sub-networks independently, considering the characteristics of accented speech. In particular, we used an end-to-end framework based on Connectionist Temporal Classification (CTC) to develop the label predictor, a very important module that directly affects ASR results. To verify the efficiency of the proposed approach, we conducted several experiments of accented speech recognition for four English accents including Australian, Canadian, British (England), and Indian accents. The experimental results showed that the proposed DANN-based model outperformed the baseline model for all accents, indicating that the end-to-end domain adversarial training effectively reduced the distribution differences between accented speech and standard speech.
Understanding and Overcoming the Challenges of Efficient Transformer Quantization

Transformer-based architectures have become the de-facto standard models for a wide range of Natural Language Processing tasks.However, their memory footprint and high latency are prohibitive for efficient deployment and inference on resource-limited devices.In this work, we explore quantization for transformers.We show that transformers have unique quantization challenges -namely, high dynamic activation ranges that are difficult to represent with a low bit fixed-point format.We establish that these activations contain structured outliers in the residual connections that encourage specific attention patterns, such as attending to the special separator token.To combat these challenges, we present three solutions based on post-training quantization and quantization-aware training, each with a different set of compromises for accuracy, model size, and ease of use.In particular, we introduce a novel quantization scheme -per-embedding-group quantization.We demonstrate the effectiveness of our methods on the GLUE benchmark using BERT, establishing state-of-the-art results for post-training quantization.Finally, we show that transformer weights and embeddings can be quantized to ultra-low bit-widths, leading to significant memory savings with a minimum accuracy loss.
Measuring Learnability through Virtual Reality Laboratory Application: A User Study

The cutting-edge technology of virtual reality has changed almost every aspect of life in e-commerce, engineering, medicine, and entertainment. This technology has also made its way to the field of education in the form of virtual laboratories. A lack of student engagement and interest towards STEM subjects is reported in the literature. Several studies have been conducted to evaluate virtual reality in education, but these studies are limited in terms of participants and subject coverage. This study aimed to assess the effectiveness of virtual laboratories to develop student’s practical learning skills for secondary school physics. For this purpose, a desktop-based virtual laboratory application was developed based on the guidelines extracted from the literature. A user study was adopted as the main research method, and it was conducted with 184 students of 4 different schools. In each school, students were divided into two groups: experimental (used the virtual laboratory application) and control (used a physical laboratory). The data were collected through an academic quiz conducted at the end of the study. The mean score of the experimental group was 7.16, compared with 5.87 for the control group. The results revealed that the students’ learning using the virtual laboratory application was better compared with the control group. Interestingly, there was no significant difference in the performance of boys and girls in both groups. The usability questionnaire was also completed by 92 students of the experimental group to assess the application interface. The mean score was 73.5 (above average) with an internal consistency of 0.76. The participants found the virtual laboratory application to be user-friendly, easy to use, and supportive in learning.
A Semiautomatic Multi-Label Color Image Segmentation Coupling Dirichlet Problem and Colour Distances

Image segmentation is an essential but critical component in low level vision, image analysis, pattern recognition, and now in robotic systems. In addition, it is one of the most challenging tasks in image processing and determines the quality of the final results of the image analysis. Colour based segmentation could hence offer more significant extraction of information as compared to intensity or texture based segmentation. In this work, we propose a new local or global method for multi-label segmentation that combines a random walk based model with a direct label assignment computed using a suitable colour distance. Our approach is a semi-automatic image segmentation technique, since it requires user interaction for the initialisation of the segmentation process. The random walk part involves a combinatorial Dirichlet problem for a weighted graph, where the nodes are the pixel of the image, and the positive weights are related to the distances between pixels: in this work we propose a novel colour distance for computing such weights. In the random walker model we assign to each pixel of the image a probability quantifying the likelihood that the node belongs to some subregion. The computation of the colour distance is pursued by employing the coordinates in a colour space (e.g., RGB, XYZ, YCbCr) of a pixel and of the ones in its neighbourhood (e.g., in a 8-neighbourhood). The segmentation process is, therefore, reduced to an optimisation problem coupling the probabilities from the random walker approach, and the similarity with respect the labelled pixels. A further investigation involves an adaptive preprocess strategy using a regression tree for learning suitable weights to be used in the computation of the colour distance. We discuss the properties of the new method also by comparing with standard random walk and k-means approaches. The experimental results carried on the White Blood Cell (WBC) dataset and GrabCut datasets show the remarkable performance of the proposed method in comparison with state-of-the-art methods, such as normalised random walk and normalised lazy random walk, with respect to segmentation quality and computational time. Moreover, it reveals to be very robust with respect to the presence of noise and to the choice of the colourspace.
Application of Deep Reinforcement Learning for Tracking Control of 3WD Omnidirectional Mobile Robot

Deep reinforcement learning, the fastest growing technique, to solve real-world complex problems by creatinga simple mathematical framework. It includes an agent, action, environment, and a reward. An agent will interactwith the environment, takes an optimal action aiming to maximize the total reward. This paper proposesthe compelling technique of deep deterministic policy gradient for solving the complex continuous actionspace of 3-wheeled omnidirectional mobile robots. Three-wheeled Omnidirectional mobile robots tracking isa difficult task because of the orientation of the wheels which makes it rotate around its own axis rather tofollow the trajectory. A deep deterministic policy gradient (DDPG) algorithm has been designed to train in environmentswith continuous action space to follow the trajectory by training the neural networks defined forthe policy and value function to maximize the reward function defined for the tracking of the trajectory. DDPGagent environment is created in the Reinforcement learning toolbox in MATLAB 2019 while for Actor and criticnetwork design deep neural network designer is used. Results are shown to illustrate the effectiveness of thetechnique with a convergence of error approximately to zero.
Skill Induction and Planning with Latent Language

We present a framework for learning hierarchical policies from demonstrations, using sparse natural language annotations to guide the discovery of reusable skills for autonomous decision-making.We formulate a generative model of action sequences in which goals generate sequences of high-level subtask descriptions, and these descriptions generate sequences of low-level actions.We describe how to train this model using primarily unannotated demonstrations by parsing demonstrations into sequences of named high-level subtasks, using only a small number of seed annotations to ground language in action.In trained models, natural language commands index a combinatorial library of skills; agents can use these skills to plan by generating high-level instruction sequences tailored to novel goals.We evaluate this approach in the ALFRED household simulation environment, providing natural language annotations for only 10% of demonstrations.It achieves task completion rates comparable to state-of-the-art models (outperforming several recent methods with access to ground-truth plans during training and evaluation) while providing structured and human-readable high-level plans. 1
Deep Learning Based Filtering Algorithm for Noise Removal in Underwater Images

Under-water sensing and image processing play major roles in oceanic scientific studies. One of the related challenges is that the absorption and scattering of light in underwater settings degrades the quality of the imaging. The major drawbacks of underwater imaging are color distortion, low contrast, and loss of detail (especially edge information). The paper proposes a method to address these issues by de-noising and increasing the resolution of the image using a model network trained on similar data. The network extracts frames from a video and filters them with a trigonometric–Gaussian filter to eliminate the noise in the image. It then applies contrast limited adaptive histogram equalization (CLAHE) to improvise the image contrast, and finally enhances the image resolution. Experimental results show that the proposed method could effectively produce enhanced images from degraded underwater images.
Zero-Shot Scene Graph Relation Prediction Through Commonsense Knowledge Integration

Relation prediction among entities in images is an important step in scene graph generation (SGG), which further impacts various visual understanding and reasoning tasks. Existing SGG frameworks, however, require heavy training yet are incapable of modeling unseen (i.e., zero-shot) triplets. In this work, we stress that such incapability is due to the lack of commonsense reasoning, i.e., the ability to associate similar entities and infer similar relations based on general understanding of the world. To fill this gap, we propose CommOnsense-integrAted sCene grapH rElation pRediction (COACHER), a framework to integrate commonsense knowledge for SGG, especially for zero-shot relation prediction. Specifically, we develop novel graph mining pipelines to model the neighborhoods and paths around entities in an external commonsense knowledge graph, and integrate them on top of state-of-the-art SGG frameworks. Extensive quantitative evaluations and qualitative case studies on both original and manipulated datasets from Visual Genome demonstrate the effectiveness of our proposed approach. The code is available at https://github.com/Wayfear/Coacher.