Reviewerly/exHarmony huggingface dataset:
Transformer-Encoder Detector Module: Using Context to Improve Robustness to Adversarial Attacks on Object Detection

Deep neural network approaches have demonstrated high performance in object recognition (CNN) and detection (Faster-RCNN) tasks, but experiments have shown that such architectures are vulnerable to adversarial attacks (FFF, UAP): low amplitude perturbations, barely perceptible by the human eye, can lead to a drastic reduction in labelling performance. This article proposes a new context module, called Transformer-Encoder Detector Module, that can be applied to an object detector to (i) improve the labelling of object instances; and (ii) improve the detector's robustness to adversarial attacks. The proposed model achieves higher mAP, F1 scores and AUC average score of up to 13% compared to the baseline Faster-RCNN detector, and an mAP score 8 points higher on images subjected to FFF or UAP attacks due to the inclusion of both contextual and visual features extracted from scene and encoded into the model. The result demonstrates that a simple ad-hoc context module can improve the reliability of object detectors significantly.
Explainable Just-In-Time Bug Prediction: Are We There Yet?

Explaining the prediction results of software bug prediction models is a challenging task, which can provide useful information for developers to understand and fix the predicted bugs. Recently, Jirayus et al.'s proposed to use two model-agnostic techniques (i.e., LIME and iBreakDown) to explain the prediction results of bug prediction models. Although their experiments on file-level bug prediction show promising results, the performance of these techniques on explaining the results of just-in-time (i.e., change-level) bug prediction is unknown. This paper conducts the first empirical study to explore the explainability of these model-agnostic techniques on just-in-time bug prediction models. Specifically, this study takes a three-step approach, 1) replicating previously widely used just-in-time bug prediction models, 2) applying Local Interpretability Model-agnostic Explanation Technique (LIME) and iBreakDown on the prediction results, and 3) manually evaluating the explanations for buggy instances (i.e. positive predictions) against the root cause of the bugs. The results of our experiment show that LIME and iBreakDown fail to explain defect prediction explanations for just-in-time bug prediction models, unlike file-level. This paper urges for new approaches for explaining the results of just-in-time bug prediction models.
Workflow models for aggregating cultural heritage data on the web: A systematic literature review

Abstract In recent years, different cultural institutions have made efforts to spread culture through the construction of a unique search interface that integrates their digital objects and facilitates data retrieval for lay users. However, integrating cultural data is not a trivial task; therefore, this work performs a systematic literature review on data aggregation workflows, in order to answer five questions: What are the projects? What are the planned steps? Which technologies are used? Are the steps performed manually, automatically, or semi‐automatically? Which perform semantic search? The searches were carried out in three databases: Networked Digital Library of Theses and Dissertations, Scopus and Web of Science. In Q01, 12 projects were selected. In Q02, 9 stages were identified: Harvesting, Ingestion, Mapping, Indexing, Storing, Monitoring, Enriching, Displaying, and Publishing LOD. In Q03, 19 different technologies were found it. In Q04, we identified that most of the solutions are semi‐automatic and, in Q05, that most of them perform a semantic search. The analysis of the workflows allowed us to identify that there is no consensus regarding the stages, their nomenclatures, and technologies, besides presenting superficial discussions. But it allowed to identify the main steps for the implementation of the aggregation of cultural data.
Undirected Complete Graph to Design New Public Key Cryptosystem

Abstract In this work, the alternative version of the ElGamal public key cryptosystem (EPKC) first is proposed. The revised EPKC depended on an extended discrete logarithm problem over matrices. More secure communications with the revised EPKC in compare to the original one are obtained. Also, a new public key cryptosystem has been designed based on undirected complete graph (UCG). The plaintext data is selected as a subset from the finite fields which is corresponded to undirected complete graph G ( V , E ), where V and E are finite sets of the vertices and edges respectively. The plaintext is encrypted using the matrix power function (MPF) and the minimum spanning tree (MST) problem of undirected complete graph (UCG). The adjacent matrix representation of MST is used to speed up the calculations. More secure on the proposed cryptosystem has been determined in compare to other asymmetric public key cryptosystems. New computational results of the revised EPKC and the UCG-based public key cryptosystem are discussed.
Evolutionary Quantization of Neural Networks with Mixed-Precision

Quantization is an effective way for reducing the memory and computation costs of deep neural networks. Most of existing methods exploit the fixed-precision quantization approach, e.g., weights and activations (i.e., output features) are represented as 8-bit values. Although mixed-precision quantization provides us a greater possibility to efficiently allocate computation resources and maintain the network performance, it is difficult to accurately solve the optimal bit-width of each layer. In this paper, we develop a novel evolutionary based method to automatically determine the bit-widths of weights and activations in each convolutional layer, namely, Evolutionary Mixed-Precision Quantization (EMQ). Specifically, the quantization intervals of weights and activations of all layers in the given network will be simultaneously encoded as an individual. The fitness of each individual is calculated as the performance of the corresponding quantized network. The optimal quantization result will be updated and elected during the evolutionary search. Extensive experiments conducted on benchmark datasets and models demonstrate the effectiveness of the proposed method over the state-of-the-art network quantization algorithms.
A Semantic Segmentation Model for Headdresses in Thangka Image Based on Line Drawing Augmentation and Spatial Prior Knowledge

Thangka is a religious scroll painting. As a window to Tibetan traditional culture and religious customs, the segmentation of important semantic objects in the images can help the public understand the image content and realize high-level semantic cognition of the visual content. This paper proposes a semantic segmentation network for the central figures’ headdresses in the portrait-type Thangka images based on the rich structure and standard composition of Thangka images. Meanwhile, it has also constructed the first pixel-level semantic annotation data set of the central figures’ headdresses in Thangka images. Firstly, an original Thangka image witnesses edge augmentation through the line drawing augmentation module. Then the Feature Extraction Network is used to extract the feature map. After that, it uses the Half-RPN network to get the region of interest (ROI). Finally, Mask R-CNN Head completes segmentation and class prediction. The experimental results show that the performance of the proposed model is 10% to 19% better than that of the state-of-the-art models such as Mask R-CNN and DeepLab V3.
Improving the Performance of an Artificial Intelligence Recommendation Engine with Deep Learning Neural Nets

The most valuable artificial intelligence application for e-commerce to social media websites these days is a smart recommendation engine that can filter panoply of information on the internet and recommend personalized products and services to each user. An efficient and reliable recommender engine (RE) increases sells and profit of the e-commerce websites, thus its performance is very crucial. Traditional RE suffers from cold-start, low accuracy, and scalability to Big Data problem. Thus, RE research has started again with great enthusiasm to explore newer techniques with deep learning artificial neural nets, as more computing power in parallel processing framework become available from latter half of this decade. In recent years deep learning (DL) artificial neural nets (ANN) have given breakthrough performance in areas like image processing and natural language processing tasks. So, its usability needs to be researched for recommender engine design also. This paper first explores the traditional ways of making a recommender engine and then evaluates the use of deep learning neural net techniques. The first contribution of this paper is expounding the theoretical foundation of different ways a RE can be built viz. content-based filtering (CBF) and collaborative filtering (CF) that comprises of complex algorithms. The second contribution of this paper is practical experiments with both traditional linear algebra techniques and deep learning auto-encoder architecture on large Movie-Lens dataset. Comparisons of the result shows deep learning methods outperform traditional methods.
Unsupervised Multimodal Image Registration with Adaptative Gradient Guidance

Multimodal image registration (MIR) is a fundamental procedure in many image-guided therapies. Recently, unsupervised learning-based methods have demonstrated promising performance over accuracy and efficiency in deformable image registration. However, the estimated deformation fields of the existing methods fully rely on the to-be-registered image pair. It is difficult for the networks to be aware of the mismatched boundaries, resulting in unsatisfactory organ boundary alignment. In this paper, we propose a novel multimodal registration framework, which elegantly leverages the deformation fields estimated from both: (i) the original to-be-registered image pair, (ii) their corresponding gradient intensity maps, and adaptively fuses them with the proposed gated fusion module. With the help of auxiliary gradient-space guidance, the network can concentrate more on the spatial relationship of the organ boundary. Experimental results on two clinically acquired CT-MRI datasets demonstrate the effectiveness of our proposed approach.
Memory-Efficient Speech Recognition on Smart Devices

Recurrent transducer models have emerged as a promising solution for speech recognition on the current and next generation smart devices. The transducer models provide competitive accuracy within a reasonable memory footprint alleviating the memory capacity constraints in these devices. However, these models access parameters from off-chip memory for every input time step which adversely effects device battery life and limits their usability on low-power devices.We address transducer model’s memory access concerns by optimizing their model architecture and designing novel recurrent cell designs. We demonstrate that i) model’s energy cost is dominated by accessing model weights from off-chip memory, ii) transducer model architecture is pivotal in determining the number of accesses to off-chip memory and just model size is not a good proxy, iii) our transducer model optimizations and novel recurrent cell reduces off-chip memory accesses by 4.5× and model size by 2× with minimal accuracy impact.
Larger Step Faster Speed: Investigating Gesture-Amplitude-based Locomotion in Place with Different Virtual Walking Speed in Virtual Reality

In this paper, we present a series of user studies to investigate the technique of gesture-amplitude-based walking-speed control for locomotion in place (LIP) in virtual reality (VR). Our 1st study suggested that compared to tapping and goose-stepping, the gesture of marching in place was significantly preferred by users across three different virtual walking speed (i.e., 1 ×, 3 ×, and 10 ×) while sitting and standing, and it yielded larger motion difference across the three speed levels. With the tracker data recorded in the 1st study, we trained a Support- Vector-Machine classification model for LIP speed control based on users' leg/foot gestures in marching in place. The overall accuracy for classifying three speed levels was above 90% for sitting and standing. With the classification model, we then compared the marching-in-place speed-control technique with the controller-based teleportation approach on a target-reaching task where users were sitting and standing. We found no significant difference between the two conditions in terms of target-reaching accuracy. More importantly, the technique of marching in place yielded significantly higher user ratings in terms of naturalness, realness, and engagement than the controller-based teleportation did.
Multi-Task Estimation of Age and Cognitive Decline from Speech

Speech is a common physiological signal that can be affected by both ageing and cognitive decline. Often the effect can be confounding, as would be the case for people at, e.g., very early stages of cognitive decline due to dementia. Despite this, the automatic predictions of age and cognitive decline based on cues found in the speech signal are generally treated as two separate tasks. In this paper, multi-task learning is applied for the joint estimation of age and the Mini-Mental Status Evaluation criteria (MMSE) commonly used to assess cognitive decline. To explore the relationship between age and MMSE, two neural network architectures are evaluated: a SincNet-based end-to-end architecture, and a system comprising of a feature extractor followed by a shallow neural network. Both are trained with single-task or multi-task targets. To compare, an SVM-based regressor is trained in a single-task setup. i-vector, x-vector and ComParE features are explored. Results are obtained on systems trained on the DementiaBank dataset and tested on an in-house dataset as well as the ADReSS dataset. The results show that both the age and MMSE estimation is improved by applying multitask learning, with state-of-the-art results achieved on the ADReSS dataset acoustic-only task.
VR-based Equipment Training for Health Professionals

It is common knowledge that health professionals' device training is a major problem, as the inadequate quality of device training puts patients at risk [2]. To find a way to counter this problem, we propose a virtual reality training simulation. The corresponding use case is exemplified by a priming procedure of a dialysis machine. This is achieved by users going through sequential interaction tasks in Virtual Reality Training using a head-mounted display. We evaluate this training method's potential within a user study, comparing it to traditional training methods, Group Training, and Video Training. Our findings demonstrate that Virtual Reality Training using a head-mounted display is a more effective and efficient method of learning how to prime a dialysis machine. Compared to other training methods, Virtual Reality Training takes longer on average, but the resulting learning effect is also higher. Furthermore, VR-training is more cost-effective than personal training and can be repeatedly performed as it dispenses with the need for teaching professionals.
Learning Stereo Matchability in Disparity Regression Networks

Learning-based stereo matching has recently achieved promising results, yet still suffers difficulties in establishing reliable matches in weakly matchable regions that are textureless, non-Lambertian, or occluded. In this paper, we address this challenge by proposing a stereo matching network that considers pixel-wise matchability. Specifically, the network jointly regresses disparity and matchability maps from 3D probability volume through expectation and entropy operations. Next, a learned attenuation is applied as the robust loss function to alleviate the influence of weakly matchable pixels in the training. Finally, a matchability-aware disparity refinement is introduced to improve the depth inference in weakly matchable regions. The proposed deep stereo matchability (DSM) framework can improve the matching result or accelerate the computation while still guaranteeing the quality. Moreover, the DSM framework is portable to many recent stereo networks. Extensive experiments are conducted on Scene Flow and KITTI stereo datasets to demonstrate the effectiveness of the proposed framework over the state-of-the-art learning-based stereo methods.
ObscureNet

In this paper, we introduce ObscureNet, an encoder-decoder architecture that effectively conceals private attributes associated with time series data generated by sensors in IoT devices, while preserving the information content of the original time series. Drawing on conditional generative models and adversarial information factorization, ObscureNet learns latent representations that are invariant to the user-specified private attributes. This allows for modifying the private attributes or generating them randomly before using the decoder to synthesize a new version of data. We present three approaches to alter private attributes at anonymization time, and show that non-deterministic approaches can prevent an adversary from re-identifying private attributes. We compare ObscureNet with the autoencoder-based anonymization methods proposed in the literature and other generative models in terms of the accuracy of sensitive and desired inferences. Our experiments on two human activity recognition datasets show that compared to the original data, the sensitive inference accuracy is reduced by 80.38% on average, while the desired inference accuracy is only reduced by 6.82%. Moreover, ObscureNet reduces the sensitive inference accuracy by an additional 13.48% on average compared to the best baseline method. We report the computation overhead of running ObscureNet on a Raspberry Pi, and corroborate that it can be used for real-time anonymization of sensor data.
Active Privacy-Utility Trade-Off Against A Hypothesis Testing Adversary

We consider a user releasing her data containing some personal information in return of a service. We model user’s personal information as two correlated random variables, one of them, called the secret variable, is to be kept private, while the other, called the useful variable, is to be disclosed for utility. We consider active sequential data release, where at each time step the user chooses from among a finite set of release mechanisms, each revealing some information about the user’s personal information, i.e., the true hypotheses, albeit with different statistics. The user manages data release in an online fashion such that maximum amount of information is revealed about the latent useful variable, while the confidence for the sensitive variable is kept below a predefined level. For the utility, we consider both the probability of correct detection of the useful variable and the mutual information (MI) between the useful variable and released data. We formulate both problems as a Markov decision process (MDP), and numerically solve them by advantage actor-critic (A2C) deep reinforcement learning (RL).
Provocations from #vanlife

Research on how lived experiences with technology intersect with home and work are core themes within HCI. Prior work has primarily focused on conventional life and work in Western countries. However, the unconventional is becoming conventional—several rising subcultures are coming into prominence due to socio-economic pressures, aided by social media. One example—#vanlife—is now practised by an estimated three million people in North America. #vanlife combines travel, home, and work by their occupants (vanlifers (vanlifers)) living full-time in cargo vans that they usually convert themselves into living spaces. We present a portrait of vanlifers’ current technology practices gleaned through ~200 hours of fieldwork and interviews. Following a thematic analysis of our data, we identified unique opportunities for integrating technology across culture, design, homesteading, offline organization, and gaming. We have distilled these opportunities into eleven provocations to inspire critical design and informed inquiry for technological interventions for #vanlife.
Lung X-Ray Image Enhancement to Identify Pneumonia with CNN

Chest x-rays have various values of intensity and high contrast. Chest X-rays require a contrast stretching process so that the image can be analyzed and diagnosed correctly. Image contrast improvement can be based on the histogram value of the image intensity. Pneumonia can be diagnosed by taking chest X-rays. Diagnosis of Pneumonia based on chest x-rays can be done automatically by a computer. Computer-based Pneumonia diagnosis requires a reliable and accurate algorithm. A reliable and accurate algorithm, namely Convolutional Neural Network. This research aimed to prove whether the chest X-ray image that was performed by contrast improvement had a significant effect in diagnosing Pneumonia. The algorithm proposed in diagnosing Pneumonia is a Convolutional Neural Network. The CLAHE repaired chest X-ray image was trained with 8 CNN architectural models. The training results of the eight CNN architectural models respectively have a loss function value of 0.0057, 0.028, 0.0964, 0.0446, 0.0473, 0.0573, 0.0979, 0.1407. The results of diagnostic testing for Pneumonia in the eight CNN architectural models were 79.65%, 79.01%, 80.29%, 76.92%, 82.53%, 80.45%, 79.81%, 78.04%, respectively. The highest accuracy result when testing is 82.53% with the CNN 35 Layers architectural model, with a description of the input image is grayscale with a size of 224x224.
Managing Traceability Information Models: Not Such a Simple Task After All?

Practitioners are poorly supported by the scientific literature when managing traceability information models (TIMs), which capture the structure and semantics of trace links. In practice, companies manage their TIMs in very different ways, even in cases where companies share many similarities. We present our findings from an in-depth focus group about TIM management with three different systems engineering companies. We find that the concrete needs of the companies as well as challenges such as scale and workflow integration are not considered by existing scientific work. We thus issue a call-to-arms for the requirements engineering and software and systems traceability communities, the two main communities for traceability research, to refocus their work on these practical problems.
Multi-View Contrastive Learning for Online Knowledge Distillation

Previous Online Knowledge Distillation (OKD) often carries out mutually exchanging probability distributions, but neglects the useful representational knowledge. We there-fore propose Multi-view Contrastive Learning (MCL) for OKD to implicitly capture correlations of feature embeddings encoded by multiple peer networks, which provide various views for understanding the input data instances. Benefiting from MCL, we can learn a more discriminative representation space for classification than previous OKD methods. Experimental results on image classification demonstrate that our MCL-OKD outperforms other state-of-the-art OKD methods by large margins without sacrificing additional inference cost. Codes are available at https://github.com/winycg/MCL-OKD.