Reviewerly/exHarmony huggingface dataset:
Resolution-Aware Knowledge Distillation for Efficient Inference

Minimizing the computation complexity is essential for the popularization of deep networks in practical applications. Nowadays, most researches attempt to accelerate deep networks by designing new network structure or compressing the network parameters. Meanwhile, transfer learning techniques such as knowledge distillation are utilized to keep the performance of deep models. In this paper, we focus on accelerating deep models and relieving the computation burden by using low-resolution (LR) images as inputs while maintaining competitive performance, which is rarely researched in the current literature. Deep networks may encounter serious performance degradation when using LR inputs because many details are unavailable from LR images. Besides, the existing approaches may fail to learn discriminative features for LR images because of the dramatic appearance variations between LR and high-resolution (HR) images. To tackle with the above problems, we propose a resolution-aware knowledge distillation (RKD) framework to narrow the cross-resolution variations by transferring knowledge from HR domain to LR domain. The proposed framework consists of a HR teacher network and a LR student network. First, we introduce a discriminator and propose an adversarial learning strategy to shrink the variations between inputs with changing resolution. Then we design a cross-resolution knowledge distillation (CRKD) loss to train discriminative student network by exploiting the knowledge of the teacher network. The CRKD loss is consisted of a resolution-aware distillation loss, a pair-wise constraint, and a maximum mean discrepancy loss. Experimental results on person re-identification, image classification, face recognition, and defect segmentation tasks demonstrate that RKD outperforms traditional knowledge distillation method by achieving better performance with lower computation complexities. Furthermore, CRKD surpasses the state-of-the-art knowledge distillation methods in transferring knowledge across different resolutions under RKD framework, especially when coping with large resolution differences.
On the Evaluation of Commit Message Generation Models: An Experimental Study

Commit messages are natural language descriptions of code changes, which are important for program understanding and maintenance. However, writing commit messages manually is time-consuming and laborious, especially when the code is updated frequently. Various approaches utilizing generation or retrieval techniques have been proposed to automatically generate commit messages. To achieve a better understanding of how the existing approaches perform in solving this problem, this paper conducts a systematic and in-depth analysis of the state-of-the-art models and datasets. We find that: (1) Different variants of the BLEU metric are used in previous works, which affects the evaluation and understanding of existing methods. (2) Most existing datasets are crawled only from Java repositories while repositories in other programming languages are not sufficiently explored. (3) Dataset splitting strategies can influence the performance of existing models by a large margin. Some models show better performance when the datasets are split by commit, while other models perform better when the datasets are split by timestamp or by project. Based on our findings, we conduct a human evaluation and find the BLEU metric that best correlates with the human scores for the task. We also collect a large-scale, information-rich, and multi-language commit message dataset MCMD and evaluate existing models on this dataset. Furthermore, we conduct extensive experiments under different dataset splitting strategies and suggest the suitable models under different scenarios. Based on the experimental results and findings, we provide feasible suggestions for comprehensively evaluating commit message generation models and discuss possible future research directions. We believe this work can help practitioners and researchers better evaluate and select models for automatic commit message generation. Our source code and data are available at https://github.com/DeepSoftwareAnalytics/CommitMsgEmpirical.
A Secure Framework for Health Record Management Using Blockchain in Cloud Environment

Abstract Health records of any type should be confidential, but simultaneously they should be accessible to an authorized user. In the current conventional health care system, health records are processed in the local database. Only authorized hospital staff can have access to the patient health records kept in health centers. The flow of information in this kind of system is center-restricted. One technology that can potentially overcome this problem is Blockchain. Blockchain technology allows sharing of data in a distributed way with benefits like privacy-preserving and secure access control. To achieve secure access control and privacy of electronic healthcare data (EHD), a secure framework for distributed sharing of the health record is proposed. The use of blockchain will remove the requirement of a trusted 3 rd party for data storage. The framework uses searchable key attribute-based encryption (ABE) for achieving fine-grain access control. The simulation results indicate that our scheme is quite effective and secure.
Looking for Info: Evaluation of Gaze Based Information Retrieval in Augmented Reality

This paper presents the results of an empirical study and a real-world deployment of a gaze-adaptive UI for Augmented Reality (AR). AR introduces an attention dilemma between focusing on the reality vs. on AR content. Past work suggested eye gaze as a technique to open information interfaces, however there is only little empirical work. We present an empirical study comparing gaze-adaptive to an always-on interface in tasks that vary focus between reality and virtual content. Across tasks, we find most participants prefer the gaze-adaptive UI and find it less distracting. When focusing on reality, the gaze UI is faster, perceived as easier and more intuitive. When focusing on virtual content, always-on is faster but user preferences are split. We conclude with the design and deployment of an interactive application in a public museum, demonstrating the promising potential in the real world.
Sign language recognition based on global-local attention

Video-level sign language recognition is still a challenging task due to the influence of sign language-independent factors and timing requirements. This paper constructs a sign language recognition framework based on global-local feature description, and proposes a three-dimensional residual global network model with attention layer and a local network model based on target detection. The global feature description is based on the whole video behavior for time series modeling. The improved timing conversion layer is used to explore the timing information of different periods and learn the video representations of different timings. In the local module the hand is located through the target detection network to highlight its key role in the whole sign language behavior, which strengthens the category differences, and compensates the global network. Experiments on two well-known Chinese sign language datasets (SLR_Dataset and DEVSIGN_D) show that the proposed method can obtain higher recognition accuracy (respectively 89.2%, 91%) and better generalization performance.
Learning to Restore Images Degraded by Atmospheric Turbulence Using Uncertainty

Atmospheric turbulence can significantly degrade the quality of images acquired by long-range imaging systems by causing spatially and temporally random fluctuations in the index of refraction of the atmosphere. Variations in the refractive index causes the captured images to be geometrically distorted and blurry. Hence, it is important to compensate for the visual degradation in images caused by atmospheric turbulence. In this paper, we propose a deep learning-based approach for restring a single image degraded by atmospheric turbulence. We make use of the epistemic uncertainty based on Monte Carlo dropouts to capture regions in the image where the network is having hard time restoring. The estimated uncertainty maps are then used to guide the network to obtain the restored image. Extensive experiments are conducted on synthetic and real images to show the significance of the proposed work.
An Incentive Mechanism for Sustainable Blockchain Storage

Miners in a blockchain system are suffering from ever-increasing storage costs, which in general have not been properly compensated by the users’ transaction fees. This reduces the incentives for the miners’ participation and may jeopardize the blockchain security. To mitigate this blockchain insufficient fee issue, we propose a Fee and Waiting Tax (FWT) mechanism, which explicitly considers the two types of negative externalities in the system. Specifically, we model the interactions between the protocol designer, users, and miners as a three-stage Stackelberg game. By characterizing the equilibrium of the game, we find that miners neglecting the negative externality in transaction selection cause they are willing to accept insufficient-fee transactions. This leads to the insufficient storage fee issue in the existing protocol (i.e., deployed in Bitcoin and Ethereum). Moreover, our proposed optimal FWT mechanism can motivate users to pay sufficient transaction fees to cover the storage costs and achieve the unconstrained social optimum. Numerical results show that the optimal FWT mechanism guarantees sufficient transaction fees and achieves an average social welfare improvement of 51.43% or more over the existing protocol. Furthermore, the optimal FWT mechanism reduces the average waiting time of low-fee transactions and all transactions by 68.49% and 61.56%, respectively.
Automated Cyber and Privacy Risk Management Toolkit

Addressing cyber and privacy risks has never been more critical for organisations. While a number of risk assessment methodologies and software tools are available, it is most often the case that one must, at least, integrate them into a holistic approach that combines several appropriate risk sources as input to risk mitigation tools. In addition, cyber risk assessment primarily investigates cyber risks as the consequence of vulnerabilities and threats that threaten assets of the investigated infrastructure. In fact, cyber risk assessment is decoupled from privacy impact assessment, which aims to detect privacy-specific threats and assess the degree of compliance with data protection legislation. Furthermore, a Privacy Impact Assessment (PIA) is conducted in a proactive manner during the design phase of a system, combining processing activities and their inter-dependencies with assets, vulnerabilities, real-time threats and Personally Identifiable Information (PII) that may occur during the dynamic life-cycle of systems. In this paper, we propose a cyber and privacy risk management toolkit, called AMBIENT (Automated Cyber and Privacy Risk Management Toolkit) that addresses the above challenges by implementing and integrating three distinct software tools. AMBIENT not only assesses cyber and privacy risks in a thorough and automated manner but it also offers decision-support capabilities, to recommend optimal safeguards using the well-known repository of the Center for Internet Security (CIS) Controls. To the best of our knowledge, AMBIENT is the first toolkit in the academic literature that brings together the aforementioned capabilities. To demonstrate its use, we have created a case scenario based on information about cyber attacks we have received from a healthcare organisation, as a reference sector that faces critical cyber and privacy threats.
An efficient parallel machine learning-based blockchain framework

The unlimited possibilities of machine learning have been shown in several successful reports and applications. However, how to make sure that the searched results of a machine learning system are not tampered by anyone and how to prevent the other users in the same network environment from easily getting our private data are two critical research issues when we immerse into powerful machine learning-based systems or applications. This situation is just like other modern information systems that confront security and privacy issues. The development of blockchain provides us an alternative way to address these two issues. That is why some recent studies have attempted to develop machine learning systems with blockchain technologies or to apply machine learning methods to blockchain systems. To show what the combination of blockchain and machine learning is capable of doing, in this paper, we proposed a parallel framework to find out suitable hyperparameters of deep learning in a blockchain environment by using a metaheuristic algorithm. The proposed framework also takes into account the issue of communication cost, by limiting the number of information exchanges between miners and blockchain.
Unsupervised Anomaly Segmentation Via Multilevel Image Reconstruction and Adaptive Attention-Level Transition

In product inspection scenarios, the objects have different characteristics of textures and different potential anomalies. Still, traditional methods cannot precisely segment structural anomalies and achieve optimal performance on all object categories. This article introduces a flexible multilevel image reconstruction (MLIR) framework for the unsupervised anomaly segmentation task. The MLIR integrates multiple generators with different downsampling ratios to serve as multiple reconstruction levels and leverage an adjustable perceptual similarity measurement to obtain the featurewise reconstruction errors, overcoming the challenge of detecting multiscale anomalies. Leveraging embeddings with different receptive fields for reconstruction, generators perform differently on the same target. Therefore, we depend on the reconstruction uncertainty to propose an adaptive attention-level transition (ALT) strategy that optimizes the detection performance automatically according to the target's category. ALT adjusts the weights of both reconstruction levels and feature measurement scales simultaneously to utilize features at a consistent level for reconstruction and anomaly detection (AD). This strategy remarkably improves the system's adaptivity for different applications. Moreover, we propose reducing the signal-to-noise ratio (SNR) of the images by adding pepper noise so that the reconstruction process is framed as an image denoising task, which improves feature extraction and image restoration robustness. We conduct extensive experiments on the MVTec AD dataset and demonstrate that the proposed method performs satisfactorily to different categories and achieves the state-of-the-art (SOTA) performance.
Human Activity Recognition with Device-Free Sensors for Well-Being Assessment in Smart Homes

Human Activity Recognition (HAR) has attracted much attention in the last two decades with applications such as remote health monitoring, security and surveillance, and smart environments. Specifically, for well-being assessment, HAR systems give us the possibility of recognizing important physical activities in the patient's daily living. For instance, using motion sensors to monitor and record the physical situations and postures of patients with chronic conditions such as arthritis and cardiovascular disease which cause limitations in mobility can be useful in behavior assessment [1]. These physical records, especially for people with disabilities or elderly people, provide caregivers with useful information for treatment. Another example is fall detection that could notify caregivers instantly when a person falls. Today, many types of sensors are used for human activity recognition, including vision-based, wearable, object-tagged and device-free. In this article, we focus on device-free sensors and give and overview of their applications in HAR for well-being assessment in smart homes, including examples from the existing literature and our own test results for simple activity recognition with some of these sensors. We will see that device-free sensors are used predominantly for fall detection, although other well-being applications such as cognitive assessment, respiration monitoring, and dementia detection have emerged as well. Let us begin by looking at various types of HAR sensors and identifying the characteristics of each of them.
Semi-HIC: A novel semi-supervised deep learning method for histopathological image classification

Histopathological images provide a gold standard for cancer recognition and diagnosis. Existing approaches for histopathological image classification are supervised learning methods that demand a large amount of labeled data to obtain satisfying performance, which have to face the challenge of limited data annotation due to prohibitive time cost. To circumvent this shortage, a promising strategy is to design semi-supervised learning methods. Recently, a novel semi-supervised approach called Learning by Association (LA) is proposed, which achieves promising performance in nature image classification. However, there are still great challenges in its application to histopathological image classification due to the wide inter-class similarity and intra-class heterogeneity in histopathological images. To address these issues, we propose a novel semi-supervised deep learning method called Semi-HIC for histopathological image classification. Particularly, we introduce a new semi-supervised loss function combining an association cycle consistency (ACC) loss and a maximal conditional association (MCA) loss, which can take advantage of a large number of unlabeled patches and address the problems of inter-class similarity and intra-class variation in histopathological images, and thereby remarkably improve classification performance for histopathological images. Besides, we employ an efficient network architecture with cascaded Inception blocks (CIBs) to learn rich and discriminative embeddings from patches. Experimental results on both the Bioimaging 2015 challenge dataset and the BACH dataset demonstrate our Semi-HIC method compares favorably with existing deep learning methods for histopathological image classification and consistently outperforms the semi-supervised LA method.
PW-MAD: Pixel-Wise Supervision for Generalized Face Morphing Attack Detection

A face morphing attack image can be verified to multiple identities, making this attack a major vulnerability to processes based on identity verification, such as border checks. Various methods have been proposed to detect face morphing attacks, however, with low generalizability to unexpected post-morphing processes. A major post-morphing process is the print and scan operation performed in many countries when issuing a passport or identity document. In this work, we address this generalization problem by adapting a pixel-wise supervision approach where we train a network to classify each pixel of the image into an attack or not, rather than only having one label for the whole image. Our pixel-wise morphing attack detection (PW-MAD) solution proved to perform more accurately than a set of established baselines. More importantly, PW-MAD shows high generalizability in comparison to related works, when evaluated on unknown re-digitized attacks. Additionally to our PW-MAD approach, we create a new face morphing attack dataset with digital and re-digitized samples, namely the LMA-DRD dataset that is publicly available for research purposes upon request.
DeepBackground: Metamorphic testing for Deep-Learning-driven image recognition systems accompanied by Background-Relevance

Recently, advances in Deep Learning (DL) have promoted the development of DL-driven image recognition systems in various fields, such as medical treatment, face detection, etc., almost achieving the same level of performance as the human brain. Nevertheless, using DL-driven image recognition systems in these safety-critical domains requires ensuring the accuracy and the stability of these systems. Recent research in this direction mainly focuses on using the image transformations for the overall image to detect the inconsistency of image recognition systems. However, the influence of the image background region (i.e., the region of the image other than the target object) on the recognition result of the systems and the robustness evaluation of the systems are not considered. To evaluate the robustness of DL-driven image recognition systems about image background region changes, this paper introduces DeepBackground, a novel metamorphic testing method for DL-driven image recognition systems. First, we define a new metric, termed Background-Relevance (BRC) to assess the influence degree of the image background region on the recognition result of the image recognition systems. DeepBackground defines a series of domain-specific metamorphic relations (MRs) combined with BRC and automatically generates many follow-up test images based on these MRs. Finally, DeepBackground detects the inconsistency of these systems and evaluates their robustness about image background changes according to BRC. Our empirical validation on 3 commercial image recognition services and 6 popular convolutional neural networks (CNNs) models shows that DeepBackground can not only evaluate the robustness of these image recognition systems about image background changes according to BRC, but also can detect their inconsistent behaviors. DeepBackground is capable of automatically generating high-quality test input images to detect the inconsistency of the image recognition systems, and evaluating the robustness of these systems about image background changes according to BRC.