Reviewerly/exHarmony huggingface dataset:
The Blessings of Unlabeled Background in Untrimmed Videos

Weakly-supervised Temporal Action Localization (WTAL) aims to detect the action segments with only video-level action labels in training. The key challenge is how to distinguish the action of interest segments from the background, which is unlabelled even on the video-level. While previous works treat the background as "curses", we consider it as "blessings". Specifically, we first use causal analysis to point out that the common localization errors are due to the unobserved confounder that resides ubiquitously in visual recognition. Then, we propose a Temporal Smoothing PCA-based (TS-PCA) deconfounder, which exploits the unlabelled background to model an observed substitute for the unobserved confounder, to remove the confounding effect. Note that the proposed deconfounder is model-agnostic and non-intrusive, and hence can be applied in any WTAL method without model re-designs. Through extensive experiments on four state-of-the-art WTAL methods, we show that the deconfounder can improve all of them on the public datasets: THUMOS-14 and ActivityNet-1.3 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
A Security- and Privacy-Preserving Approach Based on Data Disturbance for Collaborative Edge Computing in Social IoT Systems

The Internet of things (IoT) has certainly become one of the hottest technology frameworks of the year. It is deep in many industries, affecting people’s lives in all directions. The rapid development of the IoT technology accelerates the process of the era of “Internet of everything” but also changes the role of terminal equipment at the edge of the network. It has changed from a single data user to a dual role of both producing and using data. And collaborative edge computing (CEC) has been born in time. CEC itself can not only solve the problem of computing and storage but also combines with the deep learning (DL) model to make full use of edge computing ability. However, as the core of DL, the robustness of neural network is often not high. In addition, edge devices of CEC are facing a highly dynamic environment, which can easily cause the edge network to be attacked by malicious devices. Therefore, user privacy protection and security issues for CEC deserve more attention. To avoid privacy leakage and security crisis of CEC in social IoT systems, a data protection method based on data disturbance method and adversarial training viewpoint is introduced in this article. Besides, a new adversarial sample generation method based on the firefly algorithm (FA) is proposed. This method reduces the time complexity of traditional by an order for magnitude compared with traditional generative adversarial network (GAN) generation. Since sentences, information on CEC in the IoT system is characterized by a large amount of data, strict confidentiality, and high-security requirements, and they are usually high-risk information on privacy leakage. The proposed method is conducted to the sentence similarity analysis model based on a convolutional neural network (CNN) in the CEC scene to test the feasibility of the method. Compared with the original CNN, the accuracy of the model using the confrontation training method is improved by 4.8%. At the same time, the security value of our model is 2.1% higher than that of the simple CNN model, and it has the best security performance among the four comparison models. Further experiments have demonstrated that the model performs better in its capacity of resisting disturbance and can effectively help multiple organizations to implement data usage and sentence information on the requirements of user privacy protection, data security, and government regulations.
Transferable Query Selection for Active Domain Adaptation

Unsupervised domain adaptation (UDA) enables transferring knowledge from a related source domain to a fully unlabeled target domain. Despite the significant advances in UDA, the performance gap remains quite large between UDA and supervised learning with fully labeled target data. Active domain adaptation (ADA) mitigates the gap under minimal annotation cost by selecting a small quota of target samples to annotate and incorporating them into training. Due to the domain shift, the query selection criteria of prior active learning methods may be ineffective to select the most informative target samples for annotation. In this paper, we propose Transferable Query Selection (TQS), which selects the most informative samples under domain shift by an ensemble of three new criteria: transferable committee, transferable uncertainty, and transferable domainness. We further develop a randomized selection algorithm to enhance the diversity of the selected samples. Experiments show that TQS remarkably outperforms previous UDA and ADA methods on several domain adaptation datasets. Deeper analyses demonstrate that TQS can select the most informative target samples under the domain shift.
Immersive Digital Tourism: The Role of Multisensory Cues in Digital Museum Experiences

Facilitated by emerging technologies, the immersive digital museum reflects disruptive innovation in today’s tourism experience and offers a multidimensional experience different from traditional museums. To better understand how visitors respond to this innovative form of digital tourism, the current research investigates visitor experiences at the digital museum and achieves a threefold goal. First, this research delineates a three-dimensional digital museum visitor experience, namely, joviality, personal escapism, and localness. Built on this experiential framework, the present research further affirms that visual and auditory cues are the most powerful multisensory cue combination in enhancing a holistic visitor experience at the digital museum. This study also finds that emotional state and sense of presence mediate the relations between the multisensory cues and visitors’ digital museum experiences. This research contributes to the conceptualization of a digital museum experience, and provides a foundation for the future research endeavor of the new generation of digital tourism.
Strategic analysis and development plan design on digital transformation in the energy industry: A global perspective

This paper conducts strategic analysis and development plan design of energy digital transformation from a global perspective. First, this paper analyzes the reasons for the digital transformation of the energy industry, and elaborates on the current status of the digital transformation of the energy industry. Second, this paper uses the “SWOT” model to conduct a strategic analysis of the digital transformation of energy. Finally, this paper uses the “IETB” model to design the development plan. The results showed that: (1) Energy digital transformation is an effective approach to optimize the energy allocation and improve the energy scheduling. (2) The biggest barriers and uncertainties are mainly from the current weaknesses and the future threats. (3) The joint action of factors from Institution, Economy, Technology as well as Behavior achieves an effective scheme to promote the digital transformation of energy. HIGHLIGHTS Reasons for the digital transformation of energy. The development status of the digital transformation is outlined. Using the “SWOT” model to conduct a strategic analysis of the digital transformation of energy. Using the “IETB” model to design the development plan.
Machine Learning Assisted Differential Distinguishers For Lightweight Ciphers

At CRYPTO 2019, Gohr first introduces the deep learning based cryptanalysis on round-reduced SPECK. Using a deep residual network, Gohr trains several neural network based distinguishers on 8-round SPECK-32/64. The analysis follows an ‘all-in-one’ differential cryptanalysis approach, which considers all the output differences effect under the same input difference. Usually, the all-in-one differential cryptanalysis is more effective compared to the one using only one single differential trail. However, when the cipher is non-Markov or its block size is large, it is usually very hard to fully compute. Inspired by Gohr's work, we try to simulate the all-in-one differentials for non-Markov ciphers through machine learning. Our idea here is to reduce a distinguishing problem to a classification problem, so that it can be efficiently managed by machine learning. As a proof of concept, we show several distinguishers for four high profile ciphers, each of which works with trivial complexity. In particular, we show differential distinguishers for 8-round Gimli-Hash, Gimli-Cipher and Gimli-Permutation; 3-round Ascon-Permutation; 10-round Knot-256 permutation and 12-round Knot-512 permutation; and 4-round Chaskey-Permutation. Finally, we explore more on choosing an efficient machine learning model and observe that only a three layer neural network can be used. Our analysis shows the attacker is able to reduce the complexity of finding distinguishers by using machine learning techniques.
Contact Tracing App Privacy: What Data Is Shared By Europe’s GAEN Contact Tracing Apps

We describe the data transmitted to backend servers by the contact tracing apps now deployed in Europe with a view to evaluating user privacy. These apps consist of two separate components: a "client" app managed by the national public health authority and the Google/Apple Exposure Notification (GAEN) service, that on Android devices is managed by Google and is part of Google Play Services. We find that the health authority client apps are generally well behaved from a privacy point of view, although the privacy of the Irish, Polish, Danish and Latvian apps could be improved. In marked contrast, we find that the Google Play Services component of these apps is problematic from a privacy viewpoint. Even when minimally configured, Google Play Services still contacts Google servers roughly every 20 minutes, potentially allowing location tracking via IP address. In addition, the phone IMEI, hardware serial number, SIM serial number and IMSI, handset phone number etc are shared with Google, together with detailed data on phone activity. This data collection is enabled simply by enabling Google Play Services, even when all other Google services and settings are disabled, and so is unavoidable for users of GAEN-based contact tracing apps on Android.
Secure Watermarking Schemes and Their Approaches in the IoT Technology: An Overview

Information security is considered one of the most important issues in various infrastructures related to the field of data communication where most of the modern studies focus on finding effective and low-weight secure approaches. Digital watermarking is a trend in security techniques that hides data by using data embedding and data extraction processes. Watermarking technology is integrated into different frames without adding an overheard as in the conventional encryption. Therefore, it is efficient to be used in data encryption for applications that run over limited resources such as the Internet of Things (IoT). In this paper, different digital watermarking algorithms and approaches are presented. Additionally, watermarking requirements and challenges are illustrated in detail. Moreover, the common architecture of the watermarking system is described. Furthermore, IoT technology and its challenges are highlighted. Finally, the paper provides the motivations, objectives and applications of the recent secure watermarking techniques in IoT and summarises them into one table. In addition, the paper highlights the potential to apply the modified watermark algorithms to secure IoT networks.
Explainable artificial intelligence for education and training

Researchers and software users benefit from the rapid growth of artificial intelligence (AI) to an unprecedented extent in various domains where automated intelligent action is required. However, as they continue to engage with AI, they also begin to understand the limitations and risks associated with ceding control and decision-making to not always transparent artificial computer agents. Understanding of “what is happening in the black box” becomes feasible with explainable AI (XAI) methods designed to mitigate these risks and introduce trust into human-AI interactions. Our study reviews the essential capabilities, limitations, and desiderata of XAI tools developed over recent years and reviews the history of XAI and AI in education (AIED). We present different approaches to AI and XAI from the viewpoint of researchers focused on AIED in comparison with researchers focused on AI and machine learning (ML). We conclude that both groups of interest desire increased efforts to obtain improved XAI tools; however, these groups formulate different target user groups and expectations regarding XAI features and provide different examples of possible achievements. We summarize these viewpoints and provide guidelines for scientists looking to incorporate XAI into their own work.
A Temporal Logic for Asynchronous Hyperproperties

Abstract Hyperproperties are properties of computational systems that require more than one trace to evaluate, e.g., many information-flow security and concurrency requirements. Where a trace property defines a set of traces, a hyperproperty defines a set of sets of traces. The temporal logics HyperLTL and HyperCTL* have been proposed to express hyperproperties. However, their semantics are synchronous in the sense that all traces proceed at the same speed and are evaluated at the same position. This precludes the use of these logics to analyze systems whose traces can proceed at different speeds and allow that different traces take stuttering steps independently. To solve this problem in this paper, we propose an asynchronous variant of HyperLTL. On the negative side, we show that the model-checking problem for this variant is undecidable. On the positive side, we identify a decidable fragment which covers a rich set of formulas with practical applications. We also propose two model-checking algorithms that reduce our problem to the HyperLTL model-checking problem in the synchronous semantics.
How to construct physical zero-knowledge proofs for puzzles with a “single loop” condition

We propose a technique to construct physical Zero-Knowledge Proof (ZKP) protocols for puzzles that require a single loop draw feature. Our approach is based on the observation that a loop has only one hole and this property remains stable by some simple transformations. Using this trick, we can transform a simple big loop, which is visible to anyone, into the solution loop by using transformations that do not disclose any information about the solution. We illustrate our technique by applying it to construct physical ZKP protocols for two Nikoli puzzles: Slitherlink and Masyu.
Interpretability of time-series deep learning models: A study in cardiovascular patients admitted to Intensive care unit

Interpretability is fundamental in healthcare problems and the lack of it in deep learning models is currently the major barrier in the usage of such powerful algorithms in the field. The study describes the implementation of an attention layer for Long Short-Term Memory (LSTM) neural network that provides a useful picture on the influence of the several input variables included in the model. A cohort of 10,616 patients with cardiovascular diseases is selected from the MIMIC III dataset, an openly available database of electronic health records (EHRs) including all patients admitted to an ICU at Boston's Medical Centre. For each patient, we consider a 10-length sequence of 1-hour windows in which 48 clinical parameters are extracted to predict the occurrence of death in the next 7 days. Inspired from the recent developments in the field of attention mechanisms for sequential data, we implement a recurrent neural network with LSTM cells incorporating an attention mechanism to identify features driving model's decisions over time. The performance of the LSTM model, measured in terms of AUC, is 0.790 (SD = 0.015). Regard our primary objective, i.e. model interpretability, we investigate the role of attention weights. We find good correspondence with driving predictors of a transparent model (r = 0.611, 95% CI [0.395, 0.763]). Moreover, most influential features identified at the cohort-level emerge as known risk factors in the clinical context. Despite the limitations of study dataset, this work brings further evidence of the potential of attention mechanisms in making deep learning model more interpretable and suggests the application of this strategy for the sequential analysis of EHRs.
Dimensions of commonsense knowledge

Commonsense knowledge is essential for many AI applications, including those in natural language processing, visual processing, and planning. Consequently, many sources that include commonsense knowledge have been designed and constructed over the past decades. Recently, the focus has been on large text-based sources, which facilitate easier integration with neural (language) models and application to textual tasks, typically at the expense of the semantics of the sources and their harmonization. Efforts to consolidate commonsense knowledge have yielded partial success, with no clear path towards a comprehensive solution. We aim to organize these sources around a common set of dimensions of commonsense knowledge. We survey a wide range of popular commonsense sources with a special focus on their relations. We consolidate these relations into 13 knowledge dimensions. This consolidation allows us to unify the separate sources and to compute indications of their coverage, overlap, and gaps with respect to the knowledge dimensions. Moreover, we analyze the impact of each dimension on downstream reasoning tasks that require commonsense knowledge, observing that the temporal and desire/goal dimensions are very beneficial for reasoning on current downstream tasks, while distinctness and lexical knowledge have little impact. These results reveal preferences for some dimensions in current evaluation, and potential neglect of others.
Multi-Scene Building Height Estimation Method Based on Shadow in High Resolution Imagery

Accurately building height estimation from remote sensing imagery is an important and challenging task. However, the existing shadow-based building height estimation methods have large errors due to the complex environment in remote sensing imagery. In this paper, we propose a multi-scene building height estimation method based on shadow in high resolution imagery. First, the shadow of building is classified and described by analyzing the features of building shadow in remote sensing imagery. Second, a variety of shadow-based building height estimation models is established in different scenes. In addition, a method of shadow regularization extraction is proposed, which can solve the problem of mutual adhesion shadows in dense building areas effectively. Finally, we propose a method for shadow length calculation combines with the fish net and the pauta criterion, which means that the large error caused by the complex shape of building shadow can be avoided. Multi-scene areas are selected for experimental analysis to prove the validity of our method. The experiment results show that the accuracy rate is as high as 96% within 2 m of absolute error of our method. In addition, we compared our proposed approach with the existing methods, and the results show that the absolute error of our method are reduced by 1.24 m–3.76 m, which can achieve high-precision estimation of building height.
UFFDFR: Undersampling framework with denoising, fuzzy c-means clustering, and representative sample selection for imbalanced data classification

In the field of artificial intelligence, classification algorithms tend to be biased toward the majority class samples when encountering imbalanced data, resulting in low recognition rates for minority class samples. Undersampling techniques address this issue by decreasing the number of majority class samples to balance the original data distribution before the dataset is learned. However, current clustering-based undersampling methods have limitations that directly affect the original imbalanced dataset and the final classification performance. To address these problems, we propose a novel three-stage undersampling framework with denoising, fuzzy c-means clustering, and representative sample selection (UFFDFR). This framework improves the classification performance on imbalanced data by removing noise and unrepresentative samples from the majority class. Experiments on 15 different imbalanced datasets demonstrate that UFFDFR effectively removed noise and unrepresentative majority class samples and improved classification performance. Furthermore, UFFDFR outperformed three classic and three state-of-the-art clustering-based undersampling methods in terms F-measure, G-mean, and AUC for five classification algorithms, which was confirmed by the Friedman and Nemenyi post-hoc statistical tests.
Applying deep neural networks for the automatic recognition of sign language words: A communication aid to deaf agriculturists

One of the major challenges that deaf people face in modern societal life is communication. For those engaged in agricultural jobs, efficiency at work and productivity are deeply related to the quality of deciphering the sign language used by the deaf farmers. Employing sign language interpreters is not a pragmatic solution to this problem. There comes the need for developing a reliable system for automatic sign language recognition (SLR). This paper reports a work on the recognition of hand gestures for the Indian sign language (ISL) words commonly used by deaf farmers. A hybrid deep learning model with convolutional long short term memory (LSTM) network has been exploited for gesture classification. The model has attained an average classification accuracy of 76.21% on the proposed dataset of ISL words from the agricultural domain.
Application of the Internet of Things in 3E (efficiency, economy, and environment) factor-based energy management as smart and sustainable strategy

(2021). Application of the Internet of Things in 3E (efficiency, economy, and environment) factor-based energy management as smart and sustainable strategy. Energy Sources, Part A: Recovery, Utilization, and Environmental Effects. Ahead of Print.
Lipstick ain’t enough: Beyond Color Matching for In-the-Wild Makeup Transfer

Makeup transfer is the task of applying on a source face the makeup style from a reference image. Real-life makeups are diverse and wild, which cover not only color-changing but also patterns, such as stickers, blushes, and jewelries. However, existing works overlooked the latter components and confined makeup transfer to color manipulation, focusing only on light makeup styles. In this work, we propose a holistic makeup transfer framework that can handle all the mentioned makeup components. It consists of an improved color transfer branch and a novel pattern transfer branch to learn all makeup properties, including color, shape, texture, and location. To train and evaluate such a system, we also introduce new makeup datasets for real and synthetic extreme makeup. Experimental results show that our framework achieves the state of the art performance on both light and extreme makeup styles. Code is available at https://github.com/VinAIResearch/CPM.