Reviewerly/exHarmony huggingface dataset:
Learning Causal Temporal Relation and Feature Discrimination for Anomaly Detection

Weakly supervised anomaly detection is a challenging task since frame-level labels are not given in the training phase. Previous studies generally employ neural networks to learn features and produce frame-level predictions and then use multiple instance learning (MIL)-based classification loss to ensure the interclass separability of the learned features; all operations simply take into account the current time information as input and ignore the historical observations. According to investigations, these solutions are universal but ignore two essential factors, i.e., the temporal cue and feature discrimination. The former introduces temporal context to enhance the current time feature, and the latter enforces the samples of different categories to be more separable in the feature space. In this article, we propose a method that consists of four modules to leverage the effect of these two ignored factors. The causal temporal relation (CTR) module captures local-range temporal dependencies among features to enhance features. The classifier (CL) projects enhanced features to the category space using the causal convolution and further expands the temporal modeling range. Two additional modules, namely, compactness (CP) and dispersion (DP) modules, are designed to learn the discriminative power of features, where the compactness module ensures the intraclass compactness of normal features, and the dispersion module enhances the interclass dispersion. Extensive experiments on three public benchmarks demonstrate the significance of causal temporal relations and feature discrimination for anomaly detection and the superiority of our proposed method.
Adversarial Reciprocal Points Learning for Open Set Recognition

Open set recognition (OSR), aiming to simultaneously classify the seen classes and identify the unseen classes as 'unknown', is essential for reliable machine learning.The key challenge of OSR is how to reduce the empirical classification risk on the labeled known data and the open space risk on the potential unknown data simultaneously. To handle the challenge, we formulate the open space risk problem from the perspective of multi-class integration, and model the unexploited extra-class space with a novel concept Reciprocal Point. Follow this, a novel learning framework, termed Adversarial Reciprocal Point Learning (ARPL), is proposed to minimize the overlap of known distribution and unknown distributions without loss of known classification accuracy. Specifically, each reciprocal point is learned by the extra-class space with the corresponding known category, and the confrontation among multiple known categories are employed to reduce the empirical classification risk. Then, an adversarial margin constraint is proposed to reduce the open space risk by limiting the latent open space constructed by reciprocal points. To further estimate the unknown distribution from open space, an instantiated adversarial enhancement method is designed to generate diverse and confusing training samples, based on the adversarial mechanism between the reciprocal points and known classes. This can effectively enhance the model distinguishability to the unknown classes. Extensive experimental results on various benchmark datasets indicate that the proposed method is significantly superior to other existing approaches and achieves state-of-the-art performance.
A fast image encryption algorithm based on parallel compressive sensing and DNA sequence

This paper proposes a fast image encryption method using parallel compressive sensing (PCS), composite chaos and DNA sequence. Firstly, to reduce the amount of information transmission and improve computational efficiency, PCS is utilized to the image encryption scheme. The measurement matrices of PCS are generated by Logistic-Tent system (LTS). Next the compressed and encrypted image is blocked and encoded to DNA matrix. And then it is diffused with a chaotic DNA matrix created by Logistic–Sine system (LSS). Finally, the scrambled image is permuted by Tent–Sine system (TSS). It is worth noting that hash values of the original image produced by SHA-256 is adopted to generate the initial values of LTS, LSS and TSS. And the DNA encoding rules we choose are related to plaintext. Therefore, the proposed image encryption algorithm is highly sensitive to the plain image and has good performance against common attacks. Simulation results and security analysis both demonstrate that the cryptosystem has a high level of security.
Non-Local Aggregation for RGB-D Semantic Segmentation

Exploiting both RGB (2D appearance) and Depth (3D geometry) information can improve the performance of semantic segmentation. However, due to the inherent difference between the RGB and Depth information, it remains a challenging problem in how to integrate RGB-D features effectively. In this letter, to address this issue, we propose a Non-local Aggregation Network (NANet), with a well-designed Multi-modality Non-local Aggregation Module (MNAM), to better exploit the non-local context of RGB-D features at multi-stage. Compared with most existing RGB-D semantic segmentation schemes, which only exploit local RGB-D features, the MNAM enables the aggregation of non-local RGB-D information along both spatial and channel dimensions. The proposed NANet achieves comparable performances with state-of-the-art methods on popular RGB-D benchmarks, NYUDv2 and SUN-RGBD.
Unpaired Stain Transfer Using Pathology-Consistent Constrained Generative Adversarial Networks

Pathological examination is the gold standard for the diagnosis of cancer. Common pathological examinations include hematoxylin-eosin (H&E) staining and immunohistochemistry (IHC). In some cases, it is hard to make accurate diagnoses of cancer by referring only to H&E staining images. Whereas, the IHC examination can further provide enough evidence for the diagnosis process. Hence, the generation of virtual IHC images from H&E-stained images will be a good solution for current IHC examination hard accessibility issue, especially for some low-resource regions. However, existing approaches have limitations in microscopic structural preservation and the consistency of pathology properties. In addition, pixel-level paired data is hard available. In our work, we propose a novel adversarial learning method for effective Ki-67-stained image generation from corresponding H&E-stained image. Our method takes fully advantage of structural similarity constraint and skip connection to improve structural details preservation; and pathology consistency constraint and pathological representation network are first proposed to enforce the generated and source images hold the same pathological properties in different staining domains. We empirically demonstrate the effectiveness of our approach on two different unpaired histopathological datasets. Extensive experiments indicate the superior performance of our method that surpasses the state-of-the-art approaches by a significant margin. In addition, our approach also achieves a stable and good performance on unbalanced datasets, which shows our method has strong robustness. We believe that our method has significant potential in clinical virtual staining and advance the progress of computer-aided multi-staining histology image analysis.
Compatibility-Aware Web API Recommendation for Mashup Creation via Textual Description Mining

With the ever-increasing prosperity of web Application Programming Interface (API) sharing platforms, it is becoming an economic and efficient way for software developers to design their interested mashups through web API re-use. Generally, a software developer can browse, evaluate, and select his or her preferred web APIs from the API's sharing platforms to create various mashups with rich functionality. The big volume of candidate APIs places a heavy burden on software developers’ API selection decisions. This, in turn, calls for the support of intelligent API recommender systems. However, existing API recommender systems often face two challenges. First, they focus more on the functional accuracy of APIs while neglecting the APIs’ actual compatibility. This then creates incompatible mashups. Second, they often require software developers to input a set of keywords that can accurately describe the expected functions of the mashup to be developed. This second challenge tests partial developers who have little background knowledge in the fields. To tackle the above-mentioned challenges, in this article we propose a compatibility-aware and text description-driven web API recommendation approach (named WAR text ). WAR text guarantees the compatibility among the recommended APIs by utilizing the APIs’ composition records produced by historical mashup creations. Besides, WAR text entitles a software developer to type a simple text document that describes the expected mashup functions as input. Then through textual description mining, WAR text can precisely capture the developers’ functional requirements and then return a set of APIs with the highest compatibility. Finally, through a real-world mashup dataset ProgrammableWeb, we validate the feasibility of our novel approach.
The role of digital informal learning in the relationship between students' digital competence and academic engagement during the <scp>COVID</scp>‐19 pandemic

During the COVID-19 crisis, digital informal learning is important for students' academic engagement. Although scholars have highlighted the importance of students' digital competence in improving digital informal learning (DIL), the mediating role of DIL between digital competence and academic engagement has remained ambiguous. The purpose of this study is to investigate the relationship between students' digital competence and their academic engagement with the mediating role of DIL in the higher education context. This study used a descriptive correlational design, and the data were analyzed using structural equation modelling (SEM). The study sample included 308 students from Shiraz University, Iran. The results showed that digital competence positively and significantly correlated with students' DIL and their academic engagement. Furthermore, DIL, as the mediator variable, was found to mediate the relationship between students' digital competence and their academic engagement. Since higher education institutions have a key role in improving students' academic engagement, particularly in the COVID-19 pandemic, academic administrators should pay more attention to students' digital competencies and provide them with efficient and user-friendly DIL platforms that can increase their academic engagement.
Characterizing Impacts of Heterogeneity in Federated Learning upon Large-Scale Smartphone Data

Federated learning (FL) is an emerging, privacy-preserving machine learning paradigm, drawing tremendous attention in both academia and industry. A unique characteristic of FL is heterogeneity, which resides in the various hardware specifications and dynamic states across the participating devices. Theoretically, heterogeneity can exert a huge influence on the FL training process, e.g., causing a device unavailable for training or unable to upload its model updates. Unfortunately, these impacts have never been systematically studied and quantified in existing FL literature.
CryptGPU: Fast Privacy-Preserving Machine Learning on the GPU

We introduce CryptGPU, a system for privacy-preserving machine learning that implements all operations on the GPU (graphics processing unit). Just as GPUs played a pivotal role in the success of modern deep learning, they are also essential for realizing scalable privacy-preserving deep learning. In this work, we start by introducing a new interface to losslessly embed cryptographic operations over secret-shared values (in a discrete domain) into floating-point operations that can be processed by highly-optimized CUDA kernels for linear algebra. We then identify a sequence of "GPU-friendly" cryptographic protocols to enable privacy-preserving evaluation of both linear and non-linear operations on the GPU. Our microbenchmarks indicate that our private GPU-based convolution protocol is over 150× faster than the analogous CPU-based protocol; for non-linear operations like the ReLU activation function, our GPU-based protocol is around 10× faster than its CPU analog. With CryptGPU, we support private inference and training on convolutional neural networks with over 60 million parameters as well as handle large datasets like ImageNet. Compared to the previous state-of-the-art, our protocols achieve a 2× to 8× improvement in private inference for large networks and datasets. For private training, we achieve a 6× to 36× improvement over prior state-of-the-art. Our work not only showcases the viability of performing secure multiparty computation (MPC) entirely on the GPU to newly enable fast privacy-preserving machine learning, but also highlights the importance of designing new MPC primitives that can take full advantage of the GPU’s computing capabilities.
Deep Learning and Blockchain-Empowered Security Framework for Intelligent 5G-Enabled IoT

Recently, many IoT applications, such as smart transportation, healthcare, and virtual and augmented reality experiences, have emerged with fifth-generation (5G) technology to enhance the Quality of Service (QoS) and user experience. The revolution of 5G-enabled IoT supports distinct attributes, including lower latency, higher system capacity, high data rate, and energy saving. However, such revolution also delivers considerable increment in data generation that further leads to a major requirement of intelligent and effective data analytic operation across the network. Furthermore, data growth gives rise to data security and privacy concerns, such as breach and loss of sensitive data. The conventional data analytic and security methods do not meet the requirement of 5G-enabled IoT including its unique characteristic of low latency and high throughput. In this paper, we propose a Deep Learning (DL) and blockchain-empowered security framework for intelligent 5G-enabled IoT that leverages DL competency for intelligent data analysis operation and blockchain for data security. The framework’s hierarchical architecture wherein DL and blockchain operations emerge across the four layers of cloud, fog, edge, and user is presented. The framework is simulated and analyzed, employing various standard measures of latency, accuracy, and security to demonstrate its validity in practical applications.
Heterogeneous Dataflow Accelerators for Multi-DNN Workloads

Emerging AI-enabled applications such as augmented and virtual reality (AR/VR) leverage multiple deep neural network (DNN) models for various sub-tasks such as object detection, image segmentation, eye-tracking, speech recognition, and so on. Because of the diversity of the sub-tasks, the layers within and across the DNN models are highly heterogeneous in operation and shape. Diverse layer operations and shapes are major challenges for a fixed dataflow accelerator (FDA) that employs a fixed dataflow strategy on a single DNN accelerator substrate since each layer prefers different dataflows (computation order and parallelization) and tile sizes. Reconfigurable DNN accelerators (RDAs) have been proposed to adapt their dataflows to diverse layers to address the challenge. However, the dataflow flexibility in RDAs is enabled at the cost of expensive hardware structures (switches, interconnects, controller, etc.) and requires per-layer reconfiguration, which introduces considerable energy costs. Alternatively, this work proposes a new class of accelerators, heterogeneous dataflow accelerators (HDAs), which deploy multiple accelerator substrates (i.e., sub-accelerators), each supporting a different dataflow. HDAs enable coarser-grained dataflow flexibility than RDAs with higher energy efficiency and lower area cost comparable to FDAs. To exploit such benefits, hardware resource partitioning across sub-accelerators and layer execution schedule need to be carefully optimized. Therefore, we also present Herald, a framework for co-optimizing hardware partitioning and layer scheduling. Using Herald on a suite of AR/VR and MLPerf workloads, we identify a promising HDA architecture, Maelstrom, which demonstrates 65.3% lower latency and 5.0% lower energy compared to the best fixed dataflow accelerators and 22.0% lower energy at the cost of 20.7% higher latency compared to a state-of-the-art reconfigurable DNN accelerator (RDA). The results suggest that HDA is an alternative class of Pareto-optimal accelerators to RDA with strength in energy, which can be a better choice than RDAs depending on the use cases.
Deep fake geography? When geospatial data encounter Artificial Intelligence

The developing convergence of Artificial Intelligence and GIScience has raised a concern on the emergence of deep fake geography and its potentials in transforming human perception of the geographic world. Situating fake geography under the context of modern cartography and GIScience, this paper presents an empirical study to dissect the algorithmic mechanism of falsifying satellite images with non-existent landscape features. To demonstrate our pioneering attempt at deep fake detection, a robust approach is then proposed and evaluated. Our proactive study warns of the emergence and proliferation of deep fakes in geography just as “lies” in maps. We suggest timely detections of deep fakes in geospatial data and proper coping strategies when necessary. More importantly, it is encouraged to cultivate a critical geospatial data literacy and thus to understand the multi-faceted impacts of deep fake geography on individuals and human society.
A Verifiable and Fair Attribute-Based Proxy Re-Encryption Scheme for Data Sharing in Clouds

To manage outsourced encrypted data sharing in clouds, attribute-based proxy re-encryption (ABPRE) has become an elegant primitive. In ABPRE, a cloud server can transform an original recipient’s ciphertext to a new one of a shared user’s. As the transformation is computation consuming, a malicious cloud server may return an incorrect re-encrypted ciphertext to save its computation resources. Moreover, a shared user may accuse the cloud server of returning an incorrect re-encrypted ciphertext to refuse to pay the cost of using the cloud service. However, existing ABPRE schemes do not support a mechanism to achieve verifiability and fairness. In this article, a novel verifiable and fair attribute-based proxy re-encryption (VF-ABPRE) scheme is introduced to support verifiability and fairness. The verifiability enables a shared user to verify whether the re-encrypted ciphertext returned by the server is correct and the fairness ensures a cloud server escape from malicious accusation if it has indeed conducted the re-encryption operation honestly. Additionally, we conduct a performance experiment to show the efficiency and practicality of the new VF-ABPRE scheme.
A novel POI recommendation method based on trust relationship and spatial–temporal factors

Existing point-of-interesting (POI) recommendation methods lack sufficient integration of information related to the features of individual users and their corresponding contexts. Personalized location-based services built upon these methods have a few limitations such as low recommendation accuracy and untapped potential interests of users. To overcome these limitations, we propose an efficient POI recommendation method based on multiple factors (i.e., preference, social relationship, and spatial–temporal factors) to improve the recommendation accuracy and alleviate the cold-start and sparsity problems. Unlike existing methods, the proposed method considers the features of each factor. First, we identify direct and indirect trust relationships, upon which the improved trust relationship measurement methods are built respectively. Second, we fuse the comprehensive trust relationship, user preference, check-in time, and geographical location into a matrix factorization model. We pay special attention to the attraction of items, correlation between items, trajectory composed of locations, and the influence of interest-forgetting during the fusing process. Finally, we generate recommendation lists of POIs for the target users. With three real-world datasets, experimental and analytical results show that the proposed method outperforms existing methods, while alleviating the cold-start and sparsity problems that commonly hinder POI recommender systems. Theoretically, our study contributes to the effective usage of multidimensional data science and analytics for POI recommender system design. In practice, our results can be used to improve the quality of personalized POI recommendation services for websites and applications.
Blockchain Based Transaction System with Fungible and Non-Fungible Tokens for a Community-Based Energy Infrastructure

Renewable energy microgeneration is rising leading to creation of prosumer communities making it possible to extract value from surplus energy and usage flexibility. Such a peer-to-peer energy trading community requires a decentralized, immutable and access-controlled transaction system for tokenized energy assets. In this study we present a unified blockchain-based system for energy asset transactions among prosumers, electric vehicles, power companies and storage providers. Two versions of the system were implemented on Hyperledger Fabric. Assets encapsulating an identifier or unique information along with value are modelled as non-fungible tokens (NFT), while those representing value only are modelled as fungible tokens (FT). We developed the associated algorithms for token lifecycle management, analyzed their complexities and encoded them in smart contracts for performance testing. The results show that performance of both implementations are comparable for most major operations. Further, we presented a detailed comparison of FT and NFT implementations based on use-case, design, performance, advantages and disadvantages. Our implementation achieved a throughput of 448.3 transactions per second for the slowest operation (transfer) with a reasonably low infrastructure.
Sustainable Supply Chains with Blockchain, IoT and RFID: A Simulation on Order Management

The digital transformation of supply chains should revolutionize entire management processes and improve various aspects of sustainability. In particular, the plans of Industry 4.0 aim towards a digitization of several procedures by exploiting emerging technologies such as the Internet of Things, RFID and blockchain. The purpose of this study is to highlight how order and disruption events processes can be improved with the adoption of emerging technologies and how this reflects on the improvement of sustainability aspects. The study is based on the comparison of two simulation scenarios between three actors in the cheese supply chain. In particular, a first traditional scenario “as is” is simulated without the use of new technologies and is compared to a second scenario “to be” that adopts IoT, RFID and blockchain. The results show an improvement in time performance for managing both perfect and non-compliant orders. The developed framework highlights the impact of new technologies on sustainability aspects, showing further managerial implications.