Reviewerly/exHarmony huggingface dataset:
Systematic Literature Review on Cyber Situational Awareness Visualizations

The dynamics of cyber threats are increasingly complex, making it more challenging than ever for organizations to obtain in-depth insights into their cyber security status. Therefore, organizations rely on Cyber Situational Awareness (CSA) to support them in better understanding the threats and associated impacts of cyber events. Due to the heterogeneity and complexity of cyber security data, often with multidimensional attributes, sophisticated visualization techniques are needed to achieve CSA. However, there have been no previous attempts to systematically review and analyze the scientific literature on CSA visualizations. In this paper, we systematically select and review 54 publications that discuss visualizations to support CSA. We extract data from these papers to identify key stakeholders, information types, data sources, and visualization techniques. Furthermore, we analyze the level of CSA supported by the visualizations, alongside examining the maturity of the visualizations, challenges, and practices related to CSA visualizations to prepare a full analysis of the current state of CSA in an organizational context. Our results reveal certain gaps in CSA visualizations. For instance, the largest focus is on operational-level staff, and there is a clear lack of visualizations targeting other types of stakeholders such as managers, higher-level decision makers, and non-expert users. Most papers focus on threat information visualization, and there is a dearth of papers that visualize impact information, response plans, and information shared within teams. Interestingly, we find that only a few studies proposed visualizations to facilitate up to the projection level (i.e., the highest level of CSA), whereas most studies facilitated only the perception level (i.e., the lowest level of CSA). Most of the studies provide evidence of the proposed visualizations through toy examples and demonstrations, while only a few visualizations are employed in industrial practice. Based on the results that highlight the important concerns in CSA visualizations, we recommend a list of future research directions.
DSNet: A Dual-Stream Framework for Weakly-Supervised Gigapixel Pathology Image Analysis

We present a novel weakly-supervised framework for classifying whole slide images (WSIs). WSIs, due to their gigapixel resolution, are commonly processed by patch-wise classification with patch-level labels. However, patch-level labels require precise annotations, which is expensive and usually unavailable on clinical data. With image-level labels only, patch-wise classification would be sub-optimal due to inconsistency between the patch appearance and image-level label. To address this issue, we posit that WSI analysis can be effectively conducted by integrating information at both high magnification (local) and low magnification (regional) levels. We auto-encode the visual signals in each patch into a latent embedding vector representing local information, and down-sample the raw WSI to hardware-acceptable thumbnails representing regional information. The WSI label is then predicted with a Dual-Stream Network (DSNet), which takes the transformed local patch embeddings and multi-scale thumbnail images as inputs and can be trained by the image-level label only. Experiments conducted on three large-scale public datasets demonstrate that our method outperforms all recent state-of-the-art weakly-supervised WSI classification methods.
Do item-dependent context representations underlie serial order in cognition? Commentary on Logan (2021).

Logan (2021) presented an impressive unification of serial order tasks including whole report, typing, and serial recall in the form of the context retrieval and updating (CRU) model. Despite the wide breadth of the model's coverage, its reliance on encoding and retrieving context representations that consist of the previous items may prevent it from being able to address a number of critical benchmark findings in the serial order literature that have shaped and constrained existing theories. In this commentary, we highlight three major challenges that motivated the development of a rival class of models of serial order, namely positional models. These challenges include the mixed-list phonological similarity effect, the protrusion effect, and interposition errors in temporal grouping. Simulations indicated that CRU can address the mixed-list phonological similarity effect if phonological confusions can occur during its output stage, suggesting that the serial position curves from this paradigm do not rule out models that rely on interitem associations, as has been previously been suggested. The other two challenges are more consequential for the model's representations, and simulations indicated the model was not able to provide a complete account of them. We highlight and discuss how revisions to CRU's representations or retrieval mechanisms can address these phenomena and emphasize that a fruitful direction forward would be to either incorporate positional representations or approximate them with its existing representations. (PsycInfo Database Record (c) 2023 APA, all rights reserved).
Revisiting Over-Smoothness in Text to Speech

Non-autoregressive text to speech (NAR-TTS) models have attracted much attention from both academia and industry due to their fast generation speed. One limitation of NAR-TTS models is that they ignore the correlation in time and frequency domains while generating speech mel-spectrograms, and thus cause blurry and over-smoothed results. In this work, we revisit this over-smoothing problem from a novel perspective: the degree of over-smoothness is determined by the gap between the complexity of data distributions and the capability of modeling methods. Both simplifying data distributions and improving modeling methods can alleviate the problem. Accordingly, we first study methods reducing the complexity of data distributions. Then we conduct a comprehensive study on NAR-TTS models that use some advanced modeling methods. Based on these studies, we find that 1) methods that provide additional condition inputs reduce the complexity of data distributions to model, thus alleviating the over-smoothing problem and achieving better voice quality. 2) Among advanced modeling methods, Laplacian mixture loss performs well at modeling multimodal distributions and enjoys its simplicity, while GAN and Glow achieve the best voice quality while suffering from increased training or model complexity. 3) The two categories of methods can be combined to further alleviate the over-smoothness and improve the voice quality. 4) Our experiments on the multi-speaker dataset lead to similar conclusions as above and providing more variance information can reduce the difficulty of modeling the target data distribution and alleviate the requirements for model capacity.
Streaming Multi-Talker ASR with Token-Level Serialized Output Training

This paper proposes a token-level serialized output training (t-SOT), a novel framework for streaming multi-talker automatic speech recognition (ASR). Unlike existing streaming multi-talker ASR models using multiple output layers, the t-SOT model has only a single output layer that generates recognition tokens (e.g., words, subwords) of multiple speakers in chronological order based on their emission times. A special token that indicates the change of "virtual" output channels is introduced to keep track of the overlapping utterances. Compared to the prior streaming multi-talker ASR models, the t-SOT model has the advantages of less inference cost and a simpler model architecture. Moreover, in our experiments with LibriSpeechMix and LibriCSS datasets, the t-SOT-based transformer transducer model achieves the state-of-the-art word error rates by a significant margin to the prior results. For non-overlapping speech, the t-SOT model is on par with a single-talker ASR model in terms of both accuracy and computational cost, opening the door for deploying one model for both single- and multi-talker scenarios.
A digital transformation-enabled framework and strategies for public health risk response and governance: China's experience

Purpose Under uncertain circumstances, digital technologies are taken as digital transformation enablers and driving forces to integrate with medical, healthcare and emergency management research for effective epidemic prevention and control. This study aims to adapt complex systems in emergency management. Thus, a digital transformation-driven and systematic circulation framework is proposed in this study that can utilize the advantages of digital technologies to generate innovative and systematic governance. Design/methodology/approach Aiming at adapting complex systems in emergency management, a systematic circulation framework based on the interpretive research is proposed in this study that can utilize the advantages of digital technologies to generate innovative and systematic governance. The framework consists of four phases: (1) analysis of emergency management stages, (2) risk identification in the emergency management stages, (3) digital-enabled response model design for emergency management, and (4) strategy generation for digital emergency governance. A case study in China was illustrated in this study. Findings This paper examines the role those digital technologies can play in responding to pandemics and outlines a framework based on four phases of digital technologies for pandemic responses. After the phase-by-phase analysis, a digital technology-enabled emergency management framework, titled “Expected digital-enabled emergency management framework (EDEM framework)” was adapted and proposed. Moreover, the social risks of emergency management phases are identified. Then, three strategies for emergency governance and digital governance from the three perspectives, namely “Strengthening weaknesses for emergency response,” “Enhancing integration for collaborative governance,” and “Engaging foundations for emergency management” that the government can adopt them in the future, fight for public health emergency events. Originality/value The novel digital transformation-driven systematic circulation framework for public health risk response and governance was proposed. Meanwhile, an “Expected digital-enabled emergency management framework (EDEM model)” was also proposed to achieve a more effective empirical response for public health risk response and governance and contribute to studies about the government facing the COVID-19 pandemic effectively.
Adaptive Domain Interest Network for Multi-domain Recommendation

Industrial recommender systems usually hold data from multiple business scenarios and are expected to provide recommendation services for these scenarios simultaneously. In the retrieval step, the topK high-quality items selected from a large number of corpus usually need to be various for multiple scenarios. Take Alibaba display advertising system for example, not only because the behavior patterns of Taobao users are diverse, but also differentiated scenarios' bid prices assigned by advertisers vary significantly. Traditional methods either train models for each scenario separately, ignoring the cross-domain overlapping of user groups and items, or simply mix all samples and maintain a shared model which makes it difficult to capture significant diversities between scenarios. In this paper, we present Adaptive Domain Interest Network(ADIN) that adaptively handles the commonalities and diversities across scenarios, making full use of multi-scenarios data during training. Then the proposed method is able to improve the performance of each business domain by giving various topK candidates for different scenarios during online inference. Specifically, our proposed ADIN models the commonalities and diversities for different domains by shared networks and domain-specific networks, respectively. In addition, we apply the domain-specific batch normalization and design the domain interest adaptation layer for feature-level domain adaptation. A self training strategy is also incorporated to capture label-level connections across domains.ADIN has been deployed in the display advertising system of Alibaba, and obtains 1.8% improvement on advertising revenue.
Not All Parameters Should Be Treated Equally: Deep Safe Semi-supervised Learning under Class Distribution Mismatch

Deep semi-supervised learning (SSL) aims to utilize a sizeable unlabeled set to train deep networks, thereby reducing the dependence on labeled instances. However, the unlabeled set often carries unseen classes that cause the deep SSL algorithm to lose generalization. Previous works focus on the data level that they attempt to remove unseen class data or assign lower weight to them but could not eliminate their adverse effects on the SSL algorithm. Rather than focusing on the data level, this paper turns attention to the model parameter level. We find that only partial parameters are essential for seen-class classification, termed safe parameters. In contrast, the other parameters tend to fit irrelevant data, termed harmful parameters. Driven by this insight, we propose Safe Parameter Learning (SPL) to discover safe parameters and make the harmful parameters inactive, such that we can mitigate the adverse effects caused by unseen-class data. Specifically, we firstly design an effective strategy to divide all parameters in the pre-trained SSL model into safe and harmful ones. Then, we introduce a bi-level optimization strategy to update the safe parameters and kill the harmful parameters. Extensive experiments show that SPL outperforms the state-of-the-art SSL methods on all the benchmarks by a large margin. Moreover, experiments demonstrate that SPL can be integrated into the most popular deep SSL networks and be easily extended to handle other cases of class distribution mismatch.
TVT: Three-Way Vision Transformer through Multi-Modal Hypersphere Learning for Zero-Shot Sketch-Based Image Retrieval

In this paper, we study the zero-shot sketch-based image retrieval (ZS-SBIR) task, which retrieves natural images related to sketch queries from unseen categories. In the literature, convolutional neural networks (CNNs) have become the de-facto standard and they are either trained end-to-end or used to extract pre-trained features for images and sketches. However, CNNs are limited in modeling the global structural information of objects due to the intrinsic locality of convolution operations. To this end, we propose a Transformer-based approach called Three-Way Vision Transformer (TVT) to leverage the ability of Vision Transformer (ViT) to model global contexts due to the global self-attention mechanism. Going beyond simply applying ViT to this task, we propose a token-based strategy of adding fusion and distillation tokens and making them complementary to each other. Specifically, we integrate three ViTs, which are pre-trained on data of each modality, into a three-way pipeline through the processes of distillation and multi-modal hypersphere learning. The distillation process is proposed to supervise fusion ViT (ViT with an extra fusion token) with soft targets from modality-specific ViTs, which prevents fusion ViT from catastrophic forgetting. Furthermore, our method learns a multi-modal hypersphere by performing inter- and intra-modal alignment without loss of uniformity, which aims to bridge the modal gap between modalities of sketch and image and avoid the collapse in dimensions. Extensive experiments on three benchmark datasets, i.e., Sketchy, TU-Berlin, and QuickDraw, demonstrate the superiority of our TVT method over the state-of-the-art ZS-SBIR methods.
Revamped Differential-Linear Cryptanalysis on Reduced Round ChaCha

In this paper, we provide several improvements over the existing differential-linear attacks on ChaCha. ChaCha is a stream cipher which has 20 rounds. At CRYPTO 2020, Beierle et al. observed a differential in the 3.5-th round if the right pairs are chosen. They produced an improved attack using this, but showed that to achieve a right pair, we need $$2^5$$ iterations on average. In this direction, we provide a technique to find the right pairs with the help of listing. Also, we provide a strategical improvement in PNB construction, modification of complexity calculation and an alternative attack method using two input-output pairs. Using these, we improve the time complexity, reducing it to $$2^{221.95}$$ from $$2^{230.86}$$ reported by Beierle et al. for 256 bit version of ChaCha. Also, after a decade, we improve existing complexity (Shi et al. ICISC 2012) for a 6-round of 128 bit version of ChaCha by more than 11 million times and produce the first-ever attack on 6.5-round ChaCha128 with time complexity $$2^{123.04}.$$
FitNN: A Low-Resource FPGA-Based CNN Accelerator for Drones

Executing deep neural networks (DNNs) on resource-constraint edge devices, such as drones, offers low inference latency, high data privacy, and reduced network traffic. However, deploying DNNs on such devices is a challenging task. During DNN inference, intermediate results require significant data movement and frequent off-chip memory (DRAM) access, which decreases the inference speed and power efficiency. To address this issue, this article presents a field-programmable gate array (FPGA)-based convolutional neural network (CNN) accelerator, named FitNN, which improves the speed and power efficiency of CNN inference by reducing data movements. FitNN adopts a pretrained CNN of iSmart2, which is composed of depthwise and pointwise blocks in the Mobilenet structure. A cross-layer dataflow strategy is proposed to reduce off-chip data transfer of feature maps. Also, multilevel buffers are proposed to keep the most needed data on-chip (in block RAM) and avoid off-chip data reorganization and reloading. Finally, a computation core is proposed to operate the depthwise, pointwise, and max-pooling computation as soon as the data arrive without reorganization, which suits the real-life scenario of the data arriving in sequence. In our experiment, FitNN is implemented on two FPGA-based platforms (both at 150 MHz), Ultra96-V2 and PYNQ-Z1, for drone-based object detection with batch size = 1. The results show that FitNN achieves 15 frames per second (FPS) on Ultra96-V2, with power consumption of 4.69 W. On PYNQ-Z1, FitNN achieves 9 FPS with 1.9 W of power consumption. Compared with the previous FPGA-based implementation of iSmart2 CNN, FitNN increases the efficiency (FPS/W) by 2.37 times.
Full Graph Autoencoder for One-Class Group Anomaly Detection of IIoT System

With the increasing automation and integration of equipment, it is urgent to carry out anomaly detection (AD) for the large-scale system to ensure security, in virtue of Industrial Internet of Things (IIoT). Recently developed intelligent methods focus on component-level diagnosis or detection, resulting in difficulty in the health assessment of system with multisource data coupling. In addition, data-driven methods rarely emphasize the use of knowledge from the real physical system. In this article, we propose a full graph autoencoder to perform one-class group AD for the large-scale IIoT system. The proposed model takes as input data of normal status at training and only comprises several normalized graph convolutional layers, thus it is simple and fast. Different from Euclidean-based methods, the proposed model can handle various irregular structures together. For graph learning, multivariate time series are converted into graph data fused with prior knowledge. To achieve AD, we propose to reconstruct the full graph for the first time to obtain a reliable anomaly score. Besides, we extend a variational model to fully learn the graph representation. Moreover, a graph augmentation operation is employed to improve the accuracy and robustness. The proposed models are evaluated on two multisensor data sets from liquid rocket engine (LRE) systems, and the experimental results demonstrate the effectiveness and generalization of the IIoT system.
Reconstruction Regularized Deep Metric Learning for Multi-Label Image Classification

In this paper, we present a novel deep metric learning method to tackle the multi-label image classification problem. In order to better learn the correlations among images features, as well as labels, we attempt to explore a latent space, where images and labels are embedded via two unique deep neural networks, respectively. To capture the relationships between image features and labels, we aim to learn a two-way deep distance metric over the embedding space from two different views, i.e., the distance between one image and its labels is not only smaller than those distances between the image and its labels’ nearest neighbors but also smaller than the distances between the labels and other images corresponding to the labels’ nearest neighbors. Moreover, a reconstruction module for recovering correct labels is incorporated into the whole framework as a regularization term, such that the label embedding space is more representative. Our model can be trained in an end-to-end manner. Experimental results on publicly available image data sets corroborate the efficacy of our method compared with the state of the arts.
Analysis of Agriculture and Food Supply Chain through Blockchain and IoT with Light Weight Cluster Head

By 2050, the world’s population will have increased by 34%, to more than 9 billion people, needing a 70% increase in food production. Prepare more dishes with fewer ingredients. Therefore, the critical goal of manufacturers is to increase production while being ecologically benign. Supply chain systems that do not enable direct farmer-to-consumer connection and rising input costs influence data collection, security, and sharing. Constraints on data security, manipulation, and single-point failure are unfulfilled due to a lack of centralized IoT agricultural infrastructure. To address these issues, the article proposes a blockchain-based IoT model. This study also shows one-of-a-kind energy savings. The decentralization of data storage improves the supply chain’s transparency and quality through blockchain technology, thus farmers can engage more efficiently. Blockchain technology improves supply chain traceability and security. This article provides a transparent, decentralized blockchain tracking solution and proposes an intelligent model protocol for several Internet of Things (IoT) devices that monitor crop development and the agricultural environment. A new approach has resolved the bulk of the supply chain difficulties. Smart contracts were utilized to organize all transactions in decentralized supply networks. The use of blockchain technology improves transaction quality, and customers may verify the legitimacy of an item’s authenticity and legality by using the system. A total of 100 IoT nodes were distributed randomly to each 500 m2 cluster farm. The Internet of Things nodes were used to assess soil moisture, temperature, and crop disease. Network stability period and network life of the proposed method show 90.4% accuracy. The food supply chain will be more efficient and trustworthy with an intelligent model. The immutability of ledger technology and smart contract support further increases supply chain security, privacy, transparency, and trust among all stakeholders in the multi-party system. By 2050, the world’s population will need a 70% increase in food production. The food supply chain will be more efficient and trustworthy with an intelligent model. This article provides a transparent, decentralized, and intelligent model protocol for several Internet of Things (IoT) devices.