Reviewerly/exHarmony huggingface dataset:
CSI-Based Location-Independent Human Activity Recognition Using Feature Fusion

Channel state information (CSI)-based human activity recognition (HAR) has important application prospects, such as smart homes, medical monitoring, and public security. Due to the collected CSI data contains not only activity information but also activity-unrelated environmental information, the characteristics of the same activity conducted at different locations are different. The existing methods of collecting samples at a fixed location to train the HAR model can hardly work well at other locations, which limits the application prospect of CSI-based HAR. To deal with this challenge, we proposed an Attention-based feature Fusion ACTivity recognition system (AF-ACT). The proposed system extracts the semantic activity features and temporal features from different dimensions to better characterize the activity at different locations. The semantic activity features are extracted by the convolutional neural network (CNN) combined with the convolutional attention module (CBAM), and the temporal features are extracted by bidirectional gated recurrent unit (BGRU) combined with the self-attention mechanism. The semantic activity features and temporal features are fused through an attention-based feature fusion (A-Fusion) module to obtain complementary information, which will promote recognition accuracy. The proposed system is evaluated in an open environment with 12 activity training locations and ten arbitrary testing locations. The experimental results show that the system can reach the highest accuracy of 91.23% in different experimental conditions when recognizing eight categories of activities.
Dual-Evidential Learning for Weakly-supervised Temporal Action Localization

Weakly-supervised temporal action localization (WS-TAL) aims to localize the action instances and recognize their categories with only video-level labels. Despite great progress, existing methods suffer from severe action-background ambiguity, which mainly comes from background noise introduced by aggregation operations and large intra-action variations caused by the task gap between classification and localization. To address this issue, we propose a generalized evidential deep learning (EDL) framework for WS-TAL, called Dual-Evidential Learning for Uncertainty modeling (DELU), which extends the traditional paradigm of EDL to adapt to the weakly-supervised multi-label classification goal. Specifically, targeting at adaptively excluding the undesirable background snippets, we utilize the video-level uncertainty to measure the interference of background noise to video-level prediction. Then, the snippet-level uncertainty is further deduced for progressive learning, which gradually focuses on the entire action instances in an “easy-to-hard” manner. Extensive experiments show that DELU achieves state-of-the-art performance on THUMOS14 and ActivityNet1.2 benchmarks. Our code is available in github.com/MengyuanChen21/ECCV2022-DELU.
Rotational Differential-Linear Distinguishers of ARX Ciphers with Arbitrary Output Linear Masks

The rotational differential-linear attacks, proposed at EUROCRYPT 2021, is a generalization of differential-linear attacks by replacing the differential part of the attacks with rotational differentials. At EUROCRYPT 2021, Liu et al. presented a method based on Morawiecki et al.'s technique (FSE 2013) for evaluating the rotational differential-linear correlations for the special cases where the output linear masks are unit vectors. With this method, some powerful (rotational) differential-linear distinguishers with output linear masks being unit vectors against FRIET, Xoodoo, and Alzette were discovered. However, how to compute the rotational differential-linear correlations for arbitrary output masks was left open. In this work, we partially solve this open problem by presenting an efficient algorithm for computing the (rotational) differential-linear correlation of modulo additions for arbitrary output linear masks, based on which a technique for evaluating the (rotational) differential-linear correlation of ARX ciphers is derived. We apply the technique to Alzette, SipHash, ChaCha, and SPECK. As a result, significantly improved (rotational) differential-linear distinguishers including deterministic ones are identified. All results of this work are practical and experimentally verified to confirm the validity of our methods. In addition, we try to explain the experimental distinguishers employed in FSE 2008, FSE 2016, and CRYPTO 2020 against ChaCha. The predicted correlations are close to the experimental ones.
Improving Subgraph Recognition with Variational Graph Information Bottleneck

Subgraph recognition aims at discovering a compressed substructure of a graph that is most informative to the graph property. It can be formulated by optimizing Graph Information Bottleneck (GIB) with a mutual information estimator. However, GIB suffers from training instability and degenerated results due to its intrinsic optimization process. To tackle these issues, we reformulate the subgraph recognition problem into two steps: graph perturbation and subgraph selection, leading to a novel Variational Graph Information Bottleneck (VGIB) framework. VGIB first employs the noise injection to modulate the information flow from the input graph to the perturbed graph. Then, the perturbed graph is encouraged to be informative to the graph property. VGIB further obtains the desired subgraph by filtering out the noise in the perturbed graph. With the customized noise prior for each input, the VGIB objective is endowed with a tractable variational upper bound, leading to a superior empirical performance as well as theoretical properties. Extensive experiments on graph interpretation, explainability of Graph Neural Networks, and graph classification show that VGIB finds better subgraphs than existing methods <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> Code is avaliable on https://github.com/Samyu0304/VGIB.
Learning to generate line drawings that convey geometry and semantics

This paper presents an unpaired method for creating line drawings from photographs. Current methods often rely on high quality paired datasets to generate line drawings. However, these datasets often have limitations due to the subjects of the drawings belonging to a specific domain, or in the amount of data collected. Although recent work in unsupervised image-to-image translation has shown much progress, the latest methods still struggle to generate compelling line drawings. We observe that line drawings are encodings of scene information and seek to convey 3D shape and semantic meaning. We build these observations into a set of objectives and train an image translation to map photographs into line drawings. We introduce a geometry loss which predicts depth information from the image features of a line drawing, and a semantic loss which matches the CLIP features of a line drawing with its corresponding photograph. Our approach outperforms state-of-the-art un-paired image translation and line drawing generation methods on creating line drawings from arbitrary photographs.
Reconstructed Student-Teacher and Discriminative Networks for Anomaly Detection

Anomaly detection is an important problem in computer vision; however, the scarcity of anomalous samples makes this task difficult. Thus, recent anomaly detection methods have used only “normal images” with no abnormal areas for training. In this work, a powerful anomaly detection method is proposed based on student-teacher feature pyramid matching (STPM), which consists of a student and teacher network. Generative models are another approach to anomaly detection. They reconstruct normal images from an input and compute the difference between the predicted normal and the input. Unfortunately, STPM does not have the ability to generate normal images. To improve the accuracy of STPM, this work uses a student network, as in generative models, to reconstruct normal features. This improves the accuracy; however, the anomaly maps for normal images are not clean because STPM does not use anomaly images for training, which decreases the accuracy of the image-level anomaly detection. To further improve accuracy, a discriminative network trained with pseudo-anomalies from anomaly maps is used in our method, which consists of two pairs of student-teacher networks and a discriminative network. The method displayed high accuracy on the MVTec anomaly detection dataset.
Collaborative and Intelligent Resource Optimization for Computing and Caching in IoV With Blockchain and MEC Using A3C Approach

Recently, the rise of the Internet of Vehicles (IoV) has driven the broad development of intelligent transportation and smart cities. In order to promote the computing power of mobile vehicles and decrease the content delivery latency of suppliers, mobile edge computing (MEC) is recognized as a promising computational paradigm and used in vehicular networks. However, there are some essential issues to be considered: 1) privacy and authenticity of data transmission in IoV, and 2) reasonable resource allocation for collaborative computing and caching. In this paper, to solve above issues, blockchain technology is introduced and adopted to ensure the accuracy and reliability of data transmission and interaction. Meanwhile, we develop an intelligent framework of resource allocation about computing and caching for blockchain-enabled MEC systems in IoV. Through jointly considering and optimizing offloading decision of computation task carried by vehicle, caching decision, the number of offloaded consensus nodes, block interval and block size, the weighted consumption costs of energy consumption and computation overheads can be decreased, and the transactional throughput of the blockchain can be increased. Moreover, due to the continuity and dynamic of the available resources of mobile vehicles and computing servers, the optimization problem is modeled as a Markov decision process (MDP). Facing the large-scale and dynamic characteristics of the system, the asynchronous advantage actor-critic (A3C) approach is introduced to deal with the optimization problem. Simulation results show that our proposed scheme achieves significant advantages over other comparison schemes, such as the total reward of the proposed scheme is about 14% higher than that of the deep <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$Q$</tex-math></inline-formula> -network based scheme.
Interact before Align: Leveraging Cross-Modal Knowledge for Domain Adaptive Action Recognition

Unsupervised domain adaptive video action recognition aims to recognize actions of a target domain using a model trained with only out-of-domain (source) annotations. The inherent complexity of videos makes this task challenging but also provides ground for leveraging multi-modal inputs (e.g., RGB, Flow, Audio). Most previous works utilize the multi-modal information by either aligning each modality individually or learning representation via cross-modal self-supervision. Different from previous works, we find that the cross-domain alignment can be more effectively done by using cross-modal interaction first. Cross-modal knowledge interaction allows other modalities to supplement missing transferable information because of the cross-modal complementarity. Also, the most transferable aspects of data can be highlighted using cross-modal consensus. In this work, we present a novel model that jointly considers these two characteristics for domain adaptive action recognition. We achieve this by implementing two modules, where the first module exchanges complementary transferable information across modalities through the semantic space, and the second module finds the most transferable spatial region based on the consensus of all modalities. Extensive experiments validate that our proposed method can significantly outperform state-of-the-art methods on multiple benchmark datasets, including the complex fine-grained dataset EPIC-Kitchens-100.
3D Common Corruptions and Data Augmentation

We introduce a set of image transformations that can be used as corruptions to evaluate the robustness of models as well as data augmentation mechanisms for training neural networks. The primary distinction of the proposed transformations is that, unlike existing approaches such as Common Corruptions [27], the geometry of the scene is incorporated in the transformations - thus leading to corruptions that are more likely to occur in the real world. We also introduce a set of semantic corruptions (e.g. natural object occlusions. See Fig. 1). We show these transformations are ‘efficient’ (can be computed on-the-fly), ‘extendable’ (can be applied on most image datasets), expose vulnerability of existing models, and can effectively make models more robust when employed as ‘3D data augmentation’ mechanisms. The evaluations on several tasks and datasets suggest incorporating 3D information into benchmarking and training opens up a promising direction for robustness research.
Using NFTs and Blockchain for Traceability and Auctioning of Shipping Containers and Cargo in Maritime Industry

The maritime industry is a complex multi-tiered sector that plays a major role in international shipping and trade.Seaborne transportation is responsible for 90% of goods moved globally, and containerized shipments account for more than 50% of those goods.This volume of movement requires efficient tracking and traceability methods that secure the documentation process and ensure the involved stakeholders.However, due to its complexity, the maritime industry suffers from a lack of trust, lack of ownership evidence, protracted documentation procedures, and excessive data aggregation.These shortcomings are reflected in delays and elevated costs in the shipping process.To mitigate these problems, we propose in this paper a blockchain-based solution for the ownership history and traceability of shipping containers in the industry.Our solution utilizes smart contracts and non-fungible tokens (NFTs) on the Ethereum blockchain to accelerate the inefficient shipping process by digitizing ownership transfer and process documentation.Furthermore, the proposed system allows the owners to auction off their cargo in the destination ports, freeing warehouses and containers in the process.We analyze our proposed system in terms of transaction costs and smart contract security.The smart contracts are developed using the Solidity language in the Remix IDE and the code is made publicly available on GitHub.
Orientational Distribution Learning with Hierarchical Spatial Attention for Open Set Recognition

Open set recognition (OSR) aims to correctly recognize the known classes and reject the unknown classes for increasing the reliability of the recognition system. The distance-based loss is often employed in deep neural networks-based OSR methods to constrain the latent representation of known classes. However, the optimization is usually conducted using the nondirectional euclidean distance in a single feature space without considering the potential impact of spatial distribution. To address this problem, we propose orientational distribution learning (ODL) with hierarchical spatial attention for OSR. In ODL, the spatial distribution of feature representation is optimized orientationally to increase the discriminability of decision boundaries for open set recognition. Then, a hierarchical spatial attention mechanism is proposed to assist ODL to capture the global distribution dependencies in the feature space based on spatial relationships. Moreover, a composite feature space is constructed to integrate the features from different layers and different mapping approaches, and it can well enrich the representation information. Finally, a decision-level fusion method is developed to combine the composite feature space and the naive feature space for producing a more comprehensive classification result. The effectiveness of ODL has been demonstrated on various benchmark datasets, and ODL achieves state-of-the-art performance.
FedBKD: Heterogenous Federated Learning via Bidirectional Knowledge Distillation for Modulation Classification in IoT-Edge System

Benefit from the rapid evolution of artificial intelligence and wireless communication technology, diverse Internet of Things (IoT) devices with edge computing ability have widely penetrated every aspect of daily human life. However, the deviations of private datasets and the heterogeneity of local models caused by the difference in device composition and application scenarios have hampering the aggregation of global recognition model in modulation classification task, thus constraining the classification performance of intelligent IoT-edge devices severely. To address this problem, we propose a heterogenous Federated learning framework based on Bidirectional Knowledge Distillation (FedBKD) for IoT system, which integrates knowledge distillation into the local model upload (client-to-cloud) and global model download (cloud-to-client) steps of federated learning. The client-to-cloud distillation is regarded as a process of multi-teacher knowledge distillation and the global network is regarded as a student network that unifies the heterogeneous knowledge from multiple local teacher networks. A public dataset is generated by conditional variational autoencoder (CVAE) and stored in the cloud server for supporting the obtaining of heterogeneous knowledge without sharing the private data of IoT devices. The cloud-to-client distillation is single-teacher-multiple-students process, which distills the knowledge from the single global model back to multiple heterogeneous local networks and partial knowledge distillation is used in this process. We implement our FedBKD method in the modulation classification task and the simulation results have proven the effectiveness of our proposed method.
Weakly-supervised Action Transition Learning for Stochastic Human Motion Prediction

We introduce the task of action-driven stochastic human motion prediction, which aims to predict multiple plausible future motions given a sequence of action labels and a short motion history. This differs from existing works, which predict motions that either do not respect any specific action category, or follow a single action label. In particular, addressing this task requires tackling two challenges: The transitions between the different actions must be smooth; the length of the predicted motion depends on the action sequence and varies significantly across samples. As we cannot realistically expect training data to cover sufficiently diverse action transitions and motion lengths, we propose an effective training strategy consisting of combining multiple motions from different actions and introducing a weak form of supervision to encourage smooth transitions. We then design a VAE-based model conditioned on both the observed motion and the action label sequence, allowing us to generate multiple plausible future motions of varying length. We illustrate the generality of our approach by exploring its use with two different temporal encoding mod-els, namely RNNs and Transformers. Our approach out-performs baseline models constructed by adapting state-of-the-art single action-conditioned motion generation methods and stochastic human motion prediction approaches to our new task of action-driven stochastic motion prediction. Our code is available at https://github.com/wei-mao-2019/WAT.
Exploring Geometric Consistency for Monocular 3D Object Detection

This paper investigates the geometric consistency for monocular 3D object detection, which suffers from the ill-posed depth estimation. We first conduct a thorough analysis to reveal how existing methods fail to consistently localize objects when different geometric shifts occur. In particular, we design a series of geometric manipulations to diagnose existing detectors and then illustrate their vulnerability to consistently associate the depth with object apparent sizes and positions. To alleviate this issue, we propose four geometry-aware data augmentation approaches to enhance the geometric consistency of the detectors. We first modify some commonly used data augmentation methods for 2D images so that they can maintain geometric consistency in 3D spaces. We demonstrate such modifications are important. In addition, we propose a 3D-specific image perturbation method that employs the camera movement. During the augmentation process, the camera system with the corresponding image is manipulated, while the geometric visual cues for depth recovery are preserved. We show that by using the geometric consistency constraints, the proposed augmentation techniques lead to improvements on the KITTI and nuScenes monocular 3D detection benchmarks with state-of-the-art results. In addition, we demonstrate that the augmentation methods are well suited for semisupervised training and cross-dataset generalization.
High Quality Segmentation for Ultra High-resolution Images

To segment 4K or 6K ultra high-resolution images needs extra computation consideration in image segmentation. Common strategies, such as downsampling, patch cropping, and cascade model, cannot address well the balance issue between accuracy and computation cost. Motivated by the fact that humans distinguish among objects continuously from coarse to precise levels, we propose the Continuous Refinement Model (CRM) for the ultra high-resolution segmentation refinement task. CRM continuously aligns the feature map with the refinement target and aggregates features to reconstruct these image details. Besides, our CRM shows its significant generalization ability to fill the resolution gap between low-resolution training images and ultra high-resolution testing ones. We present quantitative performance evaluation and visualization to show that our proposed method is fast and effective on image segmentation refinement. Code is available at https://github.com/dvlab-research/Entity/tree/main/CRM.
An Information-Driven Genetic Algorithm for Privacy-Preserving Data Publishing

Due to the expanding requirements for data publishing and growing concerns regarding data privacy, the privacy-preserving data publishing (PPDP) problem has received considerable attention from research communities, industries, and governments. However, it is challenging to tackle the trade-off between privacy preservation and data quality maintenance in PPDP. In this paper, an information-driven genetic algorithm (ID-GA) is designed to achieve optimal anonymization based on attribute generalization and record suppression. In ID-GA, an information-driven crossover operator is designed to efficiently exchange information between different anonymization solutions; an information-driven mutation operator is proposed to promote information release during anonymization; a two-dimension selection operator is designed to identify the qualities of different anonymization solutions. Experimental results verify the advantages of ID-GA in terms of solution accuracy and convergence speed. Besides, the impacts of all the proposed components are verified.