Reviewerly/exHarmony huggingface dataset:
Toward Vehicular Digital Forensics From Decentralized Trust: An Accountable, Privacy-Preserving, and Secure Realization

With the increasing number of traffic accidents and terrorist attacks by modern vehicles, vehicular digital forensics (VDF) has gained significant attention in identifying evidence from the related digital devices. Ensuring the law enforcement agency to accurately integrate various kinds of data is a crucial point to determine the facts. However, malicious attackers or semi-honest participants may undermine the digital forensic procedures. Enabling accountability and privacy preservation while providing secure data access control in VDF is a nontrivial challenge. To mitigate this issue, in this article, we propose a blockchain-based decentralized solution for VDF named BB-VDF, in which the accountable protocols and privacy-preserving algorithm are constructed. The desirable security properties and fine-grained data access control are achieved based on smart contract and the customized cryptographic construction. Specifically, we design a distributed key-policy attribute-based encryption scheme with partially hidden access structures, named DKP-ABE-H, to realize the secure fine-grained forensics data access control. Further, a novel smart contract is designed to model the forensics procedures as a finite state machine, which guarantees accountability that each participant performs auditable cooperation under tamper resistant and traceable transactions. Systematic security analysis and extensive experimental results show the feasibility and practicability of our proposed BB-VDF scheme.
TacticFlow: Visual Analytics of Ever-Changing Tactics in Racket Sports

Event sequence mining is often used to summarize patterns from hundreds of sequences but faces special challenges when handling racket sports data. In racket sports (e.g., tennis and badminton), a player hitting the ball is considered a multivariate event consisting of multiple attributes (e.g., hit technique and ball position). A rally (i.e., a series of consecutive hits beginning with one player serving the ball and ending with one player winning a point) thereby can be viewed as a multivariate event sequence. Mining frequent patterns and depicting how patterns change over time is instructive and meaningful to players who want to learn more short-term competitive strategies (i.e., tactics) that encompass multiple hits. However, players in racket sports usually change their tactics rapidly according to the opponent's reaction, resulting in ever-changing tactic progression. In this work, we introduce a tailored visualization system built on a novel multivariate sequence pattern mining algorithm to facilitate explorative identification and analysis of various tactics and tactic progression. The algorithm can mine multiple non-overlapping multivariate patterns from hundreds of sequences effectively. Based on the mined results, we propose a glyph-based Sankey diagram to visualize the ever-changing tactic progression and support interactive data exploration. Through two case studies with four domain experts in tennis and badminton, we demonstrate that our system can effectively obtain insights about tactic progression in most racket sports. We further discuss the strengths and the limitations of our system based on domain experts' feedback.
Multimodality in meta-learning: A comprehensive survey

Meta-learning has gained wide popularity as a training framework that is more data-efficient than traditional machine learning methods. However, its generalization ability in complex task distributions, such as multimodal tasks, has not been thoroughly studied. Recently, some studies on multimodality-based meta-learning have emerged. This survey provides a comprehensive overview of the multimodality-based meta-learning landscape in terms of the methodologies and applications. We first formalize the definition of meta-learning in multimodality, along with the research challenges in this growing field, such as how to enrich the input in few-shot learning (FSL) or zero-shot learning (ZSL) in multimodal scenarios and how to generalize the models to new tasks. We then propose a new taxonomy to discuss typical meta-learning algorithms in multimodal tasks systematically. We investigate the contributions of related papers and summarize them by our taxonomy. Finally, we propose potential research directions for this promising field.
Generalized and Incremental Few-Shot Learning by Explicit Learning and Calibration without Forgetting

Both generalized and incremental few-shot learning have to deal with three major challenges: learning novel classes from only few samples per class, preventing catastrophic forgetting of base classes, and classifier calibration across novel and base classes. In this work we propose a three-stage framework that allows to explicitly and effectively address these challenges. While the first phase learns base classes with many samples, the second phase learns a calibrated classifier for novel classes from few samples while also preventing catastrophic forgetting. In the final phase, calibration is achieved across all classes. We evaluate the proposed framework on four challenging benchmark datasets for image and video few-shot classification and obtain state-of-the-art results for both generalized and incremental few shot learning.
Learning Motion-Appearance Co-Attention for Zero-Shot Video Object Segmentation

How to make the appearance and motion information interact effectively to accommodate complex scenarios is a fundamental issue in flow-based zero-shot video object segmentation. In this paper, we propose an Attentive Multi-Modality Collaboration Network (AMC-Net) to utilize appearance and motion information uniformly. Specifically, AMC-Net fuses robust information from multi-modality features and promotes their collaboration in two stages. First, we propose a Multi-Modality Co-Attention Gate (MCG) on the bilateral encoder branches, in which a gate function is used to formulate co-attention scores for balancing the contributions of multi-modality features and suppressing the redundant and misleading information. Then, we propose a Motion Correction Module (MCM) with a visualmotion attention mechanism, which is constructed to emphasize the features of foreground objects by incorporating the spatio-temporal correspondence between appearance and motion cues. Extensive experiments on three public challenging benchmark datasets verify that our proposed network performs favorably against existing state-of-the-art methods via training with fewer data. The code is released at https://github.com/isyangshu/AMC-Net.
Dance with Self-Attention: A New Look of Conditional Random Fields on Anomaly Detection in Videos

This paper proposes a novel weakly supervised approach for anomaly detection, which begins with a relation-aware feature extractor to capture the multi-scale convolutional neural network (CNN) features from a video. Afterwards, self-attention is integrated with conditional random fields (CRFs), the core of the network, to make use of the ability of self-attention in capturing the short-range correlations of the features and the ability of CRFs in learning the inter-dependencies of these features. Such a framework can learn not only the spatio-temporal interactions among the actors which are important for detecting complex movements, but also their short- and long-term dependencies across frames. Also, to deal with both local and non-local relationships of the features, a new variant of self-attention is developed by taking into consideration a set of cliques with different temporal localities. Moreover, a contrastive multi-instance learning scheme is considered to broaden the gap between the normal and abnormal instances, resulting in more accurate abnormal discrimination. Simulations reveal that the new method provides superior performance to the state-of-the-art works on the widespread UCF-Crime and Shang-haiTech datasets.
Deep Neural Backdoor in Semi-Supervised Learning: Threats and Countermeasures

Semi-Supervised Learning (SSL) is a powerful derivative for humans to discover the hidden knowledge, and will be a great substitute for data taggers. Although the availability of unlabeled data rises up a huge passion to SSL, the untrustness of unlabeled data leads to many unknown security risks. In this paper, we first identify an insidious backdoor threat of SSL where unlabeled training data are poisoned by backdoor methods migrated from supervised settings. Then, to further exploit this threat, a Deep Neural Backdoor (DeNeB) scheme is proposed, which requires less data poisoning budgets and produces stronger backdoor effectiveness. By poisoning a fraction of unlabeled training data, the DeNeB achieves the illegal manipulation on the trained model without modifying the training process. Finally, an efficient detection-and-purification defense (DePuD) framework is proposed to thwart the proposed scheme. In DePuD, we construct a deep detector to locate trigger patterns in the unlabeled training data, and perform secured SSL training with purified unlabeled data where the detected trigger patterns are obfuscated. Extensive experiments based on benchmark datasets are performed to demonstrate the huge threatening of DeNeB and the effectiveness of DePuD. To our best knowledge, this is the first work to achieve the backdoor and its defense in semi-supervised learning.
Unsupervised Real-World Super-Resolution: A Domain Adaptation Perspective

Most existing convolution neural network (CNN) based super-resolution (SR) methods generate their paired training dataset by artificially synthesizing low-resolution (LR) images from the high-resolution (HR) ones. However, this dataset preparation strategy harms the application of these CNNs in real-world scenarios due to the inherent domain gap between the training and testing data. A popular attempts towards the challenge is unpaired generative adversarial networks, which generate "real" LR counterparts from real HR images using image-to-image translation and then perform super-resolution from "real" LR→SR. Despite great progress, it is still difficult to synthesize perfect "real" LR images for super-resolution. In this paper, we firstly consider the real-world SR problem from the traditional domain adaptation perspective. We propose a novel unpaired SR training framework based on feature distribution alignment, with which we can obtain degradation-indistinguishable feature maps and then map them to HR images. In order to generate better SR images for target LR domain, we introduce several regularization losses to force the aligned feature to locate around the target domain. Our experiments indicate that our SR network obtains the state-of-the-art performance over both blind and unpaired SR methods on diverse datasets.
Robust semantic segmentation based on RGB-thermal in variable lighting scenes

Semantic segmentation is an indispensable part of the Intelligent Vehicle-Infrastructure System(IVIS) perception task. It has made considerable progress with the development of convolutional neural networks. However, the current mainstream semantic segmentation networks are designed for 3-channel RGB images captured by visible light cameras, and their accuracy and robustness are still insufficient in variable lighting environments. Therefore, this paper proposes a novel network architecture with RGB and thermal data fusion to improve the accuracy and robustness of semantic segmentation in a variable lighting environment. The experimental results verify that introducing thermal data significantly improves the adaptability and segmentation accuracy of the network to variable lighting environments. Moreover, the results demonstrate that our network is accurate and robust under variable lighting environments, and its overall performance outperforms the state-of-the-art networks.
Adaptive Transmission Scheduling in Wireless Networks for Asynchronous Federated Learning

In this paper, we study asynchronous federated learning (FL) in a wireless distributed learning network (WDLN). To allow each edge device to use its local data more efficiently via asynchronous FL, transmission scheduling in the WDLN for asynchronous FL should be carefully determined considering system uncertainties, such as time-varying channel and stochastic data arrivals, and the scarce radio resources in the WDLN. To address this, we propose a metric, called an effectivity score, which represents the amount of learning from asynchronous FL. We then formulate an Asynchronous Learning-aware transmission Scheduling (ALS) problem to maximize the effectivity score and develop three ALS algorithms, called ALSA-PI, BALSA, and BALSA-PO, to solve it. If the statistical information about the uncertainties is known, the problem can be optimally and efficiently solved by ALSA-PI. Even if not, it can be still optimally solved by BALSA that learns the uncertainties based on a Bayesian approach using the state information reported from devices. BALSA-PO suboptimally solves the problem, but it addresses a more restrictive WDLN in practice, where the AP can observe a limited state information compared with the information used in BALSA. We show via simulations that the models trained by our ALS algorithms achieve performances close to that by an ideal benchmark and outperform those by other state-of-the-art baseline scheduling algorithms in terms of model accuracy, training loss, learning speed, and robustness of learning. These results demonstrate that the adaptive scheduling strategy in our ALS algorithms is effective to asynchronous FL.
ChainFL: A Simulation Platform for Joint Federated Learning and Blockchain in Edge/Cloud Computing Environments

As a distributed computing paradigm, edge computing has become a key technology for providing timely services to mobile devices by connecting Internet of Things (IoT), cloud centers, and other facilities. By offloading compute-intensive tasks from IoT devices to edge/cloud servers, the communication and computation pressure caused by the massive data in Industrial IoT can be effectively reduced. In the process of computation offloading in edge computing, it is critical to dynamically make optimal offloading decisions to minimize the delay and energy consumption spent on the devices. Although there are a large number of task offloading-decision models, how to measure and evaluate the quality of different models and configurations is crucial. In this article, we propose a novel simulation platform named ChainFL, which can build an edge computing environment among IoT devices while being compatible with federated learning and blockchain technologies to better support the embedding of security-focused offloading algorithms. ChainFL is lightweight and compatible, and it can quickly build complex network environments by connecting devices of different architectures. Moreover, due to its distributed nature, ChainFL can also be deployed as a federated learning platform across multiple devices to enable federated learning with high security due to its embedded blockchain. Finally, we validate the versatility and effectiveness of ChainFL by embedding a complex offloading-decision model in the platform, and deploying it in an Industrial IoT environment with security risks.
SHARE: Shaping Data Distribution at Edge for Communication-Efficient Hierarchical Federated Learning

Federated learning (FL) can enable distributed model training over mobile nodes without sharing privacy-sensitive raw data. However, to achieve efficient FL, one significant challenge is the prohibitive communication overhead to commit model updates since frequent cloud model aggregations are usually required to reach a target accuracy, especially when the data distributions at mobile nodes are imbalanced. With pilot experiments, it is verified that frequent cloud model aggregations can be avoided without performance degradation if model aggregations can be conducted at edge. To this end, we shed light on the hierarchical federated learning (HFL) framework, where a subset of distributed nodes are selected as edge aggregators to conduct edge aggregations. Particularly, under the HFL framework, we formulate a communication cost minimization (CCM) problem to minimize the communication cost raised by edge/cloud aggregations with making decisions on edge aggregator selection and distributed node association. Inspired by the insight that the potential of HFL lies in the data distribution at edge aggregators, we propose SHARE, i.e., SHaping dAta distRibution at Edge, to transform and solve the CCM problem. In SHARE, we divide the original problem into two sub-problems to minimize the per-round communication cost and mean Kullback-Leibler divergence of edge aggregator data, and devise two light-weight algorithms to solve them, respectively. Extensive experiments under various settings are carried out to corroborate the efficacy of SHARE.
An Adaptive Parallel Arithmetic Optimization Algorithm for Robot Path Planning

Path planning is one of the hotspots in the research of automotive engineering. Aiming at the issue of robot path planning with the goal of finding a collision-free optimal motion path in an environment with barriers, this study proposes an adaptive parallel arithmetic optimization algorithm (APAOA) with a novel parallel communication strategy. Comparisons with other popular algorithms on 18 benchmark functions are committed. Experimental results show that the proposed algorithm performs better in terms of solution accuracy and convergence speed, and the proposed strategy can prevent the algorithm from falling into a local optimal solution. Finally, we apply APAOA to solve the problem of robot path planning.
Personalized Federated Learning by Structured and Unstructured Pruning under Data Heterogeneity

The traditional approach in FL tries to learn a single global model collaboratively with the help of many clients under the orchestration of a central server. However, learning a single global model might not work well for all clients participating in the FL under data heterogeneity. Therefore, the personalization of the global model becomes crucial in handling the challenges that arise with statistical heterogeneity and the non-IID distribution of data. Unlike prior works, in this work we propose a new approach for obtaining a personalized model from a client-level objective. This further motivates all clients to participate in federation even under statistical heterogeneity in order to improve their performance, instead of merely being a source of data and model training for the central server. To realize this personalization, we leverage finding a small subnetwork for each client by applying hybrid pruning (combination of structured and unstructured pruning), and unstructured pruning. Through a range of experiments on different benchmarks, we observed that the clients with similar data (labels) share similar personal parameters. By finding a subnetwork for each client rather than taking the average over all parameters of all clients for the entire federation as in traditional FL, we efficiently calculate the averaging on the remaining parameters of each subnetwork of each client. We call this novel parameter averaging as Sub-FedAvg. Furthermore, in our proposed approach, the clients are not required to have knowledge of any underlying data distributions or label similarities among the rest of clients. The non-IID nature of each client’s local data provides distinguishing subnetworks without sharing any data. We evaluate our method on federated image classification with real world datasets. Our method outperforms existing state-of-the-art.
Arabic Sentiment Analysis Using BERT Model

Sentiment analysis is the process of determining whether a text or a writing is positive, negative, or neutral. A lot of research has been done to improve the accuracy of sentiment analysis methods, varying from simple linear models to more complex deep neural network models. Lately, the transformer-based model showed great success in sentiment analysis and was considered as the state-of-the-art model for various languages (English, german, french, Turk, Arabic, etc.). However, the accuracy for Arabic sentiment analysis still needs improvements especially in tokenization level during data processing. In fact, the Arabic language imposes many challenges, due to its complex structure, various dialects, and resource scarcity. The improvement of the proposed approach consists of integrating an Arabic BERT tokenizer instead of a basic BERT Tokenizer. Various tests were carried out with different instances (dialect and standard). We used hyperparameters optimization by random search method to obtain the best result with different datasets. The experimental study proves the efficiency of the proposed approach in terms of classification quality and accuracy compared to Arabic BERT and AraBERT models.
Multi‐frame based adversarial learning approach for video surveillance

Foreground-background segmentation (FBS) is one of the prime tasks for automated video-based applications like traffic analysis and surveillance. The different practical scenarios like weather degraded videos, irregular moving objects, dynamic background, etc., make FBS a challenging task. The existing FBS algorithms mainly depend on one of the three different factors, namely (1) complicated training process, (2) additionally trained modules for other applications, or (3) neglect the inter-frame spatio-temporal structural dependencies. In this paper, a novel multi-frame-based adversarial learning network is proposed with multi-scale inception and residual module for FBS. As, FBS is a temporal enlightenment-based problem, a temporal encoding mechanism with decreasing variable intervals is proposed for the input frame selection. The proposed network comprises multi-scale inception and residual connection-based dense modules to learn prominent features of the foreground object(s). Also, feedback of the estimated foreground map of previous frame is utilized to exhibit more temporal consistency. Learning of the network is concentrated in different ways like cross-data, disjoint, and global training-testing for FBS. The qualitative and quantitative experimental analysis of the proposed approach is done on three benchmark datasets for FBS. Experimental analysis on three benchmark datasets proves the significance of the proposed approach as compared to state-of-the-art FBS approaches.
3DVG-Transformer: Relation Modeling for Visual Grounding on Point Clouds

Visual grounding on 3D point clouds is an emerging vision and language task that benefits various applications in understanding the 3D visual world. By formulating this task as a grounding-by-detection problem, lots of recent works focus on how to exploit more powerful detectors and comprehensive language features, but (1) how to model complex relations for generating context-aware object proposals and (2) how to leverage proposal relations to distinguish the true target object from similar proposals are not fully studied yet. Inspired by the well-known transformer architecture, we propose a relation-aware visual grounding method on 3D point clouds, named as 3DVG-Transformer, to fully utilize the contextual clues for relation-enhanced proposal generation and cross-modal proposal disambiguation, which are enabled by a newly designed coordinate-guided contextual aggregation (CCA) module in the object proposal generation stage, and a multiplex attention (MA) module in the cross-modal feature fusion stage. We validate that our 3DVG-Transformer outperforms the state-of-the-art methods by a large margin, on two point cloud-based visual grounding datasets, ScanRefer and Nr3D/Sr3D from ReferIt3D, especially for complex scenarios containing multiple objects of the same category.