Reviewerly/exHarmony huggingface dataset:
Generative knowledge-based transfer learning for few-shot health condition estimation

Abstract In the field of high-end manufacturing, it is valuable to study few-shot health condition estimation. Although transfer learning and other methods have effectively improved the ability of few-shot learning, they still cannot solve the lack of prior knowledge. In this paper, by combining data enhancement, knowledge reasoning, and transfer learning, a generative knowledge-based transfer learning model is proposed to achieve few-shot health condition estimation. First, with the effectiveness of data enhancement on machine learning, a novel batch monotonic generative adversarial network (BM-GAN) is designed for few-shot health condition data generation, which can solve the problem of insufficient data and generate simulated training data. Second, a generative knowledge-based transfer learning model is proposed with the performance advantages of the belief rule base (BRB) method on few-shot learning, which combines expert knowledge and simulated training data to obtain a generalized BRB model and then fine-tunes the generalized model with real data to obtain a dedicated BRB model. Third, through uniform sampling of NASA lithium battery data and simulating few-shot conditions, the generative transfer-belief rule base (GT-BRB) method proposed in this paper is verified to be feasible for few-shot health condition estimation and improves the estimation accuracy of the BRB method by approximately 17.3%.
FedAAR: A Novel Federated Learning Framework for Animal Activity Recognition with Wearable Sensors

Deep learning dominates automated animal activity recognition (AAR) tasks due to high performance on large-scale datasets. However, constructing centralised data across diverse farms raises data privacy issues. Federated learning (FL) provides a distributed learning solution to train a shared model by coordinating multiple farms (clients) without sharing their private data, whereas directly applying FL to AAR tasks often faces two challenges: client-drift during local training and local gradient conflicts during global aggregation. In this study, we develop a novel FL framework called FedAAR to achieve AAR with wearable sensors. Specifically, we devise a prototype-guided local update module to alleviate the client-drift issue, which introduces a global prototype as shared knowledge to force clients to learn consistent features. To reduce gradient conflicts between clients, we design a gradient-refinement-based aggregation module to eliminate conflicting components between local gradients during global aggregation, thereby improving agreement between clients. Experiments are conducted on a public dataset to verify FedAAR's effectiveness, which consists of 87,621 two-second accelerometer and gyroscope data. The results demonstrate that FedAAR outperforms the state-of-the-art, on precision (75.23%), recall (75.17%), F1-score (74.70%), and accuracy (88.88%), respectively. The ablation experiments show FedAAR's robustness against various factors (i.e., data sizes, communication frequency, and client numbers).
Dependency-based security risk assessment for cyber-physical systems

Abstract A cyber-physical attack is a security breach in cyber space that impacts on the physical environment. The number and diversity of such attacks against Cyber-Physical Systems (CPSs) are increasing at impressive rates. In times of Industry 4.0 and Cyber-Physical Systems, providing security against cyber-physical attacks is a serious challenge which calls for cybersecurity risk assessment methods capable of investigating the tight interactions and interdependencies between the cyber and the physical components in such systems. However, existing risk assessment methods do not consider this specific characteristic of CPSs. In this paper, we propose a dependency-based, domain-agnostic cybersecurity risk assessment method that leverages a model of the CPS under study that captures dependencies among the system components. The proposed method identifies possible attack paths against critical components of a CPS by taking an attacker’s viewpoint and prioritizes these paths according to their risk to materialize, thus allowing the defenders to define efficient security controls. We illustrate the workings of the proposed method by applying it to a case study of a CPS in the energy domain, and we highlight the advantages that the proposed method offers when used to assess cybersecurity risks in CPSs.
VISOR-NET: Visibility Estimation Based on Deep Ordinal Relative Learning under Discrete-Level Labels

This paper proposes a novel end-to-end pipeline that uses the ordinal information and relative relation of images for visibility estimation (VISOR-NET). By encoding ordinal information into a set of relatively ordered image pairs, VISOR-NET can learn a global ranking function effectively. Due to the lack of real scenes or continuous labels in public foggy datasets, we collect a large-scale dataset that we term Foggy Highway Visibility Images (FHVI), which are taken from real surveillance scenes, and synthesize an INDoor Foggy images dataset (INDF) with continuous annotation. This work measures the estimation effectiveness on two public datasets and our FHVI dataset as a classification task and then on the INDF dataset as a regression task. Comprehensive experiments with existing deep-learning methods demonstrate the performance of the proposed method in terms of estimation accuracy, the convergence rate, model stability, and data requirements. Moreover, this method can extend inter-level visibility estimation to intra-level visibility estimation and can realize approximate regression estimation under discrete-level labels.
A comprehensive comparison of accuracy and practicality of different types of algorithms for pre-impact fall detection using both young and old adults

This study aims to comprehensively compare the accuracy and practicality of three different types of algorithms for pre-impact fall detection using both young and old subjects. Threshold-based, conventional machine learning (SVM) and deep learning (ConvLSTM) algorithms were compared. Results showed that ConvLSTM had an accuracy of 99.16 % (sensitivity: 99.32 %, specificity: 99.01 %) and an averaged lead time of 403 ms on young subjects, which outperformed SVM (97.16 %, 385 ms) and much superior to the threshold-based algorithm (89.06 %, 333 ms). In addition, latency tests on an embedded device showed that the Lite model of ConvLSTM had a low latency of 2.1 ms, which was comparable to the threshold-based algorithm (<1 ms) but much lower than SVM (86.9 ms). The feasibility and effectiveness of applying algorithms trained on young subjects to old subjects were also validated. These findings suggested that ConvLSTM has great potential for detecting pre-impact falls and preventing fall-related injuries.
Variational quantum optimization with multibasis encodings

Despite extensive research efforts, few quantum algorithms for classical optimization demonstrate a realizable quantum advantage. The utility of many quantum algorithms is limited by high requisite circuit depth and nonconvex optimization landscapes. We tackle these challenges by introducing a variational quantum algorithm that benefits from two innovations: multibasis graph encodings using single-qubit expectation values and nonlinear activation functions. Our technique results in increased observed optimization performance and a factor-of-two reduction in requisite qubits. While the classical simulation of many qubits with traditional quantum formalism is impossible due to its exponential scaling, we mitigate this limitation with exact circuit representations using factorized tensor rings. In particular, the shallow circuits permitted by our technique, combined with efficient factorized tensor-based simulation, enable us to successfully optimize the MaxCut of the 512-vertex DIMACS library graphs on a single GPU. By improving the performance of quantum optimization algorithms while requiring fewer quantum resources and utilizing shallower, more error-resistant circuits, we offer tangible progress for variational quantum optimization.
A bi-directional attention guided cross-modal network for music based dance generation

AI choreography has been an active task over the past few years. A fundamental problem of generating dance from music is to effectively learn the correlation between two modalities and then to create long-term body movements. To bridge the gap between music and motion, we propose a novel transformer-based cross-modal framework with a bi-directional attention module, aiming at generating high-quality dance sequences according to the music. The bi-directional attention module contains two layers: one is from music to dance, and the other is from dance to music, bringing benefits to the information exchange and alignment between two modalities. Furthermore, to alleviate exposure bias in sequence-to-sequence generation, we exploit a scheduled sampling strategy that is compatible with the transformer, which reduces error accumulation in long dance generation. Experimental results on 3D dance datasets demonstrate that our approach can achieve rhythmic natural long dance sequences. • A novel transformer-based music to dance framework is proposed. • A bi-directional attention decoder effectively fuses music and pose features. • A scheduled sampling strategy helps to alleviate the exposure bias.
Eye contact avoidance in crowds: A large wearable eye-tracking study

Eye contact is essential for human interactions. We investigated whether humans are able to avoid eye contact while navigating crowds. At a science festival, we fitted 62 participants with a wearable eye tracker and instructed them to walk a route. Half of the participants were further instructed to avoid eye contact. We report that humans can flexibly allocate their gaze while navigating crowds and avoid eye contact primarily by orienting their head and eyes towards the floor. We discuss implications for crowd navigation and gaze behavior. In addition, we address a number of issues encountered in such field studies with regard to data quality, control of the environment, and participant adherence to instructions. We stress that methodological innovation and scientific progress are strongly interrelated.
Accurate Solar Wind Speed Prediction with Multimodality Information

When the solar wind passes over the Earth, it will cause geomagnetic storms, affect short-wave communications, and threaten the safety of pipelines such as electricity and oil. Accurate prediction of the solar wind speed will allow people to make adequate preparations to avoid wasting resources and affecting people’s life. Most existing methods only use single-modality data as input and do not consider the information complementarity between different modalities. This paper proposes a multimodality prediction (MMP) method that jointly learns vision and sequence information in a unified end-to-end framework for solar wind speed prediction. MMP includes three modules: Vmodule, Tmodule, and Fusion module. Vmodule, which uses pretrained GoogLeNet, is proposed to learn visual representations from the extreme ultraviolet (EUV) images. Tmodule combining one-dimensional CNN with bidirectional long short-term memory (BiLSTM) is applied for learning sequence representation from multivariate time series. Finally, a multimodality fusion method is applied to improve the overall performance. We adopt the EUV images observed by the solar dynamics observatory (SDO) satellite and the OMNIWEB dataset measured at Lagrangian point 1 (L1) to experiment. Comparative experiments have shown that the proposed MMP achieves best performance in many metrics. The ablation experiments also verify the validity of each module and the rationality of the hyperparameter setting.
Pandemic lessons: New strategies and technologies of teaching

The article describes the results of the analysis of the experience gained while organizing the educational process during the two-year pandemic of COVID-19. In particular, the problem situations that arose on the transition to emergency distance learning are considered. The main attention is paid to summarizing the existing approaches to teaching students and schoolchildren, the actions necessary to mitigate the consequences of such a transition, as well as highlighting the strategic directions for training teachers, allowing them to be prepared for similar situations. The effective use of information and communication technologies has played an important role in overcoming the learning difficulties caused by the pandemic. At the same time, not only students needed timely support, but also teachers did not always have enough professionalism to quickly switch to online learning. Not all educational organizations turned out to be prepared for an almost instantaneous transition to new forms of their activities. Based on the results of the analysis, it was possible to formulate the strategic and technological directions of possible significant research that contribute to the preparation of future and current teachers for work in a changing environment.
Sleepminting, the brand new frontier of Non Fungible Tokens fraud

Non Fungible Tokens (NFTs) are becoming a standard to represent unique and valuable items, such as a piece of art, a videogame item, or other digital or physical goods, and keep track of their provenance. Thanks to blockchain technology and the power of smart contracts, NFT holders have true ownership over them, because they are the only ones who can transfer them. However, through an attack called sleepminting, an attacker is able to impersonate another person, including an artist, and create NFTs on the artist’s behalf, while still maintaining its possession, leveraging bugs in the code of the smart contract that manages the NFTs. Therefore, the attacker can cheat concerning the provenance of an NFT and then sell the fake NFTs to unaware buyers. In this paper, we propose a study that sheds light on this phenomenon. In particular, we collect over 1.3 million events that are connected to sleepminting and analyse the events under multiple aspects. The study uncovers that, by using the sleepminting attack, some users are able to create fake NFTs of popular brands, and are able to mint them to famous personalities in the NFT field, such as well known artists and collectors.
CNLL: A Semi-supervised Approach For Continual Noisy Label Learning

The task of continual learning requires careful design of algorithms that can tackle catastrophic forgetting. However, the noisy label, which is inevitable in a real-world scenario, seems to exacerbate the situation. While very few studies have addressed the issue of continual learning under noisy labels, long training time and complicated training schemes limit their applications in most cases. In contrast, we propose a simple purification technique to effectively cleanse the online data stream that is both cost-effective and more accurate. After purification, we perform fine-tuning in a semi-supervised fashion that ensures the participation of all available samples. Training in this fashion helps us learn a better representation that results in state-of-the-art (SOTA) performance. Through extensive experimentation on 3 benchmark datasets, MNIST, CIFAR10 and CIFAR100, we show the effectiveness of our proposed approach. We achieve a 24.8% performance gain for CIFAR10 with 20% noise over previous SOTA methods. Our code is publicly available. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
TMVNet : Using Transformers for Multi-view Voxel-based 3D Reconstruction

Previous research in multi-view 3D reconstruction have used different convolution neural network (CNN) architectures to obtain a 3D voxel representation. Even though CNN works well, they have limitations in exploiting the long-range dependencies in sequence transduction tasks such as multi-view 3D reconstruction. In this paper, we propose TMVNet–a two-layer transformer encoder that can better use long-range dependencies information. In contrast to using a 2D CNN decoder by the previous approaches, our model uses a 3D CNN encoder to capture the relations between the voxels in the 3D space. Also, our proposed 3D feature fusion network aggregates 3D position feature from CNN and long-range dependencies feature from transformer together. The proposed TMVNet is trained and tested on the ShapeNet dataset. Comparison against ten state-of-the-art multi-view 3D reconstruction methods and the reported quantitative and qualitative results show-case the superiority of our method.
Unsupervised Continual Learning for Gradually Varying Domains

In Unsupervised Domain Adaptation (UDA), a network is trained on a source domain and adapted on a target domain where no labeled data is available. Existing UDA techniques consider having the entire target domain available at once, which may not be feasible during deployment in realistic settings where batches of target data are acquired over time. Continual Learning (CL) has been dealing with data constrained paradigms in a supervised manner, where batches of labeled samples are sequentially presented to the network and the network continually learns from the new data without forgetting what was previously learned. Our method for unsupervised continual learning serves as a bridge between the UDA and CL paradigms. This research addresses a gradually evolving target domain fragmented into multiple sequential batches where the model continually adapts to the gradually varying stream of data in an unsupervised manner. To tackle this challenge, we propose a source free method based on episodic memory replay with buffer management. A contrastive loss is incorporated for better alignment of the buffer samples and the continual stream of batches. Our experiments on the rotating MNIST and CORe50 datasets confirm the benefits of our unsupervised continual learning method for gradually varying domains. The codes are available at https://github.com/abutaufique/ucl-gv.git.
Transformer Decoders with MultiModal Regularization for Cross-Modal Food Retrieval

Cross-modal image-recipe retrieval has gained significant attention in recent years. Most work focuses on improving cross-modal embeddings using unimodal encoders, that allow for efficient retrieval in large-scale databases, leaving aside cross-attention between modalities which is more computationally expensive. We propose a new retrieval framework, T-Food (T ransformer Decoders with MultiModal Regularization for Cross-Modal Food Retrieval) that exploits the interaction between modalities in a novel regularization scheme, while using only unimodal encoders at test time for efficient retrieval. We also capture the intra-dependencies between recipe entities with a dedicated recipe encoder, and propose new variants of triplet losses with dynamic margins that adapt to the difficulty of the task. Finally, we leverage the power of the recent Vision and Language Pretraining (VLP) models such as CLIP for the image encoder. Our approach outperforms existing approaches by a large margin on the Recipe1M dataset. Specifically, we achieve absolute improvements of 8.1 % (72.6 R@1) and +10.9 % (44.6 R@1) on the 1k and 10k test sets respectively. The code is available here:https://github.com/mshukor/TFood.
Generative Probabilistic Novelty Detection with Isometric Adversarial Autoencoders

Learning the manifold of a complex distribution is a fundamental challenge for novelty or anomaly detection. We introduce a revised learning and inference procedure that takes into account a key underlying assumption made by the framework of generative probabilistic novelty detection. The traditional framework implies the ability to not only learn the manifold of the generative distribution of inliers but also to compute non-linear orthogonal projections onto this manifold from the ambient space. We augment the original training with priors that endow the model with this property, and prove that inference becomes easier and computationally more efficient. We show experimentally that the new framework leads to improved and more stable results.
Generic parallel data structures and algorithms to GPU superpixel image segmentation

We propose parallel implementation on GPU (graphics processing unit) system for some generic algorithms applied to superpixel image segmentation problem. The aim is to provide standard algorithms based on generic decentralized data structures that could be easily improved and customized on many optimization problems on parallel platforms. Note that superpixel segmentation are clustering algorithms applied to image processing. Two types of algorithms are presented and implemented on GPU based on common parallel data structures. Firstly, we present a parallel implementation of the well-known k-means algorithm with application to 3D data. It is based on a cellular grid subdivision of space that allows closest point findings in constant optimal time for bounded distributions. Secondly, we present an application of the parallel Boruvka minimum spanning forest algorithm to compute watershed segmentation. Both techniques are fully executed on GPU and share the same data structures that embed disjoint-set-trees and distributed-link-lists. We evaluate our k-means approach with regards to state-of-the-art methods, that are, the well known SLIC algorithm, and the adaptive segmentation approach SPASM. We argue that our implementation has the shortest execution time among the tested methods, with a near real time performance and quasi linear acceleration factor, while it provides more regular shape superpixel segmentation based on hexagonal tessellation. Watershed minimum spanning forest method is presented and evaluated accordingly to the same experimental framework.
Dual-Branch Collaborative Transformer for Virtual Try-On

Image-based virtual try-on has recently gained a lot of attention in both the scientific and fashion industry communities due to its challenging setting and practical real-world applications.While pure convolutional approaches have been explored to solve the task, Transformer-based architectures have not received significant attention yet.Following the intuition that self-and cross-attention operators can deal with long-range dependencies and hence improve the generation, in this paper we extend a Transformer-based virtual try-on model by adding a dual-branch collaborative module that can exploit cross-modal information at generation time.We perform experiments on the VITON dataset, which is the standard benchmark for the task, and on a recently collected virtual try-on dataset with multi-category clothing, Dress Code.Experimental results demonstrate the effectiveness of our solution over previous methods and show that Transformer-based architectures can be a viable alternative for virtual try-on.
Signature Detection, Restoration, and Verification: A Novel Chinese Document Signature Forgery Detection Benchmark

Offline signature forgery detection has attracted many researchers in recent years. In real situations, signatures should be detected from the signed documents and verified by the forgery detection system. There are many challenges in the pipeline. First, some signatures have low resolutions and are difficult to be detected. Second, the cropped signatures may contain irrelevant background context of the document, making the signature hard to be verified. Third, some forgery signatures are very similar to genuine ones, increasing the challenge of verification. In addition, most existing datasets do not cover all the pipeline tasks. Moreover, publicly available Chinese-based signature datasets are rare for research purposes. In this paper, we construct a novel Chinese document offline signature forgery detection benchmark, namely ChiSig, which includes all pipeline tasks, i.e., signature detection, restoration, and verification. Besides, we extensively compare different deep learning-based approaches in these three tasks. The results show that our proposed dataset can effectively provide solutions for constructing pipeline systems for Chinese document signature forgery detection.