Reviewerly/exHarmony huggingface dataset:
A Dual Self-Attention mechanism for vehicle re-Identification

Vehicle re-identification has attracted tremendous attention from computer vision communities for its extensive applications in intelligent transportation and public security, while the high inter-class similarity and the large intra-class difference between vehicles bring out great challenges for re-identification (re-ID). To tackle these challenges, we learn from the self-attention mechanism in Natural Language Processing and propose a dual self-attention module to learn different regional dependencies: static self-attention for selectively refining semantic features and dynamic self-attention (called cross-region attention) for enhancing the spatial awareness of local feature. The static self-attention refines attended pixels within the entire image and salient regions, while the cross-region attention creatively captures the position-related regional dependencies for pixels within the windscreen area. These attention modules capture long-range dependencies and relative position information between different pixels or regions for vehicle feature learning globally and locally, realizing an efficient vehicle feature embedding by concatenating these augmented features for vehicle re-ID. Extensive experiments demonstrate the effectiveness and promising performance of our approach against the state-of-the-arts.
DeepFake detection with multi-scale convolution and vision transformer

With the help of some modern image generative techniques, it is possible to generate or manipulate image or video contents without introducing any obvious visual artifacts. If these manipulated images/videos are abused, it probably has a huge negative impact on society and individuals. Thus, deepfake detection has attracted considerable attention in recent years. Although the existing methods can achieve good detection performance on high-quality datasets, they are still far from satisfactory for low-quality dataset and cross-dataset evaluation. In this paper, therefore, we propose a new CNN-based method via multi-scale convolution and vision transformer for deepfake detection. In the proposed model, we design a multi-scale module with dilation convolution and depthwise separable convolution to capture more face details and tampering artifacts at different scales. Unlike the traditional classification module, furthermore, we employ a vision transformer to further learn the global information of face features for classification. Extensive experiments demonstrate that in most cases the proposed method achieves better detection results on both high-quality and low-quality datasets compared with related modern methods, and the cross-dataset generalization capabilities of the proposed method are good. In addition, many ablation experiments are provided to verify the rationality of the proposed network.
Hybrid Quantum-Classical Neural Networks

Deep learning is one of the most successful and far-reaching strategies used in machine learning today. However, the scale and utility of neural networks is still greatly limited by the current hardware used to train them. These concerns have become increasingly pressing as conventional computers are soon expected to approach the physical limitations that will slow their performance improvements in the near future. For these reasons, scientists have begun to explore alternative computing platforms, like quantum computers, for training neural networks. In recent years, variational quantum circuits have emerged as one of the most successful approaches to quantum deep learning on noisy intermediate scale quantum devices. We propose a hybrid quantum-classical neural network architecture where each neuron is a variational quantum circuit. We empirically analyze the performance of this hybrid neural network on a series of binary classification data sets using a simulated IBM universal quantum computer and a state-of-the-art IBM universal quantum computer. On the simulated hardware, we observe that the hybrid neural network achieves around 10% higher classification accuracy and 20% better minimization of the cost function than an individual variational quantum circuit. On the quantum hardware, we observe that each model only performs well when the qubit and gate count is sufficiently small.
Optimized face detector-based intelligent face mask detection model in IoT using deep learning approach

The growth of the “Internet of Medical Things (IoMT)” allows for the collection and processing of data in healthcare systems. At the same time, it is challenging to study the requirements of public health prevention. Here, mask-wearing is considered an efficient preventive measure for avoiding virus transfer. Hence, it is necessary to implement an automated mask identification model to prevent public epidemics. The main scope of the proposed method is to design a face mask detection model with IoT using a “Single Shot Multi-box Detector (SSD)” and a hybrid deep learning method. The novelty of the proposed model is that the enhancement made in the face detection and face classification with the developed ASMFO by optimizing the parameters like the threshold in SSD, steps per execution in ResNet, and learning rate in MobileNet, which makes it more efficient and to perform better the conventional models. Here, the parameter optimization is carried out using a hybrid optimization algorithm named Adaptive Sailfish Moth Flame Optimization (ASMFO). Then, the detected face images are given to the hybrid approach named Hybrid ResMobileNet (HResMobileNet)-based classification, where the parameters are tuned using the same ASMFO algorithm for achieving accurate mask detection results. However, the suggested mask identification model with IoT based on three standard datasets is compared with the conventional meta-heuristic algorithms and existing classifiers with various measures. Thus, the experimental analysis is conducted to analyze the effectiveness of the proposed framework over different meta-heuristic algorithms and existing classifiers. The implemented ASMFO-HResMobileNet provides 18.57%, 15.67%, 17.56%, 16.24%, and 19.2% elevated accuracy than SVM, CNN, VGG16-LSTM, ResNet 50, MobileNetv2, and ResNet 50-MobileNetv2.
Financial cybercrime in the Islamic Finance Metaverse

Financial cybercrime in the metaverse has become increasingly more significant for authorities, corporations, and individuals to address, requiring new regulatory and compliance frameworks, as well as novel cybersecurity mechanisms in order to prevent these crimes. Financial cybercrimes in the metaverse have increased in the last years significantly, with either the massive stealing of cryptocurrencies from exchanges or the sale of fake or dubious NFT and other financial products that have lost significant value within a short period of time. Cybercrimes in the metaverse have taken place at significant scales and given the infancy of regulations as well as the virtual nature of these activities, only few crimes have been prosecuted. Islamic finance may represent a considerable opportunity for the metaverse via connecting the financial services and instruments to real and virtual assets free of speculation. The article provides several recommendations for regulators to address these cybercrime challenges and how Islamic finance can assist in these cybercrimes.
Time series forecasting using fuzzy cognitive maps: a survey

Among various soft computing approaches for time series forecasting, fuzzy cognitive maps (FCMs) have shown remarkable results as a tool to model and analyze the dynamics of complex systems. FCMs have similarities to recurrent neural networks and can be classified as a neuro-fuzzy method. In other words, FCMs are a mixture of fuzzy logic, neural network, and expert system aspects, which act as a powerful tool for simulating and studying the dynamic behavior of complex systems. The most interesting features are knowledge interpretability, dynamic characteristics and learning capability. The goal of this survey paper is mainly to present an overview on the most relevant and recent FCM-based time series forecasting models proposed in the literature. In addition, this article considers an introduction on the fundamentals of FCM model and learning methodologies. Also, this survey provides some ideas for future research to enhance the capabilities of FCM in order to cover some challenges in the real-world experiments such as handling non-stationary data and scalability issues. Moreover, equipping FCMs with fast learning algorithms is one of the major concerns in this area.
Identifying and evaluating barriers for the implementation of machine learning in the intensive care unit

Despite apparent promise and the availability of numerous examples in the literature, machine learning models are rarely used in practice in ICU units. This mismatch suggests that there are poorly understood barriers preventing uptake, which we aim to identify.We begin with a qualitative study with 29 interviews of 40 Intensive Care Unit-, hospital- and MedTech company staff members. As a follow-up to the study, we attempt to quantify some of the technical issues raised. To perform experiments we selected two models based on criteria such as medical relevance. Using these models we measure the loss of performance in predictive models due to drift over time, change of available patient features, scarceness of data, and deploying a model in a different context to the one it was built in.The qualitative study confirms our assumptions on the potential of AI-driven analytics for patient care, as well as showing the prevalence and type of technical blocking factors that are responsible for its slow uptake. The experiments confirm that each of these issues can cause important loss of predictive model performance, depending on the model and the issue.Based on the qualitative study and quantitative experiments we conclude that more research on practical solutions to enable AI-driven innovation in Intensive Care Units is needed. Furthermore, the general poor situation with respect to public, usable implementations of predictive models would appear to limit the possibilities for both the scientific repeatability of the underlying research and the transfer of this research into practice.It is helpful for clinicians to be able to predict what will happen to a patient in an Intensive Care Unit (ICU); accurate computer-based predictive systems could help to avoid serious illness. However, most ICUs currently make little or no use of them. Here, we try to understand why, so that barriers to their introduction can be overcome. We interview medical experts, who agree that prediction systems should be feasible. They also identify practical technical problems with using them. We investigate these issues by running experiments on example predictive systems where we change what data is used to train the system and what data it is asked to make predictions on. The experiments show that the identified issues cause problems and are worthy of further attention. This work should help to enable the use of computer-based predictive systems in ICUs.
A Multi-Robot Coverage Path Planning Method for Maritime Search and Rescue Using Multiple AUVs

In this study, we focus on the Multi-robot Coverage Path Planning (MCPP) problem for maritime Search And Rescue (SAR) missions using a multiple Autonomous Underwater Vehicle (AUV) system, with the ultimate purpose of efficiently and accurately discovering the target from sonar images taken by Side-Scan Sonar (SSS) mounted on the AUVs. Considering the specificities of real maritime SAR projects, we propose a novel MCPP method, in which the MCPP problem is transformed into two sub-problems: Area partitioning and single-AUV coverage path planning. The structure of the task area is first defined using Morse decomposition of the spike pattern. The area partitioning problem is then formulated as an AUV ordering problem, which is solved by developing a customized backtracking method to balance the workload and to avoid segmentation of the possible target area. As for the single-AUV coverage path planning problem, the SAR-A* method is adopted, which generates a path that preferentially visits the possible target areas and reduces the number of turns to guarantee the high quality of the resulting sonar images. Simulation results demonstrate that the proposed method can maintain the workload balance and significantly improve the efficiency and accuracy of discovering the target. Moreover, our experimental results indicate that the proposed method is practical and the mentioned specificities are useful for discovering targets.
Arabic abstractive text summarization using RNN-based and transformer-based architectures

Recently, the Transformer model architecture and the pre-trained Transformer-based language models have shown impressive performance when used in solving both natural language understanding and text generation tasks. Nevertheless, there is little research done on using these models for text generation in Arabic. This research aims at leveraging and comparing the performance of different model architectures, including RNN-based and Transformer-based ones, and different pre-trained language models, including mBERT, AraBERT, AraGPT2, and AraT5 for Arabic abstractive summarization. We first built an Arabic summarization dataset of 84,764 high-quality text-summary pairs. To use mBERT and AraBERT in the context of text summarization, we employed a BERT2BERT-based encoder-decoder model where we initialized both the encoder and decoder with the respective model weights. The proposed models have been tested using ROUGE metrics and manual human evaluation. We also compared their performance on out-of-domain data. Our pre-trained Transformer-based models give a large improvement in performance with ∼79% less data. We found that AraT5 scores ∼3 ROUGE higher than a BERT2BERT-based model that is initialized with AraBERT, indicating that an encoder-decoder pre-trained Transformer is more suitable for summarizing Arabic text. Also, both of these two models perform better than AraGPT2 by a clear margin, which we found to produce summaries with high readability but with relatively lesser quality. On the other hand, we found that both AraT5 and AraGPT2 are better at summarizing out-of-domain text. We released our models and dataset publicly1,.2
Per-Clip Video Object Segmentation

Recently, memory-based approaches show promising results on semi-supervised video object segmentation. These methods predict object masks frame-by-frame with the help of frequently updated memory of the previous mask. Different from this per-frame inference, we investigate an alternative perspective by treating video object segmentation as clip-wise mask propagation. In this per-clip inference scheme, we update the memory with an interval and simul-taneously process a set of consecutive frames (i.e. clip) between the memory updates. The scheme provides two potential benefits: accuracy gain by clip-level optimization and efficiency gain by parallel computation of multiple frames. To this end, we propose a new method tailored for the perclip inference. Specifically, we first introduce a clip-wise operation to refine the features based on intra-clip correlation. In addition, we employ a progressive matching mechanism for efficient information-passing within a clip. With the synergy of two modules and a newly proposed perclip based training, our network achieves state-of-the-art performance on Youtube-VOS 2018/2019 val (84.6% and 84.6%) and DAVIS 2016/2017 val (91.9% and 86.1%). Fur-thermore, our model shows a great speed-accuracy trade-off with varying memory update intervals, which leads to huge flexibility.
Spatially-Adaptive Multilayer Selection for GAN Inversion and Editing

Existing GAN inversion and editing methods work well for aligned objects with a clean background, such as portraits and animal faces, but often struggle for more difficult categories with complex scene layouts and object occlusions, such as cars, animals, and outdoor images. We propose a new method to invert and edit such complex images in the latent space of GANs, such as StyleGAN2. Our key idea is to explore inversion with a collection of layers, spatially adapting the inversion process to the difficulty of the image. We learn to predict the “invertibility” of different image segments and project each segment into a latent layer. Easier regions can be inverted into an earlier layer in the generator's latent space, while more challenging regions can be inverted into a later feature space. Experiments show that our method obtains better inversion results compared to the recent approaches on complex categories, while maintaining downstream editability. Please refer to our project page at gauravparmar.com/sam_inversion.
AutoLoss-GMS: Searching Generalized Margin-based Softmax Loss Function for Person Re-identification

Person re-identification is a hot topic in computer vision, and the loss function plays a vital role in improving the discrimination of the learned features. However, most existing models utilize the hand-crafted loss functions, which are usually sub-optimal and challenging to be designed. In this paper, we propose a novel method, AutoLoss-GMS, to search the better loss function in the space of generalized margin-based softmax loss function for person reidentification automatically. Specifically, the generalized margin-based softmax loss function is first decomposed into two computational graphs and a constant. Then a general searching framework built upon the evolutionary algorithm is proposed to search for the loss function efficiently. The computational graph is constructed with a forward method, which can construct much richer loss function forms than the backward method used in existing works. In addition to the basic in-graph mutation operations, the cross-graph mutation operation is designed to further improve the offspring's diversity. The loss-rejection protocol, equivalence-check strategy and the predictor-based promising-loss chooser are developed to improve the search efficiency. Finally, experimental results demonstrate that the searched loss functions can achieve state-of-the-art performance and be transferable across different models and datasets in person re-identification.
A New Framework for More Efficient Round-Optimal Lattice-Based (Partially) Blind Signature via Trapdoor Sampling

Blind signatures, proposed by Chaum (CRYPTO’82), are interactive protocols between a signer and a user, where a user can obtain a signature without revealing the message to be signed. Recently, Hauck et al. (EUROCRYPT’20) observed that all efficient lattice-based blind signatures following the blueprint of the original blind signature by Rükert (ASIACRYPT’10) have a flawed security proof. This puts us in a situation where all known lattice-based blind signatures have at least two of the following drawbacks: heuristic security; 1 MB or more signature size; only supporting bounded polynomially many signatures, or being based on non-standard assumptions. In this work, we construct the first round-optimal (i.e., two-round) lattice-based blind signature with a signature size roughly 100 KB that supports unbounded polynomially many signatures and is provably secure under standard assumptions. Even if we allow non-standard assumptions and more rounds, ours provide the shortest signature size while simultaneously supporting unbounded polynomially many signatures. The main idea of our work is revisiting the generic blind signature construction by Fischlin (CRYPTO’06) and optimizing the commit-then-open proof using techniques tailored to lattices. Our blind signature is also the first construction to have a formal security proof in the quantum random oracle model. Finally, our blind signature extends naturally to partially blind signatures, where the user and signer can include an agreed-upon public string in the message.
Unifying Panoptic Segmentation for Autonomous Driving

This paper aims to improve panoptic segmentation for real-world applications in three ways. First, we present a label policy that unifies four of the most popular panoptic segmentation datasets for autonomous driving. We also clean up label confusion by adding the new vehicle labels pickup and van. Full relabeling information for the popular Mapillary Vistas, IDD, and Cityscapes dataset are provided to add these new labels to existing setups. Second, we introduce Wilddash2 (WD2), a new dataset and public benchmark service for panoptic segmentation. The dataset consists of more than 5000 unique driving scenes from all over the world with a focus on visually challenging scenes, such as diverse weather conditions, lighting situations, and camera characteristics. We showcase experimental visual hazard classifiers which help to pre-filter challenging frames during dataset creation. Finally, to characterize the robustness of algorithms in out-of-distribution situations, we introduce hazard-aware and negative testing for panoptic segmentation as well as statistical significance calculations that increase confidence for both concepts. Additionally, we present a novel technique for visualizing panoptic segmentation errors. Our experiments show the negative impact of visual hazards on panoptic segmentation quality. Additional data from the WD2 dataset improves performance for visually challenging scenes and thus robustness in real-world scenarios.
SwapMix: Diagnosing and Regularizing the Over-Reliance on Visual Context in Visual Question Answering

While Visual Question Answering (VQA) has progressed rapidly, previous works raise concerns about robustness of current VQA models. In this work, we study the robustness of VQA models from a novel perspective: visual context. We suggest that the models over-rely on the visual context, i.e., irrelevant objects in the image, to make predictions. To diagnose the models' reliance on visual context and measure their robustness, we propose a simple yet effective perturbation technique, SwapMix. SwapMix perturbs the visual context by swapping features of irrelevant context objects with features from other objects in the dataset. Using SwapMix we are able to change answers to more than 45% of the questions for a representative VQA model. Additionally, we train the models with perfect sight and find that the context over-reliance highly depends on the quality of visual representations. In addition to diagnosing, SwapMix can also be applied as a data augmentation strategy during training in order to regularize the context over-reliance. By swapping the context object features, the model reliance on context can be suppressed effectively. Two representative VQA models are studied using SwapMix: a co-attention model MCAN and a large-scale pretrained model LXMERT. Our experiments on the popular GQA dataset show the effectiveness of SwapMix for both diagnosing model robustness, and regularizing the over-reliance on visual context. The code for our method is available at https://github.com/vipulgupta1011/swapmix