Reviewerly/exHarmony huggingface dataset:
Atmospheric optical turbulence mitigation using iterative image registration and least squares lucky look fusion

This paper presents an atmospheric optical turbulence mitigation method that uses a sequence of short-exposure frames. An iterative block matching registration method is proposed for image dewarping. The dewarped frames are combined in a least squares lucky look (LL) fusion process. Here image patches are weighted so as to produce a fused image that is consistent with a theoretical LL optical transfer function (OTF) model. Finally, a Wiener filter is applied to provide deconvolution of the LL OTF. We also explore the LL selectivity tradespace. As the selectivity increases, the LL OTF becomes more favorable but the signal-to-noise ratio suffers, and vice versa. A restoration algorithm is applied to simulated data for quantitative analysis and two different real-world datasets for subjective evaluation. The proposed approach provides improved results compared with the benchmark methods.
Route Planning Based on Grid-Point Optimization Lazy Theat* Algorithm

The problem of grid-point path planning is explored in the paper, the threat source model is established for the penetration strike task, and the flight path cost, various flight constraints and other factors are comprehensively considered. Planning in grid points improves the variability and accuracy of route planning. The Lazy Theat* was used to plan a flyable route between the start and end positions, and it is compared with the route planned under the square grid. The simulation results showed that the route planning under the grid points is more suitable for the actual flight trajectory.
Meta-Analysis of Nature Inspired Hybrid Cuckoo Algorithm

Diving deep into nature makes us understand it better. Nature inspires researchers world-wide to design technology based upon natural phenomenon. There are various birds such as migrating birds, cranes, and cuckoo which are used as inspiration to design algorithms. In this paper, the nature-inspired Cuckoo Algorithm, which is based on brood parasitism reproductive strategy, is discussed in detail. The Cuckoo Algorithm has various hybrids which are used in numerous applications; all these hybrids are reviewed and discussed in this paper. Meta-analysis is conducted using WoS and Scopus databases to study its variants and application domains, analyze the research trend for these hybrid algorithms and predict the future for these nature-inspired algorithms. Numerous experiments based upon correlation of related terms, year-wise analysis, article growth, etc. are conducted and described in this paper.
Offline Handwritten Signature Verification Using Decision Tree

The verification of a signature is an exigent research area as the signatures of two individuals may have similarity whereas the signatures of a person may vary at various circumstances. The accuracy of signature verification framework relies mainly upon the classifier used for the classification process and the feature extraction scheme. Keeping this perspective in sight, the goal of this study is to see how well a decision tree classifier combined with a Local Binary Pattern feature set can be utilized to construct an offline writer-independent signature verification system. To evaluate the system’s performance, two signature databases of 100 and 260 writers are used. Genuine signatures as well as random forgery signatures are utilized for the development of the desired system, while genuine signatures, as well as random forgery, unskilled forgery, and simulated forgery are used to test the performance of the developed system. In simulation study, false acceptance rate of 1.00%, 7.00% and 11.00% for random, unskilled, and simulated forgery signatures, respectively is obtained whereas the false rejection rate of 0.00% is achieved using Local Binary Pattern features.
Knowledge-Based Representation Learning for Nucleus Instance Classification From Histopathological Images

The classification of nuclei in H&E-stained histopathological images is a fundamental step in the quantitative analysis of digital pathology. Most existing methods employ multi-class classification on the detected nucleus instances, while the annotation scale greatly limits their performance. Moreover, they often downplay the contextual information surrounding nucleus instances that is critical for classification. To explicitly provide contextual information to the classification model, we design a new structured input consisting of a content-rich image patch and a target instance mask. The image patch provides rich contextual information, while the target instance mask indicates the location of the instance to be classified and emphasizes its shape. Benefiting from our structured input format, we propose Structured Triplet for representation learning, a triplet learning framework on unlabelled nucleus instances with customized positive and negative sampling strategies. We pre-train a feature extraction model based on this framework with a large-scale unlabeled dataset, making it possible to train an effective classification model with limited annotated data. We also add two auxiliary branches, namely the attribute learning branch and the conventional self-supervised learning branch, to further improve its performance. As part of this work, we will release a new dataset of H&E-stained pathology images with nucleus instance masks, containing 20,187 patches of size 1024 ×1024 , where each patch comes from a different whole-slide image. The model pre-trained on this dataset with our framework significantly reduces the burden of extensive labeling. We show a substantial improvement in nucleus classification accuracy compared with the state-of-the-art methods.
The Dual Space: Concept and Applications in Cultural Heritage

An important question in cultural heritage concerns the make process of an artifact. Understanding the make process provides insight related to the origin, techniques and craftsmanship used to make the artifact. Searching for tool marks or traces left by the artist&#x0027;s hand is one way of retrieving clues related to the make process. X-ray computed tomography (CT) is a non-destructive tool that produces volumetric images of structures inside an artifact. However, interactively searching in large volumetric images for tool marks is a difficult, tedious and time consuming task. In this article, we introduce the concept of a dual space. The governing idea is that the dual space represents the air in the interior of an object. In the context of cultural heritage, the dual space represents those materials that first belonged to the object but have been removed during the make process. Our main goal of creating the dual space is to facilitate searching, inspection and interpretation of tool marks. We provide two examples of how the dual space can be used to study the make process.
Toward Round-Efficient Verifiable Re-Encryption Mix-Net

Re-encryption mix-nets (RMNs) provide an efficient, cryptographic, anonymous channel for useful applications, such as e-voting and web browsing. Many studies have been devoted to achieving practically efficient RMN protocols, but less attention has been paid to addressing their round efficiency than to computation and communication measures. However, in many interactive cryptographic protocols, network latency governs the overall execution time. Because e-voting systems are particularly interaction intensive, the design of a round-efficient RMN protocol is of particular interest. We propose a constant-round RMN protocol in a three-party model that consists of senders, mix servers and some number of receivers. Here, the main role of the receivers is to jointly decrypt a list of ciphertexts obtained from the mixing stage. Such an explicit three-party model is most suitable for e-voting applications. We define an ideal three-party RMN in the universally composable (UC) framework. We then present a constant-round RMN protocol based on the standard assumptions and prove that it UC realizes the ideal three-party RMN with respect to a static adversary that can corrupt a minority of mix servers, disallowing receivers who collude with other players.We implement and evaluate our RMN protocol over various ranges of the numbers of senders and mix servers. Our evaluation shows that our protocol runs up to 2.5&#x00D7; faster than the Universal RMN protocol. In addition, we provide a detailed theoretical analysis of our protocol in terms of computation, transmission, and round efficiency.
Direct Speech-to-Text Translation Models as Students of Text-to-Text Models

Direct speech-to-text translation (ST) is an emerging approach that consists in performing the ST task with a single neural model.Although this paradigm comes with the promise to outperform the traditional pipeline systems, its rise is still limited by the paucity of speech-translation paired corpora compared to the large amount of speech-transcript and parallel bilingual corpora available to train previous solutions.As such, the research community focused on techniques to transfer knowledge from automatic speech recognition (ASR) and machine translation (MT) models trained on huge datasets.In this paper, we extend and integrate our recent work (Gaido et al. 2020b) analysing the best performing approach to transfer learning from MT, which is represented by knowledge distillation (KD) in sequence-to-sequence models.After the comparison of the different KD methods to understand which one is the most effective, we extend our previous analysis of the effects -both in terms of benefits and drawbacks -to different language pairs in high-resource conditions, ensuring the generalisability of our findings.Altogether, these extensions complement and complete our investigation on KD for speech translation leading to the following overall findings: i) the best training recipe involves a word-level KD training followed by a fine-tuning step on the ST task, ii) word-level KD from MT can be detrimental for gender translation and can lead to output truncation (though these problems are alleviated by the fine-tuning on the ST task), and iii) the quality of the ST student model strongly depends on the quality of the MT teacher model, although the correlation is not linear.
A Gaussian High-Dimensional Random Matrix-Based Method for Detecting Abnormal Student Behaviour in Chinese Language Classrooms

In this paper, a Gaussian high-dimensional random matrix approach is used to conduct an in-depth study and analysis of the detection of abnormal student behaviour in Chinese language classrooms, and a corresponding model is designed for practical application. The Gaussian high-dimensional random matrix technique and recurrent neural network technique are applied to build a basic technical framework for intelligent proctoring. An innovative method of human posture estimation is used to complete the proctoring task, and the structure of the Gaussian high-dimensional random matrix method is simplified and the model is compressed to achieve real-time and parallelism of the method. A PolSAR sparse representation classification (OGRM_SRC) algorithm based on orthogonal Gaussian random matrices (OGRM) is proposed to construct an observation dictionary based on polarisation features and OGRM to select typical ground target samples in polarized feature images. The system not only functions as a psychometric assessment of students but also is capable of detecting and analysing abnormal student behaviour based on the improved forest algorithm. The residuals of the pixel to be classified are calculated relative to each atom in the observation dictionary, and the minimum reconstruction residual is used as the classification criterion. The OGRM_SRC algorithm is applied to classify the pixel, and the final classification results are obtained and evaluated for accuracy. We propose a fine-grained action recognition optimisation method for recurrent neural networks, fusing temporal attention information and spatial attention information to improve the action recognition method and improve the model’s ability to recognise fine-grained abnormal behaviours such as looking left, looking right, and copying with head down. Based on this, we conducted some comparative experiments to validate the effectiveness of our work and verified that the system achieved a recognition efficiency of 76.8 FPS and a recognition accuracy of 98.5% in a graphical computing processor.
Metaverse Interface with Haptic and Rigid Sense Feedback at a Low Cost

In the metaverse era, anyone should be able to easily access the metaverse for non-discrimination. This study proposes a prototype of a metaverse interface that can be implemented at a low price. It shows reasonable performance. It includes the following 4 modules. 1. Haptic feedback: Piezo sensors have been used for tactile information of various textures, which is transported as an audio data format, and reconstructed at the fingertips by an actuator (LRA motor) attached to a glove. 2. Force feedback: The module was designed to control the servomotor through 3D modeling. It provides force feedback by controlling the servo motor to grab the hand as much as the thickness of the object on Unity, the game engine. 3. Controller: Reproduce finger movement in Unity by measuring resistance change according to finger bending using a Flex sensor. Implement Yawing/Pitching/Rolling movement in Unity through MPU-6050(IMU) sensor. 4. Pose estimation: The user's current movement captured by the webcam is used for pose estimation through the Mediapipe library. It is integrated with the hand posture received serially from the controller to render the user posture inside the Unity game. The developed interface can be applied to life logging service and mirror world service, which are one of the most important applications in the metaverse.
HGLNET: A Generic Hierarchical Global-Local Feature Fusion Network for Multi-Modal Classification

Multi-modal fusion aims to capture the semantic interactions between different modalities for many downstream classification tasks. However, previous work usually considers that each modality contributes equal information to the final classification and extracts the global features of each modality for fusion. In this paper, inspired by these two observations, we propose a generic Hierarchical Global-Local feature fusion Network (HGLNet) for multi-modal classification. Specifically, HGLNet has three merits compared to the current work. (1) HGLNet proposes a Global Gated Attention (GGA) module, which adaptively generates weights that represent the contributions of different modalities. (2) HGLNet presents a novel Cross Residual Transformer (CRT) module to capture the fine-grained local interactions. (3) HGLNet utilizes hierarchical information for multi-modal fusion. Extensive experiments on three public datasets demonstrate that HGLNet achieves competitive performance against the state-of-the-art methods for three kinds of multi-modal classification tasks.
Discovering Domain Disentanglement for Generalized Multi-Source Domain Adaptation

A typical multi-source domain adaptation (MSDA) approach aims to transfer knowledge learned from a set of labeled source domains, to an unlabeled target domain. Neverthe-less, prior works strictly assume that each source domain shares the identical group of classes with the target domain, which could hardly be guaranteed as the target label space is not observable. In this paper, we consider a more versatile setting of MSDA, namely Generalized Multi-source Domain Adaptation, wherein the source domains are partially over-lapped, and the target domain is allowed to contain novel cat-egories that are not presented in any source domains. This new setting is more elusive than any existing domain adaptation protocols due to the coexistence of the domain and cate-gory shifts across the source and target domains. To address this issue, we propose a variational domain disentanglement (VDD) framework, which decomposes the domain represen-tations and semantic features for each instance by encouraging dimension-wise independence. To identify the target sam-ples of unknown classes, we leverage online pseudo labeling, which assigns the pseudo-labels to unlabeled target data based on the confidence scores. Quantitative and qualitative experi-ments conducted on two benchmark datasets demonstrate the validity of the proposed framework.
Discriminator Modification in GAN for Text-to-Image Generation

The existing Generative Adversarial Network-based text-to-image generation methods suffer from mode collapse and training instability. This paper relieves these problems by improving the discriminator ability from three aspects. First, we propose a diversity-sensitive conditional discriminator (D-SCD), which increases the diversity of the generated images by judging the combination of the generated image and mismatched text as false. Second, for the unconditional discriminator, we propose a contrastive searching gradient penalty (CSGP) strategy to measure the realism of the generated images and to penalize the gradients for stabilizing the training process. Finally, we introduce a multi-level images similarity (MLIS) loss for the discriminator feature extractor to further promote the high-level feature similarity between the real and generated images and objects. Extensive experimental results and ablation studies demonstrate that our modifications on the discriminators can effectively improve the quality of the generated images.
DH-GCN: Saliency-Aware Complex Scene Graph Generation Using Dual-Hierarchy Graph Convolutional Network

In reality, complex scene plagues numerous scene graph generation models because realistic scene contains myriad of objects and complicated relationships. Most current methods suffer poor performance when encountering complex scenes. We find that there are two principal reasons for this phenomenon. First, the construction of graph loses sight of the hierarchy of objects. Second, there exists redundant information in feature optimization. To facilitate this issue, this paper proposes an innovative dual-hierarchy graph convolutional network (DH-GCN), which is a conceptually elegant and efficient top-down approach. In specific, DH-GCN leverages salient object detector to hierarchize objects and give gist nodes more accurate representation. Moreover, the dual-hierarchy message propagation is designed to refine the representation hierarchically and eliminate redundant information. Systematic experiments on Visual Genome dataset show the superiority of our method over strong baseline methods.
FHGN: Frame-Level Heterogeneous Graph Networks for Video Question Answering

Video Question Answering (VideoQA), aims to answer the given question based on video grounding, reasoning, and multimodal interacting. Most existing studies usually extract video-level visual and linguistic embeddings separately, then perform spatial-temporal or graph-based reasoning in a single modality. These methods lack the understanding of the interactions among different modalities and ignore the rele-vance of different frames, objects, and modalities to the question. This paper proposes Frame-level Heterogeneous Graph Networks (FHGN), to learn the semantic correlations among different modalities in each frame. Specifically, we construct a uniform frame-level heterogeneous graph to perform both inter- and intra-modality reasoning. Then a two-stage attention module is applied to measure the relevance between the question and different periods, space regions, and modalities. We conduct extensive experiments on a large-scale VideoQA dataset, and the experimental results show that our FHGN achieves the state-of-the-art performance.
A Deep-Convolution-Generative-Adversarial-Networks-based Missing Data Filling Method for Blast Furnace Gas System in Steel Industry

The integrity of monitoring data is of great significance to ensure the accuracy of data analysis and reliable operation in blast furnace gas system of steel industry. In this study, a Deep-Convolution-Generative-Adversarial-Networks (DCGAN)-based data filling method is proposed for high proportion missing in time series, and the corresponding network structure is designed. Time series are transformed non-destructively into time domain images by a Gram matrix, and their temporal characteristics are preserved. The authenticity constraint and the context similarity are established to optimize the hidden variables, and the high-precision time domain image is generated through the DCGAN. The completed time series is obtained by inversely transforming the Gram matrix. The simulation results based on the actual operating data of a steel enterprise indicate that the proposed method is capable of has high padding accuracy in the case of a high proportion of random missing data and continuous missing data.
High-Quality Image Generation from Scene Graphs with Transformer

Image generation from scene graphs has traditionally focused on predicting layout from the scene graph using graph convolutional networks firstly, then converting the layout to an image. These methods might involve complex architectures, auxiliary losses, or side information such as object location or image crops during training. We propose a new method based on transformer for end-to-end image generation from scene graphs. To convert the scene graphs to the input format required by the transformer, we propose a novel method to tokenize the scene graphs. Furthermore, a discrete autoencoder is used to map the image to abstract space. Finally we train the transformer predicting image tokens conditioned on the scene graphs in an autoregressive way. Experiments on Visual Genome and Coco-Stuff datasets show that our method significantly outperforms the state-of-the-art methods in terms of image quality.
Adaptive Multi-Order Graph Neural Networks for Human Motion Prediction

Human motion prediction aims at capturing the hidden temporal correlations between historical motion and future poses. Various graph convolution networks have been presented for encoding the spatial dependencies between joints. Empirically, the crucial shortcoming of these methods is that they fail to extract enough spatially relevant information. In this paper, we propose an adaptive multi-order context fusion architecture that consists of two components. A novel message propagation module encodes the interaction between joints, while highlighting contexts from closely related joints. An adaptive aggregation module fuses various information from different-order joint features. Our model is evaluated on Human 3.6 Million dataset. Extensive experiments show that our method achieves state-of-the-art performance on short-term and long-term predictions.
Spatial Attention Guided Local Facial Attribute Editing

Facial attribute manipulation has attracted great attention from the public due to its wide range of applications. Aiming to smoothly manipulate the attributes of real facial images, it is critical to search for a proper latent code that aligns with the domain of pre-trained GAN for faithful inversion and controls the transformation within the scope of the attribute for precise editing. Previous methods mainly focused on improving the quality of reconstruction but often ignored the editing effect. To address this issue, we first propose a mapping network to manipulate latent code which is effective for diverse situations, and design a spatial attention network to predict binary mask of the certain attribute which encourages to only alter the relevant region of images and suppress irrelevant changes. In addition, we introduce a novel latent space into the GAN inversion framework which achieves high reconstruction quality especially preserving identity features and retains the ability to edit face attributes. Our methods pave the way to semantically meaningful and disentangled manipulations on both generated images and real images. Ex-perimental results indicate a clear improvement over the cur-rent state-of-the-art methods in various metrics.