Reviewerly/exHarmony huggingface dataset:
Aerial Infrared Object Tracking via an improved Long-term Correlation Filter with optical flow estimation and SURF matching

At present, there are many excellent algorithms in the field of visual object tracking. The correlation filter algorithms are more suitable for infrared object tracking, for the tracking performance is the best. Especially, the long-term correlation tracking has received much attention, owing to its ability to handle the problems of universal tracking (e.g., slight deformation, small-displacement motion, partial occlusion and out of view). However, there are three imperfections such that it fails to solve the problem of rapid motion, it cannot cope with the problem of boundary effect perfectly, and it has poor tracking effect in the case of severe occlusion and severe deformation. Aiming at the problem of rapid motion, the FlowNet 2.0 is introduced to accomplish optical flow estimation, offering motion information and predicting trajectory change process. Aiming to address the concern of boundary effect perfectly, the adjustable Gaussian window is effective to separate the object and the background, improving classifier discrimination. Aiming at the issue of poor tracking effect in the case of severe occlusion and severe deformation, the SURF feature-based matching method is effective to accurately track object and improve the infrared object tracking performance. In addition, the ratio between average peak-to-correlation energy (APCE) and its historical average, as a further complement of maximum response, is introduced to achieve the online updating mechanism, jointly determining whether the tracker needs to be updated, the SURF matching needs to be carried out or the tracker needs to be initialized by re-detector. Our algorithm is validated on synthetic infrared aerial object image sequences, real infrared thermal aerial object image sequences and a public database named AMCOM FLIR respectively. The extensive experimental testify that our improved approach achieves an optimal effect for aerial infrared object tracking in terms of precision plot, success plot and speed.
Camp: A Two-Stage Approach to Modelling Prosody in Context

Prosody is an integral part of communication, but remains an open problem in state-of-the-art speech synthesis. There are two major issues faced when modelling prosody: (1) prosody varies at a slower rate compared with other content in the acoustic signal (e.g. segmental information and background noise); (2) determining appropriate prosody without sufficient context is an ill-posed problem. In this paper, we propose solutions to both these issues. To mitigate the challenge of modelling a slow-varying signal, we learn to disentangle prosodic information using a word level representation. To alleviate the ill-posed nature of prosody modelling, we use syntactic and semantic information derived from text to learn a context-dependent prior over our prosodic space. Our context-aware model of prosody (CAMP) outperforms the state-of-the-art technique, closing the gap with natural speech by 26%. We also find that replacing attention with a jointly-trained duration model improves prosody significantly.
A pair correlation problem, and counting lattice points with the zeta function

Abstract The pair correlation is a localized statistic for sequences in the unit interval. Pseudo-random behavior with respect to this statistic is called Poissonian behavior. The metric theory of pair correlations of sequences of the form $$(a_n \alpha )_{n \ge 1}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:msub> <mml:mrow> <mml:mo>(</mml:mo> <mml:msub> <mml:mi>a</mml:mi> <mml:mi>n</mml:mi> </mml:msub> <mml:mi>α</mml:mi> <mml:mo>)</mml:mo> </mml:mrow> <mml:mrow> <mml:mi>n</mml:mi> <mml:mo>≥</mml:mo> <mml:mn>1</mml:mn> </mml:mrow> </mml:msub> </mml:math> has been pioneered by Rudnick, Sarnak and Zaharescu. Here $$\alpha $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mi>α</mml:mi> </mml:math> is a real parameter, and $$(a_n)_{n \ge 1}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:msub> <mml:mrow> <mml:mo>(</mml:mo> <mml:msub> <mml:mi>a</mml:mi> <mml:mi>n</mml:mi> </mml:msub> <mml:mo>)</mml:mo> </mml:mrow> <mml:mrow> <mml:mi>n</mml:mi> <mml:mo>≥</mml:mo> <mml:mn>1</mml:mn> </mml:mrow> </mml:msub> </mml:math> is an integer sequence, often of arithmetic origin. Recently, a general framework was developed which gives criteria for Poissonian pair correlation of such sequences for almost every real number $$\alpha $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mi>α</mml:mi> </mml:math> , in terms of the additive energy of the integer sequence $$(a_n)_{n \ge 1}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:msub> <mml:mrow> <mml:mo>(</mml:mo> <mml:msub> <mml:mi>a</mml:mi> <mml:mi>n</mml:mi> </mml:msub> <mml:mo>)</mml:mo> </mml:mrow> <mml:mrow> <mml:mi>n</mml:mi> <mml:mo>≥</mml:mo> <mml:mn>1</mml:mn> </mml:mrow> </mml:msub> </mml:math> . In the present paper we develop a similar framework for the case when $$(a_n)_{n \ge 1}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:msub> <mml:mrow> <mml:mo>(</mml:mo> <mml:msub> <mml:mi>a</mml:mi> <mml:mi>n</mml:mi> </mml:msub> <mml:mo>)</mml:mo> </mml:mrow> <mml:mrow> <mml:mi>n</mml:mi> <mml:mo>≥</mml:mo> <mml:mn>1</mml:mn> </mml:mrow> </mml:msub> </mml:math> is a sequence of reals rather than integers, thereby pursuing a line of research which was recently initiated by Rudnick and Technau. As an application of our method, we prove that for every real number $$\theta &gt;1$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mi>θ</mml:mi> <mml:mo>&gt;</mml:mo> <mml:mn>1</mml:mn> </mml:mrow> </mml:math> , the sequence $$(n^\theta \alpha )_{n \ge 1}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:msub> <mml:mrow> <mml:mo>(</mml:mo> <mml:msup> <mml:mi>n</mml:mi> <mml:mi>θ</mml:mi> </mml:msup> <mml:mi>α</mml:mi> <mml:mo>)</mml:mo> </mml:mrow> <mml:mrow> <mml:mi>n</mml:mi> <mml:mo>≥</mml:mo> <mml:mn>1</mml:mn> </mml:mrow> </mml:msub> </mml:math> has Poissonian pair correlation for almost all $$\alpha \in {\mathbb {R}}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mi>α</mml:mi> <mml:mo>∈</mml:mo> <mml:mi>R</mml:mi> </mml:mrow> </mml:math> .
Digital Geometry on the Dual of Some Semi-regular Tessellations

There are various tessellations of the plane. There are three regular ones, each of them using a sole regular tile. The square grid is self-dual, and the two others, the hexagonal and triangular grids are duals of each other. There are eight semi-regular tessellations, they are based on more than one type of tiles. In this paper, we are interested to their dual tessellations. We show a general method to obtain coordinate system to address the tiles of these tessellations. The properties of the coordinate systems used to address the tiles are playing crucial roles. For some of those grids, including the tetrille tiling D(6, 4, 3, 4) (also called deltoidal trihexagonal tiling and it is the dual of the rhombihexadeltille, T(6, 4, 3, 4) tiling), the rhombille tiling, D(6, 3, 6, 3) (that is the dual of the hexadeltille T(6, 3, 6, 3), also known as trihexagonal tiling) and the kisquadrille tiling D(8, 8, 4) (it is also called tetrakis square tiling and it is the dual of the truncated quadrille tiling T(8, 8, 4) which is also known as Khalimsky grid) we give detailed descriptions. Moreover, we are also presenting formulae to compute the digital, i.e., path-based distance based on the length of a/the shortest path(s) through neighbor tiles for these specific grids.
High-Frequency Adversarial Defense for Speech and Audio

Recent work suggests that adversarial examples are enabled by high-frequency components in the dataset. In the speech domain where spectrograms are used extensively, masking those components seems like a sound direction for defenses against attacks. We explore a smoothing approach based on additive noise masking in priority high frequencies. We show that this approach is much more robust than the naive noise filtering approach, and a promising research direction. We successfully apply our defense on a Librispeech speaker identification task, and on the UrbanSound8K audio classification dataset.
Semantic adaptation network for unsupervised domain adaptation

Domain adaptation methods aim at learning transferable model for the unlabeled target domain. Recently, extensive domain adaptation models are proposed to align representations by minimizing distribution distance between different domains or adversarial training manner. However, most existing methods only adapt the different features in feature space and ignore the semantic features that generated by the classifier, which can lead to misclassification of target samples near the source decision boundaries. In this paper, we propose a Semantic Adaptation Network (SAN) for unsupervised domain adaptation. SAN aligns the domain representations at both feature-level and semantic-level. SAN utilizes adversarial training to align different domain representations in the feature space. In the semantic space, SAN obtains the pseudo labels of target samples by nearest source semantic representation centroid, then forces the pseudo labeled target semantic representations close to corresponding source semantic representation centroids. Experiments on ImageCLEF-DA, Office-Home and VisDA datasets validate the effectiveness and superiority of our model.
Human Face Recognition and Temperature Measurement Based on Deep Learning for Covid-19 Quarantine Checkpoint

The human temperature measurement system has been widely applying in hospitals and public areas during the widespread Covid-19 pandemic. However, the current systems in the quarantine checkpoint are only capable of measuring the human temperature; however, it can not combine with the identification of facial recognition, human temperature information, and wearing mask detection. In addition, in the hospitals as well as the public areas such as schools, libraries, train stations, airports, etc. facial recognition of employees combined with temperature measurement and masking will save the time check and update employee status immediately. This study proposes a method that combines body temperature measurement, facial recognition, and masking based on deep learning. Furthermore, the proposed method adds the ability to prevent spoofing between a real face and face-in-image recognition. A depth camera is used in the proposed system to measure and calculate the length between the human's face and camera to approach the best accuracy of facial recognition and anti-spoofing. Moreover, a low-cost thermal camera measures the human body temperature. The methodology and algorithm for the human face and body temperature recognition are validated through the experimental results.
Graph Signal Denoising Using Nested-Structured Deep Algorithm Unrolling

In this paper, we propose a deep algorithm unrolling (DAU) based on a variant of the alternating direction method of multiplier (ADMM) called Plug-and-Play ADMM (PnP-ADMM) for denoising of signals on graphs. DAU is a trainable deep architecture realized by unrolling iterations of an existing optimization algorithm which contains trainable parameters at each layer. We also propose a nested-structured DAU: Its submodules in the unrolled iterations are also designed by DAU. Several experiments for graph signal denoising are performed on synthetic signals on a community graph and U.S. temperature data to validate the proposed approach. Our proposed method outperforms alternative optimization- and deep learning-based approaches.
Joint Intent Detection and Slot Filling Based on Continual Learning Model

Slot filling and intent detection have become a significant theme in the field of natural language understanding. Even though slot filling is intensively associated with intent detection, the characteristics of the information required for both tasks are different while most of those approaches may not fully aware of this problem. In addition, balancing the accuracy of two tasks effectively is an inevitable problem for the joint learning model. In this paper, a Continual Learning Interrelated Model (CLIM) is proposed to consider semantic information with different characteristics and balance the accuracy between intent detection and slot filling effectively. The experimental results show that CLIM achieves state-of-the-art performace on slot filling and intent detection on ATIS and Snips.
Palm Vein Recognition Based on NPE and KELM

Palm veins have become a research hotspot of biometric recognition due to their own advantages of universality, uniqueness, collectability and stability. This paper proposes a palm vein recognition algorithm based on Neighborhood Preserving Embedding (NPE) and Kernel Extreme Learning Machine (KELM). The algorithm firstly performs gray-scale normalization processing on vein images, then extracts neighborhood preserving embedding dimensionality reduction features, and finally uses extreme learning machine for classification and recognition. The method is tested on the multispectral palmprint database of Hong Kong Polytechnic University. The experimental results show that this method can effectively reduce the vein dimensions to less than 30, and achieve an ideal recognition effect, if the parameters are selected appropriately. The algorithm is also verified on the palmvein database of Tongji University and FYO palmvein database for verifying the robustness, and also get ideal experimental results.
An FCNN-Based Super-Resolution Mmwave Radar Framework for Contactless Musical Instrument Interface

In this article, we propose a framework for contactless human-computer interaction (HCI) using novel tracking techniques based on deep learning-based super-resolution and tracking algorithms. Our system offers unprecedented high-resolution tracking of hand position and motion characteristics by leveraging spatial and temporal features embedded in the reflected radar waveform. Rather than classifying samples from a predefined set of hand gestures, as common in existing work on deep learning with mmWave radar, our proposed imager employs a regressive full convolutional neural network (FCNN) approach to improve localization accuracy by spatial super-resolution. While the proposed techniques are suitable for a host of tracking applications, this article focuses on their application as a musical interface to demonstrate the robustness of the gesture sensing pipeline and deep learning signal processing chain. The user can control the instrument by varying the position and velocity of their hand above the vertically-facing sensor. By employing a commercially available multiple-input-multiple-output (MIMO) radar rather than a traditional optical sensor, our framework demonstrates the efficacy of the mmWave sensing modality for fine motion tracking and offers an elegant solution to a host of HCI tasks. Additionally, we provide a freely available software package and user interface for controlling the device, streaming the data to MATLAB in real-time, and increasing accessibility to the signal processing and device interface functionality utilized in this article.
SILT: Efficient transformer training for inter-lingual inference

The ability of transformers to perform precision tasks such as question answering, Natural Language Inference (NLI) or summarizing, has enabled them to be ranked as one of the best paradigms to address Natural Language Processing (NLP) tasks. NLI is one of the best scenarios to test these architectures, due to the knowledge required to understand complex sentences and established relationships between a hypothesis and a premise. Nevertheless, these models suffer from the incapacity to generalize to other domains or from difficulties to face multilingual and interlingual scenarios. The leading pathway in the literature to address these issues involve designing and training extremely large architectures, but this causes unpredictable behaviors and establishes barriers which impede broad access and fine tuning. In this paper, we propose a new architecture called Siamese Inter-Lingual Transformer (SILT). This architecture is able to efficiently align multilingual embeddings for Natural Language Inference, allowing for unmatched language pairs to be processed. SILT leverages siamese pre-trained multi-lingual transformers with frozen weights where the two input sentences attend to each other to later be combined through a matrix alignment method. The experimental results carried out in this paper evidence that SILT allows to reduce drastically the number of trainable parameters while allowing for inter-lingual NLI and achieving state-of-the-art performance on common benchmarks.
Exploring a Makeup Support System for Transgender Passing based on Automatic Gender Recognition

How to handle gender with machine learning is a controversial topic. A growing critical body of research brought attention to the numerous issues transgender communities face with the adoption of current automatic gender recognition (AGR) systems. In contrast, we explore how such technologies could potentially be appropriated to support transgender practices and needs, especially in non-Western contexts like Japan. We designed a virtual makeup probe to assist transgender individuals with passing, that is to be perceived as the gender they identify as. To understand how such an application might support expressing transgender individuals gender identity or not, we interviewed 15 individuals in Tokyo and found that in the right context and under strict conditions, AGR based systems could assist transgender passing.
Feature Selection on Elite Hybrid Binary Cuckoo Search in Binary Label Classification

For the low optimization accuracy of the cuckoo search algorithm, a new search algorithm, the Elite Hybrid Binary Cuckoo Search (EHBCS) algorithm, is improved by feature weighting and elite strategy. The EHBCS algorithm has been designed for feature selection on a series of binary classification datasets, including low-dimensional and high-dimensional samples by SVM classifier. The experimental results show that the EHBCS algorithm achieves better classification performances compared with binary genetic algorithm and binary particle swarm optimization algorithm. Besides, we explain its superiority in terms of standard deviation, sensitivity, specificity, precision, and <math xmlns="http://www.w3.org/1998/Math/MathML" id="M1"> <mi>F</mi> </math> -measure.
Pipeline Safety Early Warning Method for Distributed Signal using Bilinear CNN and LightGBM

Oil and gas pipelines are known as the backbone of global energy, and securing their safety is crucial for energy supply. In this study, we utilized a novel machine learning method based on the spatiotemporal features of distributed optical fiber sensor signals to monitor the safety of oil and gas pipelines in real time. Encouraging empirical results on a large amount of data collected from real sites confirmed that our model could accurately locate and identify the damage events of a pipeline in real time under strong noise and various hardware conditions, and could effectively handle the signal drift problem. Furthermore, as a generalized tool, the proposed solution could be applied to other industrial inspection fields. Our codes and video demos are available at https://github.com/yyysjz1997/B-CNN_LGBM-PSEW.
Indoor 3D Human Trajectory Reconstruction Using Surveillance Camera Videos and Point Clouds

3D human trajectory reconstruction in an indoor scene is critical in various applications, such as indoor navigation and human activity recognition. This task is challenging due to occlusion and clutters of indoor scenes, flexible human body joints, and severe lack of relevant datasets. Although several methods have been proposed to reconstruct a 3D human trajectory, they either can recover only 2D positions or require human initiative cooperation. In this paper, we propose a novel framework for 3D human trajectory reconstruction in an indoor scene using monocular surveillance videos and static point clouds without any initiative cooperation. The proposed framework consists of three modules: 3D pose estimation, depth regression, and trajectory reconstruction. We first estimate 3D pose from videos. Especially, we reconstruct a half-body 3D pose to deal with the occlusion problem. Then, we propose a depth regression approach to iteratively regress the depth of a 3D pose. Unlike data-driven approaches, our depth regression approach does not require training data and can be integrated into any 3D pose model. Finally, we exploit the geometric constraints from the point cloud to optimize the 3D trajectory. We evaluated the 3D pose estimation and depth regression modules on the H3.6M datasets. Due to the lack of evaluation datasets, we also built a trajectory dataset to evaluate the trajectory reconstruction performance. Empirical evaluation shows that our framework achieves accurate trajectory reconstruction results on real-world videos.
An Intelligent Medical Imaging Approach for Various Blood Structure Classifications

Blood is a vital body fluid and can be instrumental in identifying various pathological conditions. Nowadays, a lot of people are suffering from COVID-19 and every country has its own limited testing capacity. Consequently, a system is required to help doctors analyze a patient’s blood structure including COVID-19. Therefore, in this paper, we extracted and selected blood features by proposing a new feature extraction and selection method named stepwise linear discriminant analysis (SWLDA). SWLDA emphasizes on picking confined features from blood structure images and discerning its class based on reversion value such as partial F value. SWLDA begins with picking an equivalence comprising the sole finest X variable and then puts in effort to add more Xs individually, providing the situations are adequate. The process of adding and picking is based on F value to determine which variable would be entered. Then, the picked or the default F-to-enter value is compared with the uppermost partial F value. After this step, the forward addition or backward removal begins and whether the partial test values for all the predictor variables already in the line are estimated is known. Then, the comparison is made between the lowermost partial test value (FL) and preselected or defaulting consequence levels such as F0 (i.e., if F0 &gt; FL, the variable ZL is removed, and the F test is started again; otherwise, the regression equation is adopted). Finally, the system is trained by employing support vector machine (SVM) to label the blood images. The performance of the proposed approach is assessed by employing 8 different datasets of blood structures. It is assured that the proposed method has achieved significant results under different blood structure images including COVID-19.
Deep Iterative Residual Convolutional Network for Single Image Super-Resolution

Deep convolutional neural networks (CNNs) have recently achieved great success for single image super-resolution (SISR) task due to their powerful feature representation capabilities. The most recent deep learning based SISR methods focus on designing deeper / wider models to learn the non-linear mapping between low-resolution (LR) inputs and high-resolution (HR) outputs. These existing SR methods do not take into account the image observation (physical) model and thus require a large number of network's trainable parameters with a great volume of training data. To address these issues, we propose a deep Iterative Super-Resolution Residual Convolutional Network (ISRResCNet) that exploits the powerful image regularization and large-scale optimization techniques by training the deep network in an iterative manner with a residual learning approach. Extensive experimental results on various super-resolution benchmarks demonstrate that our method with a few trainable parameters improves the results for different scaling factors in comparison with the state-of-art methods.