Reviewerly/exHarmony huggingface dataset:
POLO: Learning Ex<u>p</u>licit Cr<u>o</u>ss-Moda<u>l</u>ity Fusi<u>o</u>n for Temporal Action Localization

Temporal action localization aims at discovering action instances in untrimmed videos, where RGB and flow are two widely used feature modalities. Specifically, RGB chiefly reveals appearance and flow mainly depicts motion. Given RGB and flow features, previous methods employ the early fusion or late fusion paradigm to mine the complementarity between them. By concatenating raw RGB and flow features, the early fusion implicitly achieved complementarity by the network, but it partly discards the particularity of each modality. The late fusion independently maintains two branches to explore the particularity of each modality, but it only fuses the localization results, which is insufficient to mine the complementarity. In this work, we propose ex <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</u> licit cr <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">o</u> ss-moda <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">l</u> ity fusi <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">o</u> n (POLO) to effectively utilize the complementarity between two modalities and thoroughly explore the particularity of each modality. POLO performs cross-modality fusion via estimating the attention weight from RGB modality and employing it to flow modality (vice versa). This bridges the complementarity of one modality to supply the other. Assisted with the attention weight, POLO independently learns from RGB and flow features and explores the particularity of each modality. Extensive experiments on two benchmarks demonstrate the preferable performance of POLO.
Online Handwritten Mathematical Expression Recognition and Applications: A Survey

Handwritten mathematical expressions are an essential part of many domains, including education, engineering, and science. The pervasive availability of computationally powerful touch-screen devices, similar to the recent emergence of deep neural networks as high-quality sequence recognition models, result in the widespread adoption of online recognition of handwritten mathematical expressions. Also, a deeper study and improvement of such technologies is necessary to address the current challenges posed by the extensive usage of distance learning, and remote work due to the world pandemic. This paper delineates the state-of-the-art recognition methods along with the user's experience in pen-centric applications for operating with handwritten mathematical expressions. Recognition methods have been categorized into classes, with a description of their merits and limitations. Particular attention is paid to end-to-end approaches based on encoder-decoder architecture and multi-modal input. Evaluation protocols and open benchmark datasets are considered as well as the comparison of the recognition performance, based on open competition results. The use of handwritten math recognition is illustrated by examples of applications for various fields and platforms. A distinctive part of the survey is that we also considered how UI design relies on the use of different recognition approaches, which is aimed at helping potential researchers improve the performance of the introduced approaches toward the best responses in practical applications. Finally, this paper presents the prospective survey of future research directions in handwritten mathematical expression recognition and their applications.
Visibility-aware Trajectory Optimization with Application to Aerial Tracking

The visibility of targets determines performance and even success rate of various applications, such as active slam, exploration, and target tracking. Therefore, it is crucial to take the visibility of targets into explicit account in trajectory planning. In this paper, we propose a general metric for target visibility, considering observation distance and angle as well as occlusion effect. We formulate this metric into a differentiable visibility cost function, with which spatial trajectory and yaw can be jointly optimized. Furthermore, this visibility-aware trajectory optimization handles dynamic feasibility of position and yaw simultaneously. To validate that our method is practical and generic, we integrate it into a customized quadrotor tracking system. The experimental results show that our visibility-aware planner performs more robustly and observes targets better. In order to benefit related researches, we release our code to the public.
Physical Zero-Knowledge Proof for Ripple Effect

Ripple Effect is a logic puzzle with an objective to fill numbers into a rectangular grid divided into rooms. Each room must contain consecutive integers starting from 1 to its size. Also, if two cells in the same row or column have the same number x, the space separating the two cells must be at least x cells. In this paper, we propose a physical protocol of zero-knowledge proof for Ripple Effect puzzle using a deck of cards, which allows a prover to physically show that he/she knows a solution without revealing it. In particular, we develop a physical protocol that, given a secret number x and a list of numbers, verifies that x does not appear among the first x numbers in the list without revealing x or any number in the list.
Multi-scale skeleton adaptive weighted GCN for skeleton-based human action recognition in IoT

Skeleton-based human action recognition has become a hot topic due to its potential advantages. Graph convolution network (GCN) has obtained remarkable performances in the modeling of skeleton-based human action recognition in IoT. In order to capture robust spatial–temporal features from the human skeleton, a powerful feature extractor is essential. However, Most GCN-based methods use the fixed graph topology. Besides, only a single-scale feature is used, and the multi-scale information is ignored. In this paper, we propose a multi-scale skeleton adaptive weighted graph convolution network (MS-AWGCN) for skeleton-based action recognition. Specifically, a multi-scale skeleton graph convolution network is adopted to extract more abundant spatial features of skeletons. Moreover, we develop a simple graph vertex fusion strategy, which can learn the latent graph topology adaptively by replacing the handcrafted adjacency matrix with a learnable matrix. According to different sampling strategies, weighted learning method is adopted to enrich features while aggregating. Experiments on three large datasets illustrate that the proposed method achieves comparable performances to state-of-the-art methods. Our proposed method attains an improvement of 0.9% and 0.7% respectively over the recent GCN-based method on the NTU RGB+D and Kinetics dataset.
Enhancing Graph Neural Networks via auxiliary training for semi-supervised node classification

Graph Neural Networks (GNNs) have been successfully applied to graph analysis tasks. As a canonical task in graph analysis, node classification has achieved promising results with GNNs. However, training GNNs remains challenging when training samples are limited. For the node classification task, only the known node labels are used as supervision information to train GNNs, while the implicit information of pairwise relationships between nodes is neglected. The label-to-label relationship of node pairs is explicit in labeled nodes and not utilized. Moreover, it is found that the link relationship in a graph is correlated with the labels between nodes in some graphs, like information network. Based on that, we can extract and establish labels for pairwise nodes. With the supervision of labeled pairwise nodes, it can force the predicted labels to conform to the observed pairwise relationships and provide some useful information to boost performance. In view of this, we utilize pairwise relationships by introducing the auxiliary task, node pair classification or link prediction, and propose a novel training framework for enhancing GNNs, namely EGNN. Via jointly training node classification with the auxiliary task, our framework achieves higher classification accuracy for general GNNs models with only a little extra computational cost. Moreover, the adaptive dynamic weighting strategy is designed to balance the training pace of different tasks automatically. We conduct extensive experiments, and the evaluation results suggest the superiority of our framework.
Adaptive denoising algorithm using peak statistics-based thresholding and novel adaptive complementary ensemble empirical mode decomposition

This paper proposes an adaptive denoising methodology for noisy signals that employs a novel adaptive complementary ensemble empirical mode decomposition (NACEEMD) and a peak statistics (PS)-based thresholding technique. The key idea in this paper is the peak statistics (PS)-based thresholding technique,which breaks the traditional strategy with respect to selecting more accurate and more adaptive thresholds. The NACEEMD algorithm is proposed to decompose the noisy signal into a series of intrinsic mode functions (IMFs). At the same time, NACEEMD is also used to verify the applicability of the PS-based thresholding technique in different decomposition algorithms. The PS-based threshold is used to remove the noise inherent in noise-dominant IMFs, and the denoised signal is reconstructed by combining the denoised noise-dominant IMFs and the signal-dominant IMFs. This paper uses a various of simulated signals in various noisy environments for experiments, the experimental results indicate that the proposed algorithm outperforms traditional threshold denoising methodologies in terms of signal-to-noise ratio, root mean square error, and percent root distortion. Moreover, through real ECG signal and multi-sensor data fusion experiments, the application of the proposed algorithm in the field of engineering is explored and expanded.
MetaMixUp: Learning Adaptive Interpolation Policy of MixUp With Metalearning

MixUp is an effective data augmentation method to regularize deep neural networks via random linear interpolations between pairs of samples and their labels. It plays an important role in model regularization, semisupervised learning (SSL), and domain adaption. However, despite its empirical success, its deficiency of randomly mixing samples has poorly been studied. Since deep networks are capable of memorizing the entire data set, the corrupted samples generated by vanilla MixUp with a badly chosen interpolation policy will degrade the performance of networks. To overcome overfitting to corrupted samples, inspired by metalearning (learning to learn), we propose a novel technique of learning to a mixup in this work, namely, MetaMixUp. Unlike the vanilla MixUp that samples interpolation policy from a predefined distribution, this article introduces a metalearning-based online optimization approach to dynamically learn the interpolation policy in a data-adaptive way (learning to learn better). The validation set performance via metalearning captures the noisy degree, which provides optimal directions for interpolation policy learning. Furthermore, we adapt our method for pseudolabel-based SSL along with a refined pseudolabeling strategy. In our experiments, our method achieves better performance than vanilla MixUp and its variants under SL configuration. In particular, extensive experiments show that our MetaMixUp adapted SSL greatly outperforms MixUp and many state-of-the-art methods on CIFAR-10 and SVHN benchmarks under the SSL configuration.
Bangla Documents Classification using Transformer Based Deep Learning Models

Document classification or categorization assign documents to a predefined domain category. The improvement of document classification techniques has been noticeable worldwide recently. Many transformer-based models have been introduced for different languages, which shows significant improvement in this area of Natural Language Processing. In this paper, we have classified Bangla text documents with the most recent transformer or attention mechanism-based models. We have applied the BERT (Bidirectional Encoder Representations from Transformers) and ELECTRA (Efficiently Learning an Encoder that Classifies Token Replacements Accurately) model for Bangla text classification. Both of them are pre-training text encoders and we have applied fine-tuning approach for the downstream(classification) task. Here, we have used three different Bangla text datasets for our experiment. Both of the models provide outstanding performance for two out of three datasets we have used.
A practical perspective on knowledge space theory: ALEKS and its data

The ALEKS (Assessment and LEarning in Knowledge Spaces) educational software system is an instantiation of knowledge space theory (KST) that has been used by millions of students in mathematics, chemistry, statistics and accounting. The software employs a probabilistic assessment based on KST for placement into an appropriate course or curriculum, a learning mode in which students are guided through course material according to a knowledge structure, and regularly spaced re-assessments which are also based on KST. In each of these aspects, the interactions of the student with the system are guided by the theory and by insights learned from student data. We present several relationships between theory and data for the ALEKS system. We begin by surveying the ALEKS system and examining some practical aspects of implementing KST on a large scale. We then study the effectiveness of the ALEKS assessment using both standard statistical measures and ones adapted to the KST context. Finally, we examine the learning process in ALEKS via statistics for the learning mode and its associated knowledge structures.
Feature Aggregation Networks Based on Dual Attention Capsules for Visual Object Tracking

Tracking-by-detection algorithms have considerably enhanced tracking performance with the introduction of recent convolutional neural networks (CNNs). However, most trackers directly exploit standard scalar-output CNN features, which may not capture enough feature encoding information, instead of aggregated CNN features of vector-output form. In this paper, we propose an end-to-end feature aggregation capsule framework. First, based on the existing CNN network, we aggregate a certain number of similar position-aware CNN features into a capsule to model the feature similarity. The acquired vector-level feature capsules (rather than previous scalar-level pointwise features) are utilized for differentiation learning. We then propose a group attention module to better model the entity representation between different capsule groups thus optimizes total discriminative capability. Third, to reduce the prediction interference resulted by the side effect of dimension rising within capsules, we propose a penalty attention module. Such strategy could dynamically adjust values of neurons by estimating whether they are beneficial or harmful to tracking. Experimental results on five representative benchmarks (UAVDT, DTB70, UAV123, VOT2016 and VOT2018) demonstrate the excellent tracking performance of our dual attention based capsule tracker (DACapT). Specially, it exceeds the previous top tracker by 4.6%/1.9% in precision/success evaluations on UAVDT.
On Permutation Quadrinomials and 4-Uniform BCT

Extending previous results, we study a class of general quadrinomials over the field of size 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2m</sup> with odd m and characterize conditions under which they are permutations with 4-uniform BCT, a new and important parameter related to boomerang-style attacks. These permutations are known to have the best known nonlinearity. Numerical data also show that the inverse of these functions all have large algebraic degree, making them desirable for applications.
Interaction principles for digital puppeteering to promote teacher learning

This literature review analyzed 18 studies published between 2013 and 2020 to understand the role of virtual humans in mixed reality digital puppeteering simulations aimed at promoting teacher learning. Research questions explored: simulation design models, interactor training, and the attention researchers gave to language and race in the virtual classroom. The descriptive and logistical findings from this review resulted in three learning design models and five design principles that may guide the future use of digital puppeteering in mixed-reality virtual classrooms to promote teacher learning. Across studies, findings suggested promising results of teacher learning through mixed reality simulations and illuminated possible areas for future research. These included the documentation of interactor preparation and detailed evaluation of the interactions between teachers and avatars in virtual classrooms.
Accent and Speaker Disentanglement in Many-to-many Voice Conversion

This paper proposes an interesting voice and accent joint conversion approach, which can convert an arbitrary source speaker’s voice to a target speaker with non-native accent. This problem is challenging as each target speaker only has training data in native accent and we need to disentangle accent and speaker information in the conversion model training and re-combine them in the conversion stage. In our recognition-synthesis conversion framework, we manage to solve this problem by two proposed tricks. First, we use accent-dependent speech recognizers to obtain bottleneck features for different accented speakers. This aims to wipe out other factors beyond the linguistic information in the BN features for conversion model training. Second, we propose to use adversarial training to better disentangle the speaker and accent information in our encoder-decoder based conversion model. Specifically, we plug an auxiliary speaker classifier to the encoder, trained with an adversarial loss to wipe out speaker information from the encoder output. Experiments show that our approach is superior to the baseline. The proposed tricks are quite effective in improving accentedness and audio quality and speaker similarity are well maintained.
Evaluating the benefits of a computer-aided software engineering tool to develop and document product configuration systems

Computer-Aided Software Engineering (CASE) tools are popular software programs to support the members of the development team (including analysts, designers, coders, database administrators, and project managers) in building new software systems. Up-to-date and consistent knowledge representation and documentation is crucial for companies developing Product Configuration Systems (PCSs). The literature reports various challenges in PCS development, such as maintenance, documentation, knowledge management, resource and time management, system quality, and communication with domain experts as particularly problematic. A CASE tool tailored to the specific needs of PCS development can prove to be useful in tackling at least some of these challenges. Such a CASE tool has to support product models, which means it has to not only allow the representation of the product core architecture and the optional selectable features, but also ensure consistency between representations (views) and deliver forward or reverse engineering. This enables support and automates, at least partially, the development in general and the implementation stage. The focus and main contribution of this paper is twofold. First, we describe the view-based approach required to fully conceptualise the knowledge to generate PCS software from the CASE tool. To this end, the tool indeed includes four different views to build or edit all the required knowledge. Second, we validate this CASE tool within two case companies, wherein we evaluate its application on a project each time it is used. The results show that the use of the CASE tool increases the quality of PCS documentation and saves time and resources while also improving the PCS’s overall quality.
Privacy-Preserving Streaming Truth Discovery in Crowdsourcing With Differential Privacy

Differential privacy (DP) has gained popularity in truth discovery recently due to its strong privacy guarantee. However, existing DP mechanisms for streaming data publication are not suitable for truth discovery as they fail to consider the different reliabilities of individuals, while the DP-based approaches for truth discovery are not suitable for streaming data because they ignore the correlations between truths over time. Directly applying these existing methods to streaming crowdsourced data would lead to low accuracy of the discovered truth. To solve this problem, in this paper, we propose an edge computing based privacy-preserving truth discovery mechanism, named PrivSTD, for streaming crowdsourced data to realize high accuracy of discovered truth while protecting the privacy of workers. Specifically, edge servers are introduced between the untrusted cloud server and workers to securely calculate the local truths and workers’ reliabilities. A truth-dependent budget recycle mechanism is proposed for each edge server to adaptively determine the perturbed timestamp and allocate the privacy budget according to the changing pattern of local truths. Besides, a reliability-based perturbation mechanism is proposed to reduce the perturbation magnitude on the basis of worker's reliability. We theoretical analyze the data utility and computation cost of PrivSTD, and prove that PrivSTD can satisfy <inline-formula><tex-math notation="LaTeX">$w$</tex-math></inline-formula> -event ( <inline-formula><tex-math notation="LaTeX">$\epsilon,\delta$</tex-math></inline-formula> )-differential privacy. Extensive experimental results on synthetic and real-world datasets demonstrate that PrivSTD achieves better utility than the state-of-the-art approaches.
A Novel Robust Nonlinear Kalman Filter Based on Multivariate Laplace Distribution

In this brief, we investigate the robust state estimation of nonlinear systems with heavy-tailed noise. Based on the fact that the multivariate Laplace (ML) distribution has heavier tails than Gaussian distribution but is only determined by its mean and covariance, we utilize the ML distribution to model the heavy-tailed measurement noise. Based on the Gaussian mixture form of ML distribution, the system state and the covariance of ML distribution are inferred by variational Bayesian method, and then a closed recursive robust nonlinear Kalman filter is obtained with the nonlinear integrations calculated by numerical integration methods. Simulation results demonstrate that the proposed algorithm is insensitive to its free parameters and can obtain better estimation accuracy than related robust algorithms.