Reviewerly/exHarmony huggingface dataset:
Capturing Small, Fast-Moving Objects: Frame Interpolation via Recurrent Motion Enhancement

Interpolating video frames involving large motions remains an elusive challenge. In case that frames involve small and fast-moving objects, conventional feed-forward neural network-based approaches that estimate optical flow and synthesize in-between frames sequentially often result in loss of motion features and thus blurred boundaries. To address the problem, we propose a novel Recurrent Motion-Enhanced Interpolation Network (ReMEI-Net) by assigning attention to the motion features of small objects from both the intra-scale and inter-scale perspectives. Specifically, we add recurrent feedback blocks in the existing multi-scale autoencoder pipeline, aiming to iteratively enhance the motion information of small objects across different scales. Second, to further refine the motion features of the highly moving objects, we propose a Multi-Directional ConvLSTM (MD-ConvLSTM) block to capture the global spatial contextual information of motion from multiple directions. In this way, the coarse-scale features can be utilized to correct and enhance the fine-scale features through the feedback mechanism. Extensive experiments on various datasets demonstrate the superiority of our proposed method over state-of-the-art approaches in terms of clear locations and complete shape.
Synergistic Integration Between Machine Learning and Agent-Based Modeling: A Multidisciplinary Review

Agent-based modeling (ABM) involves developing models in which agents make adaptive decisions in a changing environment. Machine-learning (ML) based inference models can improve sequential decision-making by learning agents' behavioral patterns. With the aid of ML, this emerging area can extend traditional agent-based schemes that hardcode agents' behavioral rules into an adaptive model. Even though there are plenty of studies that apply ML in ABMs, the generalized applicable scenarios, frameworks, and procedures for implementations are not well addressed. In this article, we provide a comprehensive review of applying ML in ABM based on four major scenarios, i.e., microagent-level situational awareness learning, microagent-level behavior intervention, macro-ABM-level emulator, and sequential decision-making. For these four scenarios, the related algorithms, frameworks, procedures of implementations, and multidisciplinary applications are thoroughly investigated. We also discuss how ML can improve prediction in ABMs by trading off the variance and bias and how ML can improve the sequential decision-making of microagent and macrolevel policymakers via a mechanism of reinforced behavioral intervention. At the end of this article, future perspectives of applying ML in ABMs are discussed with respect to data acquisition and quality issues, the possible solution of solving the convergence problem of reinforcement learning, interpretable ML applications, and bounded rationality of ABM.
KDV-explorer

Kernel density visualization (KDV) is a commonly used visualization tool for many spatial analysis tasks, including disease outbreak detection, crime hotspot detection, and traffic accident hotspot detection. Although the most popular geographical information systems, e.g., QGIS, and ArcGIS, can also support this operation, these solutions are not scalable to generate a single KDV for datasets with million-scale data points, let alone to support exploratory operations (e.g., zoom in, zoom out, and panning operations) with KDV in near real-time (&lt; 5 sec). In this demonstration, we develop a near real-time visualization system, called KDV-Explorer, that is built on top of our prior study on the efficient kernel density computation. Participants will be invited to conduct some kernel density analysis on three large-scale datasets (up to 1.3 million data points), including the traffic accident dataset, crime dataset and COVID-19 dataset. We will also compare the performance of our solution and the solutions in QGIS and ArcGIS.
An Encoder-Decoder Approach to Handwritten Mathematical Expression Recognition with Multi-head Attention and Stacked Decoder

Encoder-decoder framework with attention mechanism has become a mainstream solution to handwritten mathematical expression recognition (HMER) since "watch, attend and parse (WAP)" approach was proposed in 2017, where a convolutional neural network is used as encoder and a gated recurrent unit with attention is used in decoder. Inspired by the recent success of Transformer in many applications, in this paper, we adopt the design of multi-head attention and stacked decoder in Transformer to improve the decoder part of the WAP framework for HMER. Experimental results on CROHME tasks show that multi-head attention can boost the expression recognition rate (ExpRate) of WAP from 54.32%/58.05% to 56.76%/59.72% and stacked decoder can further improve ExpRate to 57.72%/61.38% on CROHME 2016/2019 test sets.
GRAIN

Data selection methods, such as active learning and core-set selection, are useful tools for improving the data efficiency of deep learning models on large-scale datasets. However, recent deep learning models have moved forward from independent and identically distributed data to graph-structured data, such as social networks, e-commerce user-item graphs, and knowledge graphs. This evolution has led to the emergence of Graph Neural Networks (GNNs) that go beyond the models existing data selection methods are designed for. Therefore, we present GRAIN, an efficient framework that opens up a new perspective through connecting data selection in GNNs with social influence maximization. By exploiting the common patterns of GNNs, GRAIN introduces a novel feature propagation concept, a diversified influence maximization objective with novel influence and diversity functions, and a greedy algorithm with an approximation guarantee into a unified framework. Empirical studies on public datasets demonstrate that GRAIN significantly improves both the performance and efficiency of data selection (including active learning and core-set selection) for GNNs. To the best of our knowledge, this is the first attempt to bridge two largely parallel threads of research, data selection, and social influence maximization, in the setting of GNNs, paving new ways for improving data efficiency.
Diverse Receptive Field Based Adversarial Concurrent Encoder Network for Image Inpainting

Image inpainting is nowadays demanding because of its wide applications such as removing the unwanted objects from the image or recovering the old corrupted photo. Existing approaches achieved superior performance with coarse-to-fine or progressive or recurrent architectures for image inpainting regardless of computational complexity. In these types, the disturbance at the first instance or first iteration may lead to semantically unambiguous results. Also, to inpaint the image with varying hole sizes it is desirable to focus on the diverse receptive fields without deeper network i.e, network with less number of parameters. Therefore, we have proposed a lightweight adversarial concurrent encoder architecture with a diverse receptive field for image inpainting. Here, the concurrent encoder is integrated with diverse receptive fields to benefit with lower computational complexity. The proposed method is compared with state-of-the-art (SOTA) methods on Places2 and Paris Street View dataset in terms of peak signal-to-noise ratio and structural similarity index. Along with the extensive results analysis and ablation study, the proposed method proves the effectiveness in terms of less computational complexity compared to existing SOTA methods.
Concept Drift Analysis by Dynamic Residual Projection for Effectively Detecting Botnet Cyber-Attacks in IoT Scenarios

IoT devices typically stream data such as sensor values to other devices including cloud-based services. Analyzing these streams for cyber-attacks is a challenging task. This is due to the infinite nature of stream-based datatypes. Analyzing streams can require additional real-time processing and computational performance capabilities. In this article, we focus on how concept drifts affect Botnet cyber-attack detection in IoT scenarios. To reveal the result, we incorporate the concept drift analysis to detect cyber-attacks on the Bot-IoT dataset, which consists of legitimate and simulated IoT network traffics, together with various types of attacks. We designed subdatasets of the Bot-IoT to ensure the concept drift occurs that eventually complete the experiments. The detection accuracies improved 15%–26% compared with the classification models without concept drift analysis. We also gain superior performance results by comparing confusion matrices when concept drift analysis is ongoing. We propose a technique featuring a dynamic sliding window based on the residual projection to perform concept drift analysis. During the process of finding concepts in data streams, the sample number is updated dynamically by comparing the anomalous quantity obtained by the residual projection method in the current window to the previous one. In addition to the Bot-IoT dataset, our method is also applied to two popular synthetic datasets SEA Concept and UG-2C-5D. The results demonstrate the effectiveness of our method with respect to the false alarm rate, misses, and average delay.
DSA-Face: Diverse and Sparse Attentions for Face Recognition Robust to Pose Variation and Occlusion

Learning local representations is important for face recognition (FR). Recent attention-based networks emphasize few facial parts, while ignoring other potentially discriminative ones. This is more serious when there are large pose variations, occlusions (e.g. face masks), or other image quality changes. To address this, we propose Diverse and Sparse Attentions, called DSA-Face. First, a divergence loss is designed to explicitly encourage the diversity among multiple attention maps by maximizing the Euclidean distance between every pair attention maps. As a result, a Pairwise Self-Contrastive Attention (PSCA) is developed to locate diverse facial parts which provide comprehensive descriptions. Second, an Attention Sparsity Loss (ASL) is proposed to encourage sparse responses in attention maps where only discriminative parts are emphasized while distracted regions (e.g. background or face masks) are discouraged. Built upon the PSCA and ASL, the DSA-Face model is developed to learn diverse and sparse attentions, which can extract diverse discriminative local representations and suppress the focus on noisy regions. Due to the pandemic of the COVID-19, the task of masked face matching is now very important, and our model can handle this much better than previous methods, demonstrating its effectiveness and usefulness. Moreover, our model outperforms the state-of-the-art methods on several other FR benchmarks, showing that it is also general to address various challenges in FR.
Algorithmic Splitting: A Method for Dataset Preparation

Datasets that appear in publications are curated and split into training, testing, and validation sub-datasets by domain experts. Consequently, machine learning models typically perform well on such split-by-hand datasets, whereas preparing real-world datasets into curated splits, i.e., training, testing, and validation sub-datasets, require extensive effort. Usually, random repetitive splitting is carried out, practiced, and evaluated until a better score is reached on the evaluation metrics. In this paper, a novel algorithmic method is proposed for splitting datasets for machine learning models. Algorithmic Splitting utilizes deterministic dimension reduction and density-based clustering techniques for estimating the data distribution. Afterward, splitting the dataset is carried out based on the estimated data distribution. Our objective is to achieve evenly representative splits of a given dataset in a standard and algorithmic way that reduces the perplexity of random splitting using the thorough splitting method. Experiments demonstrate the potential of Algorithmic Splitting through qualitative and quantitative evaluation of MNIST, Fashion MNIST, CIFAR-10, SmallNORB, and Shapes3D.
Deep Convolutional Neural Network and Weighted Bayesian Model for Evaluation of College Foreign Language Multimedia Teaching

In colleges and universities, teaching quality evaluation is an integral part of the teaching management process. Many factors influence it, and the relationship between its evaluation index and instructional quality is complicated, abstract, and nonlinear. However, existing evaluation methods and models have flaws such as excessive subjectivity and randomness, difficulty determining the weight of indicators, easy over-fitting, slow convergence speed, and limited computing power, to name a few. Furthermore, the evaluation index system focuses primarily on teaching attitude, material, and methods, rarely taking into account preparation prior to teaching or the teaching situation throughout the teaching process, resulting in an incomplete evaluation. As a result, learning how to construct a model for objectively, truly, thoroughly, and accurately assessing the teaching quality of colleges and universities is beneficial not only to improving teaching quality but also to promoting scientific decision-making in education. This paper develops a teaching assessment model using a deep convolutional neural network and the weighted Naive Bayes algorithm. Based on the degree of influence of different characteristics on the assessment outcomes, a method to estimate the weight of each evaluation characteristic by employing the related probability of class attributes is proposed, and the corresponding weight is assigned for each evaluation index, resulting in a classification model ideal for teaching assessment that promotes standardization and intelligibility.
COMPRA: A COMPact Reactive Autonomy Framework for Subterranean MAV Based Search-And-Rescue Operations

Abstract This work establishes COMPRA, a compact and reactive autonomy framework for fast deployment of Micro Aerial Vehicles (MAVs) in subterranean Search-and- Rescue (SAR) missions. A COMPRA-enabled MAV is able to autonomously explore previously unknown areas while specific mission criteria are considered e.g. an object of interest is identified and localized, the remaining useful battery life, the overall desired exploration mission duration. The proposed architecture follows a low-complexity algorithmic design to facilitate fully on-board computations, including nonlinear control, state-estimation, navigation, exploration behavior and object localization capabilities. The framework is mainly structured around a reactive local avoidance planner, based on enhanced Potential Field concepts and using instantaneous 3D pointclouds, as well as a computationally efficient heading regulation technique, based on depth images from an instantaneous camera stream. Those techniques decouple the collision-free path generation from the dependency of a global map and are capable of handling imprecise localization occasions. Field experimental verification of the overall architecture is performed in relevant unknown Global Positioning System (GPS)-denied environments.
Efficient Deterministic Search with Robust Loss Functions for Geometric Model Fitting

Geometric model fitting is a fundamental task in computer vision, which serves as the pre-requisite of many downstream applications. While the problem has a simple intrinsic structure where the solution can be parameterized within a few degrees of freedom, the ubiquitously existing outliers are the main challenge. In previous studies, random sampling techniques have been established as the practical choice, since optimization-based methods are usually too time-demanding. This prospective study is intended to design efficient algorithms that benefit from a general optimization-based view. In particular, two important types of loss functions are discussed, i.e., truncated and l1 losses, and efficient solvers have been derived for both upon specific approximations. Based on this philosophy, a class of algorithms are introduced to perform deterministic search for the inliers or geometric model. Recommendations are made based on theoretical and experimental analyses. Compared with the existing solutions, the proposed methods are both simple in computation and robust to outliers. Extensive experiments are conducted on publicly available datasets for geometric estimation, which demonstrate the superiority of our methods compared with the state-of-the-art ones. Additionally, we apply our method to the recent benchmark for wide-baseline stereo evaluation, leading to a significant improvement of performance. Our code is publicly available at https://github.com/AoxiangFan/EifficientDeterministicSearch.
The Impact of Translating Resource-Rich Datasets to Low-Resource Languages Through Multi-Lingual Text Processing

Urdu is still considered a low-resource language despite being ranked as the world’s 10th most spoken language with nearly 230 million speakers. The scarcity of benchmark datasets in low-resource languages has led researchers to utilize more ingenious techniques to curb the issue. One such option widely adopted is to use language translation services to replicate existing datasets from resource-rich languages such as English to low-resource languages, such as Urdu. For most natural language processing tasks, including polarity assessment, words translated via Google translator from one language to another often change the meaning. It results in a polarity shift causing the system’s performance degradation, particularly for sentiment classification and emotion detection tasks. This study evaluates the effect of translation on the sentiment classification task from a resource-rich language to a low-resource language. It identifies and enlists words causing polarity shift into five distinct categories. It further finds the correlation between the language with similar roots. Our study shows 2-3 percentage points performance degradation due to polarity shift as a result of translation from resource-rich languages to low-resource languages.
Is it a good move? Mining effective tutoring strategies from human–human tutorial dialogues

To construct dialogue-based Intelligent Tutoring Systems (ITS) with sufficient pedagogical expertise, a trendy research method is to mine large-scale data collected by existing dialogue-based ITS or generated between human tutors and students to discover effective tutoring strategies. However, most of the existing research has mainly focused on the analysis of successful tutorial dialogue. We argue that, to better inform the design of dialogue-based ITS, it is also important to analyse unsuccessful tutorial dialogues and gain a better understanding of the reasons behind those failures. Therefore, our study aimed to identify effective tutoring strategies by mining a large-scale dataset of both successful and unsuccessful human–human online tutorial dialogues, and further used these tutoring strategies for predicting students' problem-solving performance. Specifically, the study adopted a widely-used educational dialogue act scheme to describe the action behind utterances made by a tutor/student in the broader context of a tutorial dialogue (e.g., asking/answering a question, providing hints). Frequent dialogue acts were identified and analysed by taking into account the prior progress that a student had made before the start of a tutorial session and the problem-solving performance the student achieved after the end of the session. Besides, we performed a sequence analysis on the inferred actions to identify prominent patterns that were closely related to students' problem-solving performance. These prominent patterns could shed light on the frequent strategies used by tutors. Lastly, we measured the power of these tutorial actions in predicting students' problem-solving performance by applying a well-established machine learning method, Gradient Tree Boosting (GTB). Through extensive analysis and evaluations, we identified a set of different action patterns that were pertinent to tutors and students across dialogues of different traits, e.g., students without prior progress in solving problems, compared to those with prior progress, were likely to receive more thought-provoking questions from their tutors. More importantly, we demonstrated that the actions taken by students and tutors during a tutorial process could not adequately predict student performance and should be considered together with other relevant factors (e.g., the informativeness of the utterances).
Application of Quantum Natural Language Processing for Language Translation

In this paper, we develop compositional vector-based semantics of positive transitive sentences using quantum natural language processing (Q-NLP) to compare the parametrized quantum circuits of two synonymous simple sentences in English and Persian. We propose a protocol based on quantum long short-term memory (Q-LSTM) for Q-NLP to perform various tasks in general but specifically for translating a sentence from English to Persian. Then, we generalize our method to use quantum circuits of sentences as an input for the Q-LSTM cell. This enables us to translate sentences in different languages. Our work paves the way toward representing quantum neural machine translation, which may demonstrate quadratic speedup and converge faster or reaches a better accuracy over classical methods.
Anchor-Based Spatio-Temporal Attention 3-D Convolutional Networks for Dynamic 3-D Point Cloud Sequences

With the rapid development of measurement technology, LiDAR and depth cameras are widely used in the perception of the 3D environment. Recent learning based methods for robot perception most focus on the image or video, but deep learning methods for dynamic 3D point cloud sequences are underexplored. Therefore, developing efficient and accurate perception method compatible with these advanced instruments is pivotal to autonomous driving and service robots. An Anchor-based Spatio-Temporal Attention 3D Convolution operation (ASTA3DConv) is proposed in this paper to process dynamic 3D point cloud sequences. The proposed convolution operation builds a regular receptive field around each point by setting several virtual anchors around each point. The features of neighborhood points are firstly aggregated to each anchor based on the spatio-temporal attention mechanism. Then, anchor-based 3D convolution is adopted to aggregate these anchors' features to the core points. The proposed method makes better use of the structured information within the local region and learns spatio-temporal embedding features from dynamic 3D point cloud sequences. Anchor-based Spatio-Temporal Attention 3D Convolutional Neural Networks (ASTA3DCNNs) are built for classification and segmentation tasks based on the proposed ASTA3DConv and evaluated on action recognition and semantic segmentation tasks. The experiments and ablation studies on MSRAction3D and Synthia datasets demonstrate the superior performance and effectiveness of our method for dynamic 3D point cloud sequences. Our method achieves the state-of-the-art performance among the methods with dynamic 3D point cloud sequences as input on MSRAction3D and Synthia datasets.