Reviewerly/exHarmony huggingface dataset:
Imitation Learning for Neural Network Autopilot in Fixed-Wing Unmanned Aerial Systems

This research identifies the feasibility of training an artificial neural network (ANN) autopilot, using supervised learning techniques, including an imitation learning framework known as the data aggregation set (DAgger) algorithm. The ANN autopilot aims to mimic a unified guidance, navigation & control (GNC) system to fly a fixed-wing UAS. Utilizing high-fidelity nonlinear 6-DOF aircraft simulations, it is shown that several modifications in existing imitation learning techniques must be considered. DAgger algorithm when applied sequentially to augment data along the desired flight trajectory to train the ANN autopilot is unable to generalize across turning and straight-line maneuvers, and hence cannot learn to fly stably, eventually terminating the simulation. Monte-Carlo methods when incorporated into the DAgger algorithm allow for sampling of random data along the desired flight trajectory, proved to be effective to train the ANN autopilot to fly indefinitely with acceptable tracking errors, however undesirable low frequency oscillations were observed in control input and aircraft states. The oscillatory behavior is dealt with by introducing a time-based moving window along the trajectory for data addition in conjunction with the standard DAgger algorithm. Different variations of DAgger algorithms are tested and compared using closed-loop 6-DOF flight simulations. 3D trajectory tracking and stability in aircraft states are evaluated with evidence supporting the idea that an ANN autopilot can behave as a unified GNC system and fly a fixed-wing UAS.
Investigation of an Innovative Approach for Identifying Human Face-Profile Using Explainable Artificial Intelligence

Human identification is a well-researched topic that keeps evolving. Advancement in technology has made it easy to train models or use ones that have been already created to detect several features of the human face. When it comes to identifying a human face from the side, there are many opportunities to advance the biometric identification research further. This paper investigates the human face identification based on their side profile by extracting the facial features and diagnosing the feature sets with geometric ratio expressions. These geometric ratio expressions are computed into feature vectors. The last stage involves the use of weighted means to measure similarity. This research addresses the problem of using an eXplainable Artificial Intelligence (XAI) approach. Findings from this research, based on a small data-set, conclude that the used approach offers encouraging results. Further investigation could have a significant impact on how face profiles can be identified. Performance of the proposed system is validated using metrics such as Precision, False Acceptance Rate, False Rejection Rate and True Positive Rate. Multiple simulations indicate an Equal Error Rate of 0.89.
Multi-platform and Multi-sensor Data Fusion Based on D-S Evidence Theory

This paper introduces two common design ideas and data preparation methods of multi-platform collaborative data fusion, and analyzes the application principle of D-S evidence theory in multi-platform collaborative data fusion. By using the advantages of multi-platform collaborative data fusion system, the shortcomings of single platform and single sensor in acquiring target information are solved, with high accuracy and credibility, expanding the scope of space detection, and enhancing the fault tolerance and complementarity of the system. The simulation results verify the effectiveness and practicability of D-S evidence theory in multi-platform collaborative data fusion.
UAS Flight Path Planning for Dynamic, Multi-Vehicle Environment

This paper proposes a new path planning method for Unmanned Aerial Systems (UAS) flying in a dynamic 3D environment shared by multiple aerial vehicles posing potential conflict risks. It primarily targets applications such as Urban Aerial Mobility (UAM). A new multi-staged algorithm is designed that combines Artificial Potential Field (AFP) method and Harmonic functions with Kalman filtering and Markov Decision Process (MDP) for dynamic path planning. It starts with estimating the aircraft traffic density in the area and then generates the UAS flight path in a way to minimize the risk of encounters. Hardware-in-the-loop simulations of the algorithm in various scenarios are presented, with a RGB-D camera (RealSense <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">™</sup> ) and Pixhawk autopilot to track the target. Numerical simulations show satisfactory results for path planning in various scenarios with increasing degree of complexity.
Study on weighted Nagar-Bardini algorithms for centroid type-reduction of general type-2 fuzzy logic systems

General type-2 fuzzy logic systems (GT2 FLSs) have drawn great attentions since the alpha-planes representation of general type-2 fuzzy sets (GT2 FSs) was proposed. The iterative of type-reduction (TR) algorithms are difficult to apply in practical applications. In the enhanced types of algorithms, the Nagar-Bardini (NB) algorithms decrease the computation complexity greatly. In terms of the Newton-Cotes quadrature formulas of numerical integration techniques, the paper extends the NB algorithms to three different forms of weighted NB (WNB) algorithms according to the comparisons between the sum operation in NB algorithms and the integral operation in continuous version of NB (CNB) algorithms. The NB algorithms just become a special case of the WNB algorithms. Four simulation examples are used to illustrate and analyze the performances of the WNB algorithms while performing the centroid TR of GT2 FLSs. It also shows that, in general, the WNB algorithms have smaller absolute error and faster convergence speed compared with the NB algorithms, which provides the potential value for T2 FLSs designers and users.
Dual Hierarchical Temporal Convolutional Network with QA-Aware Dynamic Normalization for Video Story Question Answering

Video story question answering (video story QA) is a challenging problem, as it requires a joint understanding of diverse data sources (i.e., video, subtitle, question, and answer choices). Existing approaches for video story QA have several common defects: (1) single temporal scale; (2) static and rough multimodal interaction; and (3) insufficient (or shallow) exploitation of both question and answer choices. In this paper, we propose a novel framework named Dual Hierarchical Temporal Convolutional Network (DHTCN) to address the aforementioned defects together. The proposed DHTCN explores multiple temporal scales by building hierarchical temporal convolutional network. In each temporal convolutional layer, two key components, namely AttLSTM and QA-Aware Dynamic Normalization, are introduced to capture the temporal dependency and the multimodal interaction in a dynamic and fine-grained manner. To enable sufficient exploitation of both question and answer choices, we increase the depth of QA pairs with a stack of non-linear layers, and exploit QA pairs in each layer of the network. Extensive experiments are conducted on two widely used datasets: TVQA and MovieQA, demonstrating the effectiveness of DHTCN. Our model obtains state-of-the-art results on the both datasets.
An end-to-end reinforcement learning method for automated guided vehicle path planning

Based on the Internet of Things, intelligent logistics warehouses are flourishing. As an automatic transport vehicle, AGV (Automated Guided Vehicle) plays an important role in improving the efficiency of production logistics system. Since the environment of the intelligent warehouse is dynamic and partially unknown, AGV need to have self-learning and adaptive capabilities to cope with changes in the environment. The traditional path planning algorithm is difficult to operate in unknown environment without prior map knowledge. To solve this, we propose an end-to-end AGV path planning method to make AGV obtain the optimal action from the original visual image and LIDAR information. In addition, a deep reinforcement learning method is employed to train AGV, combining priority experience replay mechanism and double deep Q network with the dueling architecture, to make AGV has a certain generalization ability for unknown environment and adaptability for a dynamic environment. Finally, our simulation experiments show the effectiveness of the proposed method.
Synthesis of complex- and hyperbolic-valued Hopfield neural networks

A complex-valued Hopfield neural network (CHNN), a multistate Hopfield model, is useful for processing multilevel data, such as image data. Several alternatives of CHNN have been proposed. A hyperbolic-valued Hopfield neural network (HHNN) improves the noise tolerance of CHNN. In this work, we propose a synthetic Hopfield neural network (SHNN), a combination of a CHNN and an HHNN. An SHNN is the first combination of Hopfield models using different algebras. Since a CHNN and an HHNN have different operators, such as addition and multiplication, they are represented by matrices and vectors to compose an SHNN. A CHNN and an HHNN have different global minima. Only the common global minima are the global minima of SHNN. Thus, an SHNN is expected to improve the noise tolerance. In fact, computer simulations support our expectation.
Panoptic Image Annotation with a Collaborative Assistant

This paper aims to reduce the time to annotate images for panoptic segmentation, which requires annotating segmentation masks and class labels for all object instances and stuff regions. We formulate our approach as a collaborative process between an annotator and an automated assistant who take turns to jointly annotate an image using a predefined pool of segments. Actions performed by the annotator serve as a strong contextual signal. The assistant intelligently reacts to this signal by annotating other parts of the image on its own, which reduces the amount of work required by the annotator. We perform thorough experiments on the COCO panoptic dataset, both in simulation and with human annotators. These demonstrate that our approach is significantly faster than the recent machine-assisted interface of [Andriluka 18 ACMMM], and $2.4\times$ to $5\times$ faster than manual polygon drawing. Finally, we show on ADE20k that our method can be used to efficiently annotate new datasets, bootstrapping from a very small amount of annotated data.
Coupled geophysics and geochemistry to record recent coastal changes of contaminated sites of the Bagnoli industrial area, Southern Italy

The paper shows the results of a geochemical - geophysical survey on the decommissioned industrial area of Bagnoli (Naples, southern Italy). Analyses of 15 priority congeners of Polycyclic Aromatic Hydrocarbons (PAHs) were carried out on 21 sediment cores, collected on the beaches surrounding the old iron and steel industrial plant that was active over the last century. Anomalous high concentrations of PAHs were detected in this area, up to 87,000 μg g−1 d.w. In particular, most congeners were embedded inside a mainly sandy layer, characterized by a dark colour and high resistivity values. The joined use of Electrical Resistivity Tomography, chemical data and direct observation of cores allowed a volumetric estimation of the polluted layer corresponding to 20,200 m3. A total of 24,100 m3 of less polluted materials cover the dark, highly polluted layer. In addition, the geophysical-environmental approach helped to disentangle the deep interactions between the beach area, coastal changes and fluid vents. The actual coastal morphology is the result of historical, anthropic-driven rapid evolution that we reconstructed by using available historical images. The achieved results underline a crucial role played by combining geophysical and geochemical methodologies for a deeper and systematic understanding of anthropogenic impacts in coastal systems affected by multiple and combined stressors.
ASTA-Net: Adaptive Spatio-Temporal Attention Network for Person Re-Identification in Videos

The attention mechanism has been widely applied to enhance pedestrian representation for person re-identification in videos. However, most existing methods learn the spatial and temporal attention separately, and thus ignore the correlation between them. In this work, we propose a novel Adaptive Spatio-Temporal Attention Network (ASTA-Net) to adaptively aggregate the spatial and temporal attention features into discriminative pedestrian representation for person re-identification in videos. Specifically, multiple Adaptive Spatio-Temporal Fusion modules within ASTA-Net are designed for exploring precise spatio-temporal attention on multi-level feature maps. They first obtain the preliminary spatial and temporal attention features via the spatial semantic relations for each frame and temporal dependencies among inconsecutive frames, then adaptively aggregate the preliminary attention features on the basis of their correlation. Moreover, an Adjacent-Frame Motion module is designed to explicitly extract motion patterns according to the feature-level variation among adjacent frames. Extensive experiments on the three widely-used datasets, i.e., MARS, iLIDS-VID and PRID2011, have demonstrated the effectiveness of the proposed approach.
Learning with large, complex data and visualizations: youth data wrangling in modeling family migration

We present a micro-analysis of youth interactions with large complex, socioeconomic datasets and data visualization tools. Middle and high school youth used georeferenced data and data visualization tools to assemble models that present their family migration histories in relation to larger socioeconomic trends in a summer program. Using screen-capture and video recordings, field notes, and artifacts, we analyzed youth’s step-by-step decision-making and interaction with data interfaces in data wrangling, which we define as the practices for selecting, interpreting, and integrating datasets in order to build meaningful data displays and tell a story with the data. We identify patterns in youth’s data wrangling trajectories and propose a conceptual model for describing the stages (Find, Relate, Challenge, Build) of youth learning to construct models and tell stories about family migration. In addition, we highlight student struggles and opportunities for learning to be explored in future learning environment designs with large, complex datasets and data interfaces.
Accurate UAV Tracking with Distance-Injected Overlap Maximization

UAV tracking is usually challenged by the dual-dynamic disturbances that arise from not only diverse moving target but also motion camera, leading to a more serious model drift issue than traditional visual tracking. In this work, we propose to alleviate this issue with distance-injected overlap maximization. Our idea is improving the accuracy of target localization by deriving a conceptually simple target localization loss and a global feature recalibration scheme in a mutual reinforced way. In particular, the target localization loss is designed by simply incorporating the normalized distance of target offset and generic semantic IoU loss, resulting in the distance-injected semantic IoU loss, and its minimal solution can alleviate the drift problem caused by camera motion. Moreover, the deep feature extractor is reconstructed and alternated with a feature recalibration network, which can leverage the global information to recalibrate significant features and suppress negligible features. Following by multi-scale feature concat, the proposed tracker can improve the discriminative capability of feature representation for UAV targets on the fly. Extensive experimental results on four benchmarks, i.e. UAV123, UAVDT, DTB70, and VisDrone, demonstrate the superiority of the proposed tracker against existing state-of-the-arts on UAV tracking.
Discriminative Spatial Feature Learning for Person Re-Identification

Person re-identification (ReID) aims to match detected pedestrian images from multiple non-overlapping cameras. Most existing methods employ a backbone CNN to extract a vectorized feature representation by performing some global pooling operations (such as global average pooling and global max pooling) on the 3D feature map (i.e., the output of the backbone CNN). Although simple and effective in some situations, the global pooling operation only focuses on the statistical properties and ignores the spatial distribution of the feature map. Hence, it can not distinguish two feature maps when they have similar response values located in totally different positions. To handle this challenge, a novel method is proposed to learn the discriminative spatial features. Firstly, a self-constrained spatial transformer network (SC-STN) is introduced to handle the misalignments caused by detection errors. Then, based on the prior knowledge that the spatial structure of a pedestrian often keeps robust in vertical orientation of images, a novel vertical convolution network (VCN) is proposed to extract the spatial feature in vertical. Extensive experimental evaluations on several benchmarks demonstrate that the proposed method achieves state-of-the-art performances by introducing only a few parameters to the backbone.
Preserving Global and Local Temporal Consistency for Arbitrary Video Style Transfer

Video style transfer is a challenging task that requires not only stylizing video frames but also preserving temporal consistency among them. Many existing methods resort to optical flow for maintaining the temporal consistency in stylized videos. However, optical flow is sensitive to occlusions and rapid motions, and its training processing speed is quite slow, which makes it less practical in real-world applications. In this paper, we propose a novel fast method that explores both global and local temporal consistency for video style transfer without estimating optical flow. To preserve the temporal consistency of the entire video (i.e., global consistency), we use structural similarity index instead of flow optical and propose a self-similarity loss to ensure the temporal structure similarity between the stylized video and the source video. Furthermore, to enhance the coherence between adjacent frames (i.e., local consistency), a self-attention mechanism is designed to attend the previous stylized frame for synthesizing the current frame. Extensive experiments demonstrate that our method generally achieves better visual results and runs faster than the state-of-the-art methods, which validates the superiority of simultaneously preserving global and local temporal consistency for video style transfer
Rectification and Super-Resolution Enhancements for Forensic Text Recognition

Retrieving text embedded within images is a challenging task in real-world settings. Multiple problems such as low-resolution and the orientation of the text can hinder the extraction of information. These problems are common in environments such as Tor Darknet and Child Sexual Abuse images, where text extraction is crucial in the prevention of illegal activities. In this work, we evaluate eight text recognizers and, to increase the performance of text transcription, we combine these recognizers with rectification networks and super-resolution algorithms. We test our approach on four state-of-the-art and two custom datasets (TOICO-1K and Child Sexual Abuse (CSA)-text, based on text retrieved from Tor Darknet and Child Sexual Exploitation Material, respectively). We obtained a 0.3170 score of correctly recognized words in the TOICO-1K dataset when we combined Deep Convolutional Neural Networks (CNN) and rectification-based recognizers. For the CSA-text dataset, applying resolution enhancements achieved a final score of 0.6960. The highest performance increase was achieved on the ICDAR 2015 dataset, with an improvement of 4.83% when combining the MORAN recognizer and the Residual Dense resolution approach. We conclude that rectification outperforms super-resolution when applied separately, while their combination achieves the best average improvements in the chosen datasets.
ADHD Intelligent Auxiliary Diagnosis System Based on Multimodal Information Fusion

The traditional medical diagnosis methods of ADHD mainly rely on scale evaluation and interview observation. The diagnosis conclusion is subjective and extremely dependent on the doctor's experience level. There is an urgent need to improve diagnosis efficiency and improve the diagnosis standard through other technical means in the clinical process. We have designed and developed the ADHD intelligent auxiliary diagnosis system with software and hardware cooperation. The system performs a set of functional test tasks, uses a camera module to capture multimodal information such as facial expressions, eye movements, limb movements, language expressions and reaction abilities of children during task completion, and uses computer vision technology to automatically extract measurable characteristics. Finally, deep learning technology is used to detect children's specific behaviors in the video, which is complementary to the existing doctor's diagnosis basis. This system was deployed in the Department of Psychology of Children's Hospital of Zhejiang University in July 2019 and has been used in actual clinical diagnosis to date. It has completed the testing and evaluation of hundreds of ADHD children.
Concept Drift Detection for Multivariate Data Streams and Temporal Segmentation of Daylong Egocentric Videos

The long and unconstrained nature of egocentric videos makes it imperative to use temporal segmentation as an important pre-processing step for many higher-level inference tasks. Activities of the wearer in an egocentric video typically span over hours and are often separated by slow, gradual changes. Furthermore, the change of camera viewpoint due to the wearer's head motion causes frequent and extreme, but, spurious scene changes. The continuous nature of boundaries makes it difficult to apply traditional Markov Random Field (MRF) pipelines relying on temporal discontinuity, whereas deep Long Short Term Memory (LSTM) networks gather context only upto a few hundred frames, rendering them ineffective for egocentric videos. In this paper, we present a novel unsupervised temporal segmentation technique especially suited for day-long egocentric videos. We formulate the problem as detecting concept drift in a time-varying, non i.i.d. sequence of frames. Statistically bounded thresholds are calculated to detect concept drift between two temporally adjacent multivariate data segments with different underlying distributions while establishing guarantees on false positives. Since the derived threshold indicates confidence in the prediction, it can also be used to control the granularity of the output segmentation. Using our technique, we report significantly improved state of the art f-measure for daylong egocentric video datasets, as well as photostream datasets derived from them: HUJI~(73.01%, 59.44%), UTEgo~(58.41%, 60.61%) and Disney~(67.63%, 68.83%).