Reviewerly/exHarmony huggingface dataset:
Motion Planning of Manipulators for Simultaneous Obstacle Avoidance and Target Tracking: An RNN Approach With Guaranteed Performance

Motion planning is a core issuein the field of robotic control, which directly affects the programming efficiency of robots. In this article, we study the motion planning problem of manipulators for simultaneous obstacle avoidance and target tracking, and propose a novel real-time planning method in a complex workspace. One important feature of the proposed method is that the robot can avoid colliding with obstacles by easily defining “virtual fences,” which are described by a group of level set functions. Thus, the feasible space can be abstracted as inequality constraints. Taking the predefined task, physical constraints, and feasible space constraints into consideration, the motion planning problem is formulated into a quadratic programming (QP) one, in which the redundant degrees of freedom are used to optimize the velocities of the robot. Then, the control command is obtained by an established recurrent neural network, which is capable of solving the QP problem in an online manner. Theoretical conduction and verification in several typical workspaces demonstrate the efficacy of the established method, such as the abilityto remove physical fences, quick rearrangement, and performance optimization.
Multimodal Clustering Networks for Self-supervised Learning from Unlabeled Videos

Multimodal self-supervised learning is getting more and more attention as it allows not only to train large networks without human supervision but also to search and retrieve data across various modalities. In this context, this paper proposes a framework that, starting from a pre-trained backbone, learns a common multimodal embedding space that, in addition to sharing representations across different modalities, enforces a grouping of semantically similar instances. To this end, we extend the concept of instance-level contrastive learning with a multimodal clustering step in the training pipeline to capture semantic similarities across modalities. The resulting embedding space enables retrieval of samples across all modalities, even from unseen datasets and different domains. To evaluate our approach, we train our model on the HowTo100M dataset and evaluate its zero-shot retrieval capabilities in two challenging domains, namely text-to-video retrieval, and temporal action localization, showing state-of-the-art results on four different datasets.
Positionless aspect based sentiment analysis using attention mechanism

Aspect-based sentiment analysis (ABSA) aims at identifying fine-grained polarity of opinion associated with a given aspect word. Several existing articles demonstrated promising ABSA accuracy using positional embedding to show the relationship between an aspect word and its context. In most cases, the positional embedding depends on the distance between the aspect word and the remaining words in the context, known as the position index sequence. However, these techniques usually employ both complex preprocessing approaches with additional trainable positional embedding and complex architectures to obtain the state-of-the-art performance. In this paper, we simplify preprocessing by including polarity lexicon replacement and masking techniques that carry the information of the aspect word’s position and eliminate the positional embedding. We then adopt a novel and concise architecture using two Bidirectional GRU along with an attention layer to classify the aspect based on its context words. Experiment results show that the simplified preprocessing and the concise architecture significantly improve the accuracy of the publicly available ABSA datasets, obtaining 81.37%, 75.39%, 80.88%, and 89.30% in restaurant 14, laptop 14, restaurant 15, and restaurant 16 respectively.
Fractional Deng Entropy and Extropy and Some Applications

Deng entropy and extropy are two measures useful in the Dempster–Shafer evidence theory (DST) to study uncertainty, following the idea that extropy is the dual concept of entropy. In this paper, we present their fractional versions named fractional Deng entropy and extropy and compare them to other measures in the framework of DST. Here, we study the maximum for both of them and give several examples. Finally, we analyze a problem of classification in pattern recognition in order to highlight the importance of these new measures.
End-to-End optimized image compression for machines, a study

An increasing share of image and video content is analyzed by machines rather than viewed by humans, and therefore it becomes relevant to optimize codecs for such applications where the analysis is performed remotely. Unfortunately, conventional coding tools are challenging to specialize for machine tasks as they were originally designed for human perception. However, neural network based codecs can be jointly trained end-to-end with any convolutional neural network (CNN)-based task model. In this paper, we propose to study an end-to-end framework enabling efficient image compression for remote machine task analysis, using a chain composed of a compression module and a task algorithm that can be optimized end-to-end. We show that it is possible to significantly improve the task accuracy when fine-tuning jointly the codec and the task networks, especially at low bitrates. Depending on training or deployment constraints, selective fine-tuning can be applied only on the encoder, decoder or task network and still achieve rate-accuracy improvements over an off-the-shelf codec and task network. Our results also demonstrate the flexibility of end-to-end pipelines for practical applications.
Edge-guided Composition Network for Image Stitching

Panorama creation is still challenging in consumer-level photography because of varying conditions of image capturing. A long-standing problem is the presence of artifacts caused by structure inconsistent image transitions. Since it is difficult to achieve perfect alignment in unconstrained shooting environment especially with parallax and object movements, image composition becomes a crucial step to produce artifact-free stitching results. Current energy-based seam-cutting image composition approaches are limited by the hand-crafted features, which are not discriminative and adaptive enough to robustly create structure consistent image transitions. In this paper, we present the first end-to-end deep learning framework named Edge Guided Composition Network (EGCNet) for the composition stage in image stitching. We cast the whole composition stage as an image blending problem, and aims to regress the blending weights to seamlessly produce the stitched image. To better preserve the structure consistency, we exploit perceptual edges to guide the network with additional geometric prior. Specifically, we introduce a perceptual edge branch to integrate edge features into the model and propose two edge-aware losses for edge guidance. Meanwhile, we gathered a general-purpose dataset for image stitching training and evaluation (namely, RISD). Extensive experiments demonstrate that our EGCNet produces plausible results with less running time, and outperforms traditional methods especially under the circumstances of parallax and object motions.
Mineral Prospectivity Mapping based on Isolation Forest and Random Forest: Implication for the Existence of Spatial Signature of Mineralization in Outliers

Known mineralized locations and randomly chosen non-mineralized locations are used traditionally as training samples in data-driven mineral prospectivity mapping (MPM). In this paper, we took advantage of (a) the variable importance and partial dependence plot, which enable interpretation of random forest (RF) modeling, and (b) the synthetic minority over-sampling technique, and investigated the efficacy of outlier-based training samples used for data-driven MPM in contrast to traditional practice of using known mineralized locations as positive training samples. The prediction maps obtained by RF modeling based on different sets of training samples suggest bias toward known mineralized locations in data-driven MPM. The proposed outlier-based training samples for data-driven MPM involve both unsupervised learning and supervised learning. The former aims at outlier detection, while the latter uses the resulting outliers as positive training samples to investigate the following: firstly, the delineation of prospective area or spatial signature of existing mineral system by avoiding the bias arising from the known mineralized locations in data-driven MPM and secondly, the coherence of spatial signature of outliers, which justifies their feasibility as positive training samples for data-driven MPM by RF modeling. Analyses of receiver operating curves and correlations of the resulting prediction maps imply that outliers derived by isolation forest show consistent spatial signature as the known mineralized location and, thus, were effective in narrowing down the prospective target areas just like in traditional data-driven MPM.
Online group streaming feature selection considering feature interaction

In real-world applications, features can be generated continuously one by one or by groups, such as image analysis and physical examination. Online streaming feature selection deals with streaming features on the fly. Existing streaming feature selection methods focus on removing irrelevant and redundant features and selecting the most relevant features, but they ignore the interaction between features. Interacting features appear to be irrelevant or weakly relevant to the class individually. However, if they are combined, they may highly correlate with the class. Features within the same group are more likely to interact with each other. Therefore, in this paper, we focus on feature interaction within and between the streaming groups and propose an Online Group Streaming Feature Selection method that can select Features to Interact with each other, named OGSFS-FI. OGSFS-FI consists of two stages: online intra-group selection and online inter-group selection. For intra-group selection, we design a new pair selection strategy that can select features interacting with each other. For inter-group selection, we use the regularization and variable selection method elastic net, which encourages a grouping effect. Extensive experiments conducted on synthetic and real-world datasets demonstrate our new method’s efficiency and effectiveness.
RepliCueAuth: Validating the Use of a Lab-Based Virtual Reality Setup for Evaluating Authentication Systems

Evaluating novel authentication systems is often costly and time-consuming. In this work, we assess the suitability of using Virtual Reality (VR) to evaluate the usability and security of real-world authentication systems. To this end, we conducted a replication study and built a virtual replica of CueAuth [52], a recently introduced authentication scheme, and report on results from: (1) a lab-based in-VR usability study (N=20) evaluating user performance; (2) an online security study (N=22) evaluating system's observation resistance through virtual avatars; and (3) a comparison between our results and those previously reported in the real-world evaluation. Our analysis indicates that VR can serve as a suitable test-bed for human-centred evaluations of real-world authentication schemes, but the used VR technology can have an impact on the evaluation. Our work is a first step towards augmenting the design and evaluation spectrum of authentication systems and offers ground work for more research to follow.
Pedestrian Street-Cross Action Recognition in Monocular Far Infrared Sequences

The early recognition and understanding of the actions performed by pedestrians in traffic scenes leads to an anticipation of pedestrian intentions in advance and helps in the process of collision warning and avoidance in the context of autonomous vehicles.An environment with low visibility conditions such as night-time, fog, heavy rain or smoke increases the number of difficult situations in traffic.A complete and original model for assessing if a pedestrian is engaged in a street cross action using only infrared monocular scene perception is proposed in this paper.The assessment of a street cross action is done by the time series analysis of features like: pedestrian motion, position of pedestrians with respect to the drivable area and their distance with respect to the ego-vehicle.The extraction of these features emerges from the combination of a deep learning based pedestrian detector with an original tracking algorithm, a semantic segmentation of the road surface and a time series long-short term memory network based action recognition.In order to validate the proposed method we introduce a new dataset named CROSSIR.It is formed of pedestrian annotations, action annotations and semantic labels for the road.The CROSSIR dataset is suitable for several common computer vision algorithms: (1) pedestrian detection and tracking algorithms because each pedestrian has a unique identifier over the frames in which it appears; (2) pedestrian action recognition;(3) semantic segmentation of the road pixels in the infrared image.
Q-learning and hyper-heuristic based algorithm recommendation for changing environments

A considerable amount of research has been devoted to solving static optimization problems via bio-inspired metaheuristic algorithms. However, most of the algorithms assume that all problem-related data remain unchanged during the optimization process, which is not a realistic assumption. Recently, dynamic optimization problems (DOPs) grabbed remarkable attention from the research community. However, the literature still lacks clear guidelines on selecting the most appropriate bio-inspired algorithm under changing environments. Due to the availability of many design choices, the selection of a suitable bio-inspired metaheuristic algorithm becomes an immediate challenge. This study proposes an algorithm recommendation architecture using Q-learning and hyper-heuristic approaches to help decision-makers select the most suitable bio-inspired algorithm for a given problem. To this end, Artificial Bee Colony (ABC), Manta Ray Foraging Optimization (MRFO), Salp Swarm Algorithm (SSA), and Whale Optimization Algorithm (WOA) are employed as low-level optimizers so that the Q-learning and hyper-heuristic automatically select the optimizer in each cycle of the optimization process. The proposed algorithms are implemented in dynamic multidimensional knapsack problems, a natural extension of the well-known 0–1 knapsack problem. The performances of the recommender and standalone bio-inspired algorithms are evaluated through a comprehensive experimental analysis including appropriate statistical tests. Thus, the significant differences among the algorithms are revealed. The obtained results point out the efficiencies of the Q-learning-based algorithm recommender and MRFO in solving the dynamic multidimensional knapsack problem.
Ninja Hands: Using Many Hands to Improve Target Selection in VR

Selection and manipulation in virtual reality often happen using an avatar's hands. However, objects outside the immediate reach require effort to select. We develop a target selection technique called Ninja Hands. It maps the movement of a single real hand to many virtual hands, decreasing the distance to targets. We evaluate Ninja Hands in two studies. The first study shows that compared to a single hand, 4 and 8 hands are significantly faster for selecting targets. The second study complements this finding by using a larger target layout with many distractors. We find no decrease in selection time across 8, 27, and 64 hands, but an increase in the time spent deciding which hand to use. Thereby, net movement time still decreases significantly. In both studies, the physical motion exerted also decreases significantly with more hands. We discuss how these findings can inform future implementations of the Ninja Hands technique.
Integrating Scene Semantic Knowledge into Image Captioning

Most existing image captioning methods use only the visual information of the image to guide the generation of captions, lack the guidance of effective scene semantic information, and the current visual attention mechanism cannot adjust the focus intensity on the image. In this article, we first propose an improved visual attention model. At each timestep, we calculated the focus intensity coefficient of the attention mechanism through the context information of the model, then automatically adjusted the focus intensity of the attention mechanism through the coefficient to extract more accurate visual information. In addition, we represented the scene semantic knowledge of the image through topic words related to the image scene, then added them to the language model. We used the attention mechanism to determine the visual information and scene semantic information that the model pays attention to at each timestep and combined them to enable the model to generate more accurate and scene-specific captions. Finally, we evaluated our model on Microsoft COCO (MSCOCO) and Flickr30k standard datasets. The experimental results show that our approach generates more accurate captions and outperforms many recent advanced models in various evaluation metrics.
Automatic segmentation of the cardiac MR images based on nested fully convolutional dense network with dilated convolution

Cardiac Magnetic Resonance Image (MRI) segmentation plays a helpful role in diagnosing cardiac disease. It is the preliminary step to estimate the functional indices such as ejection fraction (EF) and stroke volume. In this paper, we propose an automatic method for cardiac MRI segmentation based on deep learning. A nested U-shape network with Compressed Dense Blocks (CDBlocks) called BLU-Net is introduced. The Fully Convolutional Dense Network (FCD) is employed as the backbone. Compared with common dense blocks, the CDBlock reduces the connection between the input and the inner layers, and a 1 × 1 convolution is employed to compress the generated feature maps obtained by inner layers. Dilated convolution is employed in the CDBlock to obtain a larger receptive field without losing spatial resolution and reducing the loss of feature information. To learn more semantic information, an additional up-sampling path is adopted, and it makes our model more robust. Our method is evaluated on four cardiac MRI datasets, and the DSC and the HD metrics are employed in the experiment. The experimental results show that BLU-Net outperforms FCD and also outperforms some mainstream networks.
Impact of resampling methods and classification models on the imbalanced credit scoring problems

For imbalanced credit scoring, the most common solution is to balance the class distribution of the training set with a resampling method, and then train a classification model and classify the customer samples in the test set. However, it is still difficult to select the most appropriate resampling methods and classification models, and the optimal combinations of them have not been identified. Therefore, this study proposes a new benchmark models comparison framework for imbalanced credit scoring. In the framework, we introduce the index of balanced accuracy and four other evaluation measures, experimentally compare the performance of 10 benchmark resampling methods and nine benchmark classification models respectively on six credit scoring data sets, and analyze the optimal combinations of them. The experimental result shows: (1) as for benchmark resampling methods, random under-sampling (a traditional resampling method) and synthetic minority over-sampling technique combined with Wilson’s edited nearest neighbor (an intelligent resampling method) present the best performance; (2) as for benchmark classification models, logistic regression (a single classification model) and adaptive boosting (an ensemble classification model) present the best performance; (3) as for optimal combinations, random under-sampling combined with random subspace (an ensemble classification model) can obtain the most satisfactory credit scoring performance.
MASAD: A large-scale dataset for multimodal aspect-based sentiment analysis

Aspect-based sentiment analysis has obtained great success in recent years. Most of the existing work focuses on determining the sentiment polarity of the given aspect according to the given text, while little attention has been paid to the visual information as well as multimodality content for aspect-based sentiment analysis. Multimodal content is becoming increasingly popular in mainstream online social platforms and can help better extract user sentiments toward a given aspect. There are only few studies focusing on this new task: Multimodal Aspect-based Sentiment Analysis ( MASA ), which performs aspect-based sentiment analysis by integrating both texts and images. In this paper, we propose a mutimodal interaction model for MASA to learn the relationship among the text, image and aspect via interaction layers and adversarial training. Additionally, we build a new large-scale dataset for this task, named MASAD , which involves seven domains and 57 aspect categories with 38 k image-text pairs. Extensive experiments have been conducted on the proposed dataset to provide several baselines for this task. Though our models obtain significant improvement for this task, empirical results show that MASA is more challenging than textual aspect-based sentiment analysis, which indicates that MASA remains a challenging open problem and requires further efforts.
Spatiotemporal Attention for Multivariate Time Series Prediction and Interpretation

Multivariate time series modeling and prediction problems are abundant in many machine learning application domains. Accurate interpretation of the prediction outcomes from the model can significantly benefit the domain experts. In addition to isolating the important time-steps, spatial interpretation is also critical to understand the contributions of different variables on the model output. We propose a novel deep learning architecture, called spatiotemporal attention mechanism (STAM) for simultaneous learning of the most important time steps and variables. STAM is a causal (i.e., only depends on past inputs and does not use future inputs) and scalable (i.e., scales well with an increase in the number of variables) approach that is comparable to the state-of-the-art models in terms of computational tractability. We demonstrate our models’ performance on a popular public dataset and a domain-specific dataset, where the learned attention weights are validated from a domain knowledge perspective. When compared with the baseline models, the results show that STAM maintains state-of-the-art prediction accuracy while offering the benefit of accurate spatiotemporal interpretability.
Secure Text Extraction From Complex Degraded Images by Applying Steganography and Deep Learning

This work's primary goal is to secure the transmission of text hidden within the cover image using steganography over a public network of computers. Steganography is a powerful tool for concealing information within a cover image so that the concealed message remains undetectable. As a result, steganography refers to concealed writing. The secure transmission of information over a public network communication channel using steganography occurs in two stages, the first on the sender side and the second on the receiver side. In the first phase, steganography is normally applied to conceal the encrypted information within the image as a cover. The encrypted data is implanted inside the cover image using an improved least significant bit steganography method. The secret key obtained by the embedding algorithm is shared with the message retrieval algorithm on the receiver side to retrieve the message. Finally, the embedded text message is identified using a hybrid convolution regression adaptive integrated neural network (CRAINN) approach.