Reviewerly/exHarmony huggingface dataset:
A Framework for the Systematic Evaluation of Obstacle Avoidance and Object-Aware Controllers

Real-time control is an essential aspect of safe robot operation in the real world with dynamic objects. We present a framework for the analysis of object-aware con-trollers, methods for altering a robot's motion to anticipate and avoid possible collisions. This framework is focused on three design considerations: kinematics, motion profiles, and virtual constraints. Additionally, the analysis in this work relies on verification of robot behaviors using fundamental robot-obstacle experimental scenarios. To showcase the effectiveness of our method we compare three representative object-aware controllers. The comparison uses metrics originating from the design considerations. From the analysis, we find that the design of object-aware controllers often lacks kinematic considerations, continuity of control points, and stability in movement profiles. We conclude that this framework can be used in the future to design, compare, and benchmark obstacle avoidance methods.
Cure-GNN: A Robust Curvature-Enhanced Graph Neural Network Against Adversarial Attacks

Graph neural networks (GNNs) are a specialized type of deep learning models on graphs by learning aggregations over neighbor nodes. However, recent studies reveal that the performance of GNNs are severely deteriorated by injecting adversarial examples. Hence, improving the robustness of GNNs is of significant importance. Prior works are devoted to reducing the influence of direct adversaries which are adversarial attacks by positioning a node's one-hop neighbors, yet these approaches are limited in protecting GNNs from indirect adversarial attacks within a node's multi-hop neighbors. In this work, we approach this problem from a new angle by exploring the graph Ricci curvature, which can characterize the relationships of both direct and indirect links from any two nodes’ neighborhoods in the Riemannian space. We first investigate the distinguishable properties of adversarial attacks with graph Ricci curvature distribution. Then, a novel defense framework called Cure-GNN is proposed to detect and mitigate adversarial effects. Cure-GNN discerns the distinction between adversarial edges and normal edges via computing curvature, and merges it into the node features reconstructed by a residual learning framework. Extensive experiments over real-world datasets on node classification task demonstrate the efficacy of Cure-GNN and achieves superiority to the state-of-the-arts without incurring high complexity.
HearASL: Your Smartphone Can Hear American Sign Language

Sign language is expressed by movements of the hands and facial expressions, which is mainly used by the deaf community. Although some gesture recognition methods are put forward, they possess different defects and are not applicable to deal with the sign language recognition (SLR) problem. In this article, we propose an end-to-end American SLR system with built-in speakers and microphones in smartphones, which enables SLR at both word level and sentence level. The high-level idea is to use the inaudible acoustic signal to estimate channel information and capture the sign language in real time. We use channel impulse response to represent each sign language gesture, which can realize finger-level recognition. We also pay attention to conversion movements between two words and treat them as an additional label when training the sentence-level classification model. We implement a prototype system and run a series of experiments that demonstrate the promising performance of our system. Experimental results show that our approach can achieve an accuracy of 97.2% at word-level recognition and word error rate of 0.9% at sentence-level recognition, respectively.
EvAnGCN: Evolving Graph Deep Neural Network Based Anomaly Detection in Blockchain

Detecting anomalous behaviors in the blockchain is important for maintaining its integrity. An imminent challenge is to capture the evolving model of transactions in the network. Representing the network with a dynamic graph helps model the system’s time-evolving nature. However, as the graph evolves, real-world scenarios further stimulate the development of Graph Neural Networks (GNNs) to handle dynamic graph structures. In this paper, we propose a novel dynamic Graph Convolutional Network framework, namely EvAnGCN (Evolving Anomaly detection GCN), that helps detect anomalous behaviors in the blockchain. EvAnGCN exploits the time-based neighborhood feature aggregation of transactional features and the dynamic structure of the transaction network to detect anomalous nodes within the network. We conducted experiments on the Ethereum blockchain transaction dataset. Our experimental results demonstrate that EvAnGCH outperformed the baseline models.
Searching the Deployable Convolution Neural Networks for GPUs

Customizing Convolution Neural Networks (CNN) for production use has been a challenging task for DL practitioners. This paper intends to expedite the model customization with a model hub that contains the optimized models tiered by their inference latency using Neural Architecture Search (NAS). To achieve this goal, we build a distributed NAS system to search on a novel search space that consists of prominent factors to impact latency and accuracy. Since we target GPU, we name the NAS optimized models as GPUNet, which establishes a new SOTA Pareto frontier in inference latency and accuracy. Within 1ms, GPUNet is 2x faster than EfficientNet-X and FBNetV3 with even better accuracy. We also validate GPUNet on detection tasks, and GPUNet consistently outperforms EfficientNet-X and FB-NetV3 on COCO detection tasks in both latency and accuracy. All of these data validate that our NAS system is effective and generic to handle different design tasks. With this NAS system, we expand GPUNet to cover a wide range of latency targets such that DL practitioners can deploy our models directly in different scenarios.
Decision-Making Modeling in Educational Process Organization Under the Conditions of Crisis Situations Forecasting

The research deals with the problem of decision- making processes, related to the optimization of educational process. The case of shortening the length of academic semesters due to an increase in the weekly workload is considered. The mathematical model was built for the most effective use of existing audiences. It is possible to develop options for management decisions regarding the combined (face-to-face and distance learning with the use of electronic learning tools) as forms of education.
Web-Based Authoring Tool for Virtual Exhibitions

This work presents a web-based authoring tool that enables museum curators and individuals to make digital collections of exhibits accessible and explorable via the Web. It allows users to create interactive and immersive virtual 3D/VR exhibitions using a unified collaborative authoring environment based on a Human-Centered Design approach with the active participation of museum curators and end-users. Main contributions are pertinent to the fields of (a) user-designed dynamic virtual exhibitions, (b) exhibition tours, (c) visualization in web-based 3D/VR technologies, and (d) immersive navigation and interaction.
Text-to-Image Synthesis based on Object-Guided Joint-Decoding Transformer

Object-guided text-to-image synthesis aims to generate images from natural language descriptions built by two-step frameworks, i.e., the model generates the layout and then synthesizes images from the layout and captions. However, such frameworks have two issues: 1) complex structure, since generating language-related layout is not a trivial task; 2) error propagation, because the inappropriate layout will mislead the image synthesis and is hard to be revised. In this paper, we propose an object-guided joint-decoding module to simultaneously generate the image and the corresponding layout. Specially, we present the joint-decoding transformer to model the joint probability on images tokens and the corresponding layouts tokens, where layout tokens provide additional observed data to model the complex scene better. Then, we describe a novel Layout-Vqgan for layout encoding and decoding to provide more information about the complex scene. After that, we present the detail-enhanced module to enrich the language-related details based on two facts: 1) visual details could be omitted in the compression of VQGANs; 2) the joint-decoding transformer would not have sufficient generating capacity. The experiments show that our approach is competitive with previous object-centered models and can generate diverse and high-quality objects under the given layouts.
Multi-Tree Guided Efficient Robot Motion Planning

Motion Planning is necessary for robots to complete different tasks. Rapidly-exploring Random Tree (RRT) and its variants have been widely used in robot motion planning due to their fast search in the state space. However, they perform not well in many complex environments since the motion planning needs to simultaneously consider the geometry constraints and differential constraints. In this article, we propose a novel robot motion planning algorithm that utilizes multi-tree to guide the exploration and exploitation. The proposed algorithm maintains more than two trees to search the state space at first. Each tree will explore the local environments. The tree starts from the root will gradually collect information from other trees and grow towards the goal state. This simultaneous exploration and exploitation method can quickly find a feasible trajectory. We compare the proposed algorithm with other popular motion planning algorithms. The experiment results demonstrate that our algorithm performs better on different evaluation metrics.
Positional Attention Guided Transformer-Like Architecture for Visual Question Answering

Transformer architectures have recently been introduced into the field of visual question answering (VQA), due to their powerful capabilities of information extraction and fusion. However, existing Transformer-like models, including models using a single Transformer structure and large-scale pre-training generic visual-linguistic models, do not fully utilize both positional information of words in questions and positional information of objects in images, which are shown in this paper to be crucial in VQA tasks. To address this challenge, we propose a novel positional attention guided Transformer-like architecture, which can adaptively extracts positional information within and across the visual and language modalities, and use this information to guide high-level interactions in inter- and intra-modality information flows. In particular, we design and assemble three positional attention modules into a single Transformer-like model MCAN. We show that the positional information introduced in intra-modality interaction can adaptively modulate inter-modality interaction according to different inputs, which plays an important role for visual reasoning. Experimental results demonstrate that our model outperforms the state-of-the-art models and is particularly good at handling object counting questions. Overall, our model achieves the accuracy of 70.10%, 71.27%, and 71.52% on the datasets of COCO-QA, VQA v1.0 test-std and VQA v2.0 test-std, respectively.
Refined Interiors Using Augmented Reality

This paper is proposed to create the interior of our house or place of business by positioning 3D items in accordance with our preferences. In essence, programs that replicate a furniture arrangement frequently employ augmented reality technology. Furniture purchases made in physical and mortar stores are time-consuming and labor-intensive, and they do not allow for the customization of interiors as desired. A technique that uses a camera to add virtual objects to the actual world (Augmented Reality) is suggested as a solution to this issue. Systems for arranging furniture in augmented reality are helpful for examining room or building plans without having to purchase or relocate actual furniture. Virtual furniture might be put and moved about in the actual world in an augmented reality setting, giving users an interactive experience. To materialize numerous 3D items and furnishings in the real world, we created a smartphone interface for augmented reality.
The Use of Communication Strategies by Second Language Learners of Chinese in a Virtual Reality Learning Environment

Virtual reality (VR) for Language learning has been shown effective in improving students’ communication skills. However very few studies reveal the communication strategies students used in communicating in a VR environment, especially in learning Chinese as a foreign or second language (CFL/CSL). In this pilot study, a prototype Virtual Experiential Language Learning Environment (VELLE) was used to provide immersive learning experiences in developing CFL students’ communication strategies. Productive Failure was employed as an instructional approach to support the learning design. The participants were six CFL students learning in an Australian university. The main data sources included audio recordings of the students’ communication in VR and interview data. The findings showed that participants’ use of various types of communication strategies was influenced by the VELLE and Productive Failure design. In particular, the use of L1 in the problem-solving phase enriched the participants’ learning opportunities and all participants used a lesser number of resource deficit-related strategies after the instruction. Second, Students reported enhanced engagement in language use in the VR context.
An Integrated Chinese Malicious Webpages Detection Method Based on Pre-trained Language Models and Feature Fusion

This paper proposed an integrated Chinese malicious webpages detection method. Firstly, we collected and released a Chinese malicious webpages detection dataset called "ChiMalPages" containing URLs and HTML/JavaScript files, and specified the detailed types of malicious pages according to relevant laws. Secondly, we designed a feature template for Chinese webpages and ranked each feature's importance based on information gain of the Random Forest algorithm. Thirdly, we fine-tuned BERT on the external URLs classification task and text on webpages, respectively producing new models "BERT-URL" and "BERT-web-text". The performance of pre-trained models is obviously superior to the baseline models. Finally, we integrated features from manual templates, BERT-URL and BERT-web-text, and the classification F1 score reaches 79.84%, increasing by 7.37% compared with manually designed webpage features. Experiments proved that our method based on BERT is useful and not biased on detailed classes.
A Hybrid Machine Learning Approach for Credit Card Fraud Detection

The online banking system is the new trend in the developing digital world. The transferring of a large amount of currency in a millisecond is leading to fast accessing of the banking system as it saves more time at the online payment and digital shopping. The increase in rate of use of banking credit and debit card leads to a large amount of fraud in the field of finance. Machine learning has the new discovering faces in the field of the finance. So, this research work proposed a hybrid model using the logistic regression, multilayer perceptron, and the XgBoost. The study involves both the balance and imbalance dataset to conclude the result based on the accuracy precision and recall. The results show that accuracy of the model is 100%, and precision, recall, and F1-scores are 95.63%, 99.99%, and 97.76% respectively.
Rotational-XOR Differential Cryptanalysis and an Automatic Framework for AND-RX Ciphers

In this paper, a security evaluation framework for AND-RX ciphers against rotational-XOR differential cryptanalysis is proposed. This framework first models the structure of all the possible rotational-XOR differential (abbreviated to “RXD”) trails and introduces a method to calculate this structure round by round. Based on this approach, an automatic method is proposed for searching RXD trails. In this method, four strategies are proposed to derive better result and improve the efficiency. Unlike previous automations, the time complexity for this framework can be pre-computed, which is bounded by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${\mathcal{ O}}\left ({{c\cdot n\cdot R^{2}\cdot C_{n}^{n_{1}}} }\right)$ </tex-math></inline-formula> (where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula> is the block size, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n_{1}$ </tex-math></inline-formula> is the number of active bits for the starting point of automatic method, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$R$ </tex-math></inline-formula> is the length of the targeted rounds and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$c$ </tex-math></inline-formula> is a fixed constant). Under the given strategies and searching subspaces, the derived RXD trails are guaranteed to be optimal. To prove the correctness and efficiency, this framework is applied to all the ten variants for SIMON and three variants for Simeck. When compared with previous RXD trails, the best improvement is up to three rounds. To validate the correctness of the derived rotational-XOR differential trails, a concrete experiment on Simeck32 is conducted and the experimental result complies with the theoretical analysis. As far as we know, for all the variants of Simeck, current longest distinguishers over all the cryptanalytic methods are obtained in this paper.
Satellite image search in AgoraEO

The growing operational capability of global Earth Observation (EO) creates new opportunities for data-driven approaches to understand and protect our planet. However, the current use of EO archives is very restricted due to the huge archive sizes and the limited exploration capabilities provided by EO platforms. To address this limitation, we have recently proposed MiLaN, a content-based image retrieval approach for fast similarity search in satellite image archives. MiLaN is a deep hashing network based on metric learning that encodes high-dimensional image features into compact binary hash codes. We use these codes as keys in a hash table to enable real-time nearest neighbor search and highly accurate retrieval. In this demonstration, we showcase the efficiency of MiLaN by integrating it with EarthQube, a browser and search engine within AgoraEO. EarthQube supports interactive visual exploration and Query-by-Example over satellite image repositories. Demo visitors will interact with EarthQube playing the role of different users that search images in a large-scale remote sensing archive by their semantic content and apply other filters.
LASDNet: A Lightweight Anchor-Free Ship Detection Network for SAR Images

Deep convolutional neural networks (DCNN)-based methods have been applied widely to ship detection in SAR images. However, most DCNN-based ship target detectors that focus on the detection performance ignore the computation complexity. We propose a lightweight anchor-free ship detection network (LASDNet) for SAR images to tackle this problem. First, a lightweight backbone utilizing a double fusion with squeeze-and-excitation-bottleneck block under the CSPNet design (CSP-DFSEB) and three pooling blocks (i.e., EVE, FCT, and ME blocks) are constructed, which achieves a balance between accuracy and efficiency. Second, a transformer-based aggregation layer conducts feature fusion. Finally, an improved one-stage anchor-free detector FCOS is presented. The analyses of the High-Resolution SAR Images Dataset for Ship Detection and Instance Segmentation (HRSID) dataset show that the proposed detector has the second least number of parameters (1.15 MB), the lowest computation complexity (1.01 GFLOPs), and the highest average precision (59.25) compared with other state-of-the-art methods.
Scanline Homographies for Rolling-Shutter Plane Absolute Pose

Cameras on portable devices are manufactured with a rolling-shutter (RS) mechanism, where the image rows (aka. scanlines) are read out sequentially. The unknown camera motions during the imaging process cause the so-called RS effects which are solved by motion assumptions in the literature. In this work, we give a solution to the absolute pose problem free of motion assumptions. We categorically demonstrate that the only requirement is motion smoothness instead of stronger constraints on the camera motion. To this end, we propose a novel mathematical abstraction for RS cameras observing a planar scene, called the scanline-homography, a 3 × 2 matrix with 5 DOFs. We establish the relationship between a scanline-homography and the corresponding plane-homography, a 3 × 3 matrix with 6 DOFs assuming the camera is calibrated. We estimate the scanline-homographies of an RS frame using a smooth image warp powered by B-Splines, and recover the plane-homographies afterwards to obtain the scanline-poses based on motion smoothness. We back our claims with various experiments. Code and new datasets: https://bitbucket.org/clermontferrand/planarscanlinehomography/src/master/.
Fast Probabilistic Policy Reuse via Reward Function Fitting

Transfer learning has shown great potential to accelerate reinforcement learning (RL) by utilizing prior knowledge of relevant task that has been learned in the past. Policy Reuse Q-learning (PRQL) is a general policy transfer framework, which speeds up the learning process of the target task by probabilistically reusing source policies from the policy library. In this paper, we propose an improved PRQL method to achieve more fast probabilistic policy reuse in deep reinforcement learning (DRL). First, we extend the basic PRQL algorithm to DRL, proposing a probability policy reuse algorithm that builds on DRL to solve more complex problems. Second, PRQL algorithms usually use a metric based on the average gain to measure the similarity between tasks. However, it contains very limited information and must be delayed until the end of an episode to update, which is inefficient. Instead, we propose a new metric based on fitting the reward function, which can make the agent converge to the most suitable reuse policy more quickly and accurately. We demonstrate the detection accuracy, received cumulative reward, and speed of convergence of our method in three complex Markov tasks. Experimental results show that our method can consistently achieve efficient policy transfer in these tasks.