Reviewerly/exHarmony huggingface dataset:
An End-to-End Object Detector with Spatiotemporal Context Learning for Machine-Assisted Rehabilitation

Recently, object detection technologies applied in rehabilitation systems are mainly based on the ready-made technology of CNNs. This paper proposes an DETR-based detector which is an end-to-end object detector with spatiotemporal context learning for machine-assisted rehabilitation. To improve the performance of small object detection, first, the multi-level features of the RepVGG are fused with the SE attention mechanism to build a SEFP-RepVGG. To make the encoder-decoder structure more suitable, next, the value of the encoder is generated by using feature maps with more detailed information than key/query. To reduce computation, Patch Merging is finally imported to modify the feature map scale of the input encoder. The proposed detector has higher real-time performance than DETR and obtains the competitive detection accuracy on the ImageNet VID benchmark. Some typical samples from the NTU RGB-D 60 dataset are selected to build a new limb-detection dataset for further evaluation. The results show the effectiveness of the proposed detector in the rehabilitation scenarios.
Obstacle Avoidance Planning and Experimental Study of Reconfigurable Cable-Driven Parallel Robot Based on Deep Reinforcement Learning

Cable-driven parallel robot (CDPR) is widely used in the fields of hoisting and cargo handling. However, it is very easy to be interfered with and restricted by obstacles in the space, which affects the working performance of the robot. This paper takes the reconfigurable cable-driven parallel robot (RCDPR) as the research object and adopts deep reinforcement learning (DRL) to solve the obstacle avoidance planning problem. The model of RCDPR is structured, and kinematic analysis is performed to obtain the state transform function. An improved Soft AC algorithm is employed by expected SARSA and adaptive target values, which enhances the utilization of samples. An environment for RCDPR obstacle avoidance tasks is built, and then the improved Soft AC algorithm is used to train the robot to avoid obstacles in the environment. Finally, the simulation and experimental study of the paths and trajectories generated by the policy neural network are performed to verify the feasibility and effectiveness of the proposed algorithm. The results show that RCDPR can realize autonomous planning and intelligent obstacle avoidance by DRL.
Planning with Q-Values in Sparse Reward Reinforcement Learning

Learning a policy from sparse rewards is a main challenge in reinforcement learning (RL). The best solutions to this challenge have been via sample inefficient model-free RL algorithms. Model-based RL algorithms are known to be sample efficient but few of them can solve sparse settings. To address these limitations, we present PlanQ, a sample efficient model-based RL framework that resolves sparse reward settings. PlanQ leverages Q-values that encode long-term values and serve as a richer feedback signal to actions than immediate rewards. As such, PlanQ scores rollout returns from its learned model with returns containing Q-values. We verify the efficacy of the approach on robot manipulation tasks whose difficulties range from simple to complex. Our experimental results show that PlanQ enhances performance and efficiency in sparse reward settings.
An Adaptive Robust Student’s t-Based Kalman Filter Based on Multi-sensor Fusion

In practical applications, Kalman filter and its variants such as UKF may suffer from the time-varying measurement noise and process-error. Especially, when the process-error is heavy-tailed probability distribution, the Gaussian assumption would be no longer as accurately as expected. Aiming at the time-varying measurement noise and the situation with heavy-tailed process-error problems, in this paper, a new algorithm is proposed based on Student’s t-distribution and multi-sensor information fusion. The robustness of the proposed algorithm is guaranteed by the timely estimation of the measurement noise, and the adaptiveness is realized by replacing the Gaussian by the Student’s t-distribution. The Kullback-Leible Divergence (KLD) is used as the criterion for distinguishing the Gaussian distribution from the Student’s t-distribution. Finally, a challenging target tracking example is presented and the simulation results show that the proposed algorithm achieves a higher accuracy than the other algorithms.
Realtime Interpersonal Human Synchrony Detection Based on Action Segmentation

IS (Interpersonal Synchrony), where the follower (participant) tries to behave the same action along with the raiser (human or metronome), is an essential social interaction skill. The evaluation of interpersonal synchronization is valuable for early autism screening. However, the research on IS evaluation is limited, and the current approaches usually evaluate the IS task with “motion energy” that is calculated by imprecise corner detection of the participant, which is not robust in an uncontrollable clinical environment. Moreover, these approaches need to manually mark the start and the end anchor of the specified action segment, which is labor-intensive. In this paper, we construct a realtime action segmentation model to automatically recognize the human-wise action class frame by frame. A simple yet efficient backbone is utilized to classify action class straightly instead of extracting the motion features (e.g. optical flow) with high computational complexity. Specifically, given an action video, a sliding window stacks frames in a fixed window size to feed a Resnet-like action classification branch (ACB) to classify the current action label. To further improve the accuracy of action boundary and eliminate the over-segmentation noises, we incorporate a boundary prediction branch (BPB), cooperating with majority-voting strategy, to refine the action classification generated by ACB. Then we can calculate the IS overlap easily by comparing two action timelines belonging to raiser and follower. To evaluate the proposed model, we collect 200K annotated images belonging to 40 subjects who perform 2 tasks (nod and clap) in 2 conditions (interpersonal and human-metronome). The experiment results demonstrate that our model achieves 87.1% accuracy at 200 FPS and can locate the start and end of action precisely in realtime.
A Multi-sensor Combined Tracking Method for Following Robots

At present, the research on tracking methods is mainly based on visual tracking algorithm, which has reduced the accuracy at night or under the condition of insufficient light intensity. Therefore, this paper starts from the direction of multi-sensor combined tracking. Firstly, in order to verify the feasibility and performance of the multi-sensor combined tracking method proposed in this paper, a set of tracking robot system is designed. Secondly, aiming at the problem that the visual tracking method fails to track in scenes such as complete occlusion and insufficient illumination, the non-line-of-sight perception of the following target is realized based on the fusion of ultra-wide band (UWB) and inertial measurement unit (IMU) sensors. Besides, based on coordinate transformation and decision tree algorithm, this paper makes decisions on UWB and visual tracking targets to achieve combined tracking.
Rangefinder-Based Obstacle Avoidance Algorithm for Human-Robot Co-carrying

The mobile robot (follower) in the human-robot co-carrying task needs to follow the human partner (leader) online, and also needs to avoid obstacles. Most existing local reactive obstacle avoidance algorithms cannot be applicable. This paper proposes a rangefinder-based obstacle avoidance algorithm for omnidirectional mobile robots in human-robot co-carrying. The proposed obstacle avoidance algorithm is implemented using a rangefinder, and obstacle avoidance can be achieved combined with controller design on our designed co-carrying system. Finally, simulation results are carried out to show the effectiveness of our proposed obstacle avoidance algorithm.
Memory-Based STOMP for Local Path Planning

Planning and navigation of mobile robots has always been a challenging problem, which has attracted a large number of scholars, especially the research on local path planners. In order to use the past planning experience to guide future path planning, a memory-based stochastic trajectory optimization for motion planning (M-STOMP) is used to solve the local path planning problem. Firstly, the past path planning experience is continuously used to guide the subsequent planning by using memory, which is a method for continuous planning. Then, STOMP algorithm uses Gaussian distribution to generate some smooth paths in the state space, and uses optimized method to update to get a better path. Finally, this method was tested in four different scenarios which validate the proposed method. This paper gave a method for local path planning from a new perspective.
Hybrid Deep Convolutional Network for Face Alignment and Head Pose Estimation

Face alignment has been an important focus of vision research because it is the most fundamental step in face analysis, reconstruction, and applications of emotion and attention. However, face alignment still suffers from some problems, such as lack of stability and poor performance in practical applications due to occlusion, illumination, and high training costs. This paper proposes a Dual-Task Hybrid Deep Convolutional Network (DHDCN) to estimate head pose and facial landmark locations simultaneously. By connecting the multi-level features, the local features and global features can be effectively fused. Features common to both tasks are learned in the initial stages of the network, and later stages will train the two tasks independently. Although the results have some gaps compared to the state-of-the-art results, it also demonstrates the feasibility and potential of learning both tasks simultaneously.
Research on Multi-model Fusion Algorithm for Image Dehazing Based on Attention Mechanism

In recent years, the researchers of image dehazing mainly focused on deep learning algorithms. However, due to the defective network structure, and inadequate feature extraction, the deep learning algorithm still has many problems to be solved. In this paper, we fuse the physical models including haze imaging model with absorption compensation, multiple scattering imaging model and multi-scale retinex imaging model with convolutional neural network to construct the image dehazing network. Multiple scattering haze imaging model is used to describe the haze imaging process in a more consistent way with the physical imaging mechanism. And the multi-scale retinex imaging model ensures the color fidelity. In the network structure, multi-scale feature extraction module can improve network performance in terms of feature reuse. In the attention feature extraction module, the back-propagating of the important front features is used to enhance features. This method can effectively make up for the deficiency that autocorrelation features cannot share the deep-level information, which is also effective for features replenishment. The results of the comparative experiment demonstrate that our network outperforms state-of-the-art dehazing methods.
Research on Formation Obstacle Avoidance Algorithm of Multiple AUVs Based on Interfered Fluid Dynamical System

This paper focuses on planning a two-dimensional (2-D) obstacle avoidance path for the formation of autonomous underwater vehicles (AUVs) in an ocean environment with complicated static obstacles. Inspired by the natural phenomenon of a flowing stream avoiding obstacles, a novel strategy based on an interfered fluid dynamical system (IFDS) is designed. In view of the particular features of the ocean environment, the obstacles are modeled first. Then, by imitating the phenomenon of fluid flow, the IFDS method is used to quickly plan a smooth and safe path for formation AUVs, which conforms to the general characteristic of the phenomenon that running water can avoid rock and arrive at its destination. Finally, formation control is achieved using rigid graph theory. The planned route serves as a known virtual AUV to the leader AUV. The simulation results show that this method has the characteristics of curve continuity and smoothness, enhances obstacle avoidance effects, and has good performance in obstacle avoidance in 2-D path planning.
Vision-Based Fall Detection and Alarm System for Older Adults in the Family Environment

This study proposes an innovative fall detection and alarm system for the elderly in the family environment based on deep learning. The overall cost of hardware development is a camera and an edge device like a Raspberry PI or an old laptop that can detect and alert users to falls without touching the user’s body. The development idea of the system is as follows: 1. Collect the pictures of falling and normal states under different conditions; 2. The improved lightweight SSD-Mobilenet object detection model is used to train the data set and select the optimal weight; 3. Optimal results are deployed on a Raspberry PI 4B device using a lightweight inference engine Paddle Lite. The mean Average Precision of the best model is 92.7%, and the detection speed can reach 14FPS (Frames Per Second) on the development board. When the camera detects that someone has fallen for 10 s, the compiled script sends an alert signal to the default guardian’s email via the Mutt email program on Linux. The experimental results show that the fall detection system achieves satisfactory detection accuracy and comfort.
ResNet-BiGRU-Attention Based Facial Expression Analysis Model for a Humanoid Robot

In recent years, the robot industry has developed rapidly. With the advance of technology, robots have played an important role on human life in the world. In order to achieve more and more natural and intelligent human-machine interaction, the facial expression recognition model based on ResNet-BiGRU-Attention is proposed in this paper. Firstly, RNN was introduced on the basis of ResNet, the model learns shallow features through ResNet, and learns deep features through RNN. The authors use BiGRU, an enhanced version of RNN, to learn the features of data from two aspects: forward and backward. Secondly, the authors also use the attention mechanism to enhance the weight of the key features in the facial expression. The RAF-DB facial emotion dataset samples were used to train the proposed model. At last, the bipedal humanoid robot NAO was used as an experimental platform. The simulation experiment shows that the model can identify the 7 basic expressions, including happy, angry, nausea, fear, sadness, surprise, and nature. It achieves 86.02% recognition accuracy rate. The proposed model provides a reliable solution for the facial expression recognition in humanoid robot vision.
Adaptation of Students to the Digital Space of the Modern World: Problems of Legal Support in Russia

The integration of the individual into a fundamentally new virtual socio-cultural environment, associated with the modernization of existing skills or the acquisition of additional digital competencies necessary for successful functioning and self-realization in a changing society, makes it possible to speak about the emergence of a cyber form of socialization of the individual “digital socialization”. The authors identify the main directions of digital socialization and the main types of digital competencies acquired by users during the educational process using modern information technologies. The article analyzes the results of a specific sociological survey conducted by the authors, illustrating both the dynamics of the development of interests and needs of modern users of information educational resources, and problematic moments in the process of forming digital competencies. The compliance of the legal framework of the Russian Federationwith the prevailing needs and challenges of modern society in the context of the transformation of the modern institute of education is analyzed. The authors conclude that the growing demand for new sets of professional digital competencies in the modern labor market, which has changed under the influence of digitalization and automation of processes and the aftershock of the global pandemic, will inevitably entail further theoretical elaboration and correction of the understanding of the essence of professional digital competencies.
Dynamic Hand Gesture Recognition for Numeral Handwritten via A-Mode Ultrasound

In recent years, due to the defects of weak sEMG signal, insensitive to fine finger movement and serious impression by noise, researchers consider the need to use A-mode ultrasound (AUS) for gesture decoding. However, the current A-mode ultrasonic gesture recognition algorithm is still relatively basic, which can recognize the recognition function of discrete gestures. However, due to the lack of time information, A-mode ultrasound still lacks an algorithm to recognize the dynamic gesture process. Therefore, we design and experiment a deep learning algorithm model applied to AUS signal, which is a deep learning framework based on LSTM. Due to the principle of LSTM, the model sets a certain number of frames as the whole action process, and constructs the connection of each frame in the whole process, so the time correlation (time characteristic) of AUS signal is constructed. Then, the features from AUS signal are sent to the complete full connection layer to output the classification results. And because AUS signal lacks data set of dynamic gestures, we designed and tested handwritten digits 0–9 as an example of dynamic gestures. Experimental results show that this algorithm can realize the dynamic gesture classification of AUS signal and solve the defect of AUS signal lacking time information. In addition, compared with the experimental action of traditional methods, it gives the practical significance of dynamic gesture in life, which is closer to life.
Weakly Supervised Nucleus Segmentation Using Point Annotations via Edge Residue Assisted Network

Cervical cell nucleus segmentation can facilitate computer-assisted cancer diagnostics. Due to the obtaining manual annotations difficulty, weak supervision is more excellent strategy with only point annotations for this task than fully supervised one. We propose a novel weakly supervised learning model by sparse point annotations. The training phase has two major stages for the fully convolutional networks (FCN) training. In the first stage, coarse mask labels generation part obtains initial coarse nucleus regions using point annotations with a self-supervised learning manner. For refining the output nucleus masks, we retrain ERN with an additional constraint by our proposed edge residue map at the second stage. The two parts are trained jointly to improve the performance of the whole framework. As experimental results demonstrated, our model is able to resolve the confusion between foreground and background of cervical cell nucleus image with weakly-supervised point annotations. Moreover, our method can achieves competitive performance compared with fully supervised segmentation network based on pixel-wise annotations.
SCRMA: Snake-Like Robot Curriculum Rapid Motor Adaptation

Controllers for underwater snake-like robots are difficult to design because of their high DOF and complex motions. Additionally, because of the complex underwater environment and insufficient knowledge of hydrodynamics, the traditional control algorithms based on environment or robot modeling cannot work well. In this paper, we propose an SCRMA algorithm, which combines the characteristics of the RMA algorithm for rapid learning and adaptation to the environment, and uses curriculum learning and save &load exploration to accelerate the training speed. Experiments show that the SCRMA algorithm works better than other kinds of reinforcement learning algorithms nowadays.
Vision-Based Action Detection for RTI Protocol of ASD Early Screening

Autism Spectrum Disorder (ASD) is a congenital neurodevelopmental disorder, and the number of ASD has been increasing in recent decades worldwide. Early screening is essential for proper treatment and intervention in toddlers with ASD. However, manual early screening methods for ASD are costly and inefficient. Stereotyped behavior is one of the clinical manifestations of ASD toddlers. In this paper, we propose a vision-based action detection network, named OstAD, for response-to-instruction (RTI) protocol to assist professional clinicians with an early screening. Our network adopts a temporal attention branch to aggregate contextual features, and proposes a spatial attention branch to generate local frame-level features of the toddlers. Experimental results demonstrate that the proposed OstAD model can detect typical actions of ASD toddler with mAP 72.6% and 75.9% accuracy, and achieves the excellent results in the RTI screening.
Multi-robot Cooperation Learning Based on Powell Deep Deterministic Policy Gradient

Model-free deep reinforcement learning algorithms have been successfully applied to a range of challenging sequential decision making and control tasks. However, these methods could not perform well in multi-agent environments due to the instability of teammates’ strategies. In this paper, a novel reinforcement learning method called Powell Deep Deterministic Policy Gradient (PDDPG) is proposed, which integrates Powell’s unconstrained optimization method and deep deterministic policy gradient. Specifically, each agent is regarded as a one-dimensional variable and the process of multi-robot cooperation learning is corresponding to optimal vector searching. A conjugate direction in Powell-method is constructed and is used to update the policies of agents. Finally, the proposed method is validated in a dogfight-like multi-agent environment. The results suggest that the proposed method outperforms much better than independent Deep Deterministic Policy Gradient (IDDPG), revealing a promising way in realizing high-quality independent learning.
Multi-task Facial Landmark Detection Network for Early ASD Screening

Joint attention is an important skill that involves coordinating the attention of at least two individuals towards an object or event in early child development, which is usually absent in children with autism. Children’s joint attention is an essential part of the diagnosis of autistic children. To improve the effectiveness of autism screening, in this paper, we propose a multi-task facial landmark detection network to enhance the stability of gaze estimation and the accuracy of the joint attention screening result. In order to verify the proposed method, we recruit 39 toddlers aged from 16 to 32 months in this study and build a children-based facial landmarks dataset from 19 subjects. Experiments show that the accuracy of the joint attention screening result is 92.5 $$\%$$ , which demonstrates the effectiveness of our method.