Reviewerly/exHarmony huggingface dataset:
Research on Cross-Domain Study Curricula in Cyber-Physical Systems: A Case Study of Belarusian and Ukrainian Universities

The increasing importance of continuingly complex Cyber-Physical Systems (CPS) challenges and encourages universities worldwide to organize up-to-date CPS oriented educational programs. The Erasmus+ project “CybPhys” aims to support CPS oriented educational programs in Belarus and Ukraine. We put forward a hypothesis that the educational situation, training opportunities, situation of the labor market, and general economic development are related to each other and can vary depending on the region. In order to validate our hypothesis and to provide input to curricula modernizations, an in-depth study in the form of surveys was conducted in the years 2019–2020. The results of the study showed that the differences between the perceptions of stakeholders in different regions of one country were more significant than the deviations between the global evaluation marks obtained from Belarusian and Ukrainian respondents. In order to increase the synergy of the CybPhys partners, the transdisciplinary and T-shape skills approaches are introduced to the education programs of the partner’s universities. An innovative ICT based teaching and learning environment and associated teaching methodologies will be developed. The research provides valuable input to the development of industry and research-oriented cross-domain study programs in Cyber-Physical Systems focused on the needs of Belarusian and Ukrainian industries.
MM-Hand

Estimating the 3D hand pose from a monocular RGB image is important but challenging. A solution is training on large-scale RGB hand images with accurate 3D hand keypoint annotations. However, it is too expensive in practice. Instead, we develop a learning-based approach to synthesize realistic, diverse, and 3D pose-preserving hand images under the guidance of 3D pose information. We propose a 3D-aware multi-modal guided hand generative network (MM-Hand), together with a novel geometry-based curriculum learning strategy. Our extensive experimental results demonstrate that the 3D-annotated images generated by MM-Hand qualitatively and quantitatively outperform existing options. Moreover, the augmented data can consistently improve the quantitative performance of the state-of-the-art 3D hand pose estimators on two benchmark datasets. The code will be available at https://github.com/ScottHoang/mm-hand.
Activity-driven Weakly-Supervised Spatio-Temporal Grounding from Untrimmed Videos

In this paper, we study the problem of weakly-supervised spatio-temporal grounding from raw untrimmed video streams. Given a video and its descriptive sentence, spatio-temporal grounding aims at predicting the temporal occurrence and spatial locations of each query object across frames. Our goal is to learn a grounding model in a weakly-supervised fashion, without the supervision of both spatial bounding boxes and temporal occurrences during training. Existing methods have been addressed in trimmed videos, but their reliance on object tracking will easily fail due to frequent camera shot cut in untrimmed videos. To this end, we propose a novel spatio-temporal multiple instance learning framework for untrimmed video grounding. Spatial MIL and temporal MIL are mutually guided to ground each query to specific spatial regions and the occurring frames of a video. Furthermore, an activity described in the sentence is captured to use the informative contextual cues for region proposals refinement and text representation. We conduct extensive evaluation on YouCookII and RoboWatch datasets, and demonstrate our method outperforms state-of-the-art methods.
Presence, Embodied Interaction and Motivation

The use of immersive virtual environments (IVEs) for educational purposes has increased in recent years, but the mechanisms through which they contribute to learning is still unclear. Popular explanations for the learning benefits brought by IVEs come from motivation, presence and embodied perspectives; either as individual benefits or through mediation effects on each other. This paper describes an experiment designed to interrogate these approaches, and provides evidence that embodied controls and presence encourage learning in immersive virtual environments, but for distinct,non-interacting reasons, which are also not explained by motivational benefits.
Nonintrusive Fine-Grained Home Care Monitoring: Characterizing Quality of In-Home Postural Changes Using Bone-Based Human Sensing

In contrast to the physical activities of able-bodied people at home, most people who require long-term specific care (e.g., bedridden patients and patients who have difficulty walking) usually show more low-intensity slow physical activities with postural changes. Although the existing devices can detect data such as heart rate and the number of steps, they have been increasing the physical burden relying on long-term wearing. The purpose of this paper is to realize a noninvasive fine-grained home care monitoring system that is sustainable for people requiring special care. In the proposed method, we present a novel technique that integrates inexpensive camera devices and bone-based human sensing technologies to characterize the quality of in-home postural changes. We realize a local process in feature data acquisition once per second, which extends from a computer browser to Raspberry Pi. Our key idea is to regard the changes of the bounding box output by standalone pose estimation models in the shape and distance as the quality of the pose conversion, body movement, and positional changes. Furthermore, we use multiple servers to realize distributed processing that uploads data to implement home monitoring as a web service. Based on the experimental results, we conveyed our findings and advice to the subject that include where the daily living habits and the irregularity of home care timings needed improvement.
Controlling the autonomous vehicle using computer vision and cloud server

Implementing the unmanned cars has been given much importance from the recent past years. These cars have been designed by the automobile industries and are in the stage to test their rides on the roads. With being implemented with automated cars, the problem arises when there is any technical issue of how the backup works. So, in this paper, we are presenting the implementation of the miniature model of the automated robotic vehicle using the Raspberry Pi and how the data can be sent to the cloud. So that a person can be able to observe and maintain the Speed, Distance Covered and the Movement of the Vehicle on the Roads of the city. This model is designed on the applications of the safety measures to be taken for these unmanned vehicles that are almost a reality in the coming days. The Raspberry Pi development board is used as the core processor, camera module for the identification of the traffic signals and the Ultrasonic sensor for the identification of any of the vehicle nearby, and if so the direction of the vehicle changes from which it is moving and all the remaining data regarding the data posted in the cloud is well described through the paper.
Visual Relation of Interest Detection

In this paper, we propose a novel Visual Relation of Interest Detection (VROID) task, which aims to detect visual relations that are important for conveying the main content of an image, motivated from the intuition that not all correctly detected relations are really "interesting" in semantics and only a fraction of them really make sense for representing the image main content. Such relations are named Visual Relations of Interest (VROIs). VROID can be deemed as an evolution over the traditional Visual Relation Detection (VRD) task that tries to discover all visual relations in an image. We construct a new dataset to facilitate research on this new task, named ViROI, which contains 30,120 images each with VROIs annotated. Furthermore, we develop an Interest Propagation Network (IPNet) to solve VROID. IPNet contains a Panoptic Object Detection (POD) module, a Pair Interest Prediction (PaIP) module and a Predicate Interest Prediction (PrIP) module. The POD module extracts instances from the input image and also generates corresponding instance features and union features. The PaIP module then predicts the interest score of each instance pair while the PrIP module predicts that of each predicate for each instance pair. Then the interest scores of instance pairs are combined with those of the corresponding predicates as the final interest scores. All VROI candidates are sorted by final interest scores and the highest ones are taken as final results. We conduct extensive experiments to test effectiveness of our method, and the results show that IPNet achieves the best performance compared with the baselines on visual relation detection, scene graph generation and image captioning.
Pay Attention Selectively and Comprehensively

Deep neural network with multi-scale feature fusion has achieved great success in human pose estimation. However, drawbacks still exist in these methods: 1) they consider multi-scale features equally, which may over-emphasize redundant features; 2) preferring deeper structures, they can learn features with the strong semantic representation, but tend to lose natural discriminative information; 3) to attain good performance, they rely heavily on pretraining, which is time-consuming, or even unavailable practically. To mitigate these problems, we propose a novel comprehensive recalibration model called Pyramid GAting Network (PGA-Net) that is capable of distillating, selecting, and fusing the discriminative and attention-aware features at different scales and different levels (i.e., both semantic and natural levels). Meanwhile, focusing on fusing features both selectively and comprehensively, PGA-Net can demonstrate remarkable stability and encouraging performance even without pre-training, making the model can be trained truly from scratch. We demonstrate the effectiveness of PGA-Net through validating on COCO and MPII benchmarks, attaining new state-of-the-art performance. https://github.com/ssr0512/PGA-Net
Deep Learning Approach for Intelligent Named Entity Recognition of Cyber Security

In recent years, the amount of Cyber Security data generated in the form of unstructured texts, for example, social media resources, blogs, articles, and so on has exceptionally increased. Named Entity Recognition (NER) is an initial step towards converting this unstructured data into structured data which can be used by a lot of applications. The existing methods on NER for Cyber Security data are based on rules and linguistic characteristics. A Deep Learning (DL) based approach embedded with Conditional Random Fields (CRFs) is proposed in this paper. Several DL architectures are evaluated to find the most optimal architecture. The combination of Bidirectional Gated Recurrent Unit (Bi-GRU), Convolutional Neural Network (CNN), and CRF performed better compared to various other DL frameworks on a publicly available benchmark dataset. This may be due to the reason that the bidirectional structures preserve the features related to the future and previous words in a sequence.
Cross-resolution face recognition adversarial attacks

Face Recognition is among the best examples of computer vision problems where the supremacy of deep learning techniques compared to standard ones is undeniable. Unfortunately, it has been shown that they are vulnerable to adversarial examples - input images to which a human imperceptible perturbation is added to lead a learning model to output a wrong prediction. Moreover, in applications such as biometric systems and forensics, cross-resolution scenarios are easily met with a non-negligible impact on the recognition performance and adversary’s success. Despite the existence of such vulnerabilities set a harsh limit to the spread of deep learning-based face recognition systems to real-world applications, a comprehensive analysis of their behavior when threatened in a cross-resolution setting is missing in the literature. In this context, we posit our study, where we harness several of the strongest adversarial attacks against deep learning-based face recognition systems considering the cross-resolution domain. To craft adversarial instances, we exploit attacks based on three different metrics, i.e., L1, L2, and L∞, and we study the resilience of the models across resolutions. We then evaluate the performance of the systems against the face identification protocol, open- and close-set. In our study, we find that the deep representation attacks represents a much dangerous menace to a face recognition system than the ones based on the classification output independently from the used metric. Furthermore, we notice that the input image’s resolution has a non-negligible impact on an adversary’s success in deceiving a learning model. Finally, by comparing the performance of the threatened networks under analysis, we show how they can benefit from a cross-resolution training approach in terms of resilience to adversarial attacks.
Staged Animation Strategies for Online Dynamic Networks

Dynamic networks-networks that change over time-can be categorized into two types: offline dynamic networks, where all states of the network are known, and online dynamic networks, where only the past states of the network are known. Research on staging animated transitions in dynamic networks has focused more on offline data, where rendering strategies can take into account past and future states of the network. Rendering online dynamic networks is a more challenging problem since it requires a balance between timeliness for monitoring tasks-so that the animations do not lag too far behind the events-and clarity for comprehension tasks-to minimize simultaneous changes that may be difficult to follow. To illustrate the challenges placed by these requirements, we explore three strategies to stage animations for online dynamic networks: time-based, event-based, and a new hybrid approach that we introduce by combining the advantages of the first two. We illustrate the advantages and disadvantages of each strategy in representing low- and high-throughput data and conduct a user study involving monitoring and comprehension of dynamic networks. We also conduct a follow-up, think-aloud study combining monitoring and comprehension with experts in dynamic network visualization. Our findings show that animation staging strategies that emphasize comprehension do better for participant response times and accuracy. However, the notion of “comprehension” is not always clear when it comes to complex changes in highly dynamic networks, requiring some iteration in staging that the hybrid approach affords. Based on our results, we make recommendations for balancing event-based and time-based parameters for our hybrid approach.
Energy‐optimal transport trajectory planning and online trajectory modification for holonomic robots

Abstract To maximize energy efficiency, we suggest minimum‐energy trajectory planning and online trajectory modification algorithms for three‐wheeled omni‐directional mobile robots (TOMRs). First, minimum‐energy multisection trajectory planning for various constraints is performed using Pontryagin's minimum principle. Two strategies are established to formulate an efficient minimum‐energy, multisection trajectory generation algorithm: an online trajectory modification and a control algorithm that maintains energy efficiency are investigated using novel heuristic functions. Compared with the conventional trajectory using the trapezoidal velocity profile, the generated transport trajectory can save energy.
Complementary-View Co-Interest Person Detection

Fast and accurate identification of the co-interest persons, who draw joint interest of the surrounding people, plays an important role in social scene understanding and surveillance. Previous study mainly focuses on detecting co-interest persons from a single-view video. In this paper, we study a much more realistic and challenging problem, namely co-interest person~(CIP) detection from multiple temporally-synchronized videos taken by the complementary and time-varying views. Specifically, we use a top-view camera, mounted on a flying drone at a high altitude to obtain a global view of the whole scene and all subjects on the ground, and multiple horizontal-view cameras, worn by selected subjects, to obtain a local view of their nearby persons and environment details. We present an efficient top- and horizontal-view data fusion strategy to map multiple horizontal views into the global top view. We then propose a spatial-temporal CIP potential energy function that jointly considers both intra-frame confidence and inter-frame consistency, thus leading to an effective Conditional Random Field~(CRF) formulation. We also construct a complementary-view video dataset, which provides a benchmark for the study of multi-view co-interest person detection. Extensive experiments validate the effectiveness and superiority of the proposed method.
Single Image Deraining via Scale-space Invariant Attention Neural Network

Image enhancement from degradation of rainy artifacts plays a critical role in outdoor visual computing systems. In this paper, we tackle the notion of scale that deals with visual changes in appearance of rain steaks with respect to the camera. Specifically, we revisit multi-scale representation by scale-space theory, and propose to represent the multi-scale correlation in convolutional feature domain, which is more compact and robust than that in pixel domain. Moreover, to improve the modeling ability of the network, we do not treat the extracted multi-scale features equally, but design a novel scale-space invariant attention mechanism to help the network focus on parts of the features. In this way, we summarize the most activated presence of feature maps as the salient features. Extensive experiments results on synthetic and real rainy scenes demonstrate the superior performance of our scheme over the state-of-the-arts. The source code of our method can be found in: https://github.com/pangbo1997/RainRemoval.
Deep Learning Architecture to Assist With Steering a Powered Wheelchair

This paper describes a novel Deep Learning architecture to assist with steering a powered wheelchair. A rule-based approach is utilized to train and test a Long Short Term Memory (LSTM) Neural Network. It is the first time a LSTM has been used for steering a powered wheelchair. A disabled driver uses a joystick to provide desired speed and direction, and the Neural Network provides a safe direction for the wheelchair. Results from the Neural Network are mixed with desired speed and direction to avoid obstacles. Inputs originate from a joystick and from three ultrasonic transducers attached to the chair. The resultant course is a blend of desired directions and directions that steer the chair to avoid collision. A rule-based approach is used to create a training and test set for the Neural Network system and applies deep learning to predict a safe route for a wheelchair. The user can over-ride the new system if necessary.
Morpho-chronology of monogenetic scoria cones from their level contour curves. Applications to the Chichinautzin monogenetic field, Central Mexico

The genesis, distribution and hazards associated with monogenetic volcanic fields pose particular difficulties derived from their dispersed character. Establishing a relationship between the spatial and the age distributions of volcanic cones is necessary both to understand the nature of monogenetic field formation and evolution and to set supported hazard-assessment criteria. We present here a new method –specifically developed to quantify the degree of age-dependent erosion of individual scoria cones– aimed to estimate the relative ages of undated scoria cones in a monogenetic volcanic field from a contour level morphometric analysis of the field digital elevation model (DEM). This method is based on the hypothesis that age and the degree and type of erosion causing significant morphological changes on scoria cones are markedly correlated. It is assumed that, at the time of their formation, monogenetic volcanoes have diverse, yet smooth shapes, which mechanical processes of wearing gradually change in different spatial scales. This evolution is largely evidenced both as changes of the volcano slopes and as height-dependent undulations of the cone surface generated by the hydrometeorological ablation in the otherwise uniform elevation contour lines. Here, we characterize the time-dependence of the shape and size of the altitude isoline undulations by means of the Elliptical Fourier Descriptor spectra of the contour curves of volcanic cones. To test the effects of the DEM data type and resolution on this morpho-chronological dating, we analyze the correlations of published radiometric ages of cones with two datasets of contour curves from the Sierra Chichinautzin monogenetic volcanic field in Central Mexico (SCVF): one obtained from airborne LIDAR surveys (INEGI, Mexico) and the other from a satellite radar mission (DLR, Germany).
Pose-native Network Architecture Search for Multi-person Human Pose Estimation

Multi-person pose estimation has achieved great progress in recent years, even though, the precise prediction for occluded and invisible hard keypoints remains challenging. Most of the human pose estimation networks are equipped with an image classification-based pose encoder for feature extraction and a handcrafted pose decoder for high-resolution representations. However, the pose encoder might be sub-optimal because of the gap between image classification and pose estimation. The widely used multi-scale feature fusion in pose decoder is still coarse and cannot provide sufficient high-resolution details for hard keypoints. Neural Architecture Search (NAS) has shown great potential in many visual tasks to automatically search efficient networks. In this work, we present the Pose-native Network Architecture Search (PoseNAS) to simultaneously design a better pose encoder and pose decoder for pose estimation. Specifically, we directly search a data-oriented pose encoder with stacked searchable cells, which can provide an optimum feature extractor for the pose specific task. In the pose decoder, we exploit scale-adaptive fusion cells to promote rich information exchange across the multi-scale feature maps. Meanwhile, the pose decoder adopts a Fusion-and-Enhancement manner to progressively boost the high-resolution representations that are non-trivial for the precious prediction of hard keypoints. With the exquisitely designed search space and search strategy, PoseNAS can simultaneously search all modules in an end-to-end manner. PoseNAS achieves state-of-the-art performance on three public datasets, MPII, COCO, and PoseTrack, with small-scale parameters compared with the existing methods. Our best model obtains 76.7% mAP and 75.9% mAP on the COCO validation set and test set with only 33.6M parameters. Code and implementation are available at https://github.com/for-code0216/PoseNAS.