Miracle-dz/UniFAR-FTUs huggingface dataset:
IoT Device Fingerprint using Deep Learning

Device Fingerprinting (DFP) is the identification of a device without using its network or other assigned identities including IP address, Medium Access Control (MAC) address, or International Mobile Equipment Identity (IMEI) number. DFP identifies a device using information from the packets which the device uses to communicate over the network. Packets are received at a router and processed to extract the information. In this paper, we worked on the DFP using Inter Arrival Time (IAT). IAT is the time interval between the two consecutive packets received. This has been observed that the IAT is unique for a device because of different hardware and the software used for the device. The existing work on the DFP uses the statistical techniques to analyze the IAT and to further generate the information using which a device can be identified uniquely. This work presents a novel idea of DFP by plotting graphs of IAT for packets with each graph plotting 100 IATs and subsequently processing the resulting graphs for the identification of the device. This approach improves the efficiency to identify a device DFP due to achieved benchmark of the deep learning libraries in the image processing. We configured Raspberry Pi to work as a router and installed our packet sniffer application on the Raspberry Pi. The packet sniffer application captured the packet information from the connected devices in a log file. We connected two Apple devices iPad4 and iPhone 7 Plus to the router and created IAT graphs for these two devices. We used Convolution Neural Network (CNN) to identify the devices and observed the accuracy of 86.7%.
A Particle Filtering Based Approach for Transformer Winding Degradation Prognostics

When the transformer works for a long time, its winding is gradually deteriorated with time, and the fault phenomena such as winding short circuit or circuit break lead to serious power supply accidents. Under high temperature conditions, this paper analyzes the degradation process of the winding, and determines that the resonant frequency can be used as testing index of its degradation process. Therefore, the resonant frequency is used to monitor the performance degradation state of the transformer winding and realize the advance prediction, which can effectively avoid accidents. Accurate prediction of system reliability is of plenty of importance to engineering systems for accomplishing the designate function and system safety management. As the concerned system is getting complicated and more sufficient health monitoring measurement is available, the traditional reliability prediction schemes resorting to only one kind of prediction approaches, model-based or data-driven, begin to show their limitations. This paper proposes a PF prognostic method by combining traditional model approaches. The effectiveness of the proposed method is verified by thermal degradation experiments. This method improves the reliability of power system and is conducive to the rapid development of smart grid.
Work-in-Progress: Joint Network and Computing Resource Scheduling for Wireless Networked Control Systems

Real-time task scheduling for wireless networked control systems provides guarantees for the quality of service. This paper introduces a new model for joint network and computing resource scheduling (JNCRS) in real-time wireless networked control systems. This new end-to-end real-time task model considers a strict execution order of segments including the sensing, the computing and the actuating segment based on the control loop of WNCSs. The general JNCRS problem is proved to be a NP-hard problem. After dividing the JNCRS problem into four subproblems, we propose a polynomial-time optimal algorithm to solve the first subproblem where each segment has unit execution time, by checking the intervals with 100% network resource utilization and modify the deadlines of tasks. To solve the second subproblem where the computing segment is larger than one unit execution time, we define the new timing parameters of each network segment by taking into account the scheduling of the computing segments. We propose a polynomial-time optimal algorithm to check the intervals with the network resource utilization larger than or equal to 100% and modify the timing parameters of tasks based on these intervals.
Online Learning of Body Orientation Control on a Humanoid Robot Using Finite Element Goal Babbling

How can high dimensional robots learn general sets of skills from experience in the real world? Many previous approaches focus on maximizing a single utility function and require large datasets of experience to do this, something that is not possible to collect outside of simulation as every data point is expensive both in time and in a potential wear down of the robot. This paper addresses this question using a newly developed framework called Finite Element Goal Babbling (FEGB). FEGB is an online learning method that aims at providing general control over some measurable feature, in contrast to optimizing it to some given utility function. It generalizes standard goal babbling by breaking down the full learning problem into local sub-problems, and combining it with a planner that learns how to navigate between these subproblems. We test FEGB using a real humanoid robot Nao, and find that it could quickly learn to robustly control its body orientation. After only 20–30 minutes of training, the robot could freely move into any body orientation between lying on either side and on its back. Rapid learning of body orientation control in high dimensional real robots is largely an unexplored field of robotics, and although many challenges remain, FEGB shows a feasible approach to the problem.
Adaptive Task Planner for Performing Home Service Tasks in Cooperation with a Human

To perform a home service task through cooperation with a human in a real environment, a robot needs to deal with the environmental changes and accordingly plan appropriate behavior sequence. For this purpose, in this paper, we propose an adaptive task planner which is based on memory and reasoning. A robot perceives user behaviors and objects using an RGB-depth and thermal sensor. The robot stores a temporal sequence of behaviors for performing a task in its episodic memory that is realized by a sequence to sequence network. When the user command is given, the episodic memory is used to retrieve the behavior sequence to carry out the command. On the other hand, when the robot perceives user behaviors, the robot postpones its behavior till his/her behavior is stopped. Once stopped, the episodic memory retrieves the behavior sequence to conduct a task that the user has intended. A task scheduler schedules the behavior sequence from the memory and sends it to an internal simulator. The internal simulator confirms the behavior sequence to be executable and then if executable, it sends the next executable behavior to the execution module. If a behavior fails in the internal simulation test, fast forward planner generates an alternative behavior sequence to resolve the failed behavior problem. The effectiveness and applicability of the proposed planner is demonstrated by a wheel-based humanoid robot.
Mobile Robot Localization Considering Class of Sensor Observations

Localization robustness against environment dynamics is significant for robots to achieve autonomous navigation in unmodified environments. A basic method of improving the robustness of a robot is considering the sensor observations obtained from mapped obstacles and using them for localizing the robot's pose. This study proposes an observation model that considers the class of sensor observations, where “class” categorizes the sensor observations as those obtained from mapped and unmapped obstacles. In the proposed approach, the robot's pose and the class are estimated simultaneously. As a result, the robot's pose can be localized using the sensor observations obtained only from mapped obstacles. First, we evaluated the performance of the proposed approach using simulations. Further, we tested the proposed approach in a real-world mobile robot navigation competition, called “Tsukuba Challenge,” held in Japan. The robustness and effectiveness of the proposed approach against environment dynamics were verified from the experimental results.
Personal Mobility Vehicle Autonomous Navigation Through Pedestrian Flow: A Data Driven Approach for Parameter Extraction

In this paper we present a data driven approach for safe and smooth autonomous navigation of a personal mobility vehicle (PMV) when facing moving obstacles such as people and bicycles in public pedestrian paths. In a period of three months, data from five different persons driving the robotic PMV in an outdoor environment while facing pedestrians were collected. 2465 clean tracks around the vehicle together with PMVs trajectories were collected. We performed an analysis of the parameters involved for human-driven smooth navigation. Relevant parameters regarding PMV-Human interaction included distance to moving objects, passing side and velocities. Moreover, data suggests the existence of a social navigational distance for the PWv. For autonomous navigation we implemented a Frenet planner to achieve safe and smooth navigation for the passenger and pedestrians around. Experimental results in real pedestrian paths show that the PMV is capable of smoothly following its path while facing pedestrians and bicycles.
Towards View-Invariant Intersection Recognition from Videos using Deep Network Ensembles

This paper strives to answer the following question: Is it possible to recognize an intersection when seen from different road segments that constitute the intersection? An intersection or a junction typically is a meeting point of three or four road segments. Its recognition from a road segment that is transverse to or 180 degrees apart from its previous sighting is an extremely challenging and yet a very relevant problem to be addressed from the point of view of both autonomous driving as well as loop detection. This paper formulates this as a problem of video recognition and proposes a novel LSTM based Siamese style deep network for video recognition. For what is indeed a challenging problem and the limited annotated dataset available we show competitive results of recognizing intersections when approached from diverse viewpoints or road segments. Specifically, we tabulate effective recognition accuracy even as the approaches to the intersection being compared are disparate both in terms of viewpoints and weather/illumination conditions. We show competitive results on both synthetic yet highly realistic data mined from the gaming platform GTA as well as on real world data made available through Mapillary.
Machine Learning Regression Techniques for the Silage Maize Yield Prediction Using Time-Series Images of Landsat 8 OLI

Machine learning (ML) techniques have been utilized for the crop monitoring and yield estimation/prediction using remotely sensed data. However, these methods have been investigated less for yield prediction of some crops, such as silage maize, which can be cultivated at various times in different fields of an area. Inconsistency between fields for satellite-derived normalized difference vegetation index (NDVI) temporal profiles can lead to some difficulties in yield prediction methods using time series of remotely sensed data. Therefore, this research has investigated silage maize yield prediction based on time series of NDVI dataset derived from Landsat 8 OLI. This paper employed advanced ML techniques including boosted regression tree (BRT), random forest regression (RFR), support vector regression, and Gaussian process regression (GPR) approaches and compared their performance with some proposed conventional regression methods. For this purpose, the NDVI values of all silage maize fields were averaged and integrated to produce a two-dimensional dataset for each year. The ML techniques were employed 100 times and their evaluation metrics were used to evaluate their performances and also analyze their stability. Finally, all the results of each ML technique were averaged to produce silage maize yields. The comparisons between the results of these methods indicate that the BRT technique, with the average $R$ value higher than 0.87, outperforms other ones for all years. It was followed by RFR with almost same performance as GPR technique. This research demonstrated that some advanced ML approaches can predict the silage maize yield and they are less sensitive to inconsistency of NDVI time series. The results also showed that RFR was the most stable method to predict the maize yield in 2015, while it was trained using 2013–2014 dataset.
The Application Development for Measuring and Evaluating ECG Data for Home Health Care Using Smart Clothes

In this paper, we present a development of a mobile application for Electrocardiogram evaluation integrated, with smart clothing via a Bluetooth connection. The objectives of the application are wireless data collection and analysis of electrocardiogram. The analysis is aimed for detection of Q, R and S parameters. Measurement and evaluation of parameters can allow detection of regularity and rhythm of heartbeats. Early warning can help to decide for preventive actions to avoid a lot of heart diseases. The application stores recorded data in the form of the format of arrythmia database to be compatible with the professional medical software for further analysis of electrocardiogram records. Also, we proposed a simplified version of Pan and Tompkins algorithm. We reduced a delay in signal analysis at the cost of reducing the accuracy of the original algorithm. This solution will be devoted to the purposes of research and home health care instead of clinical diagnosis.
Enhanced Association With Supervoxels in Multiple Hypothesis Tracking

Remarkable progress has been made in the field of multi-object tracking. Although tracking-by-detection has recently became one of the most popular frameworks, it still has one main drawback: this approach relies heavily on the quality of detection. Thus, the missing detections caused by partial occlusion usually lead to fragment problem. To address this problem, this paper introduces supervoxels to represent objects with partial occlusion, even for missing detections. We first extract superpixels of the foreground, and then our proposed supervoxel consists of spatial-temporal sequences of superpixels. The supervoxels represent tracklets at the image level, so it is robust for initial detection. Then, we incorporate supervoxels into multiple hypotheses tracking by considering the enhanced association with supervoxels (EAS). Moreover, we propose a detection refinement method based on EAS. As our approach allows us to handle partial occlusion problems, we achieve remarkable results in crowded scenes. Finally, our experiments on both MOT15 and MOT16 benchmarks show that our EAS is competitive with the state-of-the-art trackers.
Hybrid Bio-Inspired Architecture for Walking Robots Through Central Pattern Generators Using Open Source FPGAs

In this paper we present a new robotic control approach inspired in the animal nervous system control. The system implements the binomial Brain-Peripheral Nervous System (CNS-PNS) combining the use of microprocessors as the high level control, or brain, and FPGAs as the low level control, or nervous system. Thanks to the new open source tools for FPGAs, we are able to apply them in the field of robotics in new ways that were impossible before. In this paper, we will demonstrate that our approach is not only able to control the movements of robots using digital circuits built inside an FPGA, but is also capable of generating, synthesizing and uploading them inside the FPGA in real time and on demand.
Distributed Pressure Sensing for Enabling Self-Aware Autonomous Aerial Vehicles

Autonomous aerial transportation will be a fixture of future robotic societies, simultaneously requiring more stringent safety requirements and fewer resources for characterization than current commercial air transportation. More robust, adaptable, self-state estimation will be necessary to create such autonomous systems. We present a modular, scalable, distributed pressure sensing skin for aerodynamic state estimation of a large, flexible aerostructure. This skin used a network of 22 nodes that performed in situ computation and communication of data collected from 74 pressure sensors, which were embedded into the skin panels of an ultra-lightweight 14-foot wingspan made from commutable, lattice-based subcomponents, and tested at NASA Langley Research Center's 14X22 wind tunnel. The density of the pressure sensors allowed for the use of a novel distributed algorithm to generate estimates of the wing lift contribution that were more accurate than the direct integration of the pressure distribution over the wing surface.
Work-in-Progress: From Logical Time Scheduling to Real-Time Scheduling

Scheduling is a central yet challenging problem in real-time embedded systems. The Clock Constraint Specification Language (CCSL) provides a formalism to specify logical constraints of events in real-time embedded systems. A prerequisite for the events is that they must be schedulable under constraints. That is, there must be a schedule which controls all events to occur infinitely often. Schedulability analysis of CCSL raises important algorithmic problems such as computational complexity and design of efficient decision procedures. In this work, we compare the scheduling problems of CCSL specifications to the real-time scheduling problem. We show how to encode a simple task model in CCSL and discuss some benefits and differences compared to more classical scheduling strategies.
Scan Similarity-based Pose Graph Construction method for Graph SLAM

Scan similarity-based pose graph construction method for graph SLAM is proposed. To perform delicate pose graph SLAM, front-end that constructs a graph as well as back-end that optimizes the constructed graph is an important task. Generally, there is an error accumulation phenomenon during the odometry estimation process. This paper focuses on the method of creating a high quality graph by suggesting ways to improve the graph accuracy since the accumulated errors in the graph might degrade the performance of the entire graph SLAM. We deal with one of our previous works, dynamic keyframe selection technique, based on scan similarity computation method more precisely and suggest a loop closure detection method by exploiting previously proposed 2-D laser scan descriptor. To verify objective performance of the proposed method, the experimental results of the odometry estimation are shown by using the benchmark dataset and the real world dataset. Additionally, results of the pose graph SLAM are shown for the real world dataset which include the loop clorues.
Video Motion Capture from the Part Confidence Maps of Multi-Camera Images by Spatiotemporal Filtering Using the Human Skeletal Model

This paper discusses video motion capture, namely, 3D reconstruction of human motion from multi-camera images. After the Part Confidence Maps are computed from each camera image, the proposed spatiotemporal filter is applied to deliver the human motion data with accuracy and smoothness for human motion analysis. The spatiotemporal filter uses the human skeleton and mixes temporal smoothing in two-time inverse kinematics computations. The experimental results show that the mean per joint position error was 26.1mm for regular motions and 38.8mm for inverted motions.
Detection and Pose Estimation for Short-Range Vision-Based Underwater Docking

The potential of using autonomous underwater vehicles (AUVs) for underwater exploration is confined by its limited on-board battery energy and data storage capacity. This problem has been addressed using docking systems by underwater recharging and data transfer for AUVs. In this paper, we propose a vision-based framework by addressing the detection and pose estimation problems for short-range underwater docking using these systems. For robust and credible detection of docking stations, we propose a convolutional neural network called docking neural network (DoNN). For accurate pose estimation, a perspective-n-point algorithm is integrated into our framework. In order to examine our framework in underwater docking tasks, we collected a dataset of 2D images, named underwater docking images dataset (UDID), which is the first publicly available underwater docking dataset to the best of our knowledge. In the field experiments, we first evaluate the performance of DoNN on the UDID and its deformed variations. Next, we examine the pose estimation module by ground and underwater experiments. At last, we integrate our proposed vision-based framework with an ultra-short baseline acoustic sensor, to demonstrate the efficiency and accuracy of our framework by performing experiments in a lake. The experimental results show that the proposed framework is able to detect docking stations and estimate their relative pose more efficiently and successfully, compared with the state-of-the-art baseline systems.
Semantic Grid Estimation with a Hybrid Bayesian and Deep Neural Network Approach

In an autonomous vehicle setting, we propose a method for the estimation of a semantic grid, i.e. a bird's eye grid centered on the car's position and aligned with its driving direction, which contains high-level semantic information about the environment and its actors. Each grid cell contains a semantic label with divers classes, as for instance {Road, Vegetation, Building, Pedestrian, Car…}. We propose a hybrid approach, which combines the advantages of two different methodologies: we use Deep Learning to perform semantic segmentation on monocular RGB images with supervised learning from labeled groundtruth data. We combine these segmentations with occupancy grids calculated from LIDAR data using a generative Bayesian particle filter. The fusion itself is carried out with a deep neural network, which learns to integrate geometric information from the LIDAR with semantic information from the RGB data. We tested our method on two datasets, namely the KITTI dataset, which is publicly available and widely used, and our own dataset obtained with our own platform, equipped with a LIDAR and various sensors. We largely outperform baselines which calculate the semantic grid either from the RGB image alone or from LIDAR output alone, showing the interest of this hybrid approach.
Protective Clothing Detection of Substation Workers Using S-HOG+C Operator

This paper focuses on the application of the protective clothing detection system by using S-HOG+C operator for substation workers in order to meet the practical needs in a complex environment. An image is firstly divided into three cells including a helmet, upper and lower part of a body according to the characteristics of objects. The HOG and HOC features of the three cells are extracted individually to train a classifier. A linear Support Vector Machine is used in our study. The S-HOG+C operator is used to improve the detection accuracy on the protective clothing of substation workers. Three evaluation indicators are used to analyze the performance of the S-HOG+C operator. The experimental results suggest that the S-HOG+C operator perform better than the HOG+C operator in the protective clothing detection for substation workers. This work is supported by Science and Technology Project of Guangdong Power Grid Co., Ltd. (GDKJXM20162351).