Query: A TSR visual servoing system based on a novel dynamic template matching method

Candidate papers:

abstract: The so called Tethered Space Robot (TSR) is a novel active space debris removal system. To solve its problem of non cooperative target recognition during short distance rendezvous events, this paper presents a framework for a real time visual servoing system using non calibrated monocular CMOS (Complementary Metal Oxide Semiconductor) When a small template is used for matching with a large scene, it always leads to mismatches, so a novel template matching algorithm to solve the problem is presented. Firstly, the novel matching algorithm uses a hollow annulus structure according to a FAST (Features from Accelerated Segment) algorithm and makes the method be rotation invariant. Furthermore, the accumulative deviation can be decreased by the hollow structure. The matching function is composed of grey and gradient differences between template and object image, which help it reduce the effects of illumination and noises. Then, a dynamic template update strategy is designed to avoid tracking failures brought about by wrong matching or occlusion. Finally, the system synthesizes the least square integrated predictor, realizing tracking online in complex circumstances. The results of ground experiments show that the proposed algorithm can decrease the need for sophisticated computation and improves matching accuracy.
author_names: Jia Cai

Panfeng Huang

Bin Zhang

Dongke Wang
title: A TSR Visual Servoing System Based on a Novel Dynamic Template Matching Method
venue: Sensors
year: 2015

abstract: This paper presents a delta robotic visual servoing tracking method using zero mean normalized cross correlation (ZNCC) based grayscale template matching hardware core on field programmable gate arrays (FPGAs) The concurrent FPGA based ZNCC hardware core with cascading multiplication accumulate (MAC) circuits is designed, which can largely reduce FPGA hardware resource consumptions. A compact optical imaging system with a front 45deg slant mirror and an optical filter film are proposed, which can efficiently filter out the background cluttered artifacts. The trajectory visual servoing tracking and dynamic tracking experiments based on our built up delta robotic visual tracking platform are implemented. The experimental results indicate that the presented FPGA based embedded robotic visual tracking method can efficiently improve an object trajectory tracking performance.
author_names: Zhong Chen

Shuai Li

Ning Zhang

Yu-xiang Hao

Xianmin Zhang
title: Eye to Hand Robotic Visual Tracking Based on Template Matching on FPGAs
venue: IEEE Access
year: 2019

abstract: The actuation and control of mini scale swimming robots have exhibit promising potential in limited and constrained environments, such as biomedical, micro assembly and microfluidic applications. However, closed loop control is still a challenging for the precision and repeatability in those applications. In this paper, a new type of soft micro swimming robot based on magnetoelastic composite material is designed, and the swimmer can move freely on the water surface controlled by a rotating magnetic field with visual servoing. The visual feedback control strategy is proposed based on Mean Shift Algorithm and Fast Template Matching. The algorithm includes three parts: prediction, template matching, target positioning and template updating. To understand the characteristics of the swimmer, the relationship between swimming speed and magnetic field strength and frequency is verified by the algorithm in experiments. Furthermore, the proposed path following method with visual servoing applied in this control system. The experimental results show that the algorithm runs fast, has high accuracy and good real time performance, and can provide great potential in mini scale robot applications.
author_names: Hongbiao Xiang

Tilei Zhang

Mengwei Li Campbell

Shoujun Wang

Xiuping Yang
title: Visual Servoing of Magnetic Swimming Robot Based on Mean Shift and Fast Template Matching Algorithm
venue: 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO)
year: 2019

abstract: A efficient non contact miniature circuit breaker surface detecting system was presented based on the principle of the machine imaging technology. This visual detection system is aimed at the automatic detection of the installation direction of the test button (T button) of miniature circuit breaker. By analyzing the characteristics of the T button and the detection requirements, the appropriate camera, light mode and lens were determined, and then the proper images were obtained for further processing. To solve the belt seriously jitter, this paper studies the RGB color image space of miniature circuit breaker and component labeling algorithm, and then introduces an algorithm of location of T button. Finally, character template matching was utilized to detect the installation direction of the T button. This system can greatly optimizing the efficiency, and is very suitable for the rapid detection of a large number of miniature circuit breaker. The test results show that the accuracy of detection by this method can reach over 99% and the false detection rate is lower than 1% which meets the practical production requirement.
author_names: He Ping Chen
title: A Novel Miniature Circuit Breaker Surface Detecting System Based on Machine Vision Technology
year: 2019

abstract: This paper proposes a novel nonlinear geometric hierarchical dynamic visual servoing approach to drive a quadrotor to the desired pose defined by a previously captured image of a planar target. Different from existing works, the key novelty is to extend the position based nonlinear hierarchical control to image based nonlinear hierarchical control. More specifically, by seamlessly integrating the nonlinear hierarchical control with the geometric control, and taking full advantage of the cascade property of the system, the proposed visual servoing strategy does not require the thrust force or its derivative to be measurable when compared with the existing backstepping methods, which brings much convenience for practical applications. For the attitude loop, the axis angle rotation representation is adopted to design a tracking control law on the vector space \mathfrak {so}(3) In the outer loop, perspective image moments in the virtual image plane are employed as image feedback to construct the outer loop image based visual servoing controller with geometric control and backstepping techniques. Based on Lyapunov techniques and the theory of cascade systems, it is rigorously proven that the proposed image based controller achieves asymptotic stability. Comparative experiments are conducted to show that the proposed approach has advantages of better transient performance, better steady state performance, and stronger robustness.
author_names: Xuetao Zhang

Yongchun Fang

Xuebo Zhang

Jingqi Jiang
title: A Novel Geometric Hierarchical Approach for Dynamic Visual Servoing of Quadrotors
venue: IEEE Transactions on Industrial Electronics
year: 2020

abstract: A visual servoing scheme consists of a closed loop control approach that uses visual information feedback to control the motion of a robotic system. Probably the most popular visual servoing method is image based visual servoing (IBVS) This kind of method uses geometric visual features extracted from the image to design the control law. However, extracting, matching, and tracking geometric visual features over time significantly limits the versatility of visual servoing controllers in various industrial and medical applications, in particular for "low structured" medical images, e.g. ultrasounds and optical coherence tomography modalities. To overcome the limits of conventional IBVS, one can consider novel visual servoing paradigms known as "direct" or "featureless" approaches. This paper deals with the development of a new generation of direct visual servoing methods in which the signal control inputs are the coefficients of a multiscale image representation. In particular, we consider the use of multiscale image representations that are based on discrete wavelet and shearlet transforms. Up to now, one of the main obstacles in the investigation of multiscale image representations for visual servoing schemes was the issue of obtaining an analytical formulation of the interaction matrix that links the variation of wavelet and shearlet coefficients to the spatial velocity of the camera and the robot. In this paper, we derive four direct visual servoing controllers: two that are based on subsampled respectively non subsampled wavelet coefficients and two that are based on the coefficients of subsampled respectively non subsampled discrete shearlet transforms. All proposed controllers were tested in both simulation and experimental scenarios (using a six degree of freedom Cartesian robot in an eye in hand configuration) The objective of this paper is to provide an analysis of the respective strengths and weaknesses of wavelet and shearlet based visual servoing controllers.
author_names: Lesley-Ann Duflot

Rafael Reisenhofer

Brahim Tamadazte

Nicolas Andreff

Alexandre Krupa
title: Wavelet and shearlet based image representations for visual servoing
venue: Int. J. Robotics Res.
year: 2019

abstract: The authors have developed a bed type non invasive ultrasound theragnostic system (NIUTS) that compensates for movement by tracking and following the area to be treated by stereo ultrasound imaging while irradiating high intensity focused ultrasound (HIFU) onto the affected area. In this paper, we propose a novel robust template matching method to track and follow body targets, which include tumors and stones for the NIUTS. The proposed novel robust template matching method could be applied to a motion tracking of the real human kidney based on the ultrasound images, which is the first successful report as far as I know. This robust visual servoing method could be a great tool to treat the tumors and stones precisely and safely.
author_names: Norihiro Koizumi

Takakazu Funamoto

Joonho Seo

Dongjun Lee

Hiroyuki Tsukihara

Akira Nomiya

Takashi Azuma

Kiyoshi Yoshinaka

Naohiko Sugita

Yukio Homma

Yoichiro Matsumoto

Mamoru Mitsuishi
title: A novel robust template matching method to track and follow body targets for NIUTS
venue: 2014 IEEE International Conference on Robotics and Automation (ICRA)
year: 2014

abstract: In the field of steady state visual evoked potential (SSVEP) based brain computer interface (BCI) the lengthy training time was always an obstacle to practical application. In this paper, we explored a novel method to reduce the training cost by replacing the traditional sinusoidal template or signal template with a dynamic SSVEP model and conducting a sampling training strategy. To evaluate the method, the training time and the recognition accuracy under two conditions (sine/cosine template and dynamic model template) were compared on four different algorithms. The results showed that the dynamic model based template outstripped the sinusoidal template; and for signal template based algorithms, our proposed method reduced the training time significantly while kept the decrease of performance within an insignificant range.
author_names: Xu Han

Shangen Zhang

Xiaorong Gao
title: A study on reducing training time of BCI system based on an SSVEP dynamic model
venue: 2019 7th International Winter Conference on Brain Computer Interface (BCI)
year: 2019

abstract: Aiming at deep profile control in water flooding reservoirs, a heterogeneous combination flooding system, composed of dispersed particle gel (DPG) and a zwitterionic surfactant (THSB) was developed. The mean particle diameter was adjusted and then the DPG morphology and diameter were characterized using a laser scanning, confocal microscope, and dynamic light scattering. The effects of DPG concentration, temperature, pH, and salinity on THSB interfacial tension (IFT) were investigated. A method for determining DPG concentration was proposed, and the dynamic retention and migration mechanisms of the particles were evaluated, using cores with different permeabilities. Finally, the oil displacement mechanism of the combination flooding system was explored through visual simulation experiments and scanning electron microscopy. The results showed that the particle diameter was 2.5 mm, that compared with a surfactant system, the IFT of the combination flooding system was relatively low, and that the temperature, pH, and NaCl content of solutions had little effect on THSB IFT. Since specific matching relationships between DPG particles and reservoir pore throats existed, the retention amount and injection pressure were improved with the increased core permeability. Visual simulation experiments revealed the oil recovery mechanism of the combination flooding system. This combination flooding system can be used in reservoirs under varied conditions, as the particle diameter can be adjusted on a case by case basis, and the THSB surfactant proved to be high temperature resistant and high salinity tolerant. The combination flooding system can be applied to deep profile control and oil displacement, in oil fields.
author_names: Weitao Li

Fa-lin Wei

Chunming Xiong

Jian Ouyang

Guang Zhao

Liming Shao

Mingli Dai
title: A novel binary compound flooding system based on DPG particles for enhancing oil recovery
venue: Arabian Journal of Geosciences
year: 2019

abstract: One of the important tasks of an autonomous mobile vehicle is the reliable and fast estimation of its position over time. This paper presents the development of an adaptive technique to hasten and improve the quality of correlation based template matching for monocular visual odometry systems that estimate the relative motion of ground vehicles in low textured environments. Moreover, the factors that can affect the maximum permissible vehicle driving speed were determined and the related equations were derived. The developed system uses a single downward facing monocular camera installed at an optimum location to avoid the negative effect of directional sunlight and shadows which can disturb the correlation. In addition, the normalized cross correlation method is implemented to calculate the pixel displacement between image frames. Although this method is highly effective for template matching because of its invariance to linear brightness and contrast variations, it incurs high computational cost. Thus, the optimal sizes of image template and matching search area are selected and their locations are dynamically changed according to vehicle acceleration, in order to achieve a compromise between the performance and the computational cost of correlation. The proposed technique increases the allowable vehicle driving speed and reduces the probability of template false matching. Moreover, compared to traditional full search matching techniques, the adaptive technique demonstrates high efficiency and accuracy and improves the quality and speed of the correlation with more than 87% of reduction in computational cost. (c) 2016 Institute of Electrical Engineers of Japan. Published by John Wiley Sons, Inc.
author_names: Mohammad O A Aqel

Mohammad Hamiruce Marhaban

M Iqbal Saripan

Napsiah Ismail
title: Adaptive search template matching technique based on vehicle acceleration for monocular visual odometry system
year: 2016