SUGISAKAMASANORI

Journal of Advances in Artificial Life Robotics
Vol. 4(2); September (2023), pp. 116–121
ON LINE ISSN 2435-8061; ISSN-L 2435-8061
https://alife-robotics.org/jallr.html

Research Article

Design of an Intelligent Electronic Guide Dog Based on Jetson
Nano
Zhencheng Chang, Yuran Lu, Ziyue Xiao, YanDe Xiang

College of Electronic Information and Automation, Tianjin University of Science and Technology, 300222, China

ARTICLE INFO
Article History
Received 15 November 2022
Accepted 03 November 2023
Keywords
Obstacle avoidance lJetson Nano
GPS navigation
Ultrasonic sensor
Visual identity

ABSTRACT

Blind travel has always been the core issue of social concern, and how to assist the blind to travel
safely has always been an urgent problem to be solved urgently. Although some intelligent blind
guide products have flooded into the market in recent years, such as guide boxes, guide battles,etc., but most of their functions only stay in the obstacle avoidance stage, lack of path planning,
intelligent recognition functions, blind person positioning and are not easy to carry. Based on theabove shortcomings, an intelligent mobile blind guide robot is designed in this paper. The blind
guide robot is equipped with a variety of sensors and combined with the powerful computing
power of the main control board to form a complete obstacle avoidance system to achieveautonomous navigation and obstacle avoidance in a complex environment. It controls GPS
positioning through Jetson-Nano, plans the route, detects obstacles with the help of cameras andultrasonic sensors, and gives users feedback through the alarm sound of the buzzer.After
experiments, the GPS positioning accuracy can reach 2.5m, the positioning delay time is 3s onaverage, and the camera recognition accuracy can reach 83.3% on average, which can meet theguidance needs of the blind in indoor and outdoor life.

© 2022 The Author. Published by Sugisaka Masanori at ALife Robotics Corporation Ltd.
This is an open access article distributed under the CC BY-NC 4.0 license(http://creativecommons.org/licenses/by-nc/4.0/)

easy to carry [2]. Therefore, there is an urgent need for new
guidance equipment to meet the needs of the blind today.
The constant expansion of visual impairment groups hasbrought about an increase in the demand for blind guidefacilities in the market. And that's not all, with the constantimprovement of urban construction, people's living
environment is also changing, the conventional blind guideequipment has been incapable to meet this transformation.
The scope of the guide cane is limited, the function issingle, and the guide box can not detect the unknown
danger, therefore, the guide robot was born and graduallyattracted the attention of various industries and enterprises.
At the moment, research work on guide assistance hasbeen conducted in countries on the face of the earth,focusing on the research of walking AIDS, wearable
marching AIDS, and mobile marching AIDS. as far backas 2010, Japan developed an electronic guide stick, whichuses supersonic sensors to make the visually impaired
people feel the obstacles, and to effectively remind users
through the vibration handles. The belt-like actionaccessory developed in the United States uses ultrasoundto create a panoramic map of the region, allowing the1. Project Research Background
According to the data of the World Health Organization,
there are currently about 285 million visually impairedpeople in the world, of whom 39 million are blind, and asurvey in China has found that there are 21 million visually
impaired people, of whom 23.8% are blind. As avulnerable group in society, the blind face difficulties indaily life and social interactions, among which travel is
one of the main difficulties in the life of the blind [1].
At present, the way for blind people to travel is usuallyaccompanied by family members, guide dog assistance,and guide equipment assistance. Family companionship isthe safest of all the ways for blind people to appear, but it
wastes a lot of time and energy of the companion; Guide
dog assistance is currently the most common way for blind
people to appear, but the training cycle of guide dogs islong and the cost is high; Traditional guide equipment forthe blind is low-cost and can assist the blind to avoid
obstacles, but it lacks functions such as path planning,
intelligent recognition, and blind positioning, and is not
Corresponding author’s E-mail: 2329433484@qq.com URL: www.tust.edu.cn116

visually impaired to "regain their eyes". Apart from, theforeign research and development of smart trolleys andsmart wheelchairs also provide the foundation and help forthe research and development of mobile guide AIDS.
China's research in this industry is comparatively late, theimmediate research focus is still on the guide rod, theresearch work on the blind guide robot is still inpreparation. In the final analysis, the technology is notimpeccable. Whether it is sensor technology, tracking
strategy and independent path planning capabilities, Chinaneeds to be further improved. To this end, this paper
designs an intelligent mobile guide robot, equipped with avariety of sensors, combined with the powerful computing
power of the main control board, to form a complete
obstacle avoidance system, which can realize autonomous
navigation and obstacle avoidance in complex
environments.

microcontroller directly drives the steering machine
through PWM pulse width modulation to control thesteering of the first two wheels. SCM communicates withGPS module through serial port to obtain the position
information. The ultrasonic module is directly controlled
by the microcontroller to avoid the obstacles in front of it.
The other main control board Jetson Nano controls thecamera to obtain the wide-angle line-of-sight informationin front of the camera, compare and analyze with otherinformation collected from vehicles, road signs,
pedestrians, and assist in obstacle avoidance.
The principle framework is shown in Fig. 1.

2. Scheme Design
In this paper, an intelligent mobile guide robot for the blindis designed. The robot uses STM32 as the main controller
and Jetson Nano as the auxiliary controller, and isequipped with a variety of sensors such as ultrasonic,
Bluetooth, GPS, camera, buzzer, etc., which can form acomplete obstacle avoidance system and realize
autonomous navigation and obstacle avoidance in complex
environments. And through Jetson Nano control GPS
positioning, independent route planning, ultrasonic sensor
automatic obstacle avoidance, Bluetooth module motion
control, the camera recognizes vehicles, pedestrians, roadshoulders, etc., and the buzzer alarms when a pedestrian isdetected.

Fig 1 STM32F103ZET6 chip

2.4. Main control part design
With stm32 and Jetson Nano as the main control, stm32 isresponsible for the underlying motion control, ultrasonicsensor and GPS module, Jetson Nano is responsible forvisual information processing and analysis, and the twomain control modules communicate through serial ports.
(1) Stm32f103rct6 Introduction
Using ArmCortex-M3 kernel, Harvard structure, withindependent instruction bus and data bus, widely used for
robot motion control.48KBSRAM, 256KBFLASH, 2advanced timers, 2 basic timers, 4 universal timers, 2DMA controllers (12 channels in total), 312 bit ADC, 5
serial ports, 112 bit DAC, 51 universal IO ports, and 1SDIO interface. It has the advantages of powerful function,
wide working temperature range, fast response, low power
consumption, and is extensively used in microcontroller
motion control.
(2) Jetson Nano Introduction
The Jetson Nano A02 is an open source hardware with
CUDA core, 45W low power consumption, 128 core
Maxwell GPU, and GB large memory. Compared with
Raspberry Pi or other AI edge computing units, it has
strong image processing capability, low power
consumption, and high cost performance. Manyalgorithms can obtain real-time performance, so it is usedas the main control of visual recognition.

2.1. Functional description(1) Automatic obstacle avoidance of ultrasonic sensor.
(2) Bluetooth Module Motion Control, autonomous
navigation.
(3) GPS positioning, autonomous route planning.
(4) Cameras identify vehicles, pedestrians, shoulders, etc,
and the buzzer alarm when encountering pedestrians.
2.2. System composition
The system framework of the intelligent mobile guiderobot system can be divided into the following parts: maincontrol module, motion control module, obstacle
avoidance module, positioning module, visual recognition
module and power supply module.
2.3. Work principle introduction
The microcontroller stm32f103rct6 drive motor drive platecontrols the speed of the rear two-wheel motor throughPWM wave voltage regulation, and the stm32
117

2.5. Sport control design
The servo drive system is adopted, the first two wheels use
the TBSK20 steering machine to directly receive the stm32
signal by PWM wave electric voltage regulation to control
the steering, the second two wheels use the encoder motor,
with TB6612 as the drive chip, and the PWM pulse widthmodulation technology is used to control the motor speed
by changing the duty cycle, and then control the speed of
the car. Compared with the traditional differential system,
the motor power of the robot can be directly converted into
the driving force, and the steering mechanism is controlled
by an independent servo, which has a simpler structure,high control efficiency and high stability. Secondly, thesoftware-led power system can be adjusted according to
the different ways defined by the software, or the working
conditions of the system, which is less difficult to operate
and more intelligent. At the same time, the loss of tires andaccessories is small, which meets the application needs oflong-term stable operation, and has great advantages inflexible control.
The motor drive schematic diagram is shown in Fig.2.

Fig 3 Ultrasonic module

2.7. Gps module design
The GPS positioning module uses the ATGM336H ofZhongke Microelectronics, the number of channels is 32
channels, the sensitivity of cold start acquisition is 148dBm, the tracking sensitivity is -162dBm, thepositioning accuracy is 2.5 meters (CEP50, open ground),which has high sensitivity, supports the single system
positioning of the BDS / GPS / GLONASS satellite
navigation system, and the receiver module with any
combination of multi-system joint positioning.
ATGM336H can directly replace U-blox's MAX series ofmultiple GPS modules, the main interface signal is Pin-Pincompatible, consistent installation hole, low power
consumption, built-in antenna detection circuit, withantenna short circuit protection function.
The schematic diagram of the GPS module is shown inFig.4.

Fig 2 Motor drive module

2.6. Design of obstacle avoidance moduleThe intelligent mobile guide dog is equipped with anultrasonic sensor and a vision sensor, both of which canindependently realize the obstacle avoidance function.
The ultrasonic sensor in this paper uses HC-SR04 module,
which has a non-contact distance sensing function of 2cm400cm, and the ranging accuracy is 3mm.
The principle of ultrasonic sensor ranging is to obtain
the distance information of the obstacle by calculating thetime interval between the time of sending the source soundwave and the time of receiving the echo signal. The
calculation formula is as follows [3].

Fig 4 GPS module

2.8. Visual recognition module design
With a Jetson Nano NX AI HD camera and 160 focal field
wide Angle, the optical sensor chip is SONY IMX219,
which is suitable for image data acquisition andinformation processing. The program used in this projectis the yolov3 algorithm under the Alexey AB streamlined
darknet framework, which has a strong performance in thefield of real-time target detection and can fit the computing

Test distance =（High level time × speed of sound）/ 2(The speed of sound is 340m/s.)
The schematic diagram of ultrasonic module is shownin Fig.3.

118

power of Jetson Nano. Real-time running time in Jetson
Nano can reach a detection rate of 10 to 20 frames, and cancope with 90% of the scenes in daily life [3].
The Jetson Nano interface diagram is shown in Fig.5.

sets or take a certain number of pictures in the field, andname the pictures in a certain order. The following
example uses the KITTI-road dataset as an example. The
dataset was manually annotated by using the labelIMG
tool (The labelIMG tools: heartexlabs/labelImg):(1) Configure the preset value tag file in labelIMG: Openthe predefined_classes.txt file in the data path andwrite the label name of the object to be detected tothe file. The operation is shown in Fig.7.

Fig 5 The Jetson Nano interface

2.9. Power supply module
Because there are many drive modules, the car adopts twoparts to power supply stm32 and Jetson Nano respectively.
First, 3 sections of 12V 1500mA power supply stm32, andanother 3 sections of 12V 1500mA power supply, andAMS1117 5.0 is used to supply power to Jetson Nano withlow voltage drop regulator.
The principle of antihypertensive is shown in Fig.6.

Fig 7 Enter the object label name(2) Using the labelIMG software.
The operations are shown in Fig.8 and Fig.9.

Fig 8 Click on the Open Dir

Fig 6.Anti-pressure module

3. Introduction of YOLO training algorithm
The program used in this project for target recognition isthe YoloV3 algorithm under the darknet framework
improved by AlexeyAB, which has strong performance inthe field of real-time object detection and can be trained inaccordance with Jetson Nano.
This project starts with devices with higher computing
power than Jetson Nano, including the personal computer(windows platform) using the KIT T I data set publishedby the Toyota Technical University at Chicago (TTIC) andKarlsruhe Institute of Technology (KIT). It mainly
includes data processing, training, use and deployment.
Here is the first part. Download and use open source data

Fig 9 Select the path to the downloaded image dataset or theactual image file.

(3) To mark The operations are shown in Fig.10, Fig.11
and Fig.12.

119script, trainval_percent is the proportion of the datasets
used for validation, The train_percent is the proportion of
the data sets used for training: then run the voc_label.py inthe scripts directory for format conversion, The
streamlined classes item is High-temperature resistant withthe target category tag that you want to identify. To matchthe dataset format, the set item was modified as Fig.13.

Fig 10.Select the Pascal VOC format dimension on the left sideof the window.

Fig 13 Modify the set code

This script generates both the 2007_train.txt and the2007_val.txt files. Need to be used during the training
sessions. These two files contain the path of the picturescontained in the training set randomly and the validation
set.
4. Conclusion
The intelligent electronic guide dog based on Jetson Nano
introduced in this paper is a Visual Mobile Robot, which
is used to assist the blind people to travel securely. It isequipped with an STM32 microcontroller and Jetson Nano
AI edge computing device, integrating a variety of sensors
including HC-SR04, HC-06, GPS, camera, etc., comparedwith the immediate guide box and guide stick, it has highcost performance and obvious advantages. The maincontrol module, positioning module, visual recognition
module, motion control module, power supply module andobstacle avoidance module are introduced in detail, andthe schematic diagram is given to complete the design ofthe entire guide dog.

Fig 11 Press the shortcut key "w" to drag out the correspondingbox on the image.

References
1. Computational N A I . Retracted: Innovative Design of a
Cloud Tour Guide Robot for Smart Cities under thePrinciple of System Innovation Design. [J]. Computational
intelligence and neuroscience, 2023, 2023.
2. Fan Y M, Bao T J, Tang R H. A Guide Cane System for
Assisting the Blind in Travelling in Outdoor
Environments[J].
AppliedMechanics
andMaterials,2014,3458(631-632).
3. William E, Gao T, Yuan T, et al. Data-Driven Modellingand Control for Robot Needle Insertion in Deep Anterior
Lamellar Keratoplasty. [J]. IEEE robotics and automation
letters,2022,7(2).

Fig 12 Then select the corresponding label in the pop-up
window and click OK.

Right-key the box to modify the label or delete the box
The annotation processing of the data set is completed after
the annotation of all the picture files Then all the generated
XML tag files are selected and cut to the scripts \VOCdevkit \ VOC2007 \ Annotations folder, run thecut.py under scripts \ VOCdevkit \ VOC2007, and split thedataset, Then run the script under the project to further
process the data, Open the voc_label.py script in the scripts
directory, Edit the classes item in it, Keeping it consistent
with the labels antecedently selected to make the dataset,
The proportion of segmentation can be adjusted in the120

Authors Introduction
Mr. ZhenCheng Chang
He is currently pursuing hisundergraduate degree at the School ofElectronic
Information
andAutomation, Tianjin University of
Science and Technology.

Ms. YuRan Lu

Mr. ZiYue Xiao

Mr. YanDe Xiang

She is currently pursuing her
undergraduate degree at the Schoolof Artistic design, Tianjin Universityof Science and Technology.

He is currently pursuing his
undergraduate degree at the School ofElectronic
Information
andAutomation, Tianjin University of
Science and Technology.

He is currently pursuing his
undergraduate degree at the School ofElectronic
Information
andAutomation, Tianjin University of
Science and Technology.

121