  Designing intelligent microrobots that can autonomously navigate and perform
instructed routines in blood vessels, a complex and crowded environment with
obstacles including dense cells, different flow patterns and diverse vascular
geometries, can offer enormous possibilities in biomedical applications. Here
we report a hierarchical control scheme that enables a microrobot to
efficiently navigate and execute customizable routines in blood vessels. The
control scheme consists of two highly decoupled components: a high-level
controller setting short-ranged dynamic targets to guide the microrobot to
follow a preset path and a low-level deep reinforcement learning (DRL)
controller responsible for maneuvering microrobots towards these dynamic
guiding targets. The proposed DRL controller utilizes three-dimensional (3D)
convolutional neural networks and is capable of learning control policy
directly from a coarse raw 3D sensory input. In blood vessels with rich
configurations of red blood cells and vessel geometry, the control scheme
enables efficient navigation and faithful execution of instructed routines. The
control scheme is also robust to adversarial perturbations including blood
flows. This study provides a proof-of-principle for designing data-driven
control systems for autonomous navigation in vascular networks; it illustrates
the great potential of artificial intelligence for broad biomedical
applications such as target drug delivery, blood clots clear, precision
surgery, disease diagnosis, and more.
