A Study of the Efficacy of Generative Flow Networks for Robotics and Machine Fault-Adaptation
Creators: Sufiyan, Zahin
Publisher: name: University of Alberta Library
Year: 2024

<p>In 2005, Opportunity, one of NASA’s renowned Mars rovers, faced a dire situation.<br>
It was a moment that could end a mission that had already far outlasted its expected<br>
lifespan. After clambering out of the Victoria crater, the rover started to experience<br>
an abrupt current spike in its right front wheel. As a consequence, the wheel motor<br>
started to malfunction, causing the wheel to stop turning. NASA anticipated that<br>
the $400 million dollars of investment and most importantly invaluable scientific data<br>
regarding the Martian terrain was on the brink of remaining unexplored. With the<br>
immensity of space between Mars and Earth (140 million miles, to be specific), the<br>
engineers at NASA could only detect and diagnose the malfunction; however, human<br>
intervention in the maintenance of Opportunity was an impossibility. Nevertheless,<br>
human ingenuity again succeeded when the engineers at NASA’s Jet Propulsion Laboratory<br>
came up with an unconventional workaround. They started to drive the rover<br>
backward and thus by doing so, they were able to redistribute the mechanical load<br>
and reduce the strain on the malfunctioning wheel. The impaired wheel now functioned<br>
as a rear wheel, allowing the fully functional wheels to lead and navigate the<br>
harsh and challenging surface of Mars. Due to this innovative approach, Opportunity<br>
continued to explore Mars and gathered some of the most invaluable data about the<br>
red planet for 15 Earth years instead of its initially predicted 90-day lifespan. This<br>
was a testament to human ingenuity, but also a stark reminder of the necessity for<br>
built-in machine fault adaptability in robotic systems.<br>
Our research is a step towards adding hardware fault tolerance and fault adaptability<br>
to machines. In this research, our primary focus is to investigate the efficacy of generative flow networks (GFlowNets/CFlowNets) in robotic environments, particularly<br>
in the domain of machine fault adaptation. Generative Flow Networks is an<br>
emerging algorithm with the potential to be considered as a substitute approach to<br>
the prevalent reinforcement learning methods in continuous exploratory tasks. In our<br>
work, the experimentations were done in a simulated robotic environment (Reacherv2).<br>
This environment was manipulated and modified to introduce four distinct<br>
fault environments which are reduced range of motion, increased damping, actuator<br>
damage, and structural damage. Each fault replicates actual malfunctions that<br>
are generally witnessed in real-world machines/robots that render them inoperative.<br>
The empirical evaluation of this research indicates that continuous generative flow<br>
networks indeed have the capability to add adaptive behaviors in machines under<br>
adversarial conditions in the environment. Furthermore, the comparative analysis of<br>
CFlowNets with state-of-the-art RL algorithms also provides some key insights into<br>
the performance in terms of adaptation speed and sample efficiency. Despite a few<br>
algorithmic shortcomings, our experiments confirm that CFlowNets has the potential<br>
to be deployed in a real-world machine and it can demonstrate adaptability in case of<br>
malfunctions to maintain functionality. The thesis is motivated by the idea of transforming<br>
robots into more than just mere tools, making them capable entities which<br>
are capable of autonomously overcoming certain faults and failures, thus sustaining<br>
their operation while delaying the need for maintenance. Through experimentation<br>
in simulated robotic environments, the comparative study aims to contribute to the<br>
ongoing discourse on enhancing the adaptive capacities of automated systems and<br>
machines.</p>