  Human movement disorders or paralysis lead to the loss of control of muscle
activation and thus motor control. Functional Electrical Stimulation (FES) is
an established and safe technique for contracting muscles by stimulating the
skin above a muscle to induce its contraction. However, an open challenge
remains on how to restore motor abilities to human limbs through FES, as the
problem of controlling the stimulation is unclear. We are taking a robotics
perspective on this problem, by developing robot learning algorithms that
control the ultimate humanoid robot, the human body, through electrical muscle
stimulation. Human muscles are not trivial to control as actuators due to their
force production being non-stationary as a result of fatigue and other internal
state changes, in contrast to robot actuators which are well-understood and
stationary over broad operation ranges. We present our Deep Reinforcement
Learning approach to the control of human muscles with FES, using a recurrent
neural network for dynamic state representation, to overcome the unobserved
elements of the behaviour of human muscles under external stimulation. We
demonstrate our technique both in neuromuscular simulations but also
experimentally on a human. Our results show that our controller can learn to
manipulate human muscles, applying appropriate levels of stimulation to achieve
the given tasks while compensating for advancing muscle fatigue which arises
throughout the tasks. Additionally, our technique can learn quickly enough to
be implemented in real-world human-in-the-loop settings.
