  Experimental demonstration of complex robotic behaviors relies heavily on
finding the correct controller gains. This painstaking process is often
completed by a domain expert, requiring deep knowledge of the relationship
between parameter values and the resulting behavior of the system. Even when
such knowledge is possessed, it can take significant effort to navigate the
nonintuitive landscape of possible parameter combinations. In this work, we
explore the extent to which preference-based learning can be used to optimize
controller gains online by repeatedly querying the user for their preferences.
This general methodology is applied to two variants of control Lyapunov
function based nonlinear controllers framed as quadratic programs, which have
nice theoretic properties but are challenging to realize in practice. These
controllers are successfully demonstrated both on the planar underactuated
biped, AMBER, and on the 3D underactuated biped, Cassie. We experimentally
evaluate the performance of the learned controllers and show that the proposed
method is repeatably able to learn gains that yield stable and robust
locomotion.
