Use Ctrl+R to search backward, even in Vi mode.