Change the zsh prompt to > when in vi command mode.  Disable promptsubst.

promptsubst is disabled to prevent a $ in the prompt from being expanded
if it happens to have a word after it that turns it into a variable.