Turns out you can't do that, annoying.