Fix adding Vector layers during a zoom animation

Fixes #2416

Not sure if this is the best way. Feel free to trash and do another way :)