[vector-api] High DPI (Retina) support for vector layers

Based on @fredj's work in #1367.