also optionally apply SHAPE_ENCODING to layer creation (fixes #5911)