Grow underlying buffer only when necessary. Fixes #13552