Make sure we set the right buffer readonly