bson: dont perform append if it would overflow INT_MAX.

Allow the callers to check to see if their append would fail.