A lot less clearing of the PacketBuilderBuffer caches, and making it a bit more efficient