Very minor optimization, only update buffer on server when necessary