pothos-library: only init BufferManager when empty