Remove a vector buffer