Remove chunks from queue if we don't need them anymore.