Call done() only when really done.