The United Nations says Iran has refused to allow nuclear inspectors to revisit a suspect military site .