One cliche is "beat a hasty retreat".