Repository: trusted-ai/adversarial-robustness-toolbox
Issue: ACG and APGD loss runtime error with single sample

**Describe the bug**
When generating an adversarial example from a single sample using ACG or APGD, an error occurs while calculating the loss using both DLR and crossentropy (same error for APGD and ACG). This behavior does not occur using PGD.

**To Reproduce**
Steps to reproduce the behavior:
1. Define model arch (called PPOAppendPolicy in my code):
Sequential(
  (0): Linear(in_features=29, out_features=64, bias=True)
  (1): Tanh()
  (2): Linear(in_features=64, out_features=64, bias=True)
  (3): Tanh()
  (4): Linear(in_features=64, out_features=10, bias=True)
)
2. Define pytorch classifier:
PolicyClf = PyTorchClassifier(model=PPOAppendPolicy,
                        loss=nn.CrossEntropyLoss(),
                        input_shape=PPOmodel.env.observation_space.shape, #(29,)
                        nb_classes=PPOmodel.action_space.n)   # 10  
3. Define sample to be attacked, this is a float32 np.array called obs
4.  Try attack with DLR loss:
attack = APGD(PolicyClf, loss_type="difference_logits_ratio")
obs_adv = attack.generate(np.expand_dims(obs, axis=0)) #expand dims so obs isn't interpreted as 29 separate samples
5. Error:
RuntimeError                              Traceback (most recent call last)
Cell In[46], line 2
      1 attack = APGD(PolicyClf, loss_type="difference_logits_ratio")
----> 2 obs_adv = attack.generate(np.expand_dims(obs, axis=0))

File /usr/local/lib/python3.10/dist-packages/art/attacks/evasion/auto_projected_gradient_descent.py:500, in AutoProjectedGradientDescent.generate(self, x, y, **kwargs)
    497 x_1 = x_init_batch + perturbation
    499 f_0 = self.estimator.compute_loss(x=x_k, y=y_batch, reduction="none")
--> 500 f_1 = self.estimator.compute_loss(x=x_1, y=y_batch, reduction="none")
    502 # modification for image-wise stepsize update
    503 self.eta_w_j_m_1 = eta.copy()

File /usr/local/lib/python3.10/dist-packages/art/estimators/classification/pytorch.py:747, in PyTorchClassifier.compute_loss(self, x, y, reduction, **kwargs)
    745 # Return individual loss values
    746 self._loss.reduction = reduction
--> 747 loss = self._loss(model_outputs[-1], labels_t)
    748 self._loss.reduction = prev_reduction
    750 if isinstance(x, torch.Tensor):

File /usr/local/lib/python3.10/dist-packages/art/attacks/evasion/auto_projected_gradient_descent.py:292, in AutoProjectedGradientDescent.__init__..DifferenceLogitsRatioPyTorch.__call__(self, y_pred, y_true)
    289 i_z_i_list = []
    291 for i in range(y_true.shape[0]):
--> 292     if i_y_pred_arg[i, -1] != i_y_true[i]:
    293         i_z_i_list.append(i_y_pred_arg[i, -1])
    294     else:

RuntimeError: Boolean value of Tensor with more than one value is ambiguous
6. Try with crossentropy:
attack = ACG(PolicyClf)
obs_adv = attack.generate(np.expand_dims(obs, axis=0))
7. error:
RuntimeError                              Traceback (most recent call last)
Cell In[42], line 2
      1 attack = ACG(ActionClf)
----> 2 obs_adv = attack.generate(np.expand_dims(obs, axis=0))

File /usr/local/lib/python3.10/dist-packages/art/attacks/evasion/auto_conjugate_gradient.py:519, in AutoConjugateGradient.generate(self, x, y, **kwargs)
    516 x_1 = x_init_batch + perturbation
    518 f_0 = self.estimator.compute_loss(x=x_k, y=y_batch, reduction="none")
--> 519 f_1 = self.estimator.compute_loss(x=x_1, y=y_batch, reduction="none")
    521 self.eta_w_j_m_1 = eta.copy()
    522 self.f_max_w_j_m_1 = f_0.copy()

File /usr/local/lib/python3.10/dist-packages/art/estimators/classification/pytorch.py:747, in PyTorchClassifier.compute_loss(self, x, y, reduction, **kwargs)
    745 # Return individual loss values
    746 self._loss.reduction = reduction
--> 747 loss = self._loss(model_outputs[-1], labels_t)
    748 self._loss.reduction = prev_reduction
    750 if isinstance(x, torch.Tensor):

File /usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py:1501, in Module._call_impl(self, *args, **kwargs)
   1496 # If we don't have any hooks, we want to skip the rest of the logic in
   1497 # this function, and just call forward.
   1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   1499         or _global_backward_pre_hooks or _global_backward_hooks
   1500         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1501     return forward_call(*args, **kwargs)
   1502 # Do not call functions when jit is used
   1503 full_backward_hooks, non_full_backward_hooks = [], []

File /usr/local/lib/python3.10/dist-packages/torch/nn/modules/loss.py:1174, in CrossEntropyLoss.forward(self, input, target)
   1173 def forward(self, input: Tensor, target: Tensor) -> Tensor:
-> 1174     return F.cross_entropy(input, target, weight=self.weight,
   1175                            ignore_index=self.ignore_index, reduction=self.reduction,
   1176                            label_smoothing=self.label_smoothing)

File /usr/local/lib/python3.10/dist-packages/torch/nn/functional.py:3029, in cross_entropy(input, target, weight, size_average, ignore_index, reduce, reduction, label_smoothing)
   3027 if size_average is not None or reduce is not None:
   3028     reduction = _Reduction.legacy_get_string(size_average, reduce)
-> 3029 return torch._C._nn.cross_entropy_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index, label_smoothing)

RuntimeError: only batches of spatial targets supported (3D tensors) but got targets of size: : [1]

**Expected behavior**
This approach works for PGD:
attack = PGD(PolicyClf)
obs_adv = attack.generate(np.expand_dims(obs, axis=0))

**System information (please complete the following information):**
 - UBUNTU 20.04
 - Python version 3.10.6
 - ART version 1.14.1
 - PyTorch version 2.0.0


Changed files:
art/attacks/evasion/auto_projected_gradient_descent.py, art/attacks/evasion/auto_conjugate_gradient.py

diff --git a/art/attacks/evasion/auto_conjugate_gradient.py b/art/attacks/evasion/auto_conjugate_gradient.py
index 524b73538..4e0df74ce 100644
--- a/art/attacks/evasion/auto_conjugate_gradient.py
+++ b/art/attacks/evasion/auto_conjugate_gradient.py
@@ -463,7 +463,9 @@ class AutoConjugateGradient(EvasionAttack):
 
                 # self.eta = np.full((self.batch_size, 1, 1, 1), 2 * self.eps_step).astype(ART_NUMPY_DTYPE)
                 _batch_size = x_k.shape[0]
-                eta = np.full((_batch_size, 1, 1, 1), self.eps_step).astype(ART_NUMPY_DTYPE)
+                eta = np.full((_batch_size,) + (1,) * len(self.estimator.input_shape), self.eps_step).astype(
+                    ART_NUMPY_DTYPE
+                )
                 self.count_condition_1 = np.zeros(shape=(_batch_size,))
                 gradk_1 = np.zeros_like(x_k)
                 cgradk_1 = np.zeros_like(x_k)
@@ -650,4 +652,4 @@ def get_beta(gradk, gradk_1, cgradk_1):
     betak = -(_gradk * delta_gradk).sum(axis=1) / (
         (_cgradk_1 * delta_gradk).sum(axis=1) + np.finfo(ART_NUMPY_DTYPE).eps
     )
-    return betak.reshape((_batch_size, 1, 1, 1))
+    return betak.reshape((_batch_size,) + (1,) * (len(gradk.shape) - 1))
diff --git a/art/attacks/evasion/auto_projected_gradient_descent.py b/art/attacks/evasion/auto_projected_gradient_descent.py
index 2a1362208..5eed274c0 100644
--- a/art/attacks/evasion/auto_projected_gradient_descent.py
+++ b/art/attacks/evasion/auto_projected_gradient_descent.py
@@ -458,7 +458,9 @@ class AutoProjectedGradientDescent(EvasionAttack):
 
                 # modification for image-wise stepsize update
                 _batch_size = x_k.shape[0]
-                eta = np.full((_batch_size, 1, 1, 1), self.eps_step).astype(ART_NUMPY_DTYPE)
+                eta = np.full((_batch_size,) + (1,) * len(self.estimator.input_shape), self.eps_step).astype(
+                    ART_NUMPY_DTYPE
+                )
                 self.count_condition_1 = np.zeros(shape=(_batch_size,))
 
                 for k_iter in trange(self.max_iter, desc="AutoPGD - iteration", leave=False, disable=not self.verbose):