Repository: cirKITers/masKIT

Issue:
Include number of shots in training parameters
Currently the number of shots is not parametrisable. This should be included into our training parameters for more flexibility during experiments.

Discussion:
codecov[bot]: # [Codecov](https://codecov.io/gh/cirKITers/masKIT/pull/46?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cirKITers) Report
> Merging [#46](https://codecov.io/gh/cirKITers/masKIT/pull/46?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cirKITers) (c23432d) into [main](https://codecov.io/gh/cirKITers/masKIT/commit/441d44aa48a2929063185680d6085ed7d8589017?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cirKITers) (441d44a) will **decrease** coverage by `0.13%`.
> The diff coverage is `0.00%`.

[![Impacted file tree graph](https://codecov.io/gh/cirKITers/masKIT/pull/46/graphs/tree.svg?width=650&height=150&src=pr&token=8Or2uHjmFE&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cirKITers)](https://codecov.io/gh/cirKITers/masKIT/pull/46?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cirKITers)

```diff
@@            Coverage Diff             @@
##             main      #46      +/-   ##
==========================================
- Coverage   78.60%   78.46%   -0.14%     
==========================================
  Files          15       15              
  Lines        1173     1161      -12     
  Branches      117      117              
==========================================
- Hits          922      911      -11     
+ Misses        246      245       -1     
  Partials        5        5              
```


| [Impacted Files](https://codecov.io/gh/cirKITers/masKIT/pull/46?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cirKITers) | Coverage Δ | |
|---|---|---|
| [main.py](https://codecov.io/gh/cirKITers/masKIT/pull/46/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cirKITers#diff-bWFpbi5weQ==) | `0.00% <0.00%> (ø)` | |
| [maskit/masks.py](https://codecov.io/gh/cirKITers/masKIT/pull/46/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cirKITers#diff-bWFza2l0L21hc2tzLnB5) | `100.00% <0.00%> (ø)` | |
| [tests/test\_masks.py](https://codecov.io/gh/cirKITers/masKIT/pull/46/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cirKITers#diff-dGVzdHMvdGVzdF9tYXNrcy5weQ==) | `100.00% <0.00%> (ø)` | |
| [maskit/log\_results.py](https://codecov.io/gh/cirKITers/masKIT/pull/46/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cirKITers#diff-bWFza2l0L2xvZ19yZXN1bHRzLnB5) | `0.00% <0.00%> (ø)` | |
| [tests/test\_ensembles.py](https://codecov.io/gh/cirKITers/masKIT/pull/46/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cirKITers#diff-dGVzdHMvdGVzdF9lbnNlbWJsZXMucHk=) | `100.00% <0.00%> (ø)` | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/cirKITers/masKIT/pull/46?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cirKITers).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cirKITers)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/cirKITers/masKIT/pull/46?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cirKITers). Last update [441d44a...c23432d](https://codecov.io/gh/cirKITers/masKIT/pull/46?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cirKITers). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cirKITers).


Patch:
diff --git a/main.py b/main.py
index 0532b73..2f8b8a6 100644
--- a/main.py
+++ b/main.py
@@ -86,7 +86,11 @@ def train(
     wires = train_params["wires"]
     layers = train_params["layers"]
     steps = train_params.get("steps", 1000)
-    dev = get_device(train_params.get("sim_local", True), wires=wires)
+    dev = get_device(
+        train_params.get("sim_local", True),
+        wires=wires,
+        shots=train_params.get("shots", None),
+    )
     step_size = train_params.get("step_size", None)
     if step_size:
         opt = train_params["optimizer"].value(step_size)
@@ -205,7 +209,11 @@ def test(
         pass
     elif train_params["dataset"] == "iris":
         wires = train_params["wires"]
-        dev = get_device(train_params["sim_local"], wires=wires)
+        dev = get_device(
+            train_params["sim_local"],
+            wires=wires,
+            shots=train_params.get("shots", None),
+        )
         circuit = qml.QNode(iris_circuit, dev)
         correct = 0
         N = len(test_data)
diff --git a/maskit/utils.py b/maskit/utils.py
index 4097ee8..f5045d7 100644
--- a/maskit/utils.py
+++ b/maskit/utils.py
@@ -8,15 +8,38 @@ def check_params(train_params):
     assert isinstance(train_params["optimizer"], ExtendedOptimizers)
 
 
-def cross_entropy(predictions, targets, epsilon=1e-12):
+def cross_entropy(
+    predictions: np.ndarray, targets: np.ndarray, epsilon: float = 1e-15
+) -> float:
     """
-    Computes cross entropy between targets (encoded as one-hot vectors)
-    and predictions.
-    Input: predictions (N, k) ndarray
-           targets (N, k) ndarray
-    Returns: scalar
+    Cross entropy calculation between :py:attr:`targets` (encoded as one-hot vectors)
+    and :py:attr:`predictions`. Predictions are normalized to sum up to `1.0`.
+
+    .. note::
+
+        The implementation of this function is based on the discussion on
+        `StackOverflow <https://stackoverflow.com/a/47398312/10138546>`_.
+
+        Due to ArrayBoxes that are required for automatic differentiation, we currently
+        use this implementation instead of implementations provided by sklearn for
+        example.
+
+    :param predictions: Predictions in same order as targets. In case predictions for
+        several samples are given, the weighted cross entropy is returned.
+    :param targets: Ground truth labels for supplied samples.
+    :param epsilon: Amount to clip predictions as log is not defined for `0` and `1`.
     """
+    assert (
+        predictions.shape == targets.shape
+    ), f"Shape of predictions {predictions.shape} must match targets {targets.shape}"
+    current_sum = np.sum(predictions, axis=predictions.ndim - 1)
+
+    if predictions.ndim == 1:
+        sample_count = 1
+        predictions = predictions / current_sum
+    else:
+        sample_count = predictions.shape[0]
+        predictions = predictions / current_sum[:, np.newaxis]
+
     predictions = np.clip(predictions, epsilon, 1.0 - epsilon)
-    N = predictions.shape[0]
-    ce = -np.sum(targets * np.log(predictions + 1e-9)) / N
-    return ce
+    return -np.sum(targets * np.log(predictions)) / sample_count
diff --git a/pyproject.toml b/pyproject.toml
index c987d12..cea3441 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -27,6 +27,7 @@ test = [
     "flake8",
     "flake8-bugbear",
     "black",
+    "scikit-learn",
 ]
 dev = ["pre-commit"]
 


Test patch:
diff --git a/tests/test_utils.py b/tests/test_utils.py
new file mode 100644
index 0000000..7f97e73
--- /dev/null
+++ b/tests/test_utils.py
@@ -0,0 +1,30 @@
+import numpy as np
+import pytest
+from sklearn.metrics import log_loss
+
+from maskit.utils import cross_entropy
+
+
+@pytest.mark.parametrize(
+    "predictions,targets",
+    [
+        pytest.param(
+            np.array([[0.25, 0.25, 0.25, 0.25], [0.01, 0.01, 0.01, 0.96]]),
+            np.array([[0.0, 0.0, 0.0, 1.0], [0.0, 0.0, 0.0, 1.0]]),
+        ),
+        pytest.param(np.array([0.0, 0.0, 1.0, 0.0]), np.array([0.0, 0.0, 1.0, 0.0])),
+        pytest.param(
+            np.array([0.25, 0.25, 0.25, 0.25]), np.array([0.0, 0.0, 1.0, 0.0])
+        ),
+        pytest.param(
+            np.array([0.01, 0.01, 0.01, 0.96]), np.array([0.0, 0.0, 0.0, 1.0])
+        ),
+    ],
+)
+def test_cross_entropy(predictions, targets):
+    if predictions.ndim == 1:
+        expected_value = log_loss([targets], [predictions])
+    else:
+        expected_value = log_loss(targets, predictions)
+    calculated_value = cross_entropy(predictions=predictions, targets=targets)
+    assert np.isclose(expected_value, calculated_value)