Repository: deepset-ai/haystack

Issue:
feat: `AnswerExactMatchEvaluator` input should be one answer per query instead of list of answers
We should change the inputs of the `AnswerExactMatchEvaluator` added in https://github.com/deepset-ai/haystack/pull/7050 from
`questions: List[str], ground_truth_answers: List[List[str]], predicted_answers: List[List[str]]`
to 
`ground_truth_answers: List[str], predicted_answers: List[str]`

This change will make all metrics in Haystack core and in integrations, statistical and model-based, consistent in the inputs they expect. For answers always `List[str]`, for queries always `List[str]` and for documents (contexts) always `List[List[str]]`. It also simplifies the implementation of the new metrics and will allow us to move faster.

**Describe alternatives you've considered** 
Keeping inputs as is would mean they are inconsistent with the model based metrics and the integrations of evaluation frameworks. However, the behavior would be the same as the exact match metric in Haystack 1.x and more flexible for datasets with multiple ground truth answers such as SQuAD 2.0 and multiple predicted answers like our Reader's output.


Discussion:
coveralls: ## Pull Request Test Coverage Report for [Build 8650093623](https://coveralls.io/builds/66859467)


### Details

* **0** of **0**   changed or added relevant lines in **0** files are covered.
* No unchanged relevant lines lost coverage.
* Overall coverage remained the same at **89.934%**

---



|  Totals | [![Coverage Status](https://coveralls.io/builds/66859467/badge)](https://coveralls.io/builds/66859467) |
| :-- | --: |
| Change from base [Build 8648609512](https://coveralls.io/builds/66856857): |  0.0% |
| Covered Lines: | 6254 |
| Relevant Lines: | 6954 |

---
##### 💛  - [Coveralls](https://coveralls.io)


Patch:
diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS
index 1657c912..c559d08f 100644
--- a/.github/CODEOWNERS
+++ b/.github/CODEOWNERS
@@ -4,8 +4,8 @@
 # in the repo. Unless a later match takes precedence,
 # @deepset-ai/core-engineering will be requested for review
 # when someone opens a pull request.
-*                       @deepset-ai/core-engineering
+*                       @deepset-ai/open-source-engineering
 
 # Documentation
-*.md                    @deepset-ai/documentation @deepset-ai/core-engineering
-releasenotes/notes/*    @deepset-ai/documentation @deepset-ai/core-engineering
+*.md                    @deepset-ai/documentation @deepset-ai/open-source-engineering
+releasenotes/notes/*    @deepset-ai/documentation @deepset-ai/open-source-engineering
diff --git a/docker/docker-bake.hcl b/docker/docker-bake.hcl
index ba7e681b..72cdd339 100644
--- a/docker/docker-bake.hcl
+++ b/docker/docker-bake.hcl
@@ -26,8 +26,8 @@ target "base" {
   dockerfile = "Dockerfile.base"
   tags = ["${IMAGE_NAME}:base-${IMAGE_TAG_SUFFIX}"]
   args = {
-    build_image = "python:3.10-slim"
-    base_image = "python:3.10-slim"
+    build_image = "python:3.12-slim"
+    base_image = "python:3.12-slim"
     haystack_version = "${HAYSTACK_VERSION}"
   }
   platforms = ["linux/amd64", "linux/arm64"]
diff --git a/haystack/components/evaluators/answer_exact_match.py b/haystack/components/evaluators/answer_exact_match.py
index dcd44408..db5e7225 100644
--- a/haystack/components/evaluators/answer_exact_match.py
+++ b/haystack/components/evaluators/answer_exact_match.py
@@ -6,10 +6,11 @@ from haystack.core.component import component
 @component
 class AnswerExactMatchEvaluator:
     """
-    Evaluator that checks if the predicted answers matches any of the ground truth answers exactly.
-    The result is a number from 0.0 to 1.0, it represents the proportion any predicted answer
-    that matched one of the ground truth answers.
-    There can be multiple ground truth answers and multiple predicted answers as input.
+    Evaluator that checks if predicted answers exactly match ground truth answers.
+
+    Each predicted answer is compared to one ground truth answer.
+    The final score is a number ranging from 0.0 to 1.0.
+    It represents the proportion of predicted answers that match their corresponding ground truth answer.
 
     Usage example:
     ```python
@@ -17,8 +18,8 @@ class AnswerExactMatchEvaluator:
 
     evaluator = AnswerExactMatchEvaluator()
     result = evaluator.run(
-        ground_truth_answers=[["Berlin"], ["Paris"]],
-        predicted_answers=[["Berlin"], ["Lyon"]],
+        ground_truth_answers=["Berlin", "Paris"],
+        predicted_answers=["Berlin", "Lyon"],
     )
 
     print(result["individual_scores"])
@@ -29,15 +30,15 @@ class AnswerExactMatchEvaluator:
     """
 
     @component.output_types(individual_scores=List[int], score=float)
-    def run(self, ground_truth_answers: List[List[str]], predicted_answers: List[List[str]]) -> Dict[str, Any]:
+    def run(self, ground_truth_answers: List[str], predicted_answers: List[str]) -> Dict[str, Any]:
         """
         Run the AnswerExactMatchEvaluator on the given inputs.
         `ground_truth_answers` and `retrieved_answers` must have the same length.
 
         :param ground_truth_answers:
-            A list of expected answers for each question.
+            A list of expected answers.
         :param predicted_answers:
-            A list of predicted answers for each question.
+            A list of predicted answers.
         :returns:
             A dictionary with the following outputs:
             - `individual_scores` - A list of 0s and 1s, where 1 means that the predicted answer matched one of the ground truth.
@@ -48,8 +49,8 @@ class AnswerExactMatchEvaluator:
             raise ValueError("The length of ground_truth_answers and predicted_answers must be the same.")
 
         matches = []
-        for truths, extracted in zip(ground_truth_answers, predicted_answers):
-            if set(truths) & set(extracted):
+        for truth, extracted in zip(ground_truth_answers, predicted_answers):
+            if truth == extracted:
                 matches.append(1)
             else:
                 matches.append(0)
diff --git a/haystack/core/component/sockets.py b/haystack/core/component/sockets.py
index 3cc84dea..0435fe12 100644
--- a/haystack/core/component/sockets.py
+++ b/haystack/core/component/sockets.py
@@ -83,6 +83,7 @@ class Sockets:
     def __setitem__(self, key: str, socket: Union[InputSocket, OutputSocket]):
         """
         Adds a new socket to this Sockets object.
+
         This eases a bit updating the list of sockets after Sockets has been created.
         That should happen only in the `component` decorator.
         """
diff --git a/haystack/core/pipeline/descriptions.py b/haystack/core/pipeline/descriptions.py
index 518e399d..00a6bc25 100644
--- a/haystack/core/pipeline/descriptions.py
+++ b/haystack/core/pipeline/descriptions.py
@@ -14,8 +14,9 @@ logger = logging.getLogger(__name__)
 
 def find_pipeline_inputs(graph: networkx.MultiDiGraph) -> Dict[str, List[InputSocket]]:
     """
-    Collect components that have disconnected input sockets. Note that this method returns *ALL* disconnected
-    input sockets, including all such sockets with default values.
+    Collect components that have disconnected input sockets.
+
+    Note that this method returns *ALL* disconnected input sockets, including all such sockets with default values.
     """
     return {
         name: [socket for socket in data.get("input_sockets", {}).values() if not socket.senders or socket.is_variadic]
diff --git a/haystack/core/pipeline/pipeline.py b/haystack/core/pipeline/pipeline.py
index ca3c683b..75808c53 100644
--- a/haystack/core/pipeline/pipeline.py
+++ b/haystack/core/pipeline/pipeline.py
@@ -76,6 +76,8 @@ class Pipeline:
 
     def __eq__(self, other) -> bool:
         """
+        Pipeline equality is defined by the equality of their serialized form.
+
         Equal pipelines share every metadata, node and edge, but they're not required to use
         the same node instances: this allows pipeline saved and then loaded back to be equal to themselves.
         """
@@ -108,6 +110,7 @@ class Pipeline:
     def to_dict(self) -> Dict[str, Any]:
         """
         Serializes the pipeline to a dictionary.
+
         This is meant to be an intermediate representation but it can be also used to save a pipeline to file.
 
         :returns:
@@ -189,8 +192,7 @@ class Pipeline:
 
     def dumps(self, marshaller: Marshaller = DEFAULT_MARSHALLER) -> str:
         """
-        Returns the string representation of this pipeline according to the
-        format dictated by the `Marshaller` in use.
+        Returns the string representation of this pipeline according to the format dictated by the `Marshaller` in use.
 
         :param marshaller:
             The Marshaller used to create the string representation. Defaults to `YamlMarshaller`.
@@ -201,8 +203,7 @@ class Pipeline:
 
     def dump(self, fp: TextIO, marshaller: Marshaller = DEFAULT_MARSHALLER):
         """
-        Writes the string representation of this pipeline to the file-like object
-        passed in the `fp` argument.
+        Writes the string representation of this pipeline to the file-like object passed in the `fp` argument.
 
         :param fp:
             A file-like object ready to be written to.
@@ -240,8 +241,7 @@ class Pipeline:
         callbacks: Optional[DeserializationCallbacks] = None,
     ) -> "Pipeline":
         """
-        Creates a `Pipeline` object from the string representation read from the file-like
-        object passed in the `fp` argument.
+        Creates a `Pipeline` object from the string representation read from the file-like object passed in the `fp` argument.
 
         :param data:
             The string representation of the pipeline, can be `str`, `bytes` or `bytearray`.
@@ -506,9 +506,10 @@ class Pipeline:
 
     def inputs(self) -> Dict[str, Dict[str, Any]]:
         """
-        Returns a dictionary containing the inputs of a pipeline. Each key in the dictionary
-        corresponds to a component name, and its value is another dictionary that describes the
-        input sockets of that component, including their types and whether they are optional.
+        Returns a dictionary containing the inputs of a pipeline.
+
+        Each key in the dictionary corresponds to a component name, and its value is another dictionary that describes
+        the input sockets of that component, including their types and whether they are optional.
 
         :returns:
             A dictionary where each key is a pipeline component name and each value is a dictionary of
@@ -528,9 +529,10 @@ class Pipeline:
 
     def outputs(self) -> Dict[str, Dict[str, Any]]:
         """
-        Returns a dictionary containing the outputs of a pipeline. Each key in the dictionary
-        corresponds to a component name, and its value is another dictionary that describes the
-        output sockets of that component.
+        Returns a dictionary containing the outputs of a pipeline.
+
+        Each key in the dictionary corresponds to a component name, and its value is another dictionary that describes
+        the output sockets of that component.
 
         :returns:
             A dictionary where each key is a pipeline component name and each value is a dictionary of
@@ -573,6 +575,7 @@ class Pipeline:
     def walk(self) -> Iterator[Tuple[str, Component]]:
         """
         Visits each component in the pipeline exactly once and yields its name and instance.
+
         No guarantees are provided on the visiting order.
 
         :returns:
diff --git a/haystack/core/pipeline/template.py b/haystack/core/pipeline/template.py
index 11c591a3..49141eb7 100644
--- a/haystack/core/pipeline/template.py
+++ b/haystack/core/pipeline/template.py
@@ -22,8 +22,9 @@ class PredefinedPipeline(Enum):
 
 class PipelineTemplate:
     """
-    The PipelineTemplate class enables the straightforward creation of flexible and configurable pipelines using
-    Jinja2 templated YAML files. Specifically designed to simplify the setup of complex data processing pipelines for
+    The PipelineTemplate class enables the straightforward creation of flexible and configurable pipelines using Jinja2 templated YAML files.
+
+    Specifically designed to simplify the setup of complex data processing pipelines for
     a range of NLP tasks—including question answering, retriever augmented generation (RAG), document indexing, among
     others - PipelineTemplate empowers users to dynamically generate pipeline configurations from templates and
     customize components as necessary. Its design philosophy centers on providing an accessible, yet powerful, tool
@@ -60,7 +61,9 @@ class PipelineTemplate:
 
     def __init__(self, template_content: str):
         """
-        Initialize a PipelineTemplate. Besides calling the constructor directly, a set of utility methods is provided
+        Initialize a PipelineTemplate.
+
+        Besides calling the constructor directly, a set of utility methods is provided
         for conveniently create an instance of `PipelineTemplate` from different sources. See `from_string`,
         `from_file`, `from_predefined` and `from_url`.
 
@@ -93,6 +96,7 @@ class PipelineTemplate:
     def from_file(cls, file_path: Union[Path, str]) -> "PipelineTemplate":
         """
         Create a PipelineTemplate from a file.
+
         :param file_path: The path to the file containing the template. Must contain valid Jinja2 syntax.
         :returns: An instance of `PipelineTemplate`.
         """
@@ -103,6 +107,7 @@ class PipelineTemplate:
     def from_predefined(cls, predefined_pipeline: PredefinedPipeline) -> "PipelineTemplate":
         """
         Create a PipelineTemplate from a predefined template. See `PredefinedPipeline` for available options.
+
         :param predefined_pipeline: The predefined pipeline to use.
         :returns: An instance of `PipelineTemplate `.
         """
diff --git a/haystack/core/serialization.py b/haystack/core/serialization.py
index 8edfa89a..df521564 100644
--- a/haystack/core/serialization.py
+++ b/haystack/core/serialization.py
@@ -13,8 +13,7 @@ from haystack.core.errors import DeserializationError, SerializationError
 @dataclass(frozen=True)
 class DeserializationCallbacks:
     """
-    Callback functions that are invoked in specific
-    stages of the pipeline deserialization process.
+    Callback functions that are invoked in specific stages of the pipeline deserialization process.
 
     :param component_pre_init:
         Invoked just before a component instance is
diff --git a/haystack/logging.py b/haystack/logging.py
index 6cd314d6..98e258fb 100644
--- a/haystack/logging.py
+++ b/haystack/logging.py
@@ -26,7 +26,7 @@ class PatchedLogger(typing.Protocol):
         stacklevel: int = 1,
         **kwargs: Any,
     ) -> None:
-        ...
+        """Log a debug message."""
 
     def info(
         self,
@@ -38,7 +38,7 @@ class PatchedLogger(typing.Protocol):
         stacklevel: int = 1,
         **kwargs: Any,
     ) -> None:
-        ...
+        """Log an info message."""
 
     def warn(
         self,
@@ -50,7 +50,7 @@ class PatchedLogger(typing.Protocol):
         stacklevel: int = 1,
         **kwargs: Any,
     ) -> None:
-        ...
+        """Log a warning message."""
 
     def warning(
         self,
@@ -62,7 +62,7 @@ class PatchedLogger(typing.Protocol):
         stacklevel: int = 1,
         **kwargs: Any,
     ) -> None:
-        ...
+        """Log a warning message."""
 
     def error(
         self,
@@ -74,7 +74,7 @@ class PatchedLogger(typing.Protocol):
         stacklevel: int = 1,
         **kwargs: Any,
     ) -> None:
-        ...
+        """Log an error message."""
 
     def critical(
         self,
@@ -86,7 +86,7 @@ class PatchedLogger(typing.Protocol):
         stacklevel: int = 1,
         **kwargs: Any,
     ) -> None:
-        ...
+        """Log a critical message."""
 
     def exception(
         self,
@@ -98,7 +98,7 @@ class PatchedLogger(typing.Protocol):
         stacklevel: int = 1,
         **kwargs: Any,
     ) -> None:
-        ...
+        """Log an exception message."""
 
     def fatal(
         self,
@@ -110,7 +110,7 @@ class PatchedLogger(typing.Protocol):
         stacklevel: int = 1,
         **kwargs: Any,
     ) -> None:
-        ...
+        """Log a fatal message."""
 
     def log(
         self,
@@ -123,17 +123,17 @@ class PatchedLogger(typing.Protocol):
         stacklevel: int = 1,
         **kwargs: Any,
     ) -> None:
-        ...
+        """Log a message."""
 
     def setLevel(self, level: int) -> None:
-        ...
+        """Set the logging level."""
 
 
 def patch_log_method_to_kwargs_only(func: typing.Callable) -> typing.Callable:
     """A decorator to make sure that a function is only called with keyword arguments."""
 
     @functools.wraps(func)
-    def log_only_with_kwargs(
+    def _log_only_with_kwargs(
         msg, *, _: Any = None, exc_info: Any = None, stack_info: Any = False, stacklevel: int = 1, **kwargs: Any
     ) -> Any:  # we need the `_` to avoid a syntax error
         existing_extra = kwargs.pop("extra", {})
@@ -147,14 +147,14 @@ def patch_log_method_to_kwargs_only(func: typing.Callable) -> typing.Callable:
             extra={**existing_extra, **kwargs},
         )
 
-    return log_only_with_kwargs
+    return _log_only_with_kwargs
 
 
 def patch_log_with_level_method_to_kwargs_only(func: typing.Callable) -> typing.Callable:
     """A decorator to make sure that a function is only called with keyword arguments."""
 
     @functools.wraps(func)
-    def log_only_with_kwargs(
+    def _log_only_with_kwargs(
         level,
         msg,
         *,
@@ -177,17 +177,19 @@ def patch_log_with_level_method_to_kwargs_only(func: typing.Callable) -> typing.
             extra={**existing_extra, **kwargs},
         )
 
-    return log_only_with_kwargs
+    return _log_only_with_kwargs
 
 
 def patch_make_records_to_use_kwarg_string_interpolation(original_make_records: typing.Callable) -> typing.Callable:
+    """A decorator to ensure string interpolation is used."""
+
     @functools.wraps(original_make_records)
-    def wrapper(name, level, fn, lno, msg, args, exc_info, func=None, extra=None, sinfo=None) -> Any:
+    def _wrapper(name, level, fn, lno, msg, args, exc_info, func=None, extra=None, sinfo=None) -> Any:
         safe_extra = extra or {}
         interpolated_msg = msg.format(**safe_extra)
         return original_make_records(name, level, fn, lno, interpolated_msg, (), exc_info, func, extra, sinfo)
 
-    return wrapper
+    return _wrapper
 
 
 def _patch_structlog_call_information(logger: logging.Logger) -> None:
@@ -219,11 +221,15 @@ def _patch_structlog_call_information(logger: logging.Logger) -> None:
 
 
 def getLogger(name: str) -> PatchedLogger:
+    """
+    Get the Haystack logger, a patched version of the one from the standard library.
+
+    We patch the default logger methods to make sure that they are only called with keyword arguments.
+    We enforce keyword-arguments because
+        - it brings in consistency
+        - it makes structure logging effective, not just an available feature
+    """
     logger = logging.getLogger(name)
-    # We patch the default logger methods to make sure that they are only called with keyword arguments.
-    # We enforce keyword-arguments because
-    # - it brings in consistency
-    # - it makes structure logging effective, not just an available feature
     logger.debug = patch_log_method_to_kwargs_only(logger.debug)  # type: ignore
     logger.info = patch_log_method_to_kwargs_only(logger.info)  # type: ignore
     logger.warn = patch_log_method_to_kwargs_only(logger.warn)  # type: ignore
@@ -255,7 +261,8 @@ def add_line_and_file(_: "WrappedLogger", __: str, event_dict: "EventDict") -> "
 
 
 def correlate_logs_with_traces(_: "WrappedLogger", __: str, event_dict: "EventDict") -> "EventDict":
-    """Add correlation data for logs.
+    """
+    Add correlation data for logs.
 
     This is useful if you want to correlate logs with traces.
     """
@@ -272,7 +279,8 @@ def correlate_logs_with_traces(_: "WrappedLogger", __: str, event_dict: "EventDi
 
 
 def configure_logging(use_json: Optional[bool] = None) -> None:
-    """Configure logging for Haystack.
+    """
+    Configure logging for Haystack.
 
     - If `structlog` is not installed, we keep everything as it is. The user is responsible for configuring logging
       themselves.
diff --git a/haystack/telemetry/_environment.py b/haystack/telemetry/_environment.py
index afbd252e..5a8e3350 100644
--- a/haystack/telemetry/_environment.py
+++ b/haystack/telemetry/_environment.py
@@ -16,6 +16,8 @@ _IS_DOCKER_CACHE = None
 
 def _in_podman() -> bool:
     """
+    Check if the code is running in a Podman container.
+
     Podman run would create the file /run/.containernv, see:
     https://github.com/containers/podman/blob/main/docs/source/markdown/podman-run.1.md.in#L31
     """
@@ -24,6 +26,8 @@ def _in_podman() -> bool:
 
 def _has_dockerenv() -> bool:
     """
+    Check if the code is running in a Docker container.
+
     This might not work anymore at some point (even if it's been a while now), see:
     https://github.com/moby/moby/issues/18355#issuecomment-220484748
     """
@@ -32,7 +36,7 @@ def _has_dockerenv() -> bool:
 
 def _has_docker_cgroup_v1() -> bool:
     """
-    This only works with cgroups v1
+    This only works with cgroups v1.
     """
     path = "/proc/self/cgroup"  # 'self' should be always symlinked to the actual PID
     return os.path.isfile(path) and any("docker" in line for line in open(path))
@@ -40,8 +44,9 @@ def _has_docker_cgroup_v1() -> bool:
 
 def _has_docker_cgroup_v2() -> bool:
     """
-    cgroups v2 version, inspired from
-    https://github.com/jenkinsci/docker-workflow-plugin/blob/master/src/main/java/org/jenkinsci/plugins/docker/workflow/client/DockerClient.java
+    Check if the code is running in a Docker container using the cgroups v2 version.
+
+    inspired from: https://github.com/jenkinsci/docker-workflow-plugin/blob/master/src/main/java/org/jenkinsci/plugins/docker/workflow/client/DockerClient.java
     """
     path = "/proc/self/mountinfo"  # 'self' should be always symlinked to the actual PID
     return os.path.isfile(path) and any("/docker/containers/" in line for line in open(path))
@@ -61,8 +66,9 @@ def _is_containerized() -> Optional[bool]:
 
 def collect_system_specs() -> Dict[str, Any]:
     """
-    Collects meta data about the setup that is used with Haystack, such as:
-    operating system, python version, Haystack version, transformers version,
+    Collects meta-data about the setup that is used with Haystack.
+
+    Data collected includes: operating system, python version, Haystack version, transformers version,
     pytorch version, number of GPUs, execution environment.
 
     These values are highly unlikely to change during the runtime of the pipeline,
diff --git a/pyproject.toml b/pyproject.toml
index 6b30a7dc..b2e9202f 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -139,7 +139,7 @@ extra-dependencies = [
 
 [tool.hatch.envs.test.scripts]
 e2e = "pytest e2e"
-unit = 'pytest --cov-report xml:coverage.xml --cov="haystack" -m "not integration" test'
+unit = 'pytest --cov-report xml:coverage.xml --cov="haystack" -m "not integration" test {args:test}'
 integration = 'pytest --maxfail=5 -m "integration" test'
 integration-mac = 'pytest --maxfail=5 -m "integration" test -k "not tika"'
 integration-windows = 'pytest --maxfail=5 -m "integration" test -k "not tika"'
diff --git a/releasenotes/notes/exact-match-evaluator-197bb87b65e19d0c.yaml b/releasenotes/notes/exact-match-evaluator-197bb87b65e19d0c.yaml
index ad380617..c872542b 100644
--- a/releasenotes/notes/exact-match-evaluator-197bb87b65e19d0c.yaml
+++ b/releasenotes/notes/exact-match-evaluator-197bb87b65e19d0c.yaml
@@ -1,6 +1,5 @@
 ---
 features:
   - |
-    Add `AnswerExactMatchEvaluator`, a Component that can be used to calculate the Exact Match metric
-    given a list of questions, a list of expected answers for each question and the list of predicted
-    answers for each question.
+    Add `AnswerExactMatchEvaluator`, a component that can be used to calculate the Exact Match metric
+    comparing a list of expected answers with a list of predicted answers.


Test patch:
diff --git a/test/components/evaluators/test_answer_exact_match.py b/test/components/evaluators/test_answer_exact_match.py
index 9c7b395b..2aa4fe7e 100644
--- a/test/components/evaluators/test_answer_exact_match.py
+++ b/test/components/evaluators/test_answer_exact_match.py
@@ -5,21 +5,21 @@ from haystack.components.evaluators import AnswerExactMatchEvaluator
 
 def test_run_with_all_matching():
     evaluator = AnswerExactMatchEvaluator()
-    result = evaluator.run(ground_truth_answers=[["Berlin"], ["Paris"]], predicted_answers=[["Berlin"], ["Paris"]])
+    result = evaluator.run(ground_truth_answers=["Berlin", "Paris"], predicted_answers=["Berlin", "Paris"])
 
     assert result == {"individual_scores": [1, 1], "score": 1.0}
 
 
 def test_run_with_no_matching():
     evaluator = AnswerExactMatchEvaluator()
-    result = evaluator.run(ground_truth_answers=[["Berlin"], ["Paris"]], predicted_answers=[["Paris"], ["London"]])
+    result = evaluator.run(ground_truth_answers=["Berlin", "Paris"], predicted_answers=["Paris", "London"])
 
     assert result == {"individual_scores": [0, 0], "score": 0.0}
 
 
 def test_run_with_partial_matching():
     evaluator = AnswerExactMatchEvaluator()
-    result = evaluator.run(ground_truth_answers=[["Berlin"], ["Paris"]], predicted_answers=[["Berlin"], ["London"]])
+    result = evaluator.run(ground_truth_answers=["Berlin", "Paris"], predicted_answers=["Berlin", "London"])
 
     assert result == {"individual_scores": [1, 0], "score": 0.5}
 
@@ -28,30 +28,42 @@ def test_run_with_complex_data():
     evaluator = AnswerExactMatchEvaluator()
     result = evaluator.run(
         ground_truth_answers=[
-            ["France"],
-            ["9th century", "9th"],
-            ["classical music", "classical"],
-            ["11th century", "the 11th"],
-            ["Denmark", "Iceland", "Norway"],
-            ["10th century", "10th"],
+            "France",
+            "9th century",
+            "9th",
+            "classical music",
+            "classical",
+            "11th century",
+            "the 11th",
+            "Denmark",
+            "Iceland",
+            "Norway",
+            "10th century",
+            "10th",
         ],
         predicted_answers=[
-            ["France"],
-            ["9th century", "10th century", "9th"],
-            ["classic music", "rock music", "dubstep"],
-            ["11th", "the 11th", "11th century"],
-            ["Denmark, Iceland and Norway"],
-            ["10th century", "the first half of the 10th century", "10th", "10th"],
+            "France",
+            "9th century",
+            "10th century",
+            "9th",
+            "classic music",
+            "rock music",
+            "dubstep",
+            "the 11th",
+            "11th century",
+            "Denmark, Iceland and Norway",
+            "10th century",
+            "10th",
         ],
     )
-    assert result == {"individual_scores": [1, 1, 0, 1, 0, 1], "score": 0.6666666666666666}
+    assert result == {"individual_scores": [1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1], "score": 0.3333333333333333}
 
 
 def test_run_with_different_lengths():
     evaluator = AnswerExactMatchEvaluator()
 
     with pytest.raises(ValueError):
-        evaluator.run(ground_truth_answers=[["Berlin"]], predicted_answers=[["Berlin"], ["London"]])
+        evaluator.run(ground_truth_answers=["Berlin"], predicted_answers=["Berlin", "London"])
 
     with pytest.raises(ValueError):
-        evaluator.run(ground_truth_answers=[["Berlin"], ["Paris"]], predicted_answers=[["Berlin"]])
+        evaluator.run(ground_truth_answers=["Berlin", "Paris"], predicted_answers=["Berlin"])