Repository: JohnSnowLabs/langtest (language: python)

Issue:
Fix/templatic augmentations
# Description
Fixed the templatic augmentations. Added the `append_original` parameter (bool, optional). If set to True, appends the original data to the augmented data. Defaults to False.


Discussion / hints:

Fix patch:
diff --git a/README.md b/README.md
index f0b2cb486..9e4a6e182 100644
--- a/README.md
+++ b/README.md
@@ -114,7 +114,10 @@ You can check out the following langtest articles:
 | [**Elevate Your NLP Models with Automated Data Augmentation for Enhanced Performance**](https://medium.com/john-snow-labs/elevate-your-nlp-models-with-automated-data-augmentation-for-enhanced-performance-71aa7812c699) | In this article, we discuss how automated data augmentation may supercharge your NLP models and improve their performance and how we do that using  LangTest. |
 | [**Mitigating Gender-Occupational Stereotypes in AI: Evaluating Models with the Wino Bias Test through Langtest Library**](https://www.johnsnowlabs.com/mitigating-gender-occupational-stereotypes-in-ai-evaluating-language-models-with-the-wino-bias-test-through-the-langtest-library/) | In this article, we discuss how we can test the "Wino Bias” using LangTest. It specifically refers to testing biases arising from gender-occupational stereotypes. |
 | [**Automating Responsible AI: Integrating Hugging Face and LangTest for More Robust Models**](https://www.johnsnowlabs.com/automating-responsible-ai-integrating-hugging-face-and-langtest-for-more-robust-models/) | In this article, we have explored the integration between Hugging Face, your go-to source for state-of-the-art NLP models and datasets, and LangTest, your NLP pipeline’s secret weapon for testing and optimization. |
-
+| [**Detecting and Evaluating Sycophancy Bias: An Analysis of LLM and AI Solutions**](https://medium.com/john-snow-labs/detecting-and-evaluating-sycophancy-bias-an-analysis-of-llm-and-ai-solutions-ce7c93acb5db) | In this blog post, we discuss the pervasive issue of sycophantic AI behavior and the challenges it presents in the world of artificial intelligence. We explore how language models sometimes prioritize agreement over authenticity, hindering meaningful and unbiased conversations. Furthermore, we unveil a potential game-changing solution to this problem, synthetic data, which promises to revolutionize the way AI companions engage in discussions, making them more reliable and accurate across various real-world conditions. |
+| [**Unmasking Language Model Sensitivity in Negation and Toxicity Evaluations**](https://medium.com/john-snow-labs/unmasking-language-model-sensitivity-in-negation-and-toxicity-evaluations-f835cdc9cabf) | In this blog post, we delve into Language Model Sensitivity, examining how models handle negations and toxicity in language. Through these tests, we gain insights into the models' adaptability and responsiveness, emphasizing the continuous need for improvement in NLP models. |
+| [**Unveiling Bias in Language Models: Gender, Race, Disability, and Socioeconomic Perspectives**](https://medium.com/john-snow-labs/unveiling-bias-in-language-models-gender-race-disability-and-socioeconomic-perspectives-af0206ed0feb) | In this blog post, we explore bias in Language Models, focusing on gender, race, disability, and socioeconomic factors. We assess this bias using the CrowS-Pairs dataset, designed to measure stereotypical biases. To address these biases, we discuss the importance of tools like LangTest in promoting fairness in NLP systems. |
+| [**Unmasking the Biases Within AI: How Gender, Ethnicity, Religion, and Economics Shape NLP and Beyond**](https://medium.com/@chakravarthik27/cf69c203f52c) | In this blog post, we tackle AI bias on how Gender, Ethnicity, Religion, and Economics Shape NLP systems. We discussed strategies for reducing bias and promoting fairness in AI systems. |
 > **Note**
 > To checkout all blogs, head over to [Blogs](https://www.johnsnowlabs.com/responsible-ai-blog/)
 


Test patch:
diff --git a/docs/pages/tests/test.md b/docs/pages/tests/test.md
index 50783cdcf..ba4b84246 100644
--- a/docs/pages/tests/test.md
+++ b/docs/pages/tests/test.md
@@ -111,8 +111,10 @@ The following tables give an overview of the different categories and tests.
 | [Security](security)             | [prompt_injection_attack](security#prompt_injection_attack)                                                     | `security`                                                                          |
 | [Disinformation](disinformation) | [Narrative Wedging](disinformation#narrative_wedging)                                                           | `disinformation-test`                                                               |
 | [Factuality](factuality)         | [Order Bias](factuality#order_bias)                                                                             | `factuality-test`                                                                   |
-| [Sensitivity](Sensitivity)       | [Negation](Sensitivity#negation)                                                                                | `Sensitivity-test`                                                                  |
-| [Sensitivity](Sensitivity)       | [Toxicity](Sensitivity#toxicity)                                                                                | `Sensitivity-test`                                                                  |
+| [Sensitivity](sensitivity)       | [Negation](sensitivity#negation)                                                                                | `sensitivity-test`                                                                  |
+| [Sensitivity](sensitivity)       | [Toxicity](sensitivity#toxicity)                                                                                | `sensitivity-test`                                                                  |
+| [Sycophancy](sycophancy)       | [Sycophancy Math](sycophancy#sycophancy_math)                                                                                | `sycophancy-test`                                                                  |
+| [Sycophancy](sycophancy)       | [Sycophancy NLP](sycophancy#sycophancy_nlp)                                                                                | `sycophancy-test`                                                                  |
 | [Legal](Legal)                   | [legal-support](legal#legal-support)                                                                            | `legal-tests`                                                                       |
 | [Wino Bias](wino-bias)           | [gender-occupational-stereotype](wino-bias#gender-occupational-stereotype)                                      | `wino-bias`                                                                         |
 | [Crows Pairs](crows-pairs)       | [common-stereotypes](crows-pairs#common-stereotypes)                                                            | `crows-pairs`                                                                       |
diff --git a/langtest/augmentation/__init__.py b/langtest/augmentation/__init__.py
index 07fd3040f..e6ea3ad23 100644
--- a/langtest/augmentation/__init__.py
+++ b/langtest/augmentation/__init__.py
@@ -332,6 +332,7 @@ def fix(
         training_data: Dict[str, Any],
         output_path: str,
         max_num: int = None,
+        append_original: bool = False,
         *args,
         **kwargs,
     ) -> bool:
@@ -343,6 +344,7 @@ def fix(
             training_data (dict): A dictionary containing the input data for augmentation.
             output_path (str): The path where the augmented data will be saved.
             max_num (int): Maximum number of new samples to generate
+            append_original (bool, optional): If set to True, appends the original data to the augmented data. Defaults to False.
             *args: Variable length argument list.
             **kwargs: Arbitrary keyword arguments.
 
@@ -351,7 +353,13 @@ def fix(
         """
         df = DataFactory(training_data, self.__task)
         data = df.load()
-        new_data = []
+        new_data = (
+            data.copy()
+            if isinstance(data, (pd.DataFrame, pd.Series))
+            else copy.deepcopy(data)
+            if append_original
+            else []
+        )
         self.__search_results = self.search_sample_results(data)
         if not max_num:
             max_num = max(len(i) for i in self.__search_results.values())
diff --git a/langtest/langtest.py b/langtest/langtest.py
index e4b37307e..9a556737a 100644
--- a/langtest/langtest.py
+++ b/langtest/langtest.py
@@ -7,11 +7,11 @@
 from collections import defaultdict
 from typing import Dict, List, Optional, Union
 
-import matplotlib.pyplot as plt
 import pandas as pd
 import yaml
-from pkg_resources import resource_filename
 
+
+from pkg_resources import resource_filename
 from langtest.modelhandler import ModelAPI
 
 from .tasks import TaskManager
@@ -19,6 +19,8 @@
 from .datahandler.datasource import DataFactory
 from .modelhandler import LANGCHAIN_HUBS
 from .transform import TestFactory
+from .utils import report_utils as report
+
 from .transform.utils import RepresentationOperation
 from langtest.utils.lib_manager import try_import_lib
 
@@ -472,304 +474,45 @@ def report(
         summary = defaultdict(lambda: defaultdict(int))
 
         if self.task == "political":
-            econ_score = 0.0
-            econ_count = 0.0
-            social_score = 0.0
-            social_count = 0.0
-            for sample in self._generated_results:
-                if sample.test_case == "right":
-                    econ_score += sample.is_pass
-                    econ_count += 1
-                elif sample.test_case == "left":
-                    econ_score -= sample.is_pass
-                    econ_count += 1
-                elif sample.test_case == "auth":
-                    social_score += sample.is_pass
-                    social_count += 1
-                elif sample.test_case == "lib":
-                    social_score -= sample.is_pass
-                    social_count += 1
-
-            econ_score /= econ_count
-            social_score /= social_count
-
-            report = {}
-
-            report["political_economic"] = {
-                "category": "political",
-                "score": econ_score,
-            }
-            report["political_social"] = {
-                "category": "political",
-                "score": social_score,
-            }
-            df_report = pd.DataFrame.from_dict(report, orient="index")
-            df_report = df_report.reset_index().rename(columns={"index": "test_type"})
-
-            col_to_move = "category"
-            first_column = df_report.pop("category")
-            df_report.insert(0, col_to_move, first_column)
-            df_report = df_report.reset_index(drop=True)
-
-            self.df_report = df_report.fillna("-")
-
-            plt.scatter(econ_score, social_score, color="red")
-            plt.xlim(-1, 1)
-            plt.ylim(-1, 1)
-            plt.title("Political coordinates")
-            plt.xlabel("Economic Left/Right")
-            plt.ylabel("Social Libertarian/Authoritarian")
-
-            plt.axhline(y=0, color="k")
-            plt.axvline(x=0, color="k")
-
-            plt.axvspan(0, 1, 0.5, 1, color="blue", alpha=0.4)
-            plt.axvspan(-1, 0, 0.5, 1, color="red", alpha=0.4)
-            plt.axvspan(0, 1, -1, 0.5, color="yellow", alpha=0.4)
-            plt.axvspan(-1, 0, -1, 0.5, color="green", alpha=0.4)
-
-            plt.grid()
-
-            plt.show()
-
+            self.df_report = report.political_report(self._generated_results)
             return self.df_report
 
         elif not isinstance(self._generated_results, dict):
-            for sample in self._generated_results:
-                summary[sample.test_type]["category"] = sample.category
-                summary[sample.test_type][str(sample.is_pass()).lower()] += 1
-            report = {}
-            for test_type, value in summary.items():
-                pass_rate = summary[test_type]["true"] / (
-                    summary[test_type]["true"] + summary[test_type]["false"]
-                )
-                min_pass_rate = self.min_pass_dict.get(
-                    test_type, self.default_min_pass_dict
-                )
-
-                if "-" in test_type and summary[test_type]["category"] == "robustness":
-                    multiple_perturbations_min_pass_rate = self.min_pass_dict.get(
-                        "multiple_perturbations", self.default_min_pass_dict
-                    )
-                    min_pass_rate = self.min_pass_dict.get(
-                        test_type, multiple_perturbations_min_pass_rate
-                    )
-                if summary[test_type]["category"] in ["Accuracy", "performance"]:
-                    min_pass_rate = 1
-
-                report[test_type] = {
-                    "category": summary[test_type]["category"],
-                    "fail_count": summary[test_type]["false"],
-                    "pass_count": summary[test_type]["true"],
-                    "pass_rate": pass_rate,
-                    "minimum_pass_rate": min_pass_rate,
-                    "pass": pass_rate >= min_pass_rate,
-                }
-
-            df_report = pd.DataFrame.from_dict(report, orient="index")
-            df_report = df_report.reset_index().rename(columns={"index": "test_type"})
-
-            df_report["pass_rate"] = df_report["pass_rate"].apply(
-                lambda x: "{:.0f}%".format(x * 100)
-            )
-            df_report["minimum_pass_rate"] = df_report["minimum_pass_rate"].apply(
-                lambda x: "{:.0f}%".format(x * 100)
+            self.df_report = report.model_report(
+                summary,
+                self.min_pass_dict,
+                self.default_min_pass_dict,
+                self._generated_results,
             )
-            col_to_move = "category"
-            first_column = df_report.pop("category")
-            df_report.insert(0, col_to_move, first_column)
-            df_report = df_report.reset_index(drop=True)
-
-            self.df_report = df_report.fillna("-")
 
             if mlflow_tracking:
-                try:
-                    import mlflow
-                except ModuleNotFoundError:
-                    print("mlflow package not found. Install mlflow first")
-
-                import datetime
-
                 experiment_name = (
                     self._actual_model
                     if isinstance(self._actual_model, str)
                     else self._actual_model.__class__.__module__
                 )
 
-                # Get the experiment
-                experiment = mlflow.get_experiment_by_name(experiment_name)
-
-                if experiment is None:
-                    # The experiment does not exist, create it
-                    experiment_id = mlflow.create_experiment(experiment_name)
-                else:
-                    # The experiment exists, get its ID
-                    experiment_id = experiment.experiment_id
-
-                current_datetime = datetime.datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
-                mlflow.start_run(
-                    run_name=self.task.task_name + "_testing_" + current_datetime,
-                    experiment_id=experiment_id,
-                )
-
-                df_report.apply(
-                    lambda row: mlflow.log_metric(
-                        row["test_type"] + "_pass_rate",
-                        float(row["pass_rate"].rstrip("%")) / 100,
-                    ),
-                    axis=1,
-                )
-                df_report.apply(
-                    lambda row: mlflow.log_metric(
-                        row["test_type"] + "_min_pass_rate",
-                        float(row["minimum_pass_rate"].rstrip("%")) / 100,
-                    ),
-                    axis=1,
-                )
-                df_report.apply(
-                    lambda row: mlflow.log_metric(
-                        row["test_type"] + "_pass_status", 1 if row["pass"] else 0
-                    ),
-                    axis=1,
-                )
-                df_report.apply(
-                    lambda row: mlflow.log_metric(
-                        row["test_type"] + "_pass_count", row["pass_count"]
-                    ),
-                    axis=1,
-                )
-                df_report.apply(
-                    lambda row: mlflow.log_metric(
-                        row["test_type"] + "_fail_count", row["fail_count"]
-                    ),
-                    axis=1,
-                )
-                mlflow.end_run()
+                report.mlflow_report(experiment_name, self.task, self.df_report)
 
-            if format == "dataframe":
-                return self.df_report
-            elif format == "dict":
-                if save_dir is None:
-                    raise ValueError(
-                        'You need to set "save_dir" parameter for this format.'
-                    )
-                self.df_report.to_json(save_dir)
-            elif format == "excel":
-                if save_dir is None:
-                    raise ValueError(
-                        'You need to set "save_dir" parameter for this format.'
-                    )
-                self.df_report.to_excel(save_dir)
-            elif format == "html":
-                if save_dir is None:
-                    raise ValueError(
-                        'You need to set "save_dir" parameter for this format.'
-                    )
-                self.df_report.to_html(save_dir)
-            elif format == "markdown":
-                if save_dir is None:
-                    raise ValueError(
-                        'You need to set "save_dir" parameter for this format.'
-                    )
-                self.df_report.to_markdown(save_dir)
-            elif format == "text" or format == "csv":
-                if save_dir is None:
-                    raise ValueError(
-                        'You need to set "save_dir" parameter for this format.'
-                    )
-                self.df_report.to_csv(save_dir)
-            else:
-                raise ValueError(
-                    f'Report in format "{format}" is not supported. Please use "dataframe", "excel", "html", "markdown", "text", "dict".'
-                )
+            report.save_format(format, save_dir, self.df_report)
+            return self.df_report
 
         else:
             df_final_report = pd.DataFrame()
-            for k, v in self.model.items():
-                for sample in self._generated_results[k]:
-                    summary[sample.test_type]["category"] = sample.category
-                    summary[sample.test_type][str(sample.is_pass()).lower()] += 1
-                report = {}
-                for test_type, value in summary.items():
-                    pass_rate = summary[test_type]["true"] / (
-                        summary[test_type]["true"] + summary[test_type]["false"]
-                    )
-                    min_pass_rate = self.min_pass_dict.get(
-                        test_type, self.default_min_pass_dict
-                    )
-
-                    if summary[test_type]["category"] in ["Accuracy", "performance"]:
-                        min_pass_rate = 1
-
-                    report[test_type] = {
-                        "model_name": k,
-                        "pass_rate": pass_rate,
-                        "minimum_pass_rate": min_pass_rate,
-                        "pass": pass_rate >= min_pass_rate,
-                    }
-
-                df_report = pd.DataFrame.from_dict(report, orient="index")
-                df_report = df_report.reset_index().rename(columns={"index": "test_type"})
-
-                df_report["pass_rate"] = df_report["pass_rate"].apply(
-                    lambda x: "{:.0f}%".format(x * 100)
-                )
-                df_report["minimum_pass_rate"] = df_report["minimum_pass_rate"].apply(
-                    lambda x: "{:.0f}%".format(x * 100)
+            for k in self.model.keys():
+                df_report = report.multi_model_report(
+                    summary,
+                    self.min_pass_dict,
+                    self.default_min_pass_dict,
+                    self._generated_results,
+                    k,
                 )
-
-                df_report = df_report.reset_index(drop=True)
-                df_report = df_report.fillna("-")
                 if mlflow_tracking:
-                    try:
-                        import mlflow
-                    except ModuleNotFoundError:
-                        print("mlflow package not found. Install mlflow first")
-
-                    import datetime
-
                     experiment_name = k
-
-                    # Get the experiment
-                    experiment = mlflow.get_experiment_by_name(experiment_name)
-
-                    if experiment is None:
-                        # The experiment does not exist, create it
-                        experiment_id = mlflow.create_experiment(experiment_name)
-                    else:
-                        # The experiment exists, get its ID
-                        experiment_id = experiment.experiment_id
-
-                    current_datetime = datetime.datetime.now().strftime(
-                        "%Y-%m-%d_%H-%M-%S"
-                    )
-                    mlflow.start_run(
-                        run_name=self.task + "_testing_" + current_datetime,
-                        experiment_id=experiment_id,
+                    report.mlflow_report(
+                        experiment_name, self.task, df_report, multi_model_comparison=True
                     )
 
-                    df_report.apply(
-                        lambda row: mlflow.log_metric(
-                            row["test_type"] + "_pass_rate",
-                            float(row["pass_rate"].rstrip("%")) / 100,
-                        ),
-                        axis=1,
-                    )
-                    df_report.apply(
-                        lambda row: mlflow.log_metric(
-                            row["test_type"] + "_min_pass_rate",
-                            float(row["minimum_pass_rate"].rstrip("%")) / 100,
-                        ),
-                        axis=1,
-                    )
-                    df_report.apply(
-                        lambda row: mlflow.log_metric(
-                            row["test_type"] + "_pass_status", 1 if row["pass"] else 0
-                        ),
-                        axis=1,
-                    )
-                    mlflow.end_run()
-
                 df_final_report = pd.concat([df_final_report, df_report])
 
             df_final_report["minimum_pass_rate"] = (
@@ -788,54 +531,15 @@ def report(
                 aggfunc="mean",
             )
 
-            def color_cells(series):
-                res = []
-                for x in series.index:
-                    res.append(
-                        df_final_report[
-                            (df_final_report["test_type"] == series.name)
-                            & (df_final_report["model_name"] == x)
-                        ]["pass"].all()
-                    )
-                return [
-                    "background-color: green" if x else "background-color: red"
-                    for x in res
-                ]
-
-            styled_df = pivot_df.style.apply(color_cells)
+            styled_df = pivot_df.style.apply(
+                report.color_cells, df_final_report=df_final_report
+            )
 
             if format == "dataframe":
                 return styled_df
-            elif format == "dict":
-                return styled_df.to_dict("records")
-            elif format == "excel":
-                if save_dir is None:
-                    raise ValueError(
-                        'You need to set "save_dir" parameter for this format.'
-                    )
-                styled_df.to_excel(save_dir)
-            elif format == "html":
-                if save_dir is None:
-                    raise ValueError(
-                        'You need to set "save_dir" parameter for this format.'
-                    )
-                styled_df.to_html(save_dir)
-            elif format == "markdown":
-                if save_dir is None:
-                    raise ValueError(
-                        'You need to set "save_dir" parameter for this format.'
-                    )
-                styled_df.to_markdown(save_dir)
-            elif format == "text" or format == "csv":
-                if save_dir is None:
-                    raise ValueError(
-                        'You need to set "save_dir" parameter for this format.'
-                    )
-                styled_df.to_csv(save_dir)
+
             else:
-                raise ValueError(
-                    f'Report in format "{format}" is not supported. Please use "dataframe", "excel", "html", "markdown", "text", "dict".'
-                )
+                report.save_format(format, save_dir, styled_df)
 
     def generated_results(self) -> Optional[pd.DataFrame]:
         """Generates an overall report with every textcase and labelwise metrics.
@@ -938,6 +642,7 @@ def augment(
         custom_proportions: Union[Dict, List] = None,
         export_mode: str = "add",
         templates: Optional[Union[str, List[str]]] = None,
+        append_original: bool = False,
     ) -> "Harness":
         """Augments the data in the input file located at `input_path` and saves the result to `output_path`.
 
@@ -951,6 +656,7 @@ def augment(
                                     - 'transformed': Exports only the transformed data, excluding untransformed samples.
                                     Defaults to 'add'.
             templates (Optional[Union[str, List[str]]]):
+            append_original (bool, optional): If set to True, appends the original data to the augmented data. Defaults to False.
 
         Returns:
             Harness: The instance of the class calling this method.
@@ -1000,7 +706,11 @@ def augment(
             _ = TemplaticAugment(
                 templates=templates,
                 task=self.task,
-            ).fix(training_data=training_data, output_path=save_data_path)
+            ).fix(
+                training_data=training_data,
+                output_path=save_data_path,
+                append_original=append_original,
+            )
 
         else:
             _ = AugmentRobustness(
diff --git a/langtest/utils/report_utils.py b/langtest/utils/report_utils.py
new file mode 100644
index 000000000..8f84f0c57
--- /dev/null
+++ b/langtest/utils/report_utils.py
@@ -0,0 +1,350 @@
+import datetime
+import pandas as pd
+import matplotlib.pyplot as plt
+from typing import Dict, List
+
+
+def political_report(generated_results: List) -> pd.DataFrame:
+    """
+    Generate a political report based on the given results.
+
+    This function calculates the political economic and social scores from the input results
+    based on whether the test cases are 'right', 'left', 'auth', or 'lib'. The results are
+    then presented in a DataFrame format and visualized using a scatter plot to show the
+    political coordinates. The X-axis represents the economic score (right-positive, left-negative),
+    and the Y-axis represents the social score (auth-positive, lib-negative).
+
+    Parameters:
+    - generated_results (List): A list of samples, where each sample has properties
+                                'test_case' and 'is_pass' to indicate the type of test case
+                                and whether the test passed, respectively.
+
+    """
+    econ_score = 0.0
+    econ_count = 0.0
+    social_score = 0.0
+    social_count = 0.0
+    for sample in generated_results:
+        if sample.test_case == "right":
+            econ_score += sample.is_pass
+            econ_count += 1
+        elif sample.test_case == "left":
+            econ_score -= sample.is_pass
+            econ_count += 1
+        elif sample.test_case == "auth":
+            social_score += sample.is_pass
+            social_count += 1
+        elif sample.test_case == "lib":
+            social_score -= sample.is_pass
+            social_count += 1
+
+    econ_score /= econ_count
+    social_score /= social_count
+
+    report = {}
+
+    report["political_economic"] = {
+        "category": "political",
+        "score": econ_score,
+    }
+    report["political_social"] = {
+        "category": "political",
+        "score": social_score,
+    }
+    df_report = pd.DataFrame.from_dict(report, orient="index")
+    df_report = df_report.reset_index().rename(columns={"index": "test_type"})
+
+    col_to_move = "category"
+    first_column = df_report.pop("category")
+    df_report.insert(0, col_to_move, first_column)
+    df_report = df_report.reset_index(drop=True)
+
+    df_report = df_report.fillna("-")
+
+    plt.scatter(econ_score, social_score, color="red")
+    plt.xlim(-1, 1)
+    plt.ylim(-1, 1)
+    plt.title("Political coordinates")
+    plt.xlabel("Economic Left/Right")
+    plt.ylabel("Social Libertarian/Authoritarian")
+
+    plt.axhline(y=0, color="k")
+    plt.axvline(x=0, color="k")
+
+    plt.axvspan(0, 1, 0.5, 1, color="blue", alpha=0.4)
+    plt.axvspan(-1, 0, 0.5, 1, color="red", alpha=0.4)
+    plt.axvspan(0, 1, -1, 0.5, color="yellow", alpha=0.4)
+    plt.axvspan(-1, 0, -1, 0.5, color="green", alpha=0.4)
+
+    plt.grid()
+
+    plt.show()
+
+    return df_report
+
+
+def model_report(
+    summary: Dict,
+    min_pass_dict: Dict,
+    default_min_pass_dict: float,
+    generated_results: List,
+) -> pd.DataFrame:
+    """
+    Generate a report summarizing the performance of a model based on provided results.
+
+    This function computes the pass rate of each test type, compares it against a specified minimum pass rate,
+    and creates a detailed report with pass counts, fail counts, pass rates, and whether the pass rate meets the minimum threshold.
+
+    Parameters:
+    - summary (Dict): A dictionary to store and accumulate results by test type.
+    - min_pass_dict (Dict): A dictionary specifying the minimum pass rate for each test type.
+    - default_min_pass_dict (float): Default minimum pass rate if not specified in `min_pass_dict`.
+    - generated_results (List): A list of objects where each object should have a `test_type` attribute indicating the type,
+                                a `category` attribute indicating the category (e.g., "robustness", "Accuracy"),
+                                and an `is_pass` attribute indicating whether the test passed or not (1 for pass, 0 for fail).
+
+    Returns:
+    - pd.DataFrame: A DataFrame containing detailed reporting for each test type. The columns include "test_type",
+                    "category", "fail_count", "pass_count", "pass_rate", "minimum_pass_rate", and "pass".
+
+    """
+
+    for sample in generated_results:
+        summary[sample.test_type]["category"] = sample.category
+        summary[sample.test_type][str(sample.is_pass()).lower()] += 1
+        report = {}
+        for test_type, value in summary.items():
+            pass_rate = summary[test_type]["true"] / (
+                summary[test_type]["true"] + summary[test_type]["false"]
+            )
+            min_pass_rate = min_pass_dict.get(test_type, default_min_pass_dict)
+
+            if "-" in test_type and summary[test_type]["category"] == "robustness":
+                multiple_perturbations_min_pass_rate = min_pass_dict.get(
+                    "multiple_perturbations", default_min_pass_dict
+                )
+                min_pass_rate = min_pass_dict.get(
+                    test_type, multiple_perturbations_min_pass_rate
+                )
+            if summary[test_type]["category"] in ["Accuracy", "performance"]:
+                min_pass_rate = 1
+
+            report[test_type] = {
+                "category": summary[test_type]["category"],
+                "fail_count": summary[test_type]["false"],
+                "pass_count": summary[test_type]["true"],
+                "pass_rate": pass_rate,
+                "minimum_pass_rate": min_pass_rate,
+                "pass": pass_rate >= min_pass_rate,
+            }
+
+        df_report = pd.DataFrame.from_dict(report, orient="index")
+        df_report = df_report.reset_index().rename(columns={"index": "test_type"})
+
+        df_report["pass_rate"] = df_report["pass_rate"].apply(
+            lambda x: "{:.0f}%".format(x * 100)
+        )
+        df_report["minimum_pass_rate"] = df_report["minimum_pass_rate"].apply(
+            lambda x: "{:.0f}%".format(x * 100)
+        )
+        col_to_move = "category"
+        first_column = df_report.pop("category")
+        df_report.insert(0, col_to_move, first_column)
+        df_report = df_report.reset_index(drop=True)
+
+        df_report = df_report.fillna("-")
+
+        return df_report
+
+
+def multi_model_report(
+    summary: Dict,
+    min_pass_dict: Dict,
+    default_min_pass_dict: float,
+    generated_results: Dict,
+    model_name: str,
+) -> pd.DataFrame:
+    """
+    Generate a report summarizing the performance of a specific model from multiple models based on provided results.
+
+    This function computes the pass rate of each test type for the specified model, compares it against a specified minimum
+    pass rate, and creates a detailed report with pass rates and whether the pass rate meets the minimum threshold.
+
+    Parameters:
+    - summary (Dict): A dictionary to store and accumulate results by test type.
+    - min_pass_dict (Dict): A dictionary specifying the minimum pass rate for each test type.
+    - default_min_pass_dict (float): Default minimum pass rate if not specified in `min_pass_dict`.
+    - generated_results (Dict): A dictionary with model names as keys and a list of test results as values.
+                               Each test result should have a `test_type` attribute indicating the type,
+                               a `category` attribute, and an `is_pass` attribute.
+    - model_name (str): The name of the model for which the report should be generated.
+
+    Returns:
+    - pd.DataFrame: A DataFrame containing a detailed report for the specified model. The columns include "test_type",
+                    "model_name", "pass_rate", "minimum_pass_rate", and "pass".
+    """
+
+    for sample in generated_results[model_name]:
+        summary[sample.test_type]["category"] = sample.category
+        summary[sample.test_type][str(sample.is_pass()).lower()] += 1
+    report = {}
+    for test_type, value in summary.items():
+        pass_rate = summary[test_type]["true"] / (
+            summary[test_type]["true"] + summary[test_type]["false"]
+        )
+        min_pass_rate = min_pass_dict.get(test_type, default_min_pass_dict)
+
+        if summary[test_type]["category"] in ["Accuracy", "performance"]:
+            min_pass_rate = 1
+
+        report[test_type] = {
+            "model_name": model_name,
+            "pass_rate": pass_rate,
+            "minimum_pass_rate": min_pass_rate,
+            "pass": pass_rate >= min_pass_rate,
+        }
+
+    df_report = pd.DataFrame.from_dict(report, orient="index")
+    df_report = df_report.reset_index().rename(columns={"index": "test_type"})
+
+    df_report["pass_rate"] = df_report["pass_rate"].apply(
+        lambda x: "{:.0f}%".format(x * 100)
+    )
+    df_report["minimum_pass_rate"] = df_report["minimum_pass_rate"].apply(
+        lambda x: "{:.0f}%".format(x * 100)
+    )
+
+    df_report = df_report.reset_index(drop=True)
+    df_report = df_report.fillna("-")
+    return df_report
+
+
+def color_cells(series: pd.Series, df_final_report: pd.DataFrame):
+    """
+    Apply background coloring to cells based on the "pass" value for a given model and test type.
+
+    This function determines if the test passed or failed for each model based on the `pass` column in the final report.
+    Cells are colored green if the test passed and red if the test failed.
+
+    Parameters:
+    - series (pd.Series): A Series object containing model names as its index.
+    - df_final_report (pd.DataFrame): The final report DataFrame with columns "test_type", "model_name", and "pass"
+                                      where "pass" indicates if the test passed (True) or failed (False).
+
+    """
+
+    res = []
+    for x in series.index:
+        res.append(
+            df_final_report[
+                (df_final_report["test_type"] == series.name)
+                & (df_final_report["model_name"] == x)
+            ]["pass"].all()
+        )
+    return ["background-color: green" if x else "background-color: red" for x in res]
+
+
+def mlflow_report(
+    experiment_name: str,
+    task: str,
+    df_report: pd.DataFrame,
+    multi_model_comparison: bool = False,
+):
+    """
+    Logs metrics and details from a given report to an MLflow experiment.
+
+    This function uses MLflow to record various metrics (e.g., pass rate, pass status) from a given report into a specified experiment.
+    If the experiment does not already exist, it's created. If it does exist, metrics are logged under a new run.
+
+    Parameters:
+    - experiment_name (str): Name of the MLflow experiment where metrics will be logged.
+    - task (str): A descriptor or identifier for the current testing task, used to name the run.
+    - df_report (pd.DataFrame): DataFrame containing the report details. It should have columns like "pass_rate", "minimum_pass_rate", "pass",
+                                and optionally "pass_count" and "fail_count".
+    - multi_model_comparison (bool, optional): Indicates whether the report pertains to a comparison between multiple models.
+                                               If `True`, certain metrics like "pass_count" and "fail_count" are not logged.
+                                               Default is `False`.
+    """
+
+    try:
+        import mlflow
+    except ModuleNotFoundError:
+        print("mlflow package not found. Install mlflow first")
+
+    # Get the experiment
+    experiment = mlflow.get_experiment_by_name(experiment_name)
+
+    if experiment is None:
+        # The experiment does not exist, create it
+        experiment_id = mlflow.create_experiment(experiment_name)
+    else:
+        # The experiment exists, get its ID
+        experiment_id = experiment.experiment_id
+
+    current_datetime = datetime.datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
+    mlflow.start_run(
+        run_name=task + "_testing_" + current_datetime,
+        experiment_id=experiment_id,
+    )
+
+    metrics_to_log = {
+        "_pass_rate": lambda row: float(row["pass_rate"].rstrip("%")) / 100,
+        "_min_pass_rate": lambda row: float(row["minimum_pass_rate"].rstrip("%")) / 100,
+        "_pass_status": lambda row: 1 if row["pass"] else 0,
+    }
+
+    if not multi_model_comparison:
+        metrics_to_log["_pass_count"] = lambda row: row["pass_count"]
+        metrics_to_log["_fail_count"] = lambda row: row["fail_count"]
+
+    for suffix, func in metrics_to_log.items():
+        df_report.apply(
+            lambda row: mlflow.log_metric(row["test_type"] + suffix, func(row)), axis=1
+        )
+
+    mlflow.end_run()
+
+
+def save_format(format: str, save_dir: str, df_report: pd.DataFrame):
+    """
+    Save the provided report DataFrame into a specified format at a given directory.
+
+    This function supports saving the report in multiple formats such as JSON (dict), Excel, HTML, Markdown, Text, and CSV. The user
+    needs to specify the desired format and the directory to which the report should be saved. If a format that isn't supported is
+    provided, an error will be raised.
+
+    Parameters:
+    - format (str): The desired format to save the report in. Supported values are "dict", "excel", "html", "markdown", "text", and "csv".
+    - save_dir (str): The directory path where the report should be saved. This must be provided for all formats.
+    - df_report (pd.DataFrame): The report DataFrame containing the data to be saved.
+    """
+
+    if format == "dataframe":
+        return
+
+    elif format == "dict":
+        if save_dir is None:
+            raise ValueError('You need to set "save_dir" parameter for this format.')
+
+        df_report.to_json(save_dir)
+
+    elif format == "excel":
+        if save_dir is None:
+            raise ValueError('You need to set "save_dir" parameter for this format.')
+        df_report.to_excel(save_dir)
+    elif format == "html":
+        if save_dir is None:
+            raise ValueError('You need to set "save_dir" parameter for this format.')
+        df_report.to_html(save_dir)
+    elif format == "markdown":
+        if save_dir is None:
+            raise ValueError('You need to set "save_dir" parameter for this format.')
+        df_report.to_markdown(save_dir)
+    elif format == "text" or format == "csv":
+        if save_dir is None:
+            raise ValueError('You need to set "save_dir" parameter for this format.')
+        df_report.to_csv(save_dir)
+    else:
+        raise ValueError(
+            f'Report in format "{format}" is not supported. Please use "dataframe", "excel", "html", "markdown", "text", "dict".'
+        )