Repository: pegasystems/pega-datascientist-tools

Issue:
is_used is always true when there are multiple treatments
### pdstools version checks

- [x] I have checked that this issue has not already been reported.

- [x] I have confirmed this bug exists on the [latest version](https://pypi.org/project/pdstools/) of pdstools.


### Issue description

We are trying to determine whether an action is active (response count has increased over snapshots)[here](https://github.com/pegasystems/pega-datascientist-tools/blob/master/python/pdstools/adm/Aggregates.py#L548) but the logic is broken when an action has multipe treatments

### Reproducible example

```Python
.
```

### Expected behavior

could be an inline .over() for treatments first.

### Installed versions

.

Discussion:
@operdeck: Over isnt the solution but taking the (optional) Treatment column into account is.
Fixed in https://github.com/pegasystems/pega-datascientist-tools/pull/375

Solution patch:
diff --git a/python/pdstools/adm/Aggregates.py b/python/pdstools/adm/Aggregates.py
index 3dd21a3c..b0455003 100644
--- a/python/pdstools/adm/Aggregates.py
+++ b/python/pdstools/adm/Aggregates.py
@@ -538,7 +538,8 @@ class Aggregates:
             (
                 model_data.group_by(
                     ([] if grouping is None else grouping)
-                    + ["Name"]
+                    + ["Name"] 
+                    + (["Treatment"] if "Treatment" in self.datamart.context_keys else [])
                     + (["Issue"] if "Issue" in self.datamart.context_keys else [])
                     + (["Group"] if "Group" in self.datamart.context_keys else [])
                 )
@@ -905,7 +906,7 @@ class Aggregates:
             A Polars LazyFrame containing the global predictor overview with the following fields:
 
             - PredictorName - The name of the predictor
-            - Response Count - The total number of responses for this predictor
+            - Response Count Min/Max - The total number of responses for this predictor
             - Positives - The total number of positive responses for this predictor
             - Min, Mean, Median, Max - The min, mean, median and max performance of the predictor (AUC)
         """
@@ -919,7 +920,8 @@ class Aggregates:
             .agg(
                 [
                     # weighted performance
-                    pl.sum("ResponseCount").alias("Response Count"),
+                    pl.min("ResponseCount").alias("Response Count Min"),
+                    pl.max("ResponseCount").alias("Response Count Max"),
                     pl.col("ModelID")
                     .filter(EntryType="Active")
                     .n_unique()
diff --git a/python/pdstools/reports/HealthCheck.qmd b/python/pdstools/reports/HealthCheck.qmd
index a8940821..e104e325 100644
--- a/python/pdstools/reports/HealthCheck.qmd
+++ b/python/pdstools/reports/HealthCheck.qmd
@@ -200,9 +200,9 @@ else:
 
 ```
 
-This document gives a global overview of the Adaptive models and (if available) the predictors of the models. It is generated from a Python markdown (Quarto) file in the [Pega Data Scientist Tools](https://github.com/pegasystems/pega-datascientist-tools). This is open-source software and comes without guarantees. Off-line reports for individual models can be created as well, see [Wiki](https://github.com/pegasystems/pega-datascientist-tools/wiki).
+This document gives a global overview of the Adaptive models and (if available) the predictors of the models. It is generated from a Python markdown (Quarto) file in the [Pega Data Scientist Tools](https://github.com/pegasystems/pega-datascientist-tools). PDS Tools is open-source software and comes without guarantees. Off-line reports for Naive Bayes Adaptive models can be created as well, see [Wiki](https://github.com/pegasystems/pega-datascientist-tools/wiki).
 
-We provide guidance and best practices where possible. However these are generic guidelines and may or may not be applicable to the specific use case and situation of the implementation. The recommendations are strongly geared towards the CDH use cases and may not apply to, for example, Process AI.
+We provide guidance and best practices as much as possible. However these guidelines are generic and may or may not be applicable to a specific use case or implementation. The recommendations are strongly geared towards the CDH and may not apply to, for example, Process AI.
 
 ```{python}
 # Start with a global bubble chart. Maybe later replace by
@@ -233,7 +233,7 @@ except Exception as e:
 ```
 
 ::: {.callout-tip title="Guidance"}
-The [Plotly](https://plotly.com/python/) charts have [user controls for panning, zooming etc](https://plotly.com/chart-studio-help/zoom-pan-hover-controls/) but note that these interactive plots do not render well in portals like Sharepoint or Box. It is preferable to view them from a browser.
+The charts are built with [Plotly](https://plotly.com/python/) and have [user controls for panning, zooming etc](https://plotly.com/chart-studio-help/zoom-pan-hover-controls/) but note that these interactive plots do not render well in portals like Sharepoint or Box. It is preferable to view them from a browser.
 :::
 
 # Overview of the Channels
@@ -244,7 +244,7 @@ If a channel has two model configurations with a naming pattern like “Adm_1234
 
 The "OmniChannel" percentage is an indicator of the overlap of actions between channels. The general recommendation is to have cross-channel actions and channel-specific treatments. This metric counts the overlap between the actions in one channel and all the other channels. It takes the average overlap with the other channels. At 100% the actions are truly cross-channel and at 0% the actions are specific to this one channel.
 
-In the view below, we look at the last 30 days of data (or shorter if the data is for less than 30 days).
+In the view below, we look at the last **30 days** of data (or shorter if the data is for less than 30 days).
 
 ```{python}
 df_channel_overview = (
@@ -397,7 +397,7 @@ configurations it seems that the framework **{framework_usage}**.
 :::
 
 ::: callout-note
-The total number of responses and positives are summed up over the models for that channel. If the models are configured differently, e.g. in a serial manner (shadow pattern for example) rather than in parallel (independently learning champion challenger setup) the actual counts per channel may be different. Ultimately, the real counts per channel can only be derived from interaction history data.
+In the ADM models, the total number of responses and positives are summed up over the models for that channel. If the models are configured differently, e.g. in a serial manner (shadow pattern for example) rather than in parallel (independently learning champion challenger setup) the actual counts per channel may be off. Ultimately, the real counts per channel can only be derived from **Interaction History data**.
 :::
 
 ## Exclude unused Channels
@@ -406,20 +406,20 @@ Channels with very few or no responses will be excluded from subsequent analyses
 
 # Overview of the Predictions
 
-When data from Prediction Studio is made available to this notebook, we show an overview of the predictions in the system. A Prediction is a construct around models that allows for different modeling patterns and supports an overall monitoring of engagement lift, Performance etc.
+When data from Prediction Studio is available, we show an overview of the predictions in the system. A Prediction is a construct around models that allows for different modeling patterns and supports an overall monitoring of engagement lift, Performance etc.
 
 ::: {.content-hidden when-meta="analysis.predictions"}
 However, Prediction Data is **not available**. This is either because the data is not captured or because Predictions are not used, for example when the NBAD framework is not used.
 :::
 
 ::: {.content-hidden unless-meta="analysis.predictions"}
-Predictions monitor performance in a different way than ADM models do. The measurement of Lift depends on having a control group for which the arbitration priority is random rather than model driven.
+Predictions measure performance in a different way than ADM models do. The measurement of Lift depends on having a control group for which the arbitration priority is random rather than model driven.
 
 Lift in Predictions is the lift in *engagement* (clicks, accepts) and is defined as:
 
 $Lift = \frac{CTR_{test} - CTR_{control}}{CTR_{control}}$
 
-with "test" the model group and "control" random test group. A lift of 0% means the model group is doing no better than the random group. A lift of 100% means the models are giving two times the number of positive responses. Lift can be negative, up to -100%, which would mean the model group never results in any positive response. By default the size of the random group is 1-2% and is configured in Prediction Studio and Impact Analyzer (when using NBAD).
+with "test" the model group and "control" random test group. A lift of 0% means the model group is doing no better than the random group. A lift of 100% means the models are giving two times the number of positive responses. Lift can be negative, up to -100%, which would mean the model group never results in any positive response. By default the size of the random group is 1-2% and is configured in Prediction Studio and Impact Analyzer (when using NBA Designer).
 
 ```{python}
 if prediction_file_path:
@@ -521,6 +521,7 @@ try:
 
     plt = predictions.plot.responsecount_trend("1w")
     plt.update_layout(autosize=True, height=400, )
+    plt.update_yaxes(title="Weekly Responses")
 
     plt.show()
 except Exception as e:
@@ -1303,7 +1304,7 @@ if datamart.predictor_data is not None:
             # .with_columns(MeanPlotData=pl.col("Mean")),
             rowname_col="PredictorName",
             cdh_guidelines=cdh_guidelines,
-            highlight_limits={"Responses": "Response Count"},
+            highlight_limits={"Responses":[ "Response Count Min", "Response Count Max"]},
         )
         .tab_options(container_height="400px", container_overflow_y=True)
         .tab_spanner(
@@ -1544,7 +1545,7 @@ display(
 
 ### Models that have never been used
 
-These models have no responses at all: no positives but also no negatives. The models for these actions/treatments exist, so they must have been created in the evaluation of the actions/treatments, but they were never selected to show to the customer, so never received any responses.
+These models have no responses at all: no positives but also no negatives. The models for these actions/treatments exist, so they must have been created during decisioning, but they were never selected to show to the customer, so never received any responses.
 
 Often these represent actions that never made it into production and were only used to test out logic. But it could also be that the response mechanism is broken. It could for example be caused by outcome labels that are returned by the channel application not matching the configuration of the adaptive models.
 
@@ -1552,7 +1553,7 @@ Often these represent actions that never made it into production and were only u
 
 These models have been used but never received a “positive” response. This means the action/treatments that these models represent have been selected, so an “impression” has been made to the customer, but they never received a “positive” response.
 
-This could be because no customer ever found the proposition attractive, but it could also be because the response loop is not working correctly. Possibly Pega “believes” the user received the action but the actual channel application did never show it.
+This could be because no customer ever found the proposition attractive, but it could also be because the response loop is not working correctly. Possibly the response loop is set up so that Pega decisions automatically are counted as “negatives“ even if the channel application did never show it to the user.
 
 ### Models that are still in an immature phase of learning
 
@@ -1632,6 +1633,11 @@ except Exception as e:
 
 ## Number of Responses over Time
 
+::: {.callout-tip title="Guidance"}
+-   Look out for consecutive days without any responses
+-   Response counts should be cumulative, so negative spikes in the delta view may indicate that models have been 'reset'
+:::
+
 ```{python}
 facets = "Configuration"
 unique_count = len(datamart.unique_configurations)
@@ -1648,7 +1654,8 @@ try:
 
     response_counts = fig_update_facet_height(response_counts, facet_cols, 200, 250)
     response_counts.for_each_xaxis(lambda xaxis: xaxis.update(title=""))
-    response_counts.update_layout(yaxis_title="Response Count")
+    response_counts.update_layout(yaxis_title="Responses")
+    response_counts.for_each_yaxis(lambda yaxis: yaxis.update(title="Responses"))
 
     response_counts.show()
 except ValueError as e:
diff --git a/python/pdstools/reports/ModelReport.qmd b/python/pdstools/reports/ModelReport.qmd
index a3e9e52a..6874fde5 100644
--- a/python/pdstools/reports/ModelReport.qmd
+++ b/python/pdstools/reports/ModelReport.qmd
@@ -193,7 +193,7 @@ except Exception as e:
 ```
 
 ::: callout-tip
-The charts (built with [Plotly](https://plotly.com/python/)) have [user controls for panning, zooming etc](https://plotly.com/chart-studio-help/zoom-pan-hover-controls/). These interactive plots do not render well in portals like Sharepoint or Box. It is preferable to view them from a browser.
+The charts are built with [Plotly](https://plotly.com/python/) and have [user controls for panning, zooming etc](https://plotly.com/chart-studio-help/zoom-pan-hover-controls/) but note that these interactive plots do not render well in portals like Sharepoint or Box. It is preferable to view them from a browser.
 :::
 
 ## Model Performance


Test patch:
diff --git a/python/tests/test_Aggregates.py b/python/tests/test_Aggregates.py
index c21b801b..56306080 100644
--- a/python/tests/test_Aggregates.py
+++ b/python/tests/test_Aggregates.py
@@ -92,7 +92,6 @@ def dm_minimal():
 
     return ADMDatamart(model_df=data.lazy())
 
-
 def test_init(dm_aggregates):
     assert dm_aggregates
 
@@ -112,7 +111,7 @@ def test_predictors_overview(dm_aggregates):
 
 def test_predictors_global_overview(dm_aggregates):
     assert dm_aggregates.predictors_global_overview().collect().height == 89
-    assert dm_aggregates.predictors_global_overview().collect().width == 7
+    assert dm_aggregates.predictors_global_overview().collect().width == 8
 
 def test_summary_by_channel(dm_aggregates):
     summary_by_channel = dm_aggregates.summary_by_channel().collect()
@@ -166,17 +165,65 @@ def test_summary_by_channel(dm_aggregates):
     ]
 
 def test_summary_by_channel_timeslices(dm_minimal):
-    s1 = dm_minimal.aggregates.summary_by_channel(start_date=datetime(2033, 1, 1), end_date=datetime(2033, 1, 31)).collect()
+    s1 = dm_minimal.aggregates.summary_by_channel(start_date=datetime(2033, 1, 1), end_date=datetime(2033, 1, 31), debug=True).collect()
+    
     assert s1["Actions"].to_list() == [1,2]
     assert s1["New Actions"].to_list() == [1,2]
     assert s1["Used Actions"].to_list() == [1,1]
     assert s1["isValid"].to_list() == [False,True]
+
     s2 = dm_minimal.aggregates.summary_by_channel(start_date=datetime(2033, 2, 1), end_date=datetime(2033, 2, 28)).collect()
     assert s2["Actions"].to_list() == [1,2]
     assert s2["New Actions"].to_list() == [0,1]
     assert s2["Used Actions"].to_list() == [1,1]
     assert s2["isValid"].to_list() == [False,True]
 
+def test_used_actions():
+    events = [
+        # A1 and A2 are both used in january
+        ("20330101", "A", "A1", 100, 1000, "Mobile", "Inbound"),
+        ("20330101", "A", "A2", 500, 1000, "Mobile", "Inbound"),
+        ("20330115", "A", "A1", 150, 1500, "Mobile", "Inbound"),
+        ("20330115", "A", "A2", 600, 1200, "Mobile", "Inbound"),    
+        # But neither is used in february    
+        # See issue 370: if there are treatments, with different counts, that should not lead to flagging as used action
+        ("20330201", "A", "A1", 150, 1500, "Mobile", "Inbound"),
+        ("20330201", "A", "A2", 600, 1200, "Mobile", "Inbound"),
+        ("20330228", "A", "A1", 150, 1500, "Mobile", "Inbound"),
+        ("20330228", "A", "A2", 600, 1200, "Mobile", "Inbound"),
+        # Two are used in march
+        ("20330301", "B", "A1", 150, 1500, "Mobile", "Inbound"),
+        ("20330301", "A", "A2", 600, 1200, "Mobile", "Inbound"),
+        ("20330328", "B", "A1", 160, 1600, "Mobile", "Inbound"),
+        ("20330328", "A", "A2", 700, 1400, "Mobile", "Inbound"),
+    ]
+
+    data = pl.DataFrame(
+        modeldata_from_scratch().collect().to_dicts()[0]
+        | {
+            v: [e[i] for e in events]
+            for i, v in enumerate(
+                [
+                    "SnapshotTime",
+                    "Name",
+                    "Treatment",
+                    "Positives",
+                    "ResponseCount",
+                    "Channel",
+                    "Direction",
+                ]
+            )
+        }
+    ).with_columns(ModelID=pl.format("ModelID_{}", "Name"))
+
+    dm_used = ADMDatamart(model_df=data.lazy())
+
+    summary = dm_used.aggregates.summary_by_channel().collect()
+    assert summary['Used Actions'].to_list() == [2]
+
+    summary = dm_used.aggregates.summary_by_channel(by_period="1mo").collect()
+    assert summary['Used Actions'].to_list() == [1,0,2]
+
 def test_custom_channel_mapping(dm_aggregates):
     dm_aggregates.datamart.model_data = dm_aggregates.datamart.model_data.with_columns(
         Channel=pl.when(pl.col.Channel == "SMS")