Repository: deepchecks/deepchecks

Issue:
[FEAT] [CV] no example for condition in class performance and for simple model
Add example of using conditions in the examples of Class Performance and Simple Model Comparison.


Discussion:
saw more issues in other notebooks


Resolving patch:
diff --git a/docs/source/conf.py b/docs/source/conf.py
--- a/docs/source/conf.py
+++ b/docs/source/conf.py
@@ -101,7 +101,7 @@
     "examples/checks/performance/index": "../../../examples/tabular/checks/performance/examples/index.html",
     "user-guide/supported_models": "..//user-guide/tabular/supported_models.html",
     "examples/guides/create_a_custom_suite": "../../user-guide/general/customizations/examples/plot_create_a_custom_suite.html",
-    "examples/guides/export_outputs_to_wandb": "../..//user-guide/general/exporting_results/examples/plot_exports_output_to_wandb.html",
+    "examples/guides/export_outputs_to_wandb": "../../user-guide/general/exporting_results/examples/plot_exports_output_to_wandb.html",
     "examples/guides/save_suite_result_as_html": "../../user-guide/general/exporting_results/examples/plot_save_suite_results_as_html.html",
 }
 imgmath_image_format = 'svg'
diff --git a/docs/source/examples/vision/checks/performance/source/plot_class_performance.py b/docs/source/examples/vision/checks/performance/source/plot_class_performance.py
--- a/docs/source/examples/vision/checks/performance/source/plot_class_performance.py
+++ b/docs/source/examples/vision/checks/performance/source/plot_class_performance.py
@@ -74,7 +74,6 @@
 # the COCO dataset defined it - mean of the average precision per class, over
 # the range [0.5, 0.95, 0.05] of IoU thresholds.
 
-import numpy as np
 from deepchecks.vision.datasets.detection import coco
 
 #%%
@@ -88,9 +87,24 @@
 train_ds = coco.load_dataset(train=True, object_type='VisionData')
 test_ds = coco.load_dataset(train=False, object_type='VisionData')
 
-#%%#
+#%%
 # Run the check
 # -------------
 
 check = ClassPerformance(show_only='best')
 check.run(train_ds, test_ds, yolo)
+
+#%%
+# Define a Condition
+# ==================
+# We can also define a condition to validate that our model performance is above a certain threshold.
+# The condition is defined as a function that takes the results of the check as input and
+# returns a ConditionResult object.
+
+check = ClassPerformance(show_only='worst')
+check.add_condition_test_performance_not_less_than(0.2)
+result = check.run(train_ds, test_ds, yolo)
+result
+
+#%%
+# We detected that for several classes our model performance is below the threshold.
diff --git a/docs/source/examples/vision/checks/performance/source/plot_simple_model_comparison.py b/docs/source/examples/vision/checks/performance/source/plot_simple_model_comparison.py
--- a/docs/source/examples/vision/checks/performance/source/plot_simple_model_comparison.py
+++ b/docs/source/examples/vision/checks/performance/source/plot_simple_model_comparison.py
@@ -81,3 +81,31 @@ class has equal probability.
 # The value is a dataframe that contains the metrics' values for each class and dataset:
 
 result.value.sort_values(by=['Class', 'Metric']).head(10)
+
+#%%
+# Define a condition
+# ==================
+# We can define on our check a condition that will validate our model is better than
+# the simple model by a given margin called gain. For classification we check the gain
+# for each class separately and if there is a class that doesn't pass the defined gain
+# the condition will fail.
+#
+# The performance gain is the percent of the improved performance out of the
+# "remaining" unattained performance. Its purpose is to reflect the significance of
+# the said improvement. Take for example for a metric between 0 and 1. A change of
+# only 0.03 that takes us from 0.95 to 0.98 is highly significant (especially in an
+# imbalance scenario), but improving from 0.1 to 0.13 is not a great achievement.
+#
+# The gain is calculated as: :math:`gain = \frac{\text{model score} - \text{simple score}}
+# {\text{perfect score} - \text{simple score}}`
+#
+# Let's add a condition to the check and see what happens when it fails:
+
+check = SimpleModelComparison(strategy='stratified')
+check.add_condition_gain_not_less_than(min_allowed_gain=0.99)
+result = check.run(train_ds, test_ds, mnist_model)
+result
+
+#%%
+# We detected that for several classes our gain did not passed the target gain we
+# defined, therefore it failed.


Test patch: