Repository: deepchecks/deepchecks

Issue:
[FEAT] Add force permutation option in UnusedFeatures check
`UnusedFeatures` looks for the feature importance attr if possible, which is bias in some [sklearn models](https://explained.ai/rf-importance/).
I suggest adding an option to override this behaviour. This is already an option in `.utils.features.calculate_feature_importance`.





Discussion:
Thanks for the suggestion @ShaharKSegal. 
Because we are using lru cache to compute the FI only once, we should also enable a global flag (similar to _PERMUTATION_IMPORTANCE_TIMEOUT) that forces all FI calculations for to use permutation importance.
@noamzbr Is that an axiom? A user will always want the FI option to be the same across all checks?
My axiom is that permutation importance is always better than the internal sklearn one (as @ShaharKSegal said), so if we computed it once for some reason, makes sense that all other checks should use this result as well
Shouldn't this force permutation importance yes/no be somehow integrated in the model context structure?

Resolving patch:
diff --git a/deepchecks/checks/methodology/unused_features.py b/deepchecks/checks/methodology/unused_features.py
--- a/deepchecks/checks/methodology/unused_features.py
+++ b/deepchecks/checks/methodology/unused_features.py
@@ -23,6 +23,7 @@
 
 from deepchecks.base.check_context import CheckRunContext
 from deepchecks import Dataset, CheckResult, TrainTestBaseCheck, ConditionResult, ConditionCategory
+from deepchecks.utils.typing import BasicModel
 
 
 __all__ = ['UnusedFeatures']
@@ -64,8 +65,26 @@ def __init__(self, feature_importance_threshold: float = 0.2, feature_variance_t
         self.n_top_unused_to_show = n_top_unused_to_show
         self.random_state = random_state
 
-    def run_logic(self, context: CheckRunContext) -> CheckResult:
-        """Run check.
+    def run(self,
+            train_dataset: Dataset,
+            test_dataset: Dataset,
+            model: BasicModel = None,
+            feature_importance_force_permutation: bool = False,
+            feature_importance_timeout: int = None) -> CheckResult:
+        """Run the check.
+
+        Parameters
+        ----------
+        train_dataset : Dataset
+            dataset representing data an estimator was fitted on
+        test_dataset : Dataset
+            dataset representing data an estimator predicts on
+        model : BasicModel
+            A scikit-learn-compatible fitted estimator instance
+        feature_importance_force_permutation : bool , default: False
+            force calculation of permutation features importance
+        feature_importance_timeout : int , default: None
+            timeout in second for the permutation features importance calculation
 
         Returns
         -------
@@ -79,6 +98,13 @@ def run_logic(self, context: CheckRunContext) -> CheckResult:
             If neither train dataset nor test dataset exist, or either of the dataset objects are
             not a Dataset instance with a label.
         """
+        c = CheckRunContext(train_dataset, test_dataset, model,
+                            feature_importance_force_permutation=feature_importance_force_permutation,
+                            feature_importance_timeout=feature_importance_timeout)
+        return self.run_logic(c)
+
+    def run_logic(self, context: CheckRunContext) -> CheckResult:
+        """Run check."""
         if context.have_test():
             dataset = context.test
         else:


Test patch: