internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: deepchecks/deepchecks

Issue:
[BUG] some randomness occurring in tabular sample
**Describe the bug**
The results are not the same even when setting random states

**To Reproduce**
Run model error / performance report a couple of times

**Expected behavior**
Same results


Discussion:

Resolving patch:
diff --git a/docs/source/tutorials/tabular/plot_quickstart_in_5_minutes.py b/docs/source/tutorials/tabular/plot_quickstart_in_5_minutes.py
--- a/docs/source/tutorials/tabular/plot_quickstart_in_5_minutes.py
+++ b/docs/source/tutorials/tabular/plot_quickstart_in_5_minutes.py
@@ -43,7 +43,7 @@
 df_train, df_test = train_test_split(iris_df, stratify=iris_df[label_col], random_state=0)
 
 # Train Model
-rf_clf = RandomForestClassifier()
+rf_clf = RandomForestClassifier(random_state=0)
 rf_clf.fit(df_train.drop(label_col, axis=1), df_train[label_col]);
 
 #%%


Test patch:
Repository: deepchecks/deepchecks

Issue:
[BUG] Warning has been seen to appear below OK checks in suite condition table
**Describe the bug**
In a certain case, the condition table has been seen to have a warning appear blow the OK checks

**Expected behavior**
Condition order in the table should be:
Fail with Display
Fail without Display
Warning with Display
Warning without Display
Pass with Display
Pass without Display

Other then that order should conform to order of checks in the check_result. 

**Screenshots**
![image](https://user-images.githubusercontent.com/17730502/159263773-6c2acb34-5648-44c8-8447-c6837f8840f0.png)
![image](https://user-images.githubusercontent.com/17730502/159263786-393647e8-d3ac-4b04-a8f8-51079b6da8a8.png)




Discussion:
fixed in #928 because it was cause by using category warn but pass=true
I was mistaken

Resolving patch:
diff --git a/deepchecks/core/display_pandas.py b/deepchecks/core/display_pandas.py
--- a/deepchecks/core/display_pandas.py
+++ b/deepchecks/core/display_pandas.py
@@ -96,7 +96,8 @@ def get_conditions_table(check_results: Union['CheckResult', List['CheckResult']
                 link = f'<a href=#{check_id}>{check_header}</a>'
             else:
                 link = check_header
-                sort_value = 1 if sort_value == 1 else 5  # if it failed but has no display still show on top
+                # if it has no display show on bottom for the category (lower priority)
+                sort_value += 0.1
             table.append([icon, link, cond_result.name,
                          cond_result.details, sort_value])
 
diff --git a/deepchecks/utils/typing.py b/deepchecks/utils/typing.py
--- a/deepchecks/utils/typing.py
+++ b/deepchecks/utils/typing.py
@@ -46,7 +46,6 @@ class BasicModel(Protocol):
 
     def predict(self, X) -> List[Hashable]:
         """Predict on given X."""
-        ...
 
 
 @runtime_checkable
@@ -55,4 +54,3 @@ class ClassificationModel(BasicModel, Protocol):
 
     def predict_proba(self, X) -> List[Hashable]:
         """Predict probabilities on given X."""
-        ...


Test patch:
Repository: deepchecks/deepchecks

Issue:
[BUG] [CV] typo in context prediction validation
**Describe the bug**
it says batch_to_images instead of infer_on_batch
![Screen Shot 2022-03-30 at 17 56 08](https://user-images.githubusercontent.com/26321553/160865194-9c29434c-3644-44d3-913c-15ef3aab2375.png)




Discussion:

Resolving patch:
diff --git a/deepchecks/vision/context.py b/deepchecks/vision/context.py
--- a/deepchecks/vision/context.py
+++ b/deepchecks/vision/context.py
@@ -97,7 +97,7 @@ def __init__(self,
                         msg = f'infer_on_batch() was not implemented in {dataset_type} ' \
                            f'dataset, some checks will not run'
                     except ValidationError as ex:
-                        msg = f'batch_to_images() was not implemented correctly in {dataset_type}, the ' \
+                        msg = f'infer_on_batch() was not implemented correctly in {dataset_type}, the ' \
                            f'validation has failed with the error: {ex}. To test your prediction formatting use the ' \
                            f'function `vision_data.validate_prediction(batch, model, device)`'
 


Test patch:
Repository: deepchecks/deepchecks

Issue:
[BUG] [Vision] Old Kaggle Kernels don't work with deepchecks vision
**Describe the bug**
error importing vision on `kaggle kernel version from 2020-06-15`
```
----> 1 from deepchecks import vision

~/.local/lib/python3.7/site-packages/deepchecks/vision/__init__.py in <module>
     12 import logging
     13 from .batch_wrapper import Batch
---> 14 from .vision_data import VisionData
     15 from .context import Context
     16 from .suite import Suite

~/.local/lib/python3.7/site-packages/deepchecks/vision/vision_data.py in <module>
    530 
    531 
--> 532 class IndicesSequentialSampler(Sampler[int]):
    533     """Samples elements sequentially from a given list of indices, without replacement.
    534 

TypeError: 'type' object is not subscriptable

```
**To Reproduce**
Steps to reproduce the behavior:
1. Go to a kaggle notebook with an old version 
2. install deepchecks with `!pip install --user “git+https://github.com/deepchecks/deepchecks.git@main#egg=deepchecks[vision]”`
3. import deepchecks vision `from deepchecks import vision`

**Expected behavior**
No error, expected to work.

**Environment (please complete the following information):**
 - OS: kaggle kernel version from 2020-06-15
 - Python Version: 3.7.6
 - Deepchecks Version: 6.0



Discussion:

Resolving patch:
diff --git a/deepchecks/vision/vision_data.py b/deepchecks/vision/vision_data.py
--- a/deepchecks/vision/vision_data.py
+++ b/deepchecks/vision/vision_data.py
@@ -496,16 +496,19 @@ def _get_data_loader_copy(data_loader: DataLoader, n_samples: int = None, shuffl
     @staticmethod
     def _get_data_loader_props(data_loader: DataLoader):
         """Get properties relevant for the copy of a DataLoader."""
-        return {
-            'num_workers': data_loader.num_workers,
-            'collate_fn': data_loader.collate_fn,
-            'pin_memory': data_loader.pin_memory,
-            'timeout': data_loader.timeout,
-            'worker_init_fn': data_loader.worker_init_fn,
-            'prefetch_factor': data_loader.prefetch_factor,
-            'persistent_workers': data_loader.persistent_workers,
-            'dataset': copy(data_loader.dataset)
-        }
+        attr_list = ['num_workers',
+                     'collate_fn',
+                     'pin_memory',
+                     'timeout',
+                     'worker_init_fn',
+                     'prefetch_factor',
+                     'persistent_workers']
+        aval_attr = {}
+        for attr in attr_list:
+            if hasattr(data_loader, attr):
+                aval_attr[attr] = getattr(data_loader, attr)
+        aval_attr['dataset'] = copy(data_loader.dataset)
+        return aval_attr
 
     @staticmethod
     def _get_data_loader_sequential(data_loader: DataLoader):
@@ -529,7 +532,7 @@ def _get_data_loader_sequential(data_loader: DataLoader):
         return data_loader.__class__(**props), sampler
 
 
-class IndicesSequentialSampler(Sampler[int]):
+class IndicesSequentialSampler(Sampler):
     """Samples elements sequentially from a given list of indices, without replacement.
 
     Args:


Test patch:
Repository: deepchecks/deepchecks

Issue:
Upgrade gpu torch vision
#### Reference Issues/PRs

Example: Fixes #1234. See also #3456.
Please use keywords (e.g., Fixes) to create link to the issues or pull requests
you resolved, so that they will automatically be closed when your pull request
is merged. See https://github.com/blog/1506-closing-issues-via-pull-requests


#### What does this implement/fix? Explain your changes.


#### Any other comments?



---
Thanks for contributing a pull request! Please ensure you have taken a look at
the contribution guidelines: https://github.com/deepchecks/deepchecks/blob/main/CONTRIBUTING.rst



Discussion:

Resolving patch:
diff --git a/docs/source/conf.py b/docs/source/conf.py
--- a/docs/source/conf.py
+++ b/docs/source/conf.py
@@ -146,6 +146,8 @@
         plotly_sg_scraper,
     ),
     "pypandoc": True,
+    "default_thumb_file": os.path.join(os.path.dirname(os.path.abspath(__file__)),
+                                       "_static/sphx_glr_deepchecks_icon.png"),
 }
 
 # Add any paths that contain templates here, relative to this directory.
diff --git a/docs/source/tutorials/vision/plot_extractors_validating.py b/docs/source/tutorials/vision/plot_extractors_validating.py
--- a/docs/source/tutorials/vision/plot_extractors_validating.py
+++ b/docs/source/tutorials/vision/plot_extractors_validating.py
@@ -1,12 +1,12 @@
 # -*- coding: utf-8 -*-
 """
-Test Your Extractors In Development
-===================================
-"Extractors" are used to transform the structure of your data to the
-structure required for deepchecks. To help ensure your extractors work
-as intended, deepchecks has built-in helper functions for extractor
-validation. This guide will demonstrate how to use this helper function
-to implement formatters for your own data, step by step.
+Test Your Vision Data Class During Development
+================================================
+"Data Classes" are used to transform the structure of your data to the
+structure required for deepchecks. To help ensure they work
+as intended, deepchecks has built-in helper functions for
+validating their structure. This guide will demonstrate how to use this helper function
+to implement them for your own data, step by step.
 
 
 **Structure:**


Test patch:
Repository: deepchecks/deepchecks

Issue:
[BUG][CV] Model Error Analysis - the exception for task OTHER is different than other checks
**Describe the bug**
When running a OTHER task, the Model Error Analysis exception is different from the rest of the checks. Need to standardize it.

**To Reproduce**
Run a "OTHER" task in the model performance suite. 


Discussion:

Resolving patch:
diff --git a/deepchecks/vision/checks/performance/model_error_analysis.py b/deepchecks/vision/checks/performance/model_error_analysis.py
--- a/deepchecks/vision/checks/performance/model_error_analysis.py
+++ b/deepchecks/vision/checks/performance/model_error_analysis.py
@@ -85,6 +85,7 @@ def __init__(self,
 
     def initialize_run(self, context: Context):
         """Initialize property and score lists."""
+        context.assert_task_type(TaskType.CLASSIFICATION, TaskType.OBJECT_DETECTION)
         self._train_properties = defaultdict(list)
         self._test_properties = defaultdict(list)
         self._train_scores = []
@@ -120,7 +121,7 @@ def scoring_func(predictions, labels):
             def scoring_func(predictions, labels):
                 return per_sample_mean_iou(predictions, labels)
         else:
-            raise DeepchecksValueError(f'Unsupported task type {dataset.task_type}')
+            raise DeepchecksValueError(f'Should not reach here! Unsupported task type {dataset.task_type}')
 
         if isinstance(predictions, torch.Tensor):
             predictions = predictions.cpu().detach().numpy()


Test patch:
Repository: deepchecks/deepchecks

Issue:
[BUG] [CV] Image property drift - fix error
**Describe the bug**
fix error in image property drift

**To Reproduce**
Steps to reproduce the behavior:
1. Go to https://www.kaggle.com/code/itay94/notebookc2fb7a0a35

**Screenshots**
![image](https://user-images.githubusercontent.com/20860465/161570603-b7535655-1beb-4aa0-b7a7-4c7d56d17b94.png)

**Environment (please complete the following information):**
 - OS: Kaggle
 - Python Version: Kaggle
 - Deepchecks Version: 0.6.0




Discussion:
and shouldn't fail condition...
Desired remedy:
1. For the drift check - if not enough samples, raise an exception (like the one raised in Model Error Analysis if r squared is to low)
2. There should be a forth state of condition, signifying a python exception and outputting the exception.

Resolving patch:
diff --git a/deepchecks/vision/checks/distribution/image_property_drift.py b/deepchecks/vision/checks/distribution/image_property_drift.py
--- a/deepchecks/vision/checks/distribution/image_property_drift.py
+++ b/deepchecks/vision/checks/distribution/image_property_drift.py
@@ -139,10 +139,10 @@ def compute(self, context: Context) -> CheckResult:
         df_train = pd.DataFrame(self._train_properties)
         df_test = pd.DataFrame(self._test_properties)
         if len(df_train) < self.min_samples or len(df_test) < self.min_samples:
-            return CheckResult(
-                value=None,
-                display=f'Not enough samples to calculate drift score, min {self.min_samples} samples required.',
-                header='Image Property Drift'
+            raise NotEnoughSamplesError(
+                f'Not enough samples to calculate drift score, minimum {self.min_samples} samples required'
+                f', but got {len(df_train)} and {len(df_test)} samples in the train and test datasets.'
+                'Use \'min_samples\' parameter to change the requirement.'
             )
 
         figures = {}


Test patch:
Repository: deepchecks/deepchecks

Issue:
[BUG] TrainTestFeatureDrift doesn't display anything for dataframe data
**Describe the bug**
When running the check on naïve iris dataframe, we get no display at all, rather than a display showing the exact same distribution which will happen if constructing a Dataset from the dataframe prior.

**To Reproduce**

```
import pandas as pd
from deepchecks.tabular.dataset import Dataset
from deepchecks.tabular.checks import TrainTestFeatureDrift

iris = pd.read_csv('https://raw.githubusercontent.com/mwaskom/seaborn-data/master/iris.csv')
TrainTestFeatureDrift().run(iris, iris)

ds = Dataset(iris)
TrainTestFeatureDrift().run(ds , ds )

```

**Expected behavior**
Exact same behavior for the two cases. 

**Screenshots**
![image](https://user-images.githubusercontent.com/17730502/160836976-ca1b04df-6627-4c58-b634-9ddcc0575690.png)



Discussion:

Resolving patch:
diff --git a/deepchecks/tabular/checks/distribution/whole_dataset_drift.py b/deepchecks/tabular/checks/distribution/whole_dataset_drift.py
--- a/deepchecks/tabular/checks/distribution/whole_dataset_drift.py
+++ b/deepchecks/tabular/checks/distribution/whole_dataset_drift.py
@@ -91,7 +91,6 @@ def run_logic(self, context: Context) -> CheckResult:
         """
         train_dataset = context.train
         test_dataset = context.test
-        features = train_dataset.features
         cat_features = train_dataset.cat_features
         numerical_features = train_dataset.numerical_features
 
@@ -104,8 +103,8 @@ def run_logic(self, context: Context) -> CheckResult:
         </span>
         """
 
-        values_dict, displays = run_whole_dataset_drift(train_dataframe=train_dataset.data[features],
-                                                        test_dataframe=test_dataset.data[features],
+        values_dict, displays = run_whole_dataset_drift(train_dataframe=train_dataset.features_columns,
+                                                        test_dataframe=test_dataset.features_columns,
                                                         numerical_features=numerical_features,
                                                         cat_features=cat_features,
                                                         sample_size=sample_size, random_state=self.random_state,


Test patch:
diff --git a/deepchecks/tabular/checks/distribution/train_test_feature_drift.py b/deepchecks/tabular/checks/distribution/train_test_feature_drift.py
--- a/deepchecks/tabular/checks/distribution/train_test_feature_drift.py
+++ b/deepchecks/tabular/checks/distribution/train_test_feature_drift.py
@@ -105,6 +105,8 @@ def run_logic(self, context: Context) -> CheckResult:
         train_dataset: Dataset = context.train
         test_dataset: Dataset = context.test
         features_importance = context.features_importance
+        train_dataset.assert_features()
+        test_dataset.assert_features()
 
         train_dataset = train_dataset.select(
                 self.columns, self.ignore_columns
Repository: deepchecks/deepchecks

Issue:
[BUG] [CV] confusion metrix mixed index bug
**Describe the bug**
Confusion metrix throws error when there are mixed indexes (mixed labels?)(needs to look into bit more)

**To Reproduce**
Run confusion metrix with the following notebook:
https://www.kaggle.com/code/itay94/using-deepchecks-to-validate-ssdlite-model?scriptVersionId=92088847

**Expected behavior**
Cast the str to int or don't support str

**Screenshots**
![screen_shot_2022-04-05_at_17 44 45](https://user-images.githubusercontent.com/26321553/161785939-bafedb2f-1aed-49c2-8ac8-2ac40a45bee5.png)



Discussion:

Resolving patch:
diff --git a/deepchecks/vision/checks/performance/confusion_matrix.py b/deepchecks/vision/checks/performance/confusion_matrix.py
--- a/deepchecks/vision/checks/performance/confusion_matrix.py
+++ b/deepchecks/vision/checks/performance/confusion_matrix.py
@@ -99,6 +99,7 @@ def compute(self, context: Context, dataset_kind: DatasetKind = None) -> CheckRe
 
         dataset = context.get_data_by_kind(dataset_kind)
         matrix = pd.DataFrame(self.matrix).T
+        matrix = matrix.rename(index={-1: 'no-overlapping'}, columns={-1: 'no-overlapping'})
         matrix.replace(np.nan, 0, inplace=True)
 
         classes = sorted(
@@ -176,7 +177,7 @@ def update_object_detection(self, predictions, labels):
                 # detections are empty, update matrix for labels
                 for label in image_labels:
                     label_class = int(label[0].item())
-                    self.matrix[label_class]['no-overlapping'] += 1
+                    self.matrix[label_class][-1] += 1
                 continue
 
             list_of_ious = (
@@ -210,12 +211,12 @@ def update_object_detection(self, predictions, labels):
                     detected_class = int(image_detections[detection_index][5])
                     self.matrix[label_class][detected_class] += 1
                 else:
-                    self.matrix[label_class]['no-overlapping'] += 1
+                    self.matrix[label_class][-1] += 1
 
             for detection_index, detection in enumerate(detections_passed_threshold):
                 if n_of_matches > 0 and not (matches[:, 1] == detection_index).any():
                     detected_class = int(detection[5])
-                    self.matrix['no-overlapping'][detected_class] += 1
+                    self.matrix[-1][detected_class] += 1
 
     def update_classification(self, predictions, labels):
         """Update the confusion matrix by batch for classification task."""


Test patch:
Repository: deepchecks/deepchecks

Issue:
[BUG] Model Error Analysis tabular example doesn't catch anything
**Describe the bug**
Seems that after updating the log loss function, the example ceased catching the weak segments. 



Discussion:

Resolving patch:
diff --git a/deepchecks/utils/performance/error_model.py b/deepchecks/utils/performance/error_model.py
--- a/deepchecks/utils/performance/error_model.py
+++ b/deepchecks/utils/performance/error_model.py
@@ -128,7 +128,7 @@ def error_model_display(error_fi: pd.Series,
     dataset : tabular.Dataset
         Dataset to create display from
     model : Optional[Any]
-        Originial model for calculating the score on tabular data (Must come with scorer)
+        Original model for calculating the score on tabular data (Must come with scorer)
     scorer : Optional[Callable]
         Scorer to calculate the output values of the segments (Must come with model)
     max_features_to_show : int
@@ -179,9 +179,14 @@ def error_model_display(error_fi: pd.Series,
 
             # Partition data into two groups - weak and ok:
 
-            in_segment_indicis = cum_sum_ratio <= min_segment_size
-            weak_categories = error_per_segment_ser.index[in_segment_indicis]
-            ok_categories = error_per_segment_ser.index[~in_segment_indicis]
+            # In cum_sum_ratio the index is the categories sorted from "weakest" (the highest model error) to
+            # strongest, and the value is the cumulative fraction of all data.
+            # The weak segment contains all the weakest categories until they reach together a fraction of data of at
+            # least min_segment_size.
+            first_weakest_category_to_pass_min_segment_size = np.where(cum_sum_ratio.values >= min_segment_size)[0][0]
+            in_segment_indices = np.arange(len(cum_sum_ratio)) <= first_weakest_category_to_pass_min_segment_size
+            weak_categories = error_per_segment_ser.index[in_segment_indices]
+            ok_categories = error_per_segment_ser.index[~in_segment_indices]
 
             # Calculate score for each group and assign label and color
             if scorer:
diff --git a/deepchecks/vision/vision_data.py b/deepchecks/vision/vision_data.py
--- a/deepchecks/vision/vision_data.py
+++ b/deepchecks/vision/vision_data.py
@@ -316,7 +316,7 @@ def copy(self, n_samples: int = None, shuffle: bool = False, random_state: int =
         ----------
         n_samples : int , default: None
             take only this number of samples to the copied DataLoader. The samples which will be chosen are affected
-            by random_state (fixed random state will return consistent sampels).
+            by random_state (fixed random state will return consistent samples).
         shuffle : bool, default: False
             Whether to shuffle the samples order. The shuffle is affected random_state (fixed random state will return
             consistent order)


Test patch: