internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: automl/auto-sklearn

Issue:
Add warning if dependencies are not met
There should be a warning if one of the following dependencies is not met:
- scikit-learn==0.17
- smac==0.0.1
- lockfile>=0.10
- ConfigSpace>=0.2.1
- pyrfr==0.2.1



Discussion:
Any ETA on an upgrade to scikit-learn?

Given the amount of necessary changes before we can do that, I expect that it will take us at least two more months to do so.

I need to mention you @Ayaro before I can assign this issue to you.

Hm, doesn't work. I think I need to add you to the organization.

Closed by #188 


Resolving patch:
diff --git a/autosklearn/util/dependencies.py b/autosklearn/util/dependencies.py
--- a/autosklearn/util/dependencies.py
+++ b/autosklearn/util/dependencies.py
@@ -33,7 +33,7 @@ def _verify_package(name, operation, version):
     try:
         module = pkg_resources.get_distribution(name)
     except pkg_resources.DistributionNotFound:
-        raise MissingPackageError(name) from None
+        raise MissingPackageError(name)
 
     if not operation:
         return


Test patch:
Repository: automl/auto-sklearn

Issue:
Error while including estimators but excluding preprocessors
Sorry if I'm saying nosense or not using GitHub correctly, it's my first time using it.

I was trying today to use AutoML while including some estimators and excluding some preprocessors simultaneously.
I got an error concerning the line 28-31 of autosklearn/util/pipeline.py:

if include_estimators is not None and \
            exclude_preprocessors is not None:
        raise ValueError('Cannot specify include_estimators and '
                         'exclude_estimators.')

The same happens at line 550 of smbo.py

Shouldn't be exclude_estimators instead of exclude_preprocessors? I think what the code is trying to do is to check the mutual exclusivity of include and exclude arguments, but it checks for the wrong argument.

Sorry again if what I'm saying is incorrect, which is probably the case considering my code expertise. Thank you for your awesome work in this.



Discussion:

Resolving patch:
diff --git a/autosklearn/smbo.py b/autosklearn/smbo.py
--- a/autosklearn/smbo.py
+++ b/autosklearn/smbo.py
@@ -530,7 +530,6 @@ def run_smbo(self):
 
         self.scenario = Scenario(scenario_dict)
 
-
         # TODO rebuild target algorithm to be it's own target algorithm
         # evaluator, which takes into account that a run can be killed prior
         # to the model being fully fitted; thus putting intermediate results
@@ -545,8 +544,9 @@ def run_smbo(self):
             include['preprocessor'] = self.include_preprocessors
         elif self.exclude_preprocessors is not None:
             exclude['preprocessor'] = self.exclude_preprocessors
+
         if self.include_estimators is not None and \
-                self.exclude_preprocessors is not None:
+                self.exclude_estimators is not None:
             raise ValueError('Cannot specify include_estimators and '
                              'exclude_estimators.')
         elif self.include_estimators is not None:
diff --git a/autosklearn/util/pipeline.py b/autosklearn/util/pipeline.py
--- a/autosklearn/util/pipeline.py
+++ b/autosklearn/util/pipeline.py
@@ -25,8 +25,9 @@ def get_configuration_space(info,
         include['preprocessor'] = include_preprocessors
     elif exclude_preprocessors is not None:
         exclude['preprocessor'] = exclude_preprocessors
+
     if include_estimators is not None and \
-            exclude_preprocessors is not None:
+            exclude_estimators is not None:
         raise ValueError('Cannot specify include_estimators and '
                          'exclude_estimators.')
     elif include_estimators is not None:


Test patch:
Repository: automl/auto-sklearn

Issue:
automl.sprint_statistics    output error
![default](https://user-images.githubusercontent.com/6996297/35727228-614de816-0842-11e8-84e3-c2b050dda68f.png)

`timtout` and `memout` number error


Discussion:
Thanks a lot for reporting this, this is indeed a bug!

Resolving patch:
diff --git a/autosklearn/automl.py b/autosklearn/automl.py
--- a/autosklearn/automl.py
+++ b/autosklearn/automl.py
@@ -771,10 +771,10 @@ def sprint_statistics(self):
         num_crash = sum([s == 'Crash' for s in cv_results['status']])
         sio.write('  Number of crashed target algorithm runs: %d\n' % num_crash)
         num_timeout = sum([s == 'Timeout' for s in cv_results['status']])
-        sio.write('  Number of target algorithms that exceeded the memory '
+        sio.write('  Number of target algorithms that exceeded the time '
                   'limit: %d\n' % num_timeout)
         num_memout = sum([s == 'Memout' for s in cv_results['status']])
-        sio.write('  Number of target algorithms that exceeded the time '
+        sio.write('  Number of target algorithms that exceeded the memory '
                   'limit: %d\n' % num_memout)
         return sio.getvalue()
 


Test patch:
Repository: automl/auto-sklearn

Issue:
Something strange about the code in method load_ensemble
I have checked the code and find something strange in file backend, method load_ensemble.
line 387, sort result seems like this in the picture
![1](https://user-images.githubusercontent.com/42017222/46990935-1ecd2480-d136-11e8-8582-a8184a446949.PNG)
pick the last one seems not correct, Could you have a look at this? 





    
Something strange about the code in method load_ensemble
I have checked the code and find something strange in file backend, method load_ensemble.
line 387, sort result seems like this in the picture
![1](https://user-images.githubusercontent.com/42017222/46990935-1ecd2480-d136-11e8-8582-a8184a446949.PNG)
pick the last one seems not correct, Could you have a look at this? 





    


Discussion:

Resolving patch:
diff --git a/autosklearn/util/backend.py b/autosklearn/util/backend.py
--- a/autosklearn/util/backend.py
+++ b/autosklearn/util/backend.py
@@ -407,9 +407,10 @@ def save_ensemble(self, ensemble, idx, seed):
         except Exception:
             pass
 
-        filepath = os.path.join(self.get_ensemble_dir(),
-                                '%s.%s.ensemble' % (str(seed),
-                                                    str(idx)))
+        filepath = os.path.join(
+            self.get_ensemble_dir(),
+            '%s.%s.ensemble' % (str(seed), str(idx).zfill(10))
+        )
         with tempfile.NamedTemporaryFile('wb', dir=os.path.dirname(
                 filepath), delete=False) as fh:
             pickle.dump(ensemble, fh)


Test patch:
Repository: automl/auto-sklearn

Issue:
RM print statement
There seems to be an unneccary print statement in the code:
```
/usr/local/lib/python3.6/dist-packages/sklearn/ensemble/weight_boosting.py:29: DeprecationWarning: numpy.core.umath_tests is an internal NumPy module and should not be imported. It will be removed in a future NumPy release.
  from numpy.core.umath_tests import inner1d
[WARNING] [2019-05-14 15:52:03,539:EnsembleBuilder(1):d74860caaa557f473ce23908ff7ba369] No models better than random - using Dummy Score!
[WARNING] [2019-05-14 15:52:03,571:EnsembleBuilder(1):d74860caaa557f473ce23908ff7ba369] No models better than random - using Dummy Score!
1
['/tmp/autosklearn_tmp_707_2898/.auto-sklearn/ensembles/1.0000000000.ensemble', '/tmp/autosklearn_tmp_707_2898/.auto-sklearn/ensembles/1.0000000001.ensemble', '/tmp/autosklearn_tmp_707_2898/.auto-sklearn/ensembles/1.0000000002.ensemble', '/tmp/autosklearn_tmp_707_2898/.auto-sklearn/ensembles/1.0000000003.ensemble', '/tmp/autosklearn_tmp_707_2898/.auto-sklearn/ensembles/1.0000000004.ensemble', '/tmp/autosklearn_tmp_707_2898/.auto-sklearn/ensembles/1.0000000005.ensemble', '/tmp/autosklearn_tmp_707_2898/.auto-sklearn/ensembles/1.0000000006.ensemble', '/tmp/autosklearn_tmp_707_2898/.auto-sklearn/ensembles/1.0000000007.ensemble', '/tmp/autosklearn_tmp_707_2898/.auto-sklearn/ensembles/1.0000000008.ensemble', '/tmp/autosklearn_tmp_707_2898/.auto-sklearn/ensembles/1.0000000009.ensemble', '/tmp/autosklearn_tmp_707_2898/.auto-sklearn/ensembles/1.0000000010.ensemble', '/tmp/autosklearn_tmp_707_2898/.auto-sklearn/ensembles/1.0000000011.ensemble']
```
For example code, please see:
https://colab.research.google.com/drive/1Au5sGCegoGLLxqrIs85GCCW0oWIHqwQw


Discussion:

Resolving patch:
diff --git a/autosklearn/util/backend.py b/autosklearn/util/backend.py
--- a/autosklearn/util/backend.py
+++ b/autosklearn/util/backend.py
@@ -409,7 +409,6 @@ def load_ensemble(self, seed):
             self.logger.warning('Directory %s does not exist' % ensemble_dir)
             return None
 
-        print(seed)
         if seed >= 0:
             indices_files = glob.glob(
                 os.path.join(glob.escape(ensemble_dir), '%s.*.ensemble' % seed)
@@ -422,7 +421,6 @@ def load_ensemble(self, seed):
 
         with open(indices_files[-1], 'rb') as fh:
             ensemble_members_run_numbers = pickle.load(fh)
-        print(indices_files)
 
         return ensemble_members_run_numbers
 


Test patch:
Repository: automl/auto-sklearn

Issue:
[question!] No models better than random - using Dummy Score!
[WARNING] [2019-11-07 18:29:07,226:EnsembleBuilder(1):d74860caaa557f473ce23908ff7ba369] No models better than random - using Dummy Score!

what I should do to deal with this problem?


Discussion:

Resolving patch:
diff --git a/autosklearn/ensemble_builder.py b/autosklearn/ensemble_builder.py
--- a/autosklearn/ensemble_builder.py
+++ b/autosklearn/ensemble_builder.py
@@ -375,9 +375,14 @@ def get_n_best_preds(self):
             ],
             key=lambda x: x[1],
         )))
+        # number of models available
+        num_keys = len(sorted_keys)
         # remove all that are at most as good as random
         # note: dummy model must have run_id=1 (there is not run_id=0)
-        dummy_score = list(filter(lambda x: x[2] == 1, sorted_keys))[0]
+        dummy_scores = list(filter(lambda x: x[2] == 1, sorted_keys))
+        # number of dummy models
+        num_dummy = len(dummy_scores)
+        dummy_score = dummy_scores[0]
         self.logger.debug("Use %f as dummy score" %
                           dummy_score[1])
         sorted_keys = filter(lambda x: x[1] > dummy_score[1], sorted_keys)
@@ -385,8 +390,13 @@ def get_n_best_preds(self):
         sorted_keys = list(filter(lambda x: x[2] > 1, sorted_keys))
         if not sorted_keys:
             # no model left; try to use dummy score (num_run==0)
-            self.logger.warning("No models better than random - "
-                                "using Dummy Score!")
+            # log warning when there are other models but not better than dummy model
+            if num_keys > num_dummy:
+                self.logger.warning("No models better than random - "
+                                    "using Dummy Score!"
+                                    "Number of models besides current dummy model: %d. Number of dummy models: %d",
+                                    num_keys - 1,
+                                    num_dummy)
             sorted_keys = [
                 (k, v["ens_score"], v["num_run"]) for k, v in self.read_preds.items()
                 if v["seed"] == self.seed and v["num_run"] == 1


Test patch:
diff --git a/test/test_metalearning/pyMetaLearn/test_meta_features_sparse.py b/test/test_metalearning/pyMetaLearn/test_meta_features_sparse.py
--- a/test/test_metalearning/pyMetaLearn/test_meta_features_sparse.py
+++ b/test/test_metalearning/pyMetaLearn/test_meta_features_sparse.py
@@ -6,12 +6,12 @@
 # Make the super class importable
 sys.path.append(os.path.dirname(__file__))
 
-import arff
-import numpy as np
-from scipy import sparse
-from sklearn.impute import SimpleImputer
+import arff  # noqa: E402
+import numpy as np  # noqa: E402
+from scipy import sparse  # noqa: E402
+from sklearn.impute import SimpleImputer  # noqa: E402
 
-from sklearn.preprocessing import StandardScaler
+from sklearn.preprocessing import StandardScaler  # noqa: E402
 
 from autosklearn.pipeline.components.data_preprocessing.\
     data_preprocessing import DataPreprocessor  # noqa: E402
Repository: automl/auto-sklearn

Issue:
Ranks for metrics whose optimum is 0 like mean_absolute_error is in reverse order
## Describe the bug ##
There are various metrics whose optimum value is 0 and for them greater the value, worst are the results. But this is not considered while calculating ranks in cv_results. While calculating ranks, it is always considered that greater the value, better the results. 

## To Reproduce ##
Steps to reproduce the behaviour:
Run any regression problem with metric mean_absolute_error.

Results from cv_results_
```
>>> cv_results_[["mean_test_score","rank_test_scores", "status"]]
   mean_test_score  rank_test_scores   status
0     2.354041e+00                 6  Success
1     2.649970e+00                 4  Success
2     2.758945e+00                 3  Success
3     2.147484e+09                 1  Timeout
4     2.581370e+00                 5  Success
5     2.269768e+00                 7  Success
6     2.147484e+09                 1  Timeout
```

## Expected Result ##
```
>>> cv_results_[["mean_test_score","rank_test_scores", "status"]]
   mean_test_score  rank_test_scores   status
0     2.354041e+00                 2  Success
1     2.649970e+00                 4  Success
2     2.758945e+00                 5  Success
3     2.147484e+09                 6  Timeout
4     2.581370e+00                 3  Success
5     2.269768e+00                 1  Success
6     2.147484e+09                 6  Timeout
```

## Environment and installation: ##

Please give details about your installation:

* Is your installation in a virtual environment or conda environment? virtualenviron
* Python version - 3.7
* Auto-sklearn version - From development branch


Discussion:

Resolving patch:
diff --git a/autosklearn/automl.py b/autosklearn/automl.py
--- a/autosklearn/automl.py
+++ b/autosklearn/automl.py
@@ -1268,8 +1268,8 @@ def cv_results_(self):
 
         results['mean_fit_time'] = np.array(mean_fit_time)
         results['params'] = params
-        results['rank_test_scores'] = scipy.stats.rankdata(1 - results['mean_test_score'],
-                                                           method='min')
+        rank_order = -1 * self._metric._sign * results['mean_test_score']
+        results['rank_test_scores'] = scipy.stats.rankdata(rank_order, method='min')
         results['status'] = status
         results['budgets'] = budgets
 


Test patch:
Repository: automl/auto-sklearn

Issue:
Check if test requirement `flaky` can be removed
We currently have a test dependancy [flaky](https://pypi.org/project/flaky/) used to annotate a test `KernelPCAComponentTest::test_default_configuration_classify()`. This is the only place it's used.


Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -30,7 +30,6 @@
         "mypy",
         "pytest-xdist",
         "pytest-timeout",
-        "flaky",
         "openml",
         "pre-commit",
         "pytest-cov",


Test patch:
diff --git a/test/test_pipeline/components/feature_preprocessing/test_kernel_pca.py b/test/test_pipeline/components/feature_preprocessing/test_kernel_pca.py
--- a/test/test_pipeline/components/feature_preprocessing/test_kernel_pca.py
+++ b/test/test_pipeline/components/feature_preprocessing/test_kernel_pca.py
@@ -1,7 +1,5 @@
 import unittest
 
-import pytest
-
 from sklearn.linear_model import RidgeClassifier
 from autosklearn.pipeline.components.feature_preprocessing.kernel_pca import \
     KernelPCA
@@ -25,7 +23,6 @@ def test_default_configuration_sparse(self):
         self.assertEqual(transformation.shape[0], original.shape[0])
         self.assertFalse((transformation == 0).all())
 
-    @pytest.mark.flaky()
     def test_default_configuration_classify(self):
         for i in range(5):
             X_train, Y_train, X_test, Y_test = get_dataset(dataset='digits',
Repository: automl/auto-sklearn

Issue:
`show_models` and `leaderboard` show different rankings.



Discussion:

Resolving patch:
diff --git a/autosklearn/automl.py b/autosklearn/automl.py
--- a/autosklearn/automl.py
+++ b/autosklearn/automl.py
@@ -2184,21 +2184,20 @@ def has_key(rv, key):
 
         table = pd.DataFrame.from_dict(table_dict, orient="index")
         table.sort_values(by="cost", inplace=True)
+        table["rank"] = np.arange(1, len(table.index) + 1)
 
         # Check which resampling strategy is chosen and selecting the appropriate models
         is_cv = self._resampling_strategy == "cv"
         models = self.cv_models_ if is_cv else self.models_
 
-        rank = 1  # Initializing rank for the first model
         for (_, model_id, _), model in models.items():
             model_dict = {}  # Declaring model dictionary
 
             # Inserting model_id, rank, cost and ensemble weight
             model_dict["model_id"] = table.loc[model_id]["model_id"].astype(int)
-            model_dict["rank"] = rank
+            model_dict["rank"] = table.loc[model_id]["rank"].astype(int)
             model_dict["cost"] = table.loc[model_id]["cost"]
             model_dict["ensemble_weight"] = table.loc[model_id]["ensemble_weight"]
-            rank += 1  # Incrementing rank by 1 for the next model
 
             # The steps in the models pipeline are as follows:
             # 'data_preprocessor': DataPreprocessor,


Test patch:
Repository: alltheplaces/alltheplaces

Issue:
Blick
https://www.dickblick.com/stores/


Discussion:

Resolving patch:
diff --git a/locations/spiders/dick_blick.py b/locations/spiders/dick_blick.py
new file mode 100644
--- /dev/null
+++ b/locations/spiders/dick_blick.py
@@ -0,0 +1,56 @@
+# -*- coding: utf-8 -*-
+import scrapy
+import re
+from scrapy.utils.url import urljoin_rfc
+from scrapy.utils.response import get_base_url
+
+from locations.items import GeojsonPointItem
+
+class ApplebeesSpider(scrapy.Spider):
+    name = "dick_blick"
+    allowed_domains = ["www.dickblick.com"]
+    start_urls = (
+        'https://www.dickblick.com/stores/',
+    )
+
+    def parse_store(self, response):
+        contacts = response.xpath('//ul[@class="contact"]/li/span/text()').extract()
+
+        properties = {
+            'addr:full': contacts[0],
+            'addr:city': contacts[1],
+            'addr:state': contacts[2],
+            'addr:postcode': contacts[3],
+            'phone': contacts[4],
+            'ref': response.url,
+            'website': response.url,
+        }
+
+        day_groups = response.xpath('//ul[@class="hours"]/li[@class="storehours"]/text()').extract()
+
+        opening_hours = []
+        for day_group in day_groups:
+            match = re.match(r'(.*): (\d+)-(\d+)', day_group)
+            days, f_hr, t_hr = match.groups()
+            f_hr = int(f_hr)
+            t_hr = int(t_hr) + 12
+            opening_hours.append('{} {:02d}:00-{:02d}:00'.format(days, f_hr, t_hr))
+
+        if opening_hours:
+            properties['opening_hours'] = '; '.join(opening_hours)
+
+        yield GeojsonPointItem(
+            properties=properties,
+        )
+
+    def parse_state(self, response):
+        base_url = get_base_url(response)
+        urls = response.xpath('//div/ul[@class="storelist"]/li/a/@href').extract()
+        for path in urls:
+            yield scrapy.Request(urljoin_rfc(base_url, path), callback=self.parse_store)
+
+    def parse(self, response):
+        base_url = get_base_url(response)
+        urls = response.xpath('//div[@class="statechooser"]/select/option/@value').extract()
+        for path in urls:
+            yield scrapy.Request(urljoin_rfc(base_url, path), callback=self.parse_state)


Test patch:
Repository: alltheplaces/alltheplaces

Issue:
Sweetgreen
http://www.sweetgreen.com/locations/


Discussion:

Resolving patch:
diff --git a/locations/spiders/sweetgreen.py b/locations/spiders/sweetgreen.py
new file mode 100644
--- /dev/null
+++ b/locations/spiders/sweetgreen.py
@@ -0,0 +1,34 @@
+# -*- coding: utf-8 -*-
+import scrapy
+import json
+from scrapy.utils.url import urljoin_rfc
+from scrapy.utils.response import get_base_url
+
+from locations.items import GeojsonPointItem
+
+class SweetgreenSpider(scrapy.Spider):
+    name = "sweetgreen"
+    allowed_domains = ["www.sweetgreen.com"]
+    start_urls = (
+        'http://www.sweetgreen.com/locations/',
+    )
+
+    def parse(self, response):
+        location_hrefs = response.xpath('//a[contains(@class, "location")]')
+        for location_href in location_hrefs:
+            properties = {
+                'name': location_href.xpath('@title')[0].extract(),
+                'addr:full': location_href.xpath('@data-street')[0].extract(),
+                'phone': location_href.xpath('@data-phone')[0].extract(),
+                'ref': location_href.xpath('@id')[0].extract(),
+            }
+
+            lon_lat = [
+                float(location_href.xpath('@data-longitude')[0].extract()),
+                float(location_href.xpath('@data-latitude')[0].extract()),
+            ]
+
+            yield GeojsonPointItem(
+                properties=properties,
+                lon_lat=lon_lat,
+            )


Test patch:
Repository: alltheplaces/alltheplaces

Issue:
Planned Parenthood
https://www.plannedparenthood.org/health-center


Discussion:

Resolving patch:
diff --git a/locations/spiders/planned_parenthood.py b/locations/spiders/planned_parenthood.py
new file mode 100644
--- /dev/null
+++ b/locations/spiders/planned_parenthood.py
@@ -0,0 +1,53 @@
+# -*- coding: utf-8 -*-
+import scrapy
+import json
+import re
+
+from locations.items import GeojsonPointItem
+
+
+class PlannedParenthoodSpider(scrapy.Spider):
+    name = "planned_parenthood"
+    allowed_domains = ["www.plannedparenthood.org"]
+    start_urls = (
+        'https://www.plannedparenthood.org/health-center',
+    )
+
+    def parse(self, response):
+        state_urls = response.xpath('//ul[@class="quicklist-list"]/li/a/@href').extract()
+        for path in state_urls:
+            yield scrapy.Request(
+                response.urljoin(path),
+                callback=self.parse_state,
+            )
+
+    def parse_state(self, response):
+        venue_urls = response.xpath('//ul[@class="quicklist-list"]/li/p/a/@href').extract()
+        for path in venue_urls:
+            yield scrapy.Request(
+                response.urljoin(path),
+                callback=self.parse_venue,
+            )
+
+    def parse_venue(self, response):
+        properties = {
+            'addr:full': response.xpath('//*[@itemprop="streetAddress"]/text()')[0].extract(),
+            'addr:city': response.xpath('//*[@itemprop="addressLocality"]/text()')[0].extract(),
+            'addr:state': response.xpath('//*[@itemprop="addressRegion"]/text()')[0].extract(),
+            'addr:postcode': response.xpath('//*[@itemprop="postalCode"]/text()')[0].extract(),
+            'phone': response.xpath('//a[@itemprop="telephone"][@data-link]/text()')[0].extract(),
+            'ref': response.url,
+            'website': response.url,
+        }
+
+        map_image_url = response.xpath('//img[@class="address-map"]/@src')[0].extract()
+        match = re.search(r"center=(.*?),(.*?)&zoom", map_image_url)
+        lon_lat = [
+            float(match.group(2)),
+            float(match.group(1)),
+        ]
+
+        yield GeojsonPointItem(
+            properties=properties,
+            lon_lat=lon_lat,
+        )


Test patch: