internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: ansible/ansible-modules-core

Issue:
Service restart on Solaris is asynchronous
<!--- Verify first that your issue/request is not already reported in GitHub -->
##### ISSUE TYPE
- Bug Report
##### COMPONENT NAME

service (system/service.py)
##### ANSIBLE VERSION

```
ansible 2.1.0.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/usr/share/ansible']
```
##### CONFIGURATION

n/a
##### OS / ENVIRONMENT

From CentOS 7, managing Solaris 11.3 client
##### SUMMARY

Service restart on Solaris with SMF runs asynchronously, potentially causing subsequent tasks to fail if the service does not restart in time.
Analysis of code suggests the '-s' flag should also be used for svcadm restart as with other subcommands (incl. restart when the service is offline). This would mimic the behaviour on other platforms.

```
        if self.action == 'start':
            subcmd = "enable -rst"
        elif self.action == 'stop':
            subcmd = "disable -st"
        elif self.action == 'reload':
            subcmd = "refresh"
        elif self.action == 'restart' and status == 'online':
            subcmd = "restart"  # should be "restart -s"
        elif self.action == 'restart' and status != 'online':
            subcmd = "enable -rst"
```

(same could apply to refresh as well)
##### STEPS TO REPRODUCE

E.g. Restarting MySQL and then creating a new database:

```
- service: name="mysql:version_56" state=restarted

- mysql_db: name="exampledb" state=present
```
##### EXPECTED RESULTS

MySQL restarts and database is created.
##### ACTUAL RESULTS

mysql_db task fails, unable to connect to MySQL (as it is still restarting).

```
TASK [mysql-config : Restart mysql service] ************************************
changed: [solclient.cf.ac.uk]

TASK [mysql-config : Create mysql database] ************************************
failed: [solclient.cf.ac.uk] (item={u'collation': u'utf8_unicode_ci', u'name': u'exampledb', u'encoding': u'utf8'}) => {"failed": true, "item": {"collation": "utf8_unicode_ci", "encoding": "utf8", "name": "exampledb"}, "msg": "unable to connect to database, check login_user and login_password are correct or /root/.my.cnf has the credentials. Exception message: (2002, \"Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2)\")"}
```



Discussion:
<!-- @aderixon: Thanks for filing this bug report! -->

@ansible ping, this issue is waiting for your response.
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)

<!--- boilerplate: issue_bug_new --->


Resolving patch:
diff --git a/system/service.py b/system/service.py
--- a/system/service.py
+++ b/system/service.py
@@ -1365,9 +1365,9 @@ def service_control(self):
         elif self.action == 'stop':
             subcmd = "disable -st"
         elif self.action == 'reload':
-            subcmd = "refresh"
+            subcmd = "refresh -s"
         elif self.action == 'restart' and status == 'online':
-            subcmd = "restart"
+            subcmd = "restart -s"
         elif self.action == 'restart' and status != 'online':
             subcmd = "enable -rst"
 


Test patch:
Repository: ansible/ansible-modules-core

Issue:
Documentation Bug: AWS EC2 idempotency
**Summary:**
Idempotency is not documented in the EC2 module even though it's supported.

**Description:**
The AWS EC2 cloud module supports idempotency, by passing 'id' to the module.
This used to be documented, but in commit c6b0d469, the following lines were removed. 

``` diff
-  id:
-    description:
-      - identifier for this instance or set of instances, so that the module will be idempotent with respect to EC2 instances. This identifier is valid for at least 24 hours after the termination of the instance, and should not be reused for another call later on. For details, see the desc
-    required: false
-    default: null
-    aliases: []
```

**Expected Results:**
Idempotency should be documented.

**Actual Results:**
Ablility to use the ec2 module idempotently is not documented.



Discussion:
+1

Hi @mistaka0s -- thanks for this bug report/feature idea, and thanks as well to @adenot for adding your support for this. I'm labelling it as a bug report since it looks like the documentation was removed which seems less than ideal IMO :)

@tgerla @lwade --  thanks in advance for taking a look at this bug report and for your ongoing work in maintaining this module.

[module: cloud/amazon/ec2.py]

@ansible, ping. This issue is still waiting on your response.
[click here for bot help](https://github.com/jctanner/ansibullbot/blob/ISSUE_TRIAGE_CLASS/ISSUE_HELP.md)

<!--- boilerplate: issue_renotify_maintainer --->

@ansible, ping. This issue is still waiting on your response.
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)

<!--- boilerplate: issue_renotify_maintainer --->

+1

@ansible, ping. This issue is still waiting on your response.
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)

<!--- boilerplate: issue_renotify_maintainer --->

@ansible, ping. This issue is still waiting on your response.
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)

<!--- boilerplate: issue_renotify_maintainer --->


Resolving patch:
diff --git a/cloud/amazon/ec2.py b/cloud/amazon/ec2.py
--- a/cloud/amazon/ec2.py
+++ b/cloud/amazon/ec2.py
@@ -29,6 +29,13 @@
     required: false
     default: null
     aliases: ['keypair']
+  id:        
+    version_added: "1.1"
+    description:        
+      - identifier for this instance or set of instances, so that the module will be idempotent with respect to EC2 instances. This identifier is valid for at least 24 hours after the termination of the instance, and should not be reused for another call later on. For details, see the description of client token at U(http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Run_Instance_Idempotency.html).        
+    required: false        
+    default: null        
+    aliases: []        
   group:
     description:
       - security group (or list of groups) to use with the instance


Test patch:
Repository: mindsdb/lightwood

Issue:
ModuleNotFoundError: No module named '_lzma'
I've tried to test lightwood with [home rentals
 example](https://github.com/mindsdb/lightwood/blob/master/docs/examples/home_rentals.py) but got ModuleNotFoundError: No module named '_lzma'.

Screenshot:
![Screenshot from 2019-07-24 18-44-02](https://user-images.githubusercontent.com/7192539/61812426-c6088600-ae43-11e9-995f-2c92ea82e430.png)

It looks like _lzma is a dependency to pandas, but it should be included with Python 3.x version.




Discussion:
Weird that this would happen. Did you update/downgrade the pandas version after installing lightwood ?
No, just plain install of lightwood
hmh, weird, can't really replicate,  we should consider adding lzma as a dependency I guess.

What's the OS and python version for this ?
Python version 3.7, Ubuntu 18.04, lightwood 0.7.6. I've upgraded to the latest 0.7.9 but the same issue
With pandas 0.24 this works. But with version 0.25 that is a dependency in lightwood, it doesn't. Ok, so here is the explanation why [pandas issue](https://github.com/pandas-dev/pandas/issues/27543#issuecomment-514305279)

Resolving patch:
diff --git a/lightwood/__about__.py b/lightwood/__about__.py
--- a/lightwood/__about__.py
+++ b/lightwood/__about__.py
@@ -1,6 +1,6 @@
 __title__ = 'lightwood'
 __package_name__ = 'mindsdb'
-__version__ = '0.9.0'
+__version__ = '0.9.1'
 __description__ = "Lightwood's goal is to make it very simple for developers to use the power of artificial neural networks in their projects."
 __email__ = "jorge@mindsdb.com"
 __author__ = 'MindsDB Inc'


Test patch:
Repository: mindsdb/lightwood

Issue:
installation error in python 3.8.1
The installation fails like this
`ERROR: Could not find a version that satisfies the requirement torch>=1.3.0 (from pyro-ppl>=0.4.1->lightwood) (from versions: 0.1.2, 0.1.2.post1, 0.1.2.post2)
ERROR: No matching distribution found for torch>=1.3.0 (from pyro-ppl>=0.4.1->lightwood)`


Discussion:
Hi @Karlheinzniebuhr and thanks for reporting the issue. There was a similar issue on Windows that we have already resolved. This one I think is related with a Pytorch that still doesn't [support](https://github.com/pytorch/pytorch/issues/21741) python 3.8 version, so please use another python 3.7.x version.
I've installed the latest torch version and it works just fine on python 3.8.1

@Karlheinzniebuhr On windows10 with python3.8.1 version I am getting the same error that you report when installing torch. Can you please explain how did you installed it?
Sure,
`pip install torch===1.3.1 torchvision===0.4.2 -f https://download.pytorch.org/whl/torch_stable.html`
Pytroch *should* have a new release "early this week" based on a few issues I read related to various topics. Will wait for those and hopefully the new updates will work with python 3.8.x

currently python 3.8 doesn't run mindsdb on linux either.

Resolving patch:
diff --git a/lightwood/__about__.py b/lightwood/__about__.py
--- a/lightwood/__about__.py
+++ b/lightwood/__about__.py
@@ -1,6 +1,6 @@
 __title__ = 'lightwood'
 __package_name__ = 'mindsdb'
-__version__ = '0.15.8'
+__version__ = '0.16.0'
 __description__ = "Lightwood's goal is to make it very simple for developers to use the power of artificial neural networks in their projects."
 __email__ = "jorge@mindsdb.com"
 __author__ = 'MindsDB Inc'
diff --git a/lightwood/mixers/nn/nn.py b/lightwood/mixers/nn/nn.py
--- a/lightwood/mixers/nn/nn.py
+++ b/lightwood/mixers/nn/nn.py
@@ -16,7 +16,6 @@
 
 
 class NnMixer:
-
     def __init__(self, dynamic_parameters, is_categorical_output=False):
         self.is_categorical_output = is_categorical_output
         self.net = None


Test patch:
Repository: mindsdb/lightwood

Issue:
Stop using pickle
Apparently torch 1.5 will no longer support pickling, thus we'll have to switch to `torch.save` instead.


Discussion:
Apparently torch still supports pickling, but it might be potentially relevant to look into this ?

I should really provide sources for these sort of claims next time, because I forgot where I took this from.

Also, quick note, this should read "dill" not "pickle".

Resolving patch:
diff --git a/lightwood/api/predictor.py b/lightwood/api/predictor.py
--- a/lightwood/api/predictor.py
+++ b/lightwood/api/predictor.py
@@ -1,8 +1,6 @@
 import traceback
 import time
 
-import dill
-import pickle
 import pandas
 import numpy as np
 import torch
@@ -24,7 +22,7 @@ def __init__(self, config=None, output=None, load_from_path=None):
         """
         if load_from_path is not None:
             with open(load_from_path, 'rb') as pickle_in:
-                self_dict = pickle.load(pickle_in)
+                self_dict = torch.load(pickle_in)
             self.__dict__ = self_dict
             self.convert_to_device()
             return
@@ -229,5 +227,5 @@ def save(self, path_to):
 
             # Dump everything relevant to cpu before saving
             self.convert_to_device("cpu")
-            pickle.dump(self.__dict__, f)
+            torch.save(self.__dict__, f)
             self.convert_to_device()


Test patch:
Repository: mindsdb/lightwood

Issue:
Error when using 'force_disable_cache' in time series task
* Lightwood version: staging (post 0.51.0 release)

With us_consumption.csv dataset, training fails when using `advanced_args={'force_disable_cache': True}`, we get the following stack trace:

```
ERROR:mindsdb-logger-802183d6-3bdb-11eb-a895-4cedfb644c83---no_report:/MindsDB/mindsdb_native/mindsdb_native/libs/phases/model_interface/lightwood_backend.py:418 - Traceback (most recent call last):
  File "/MindsDB/mindsdb_native/mindsdb_native/libs/phases/model_interface/lightwood_backend.py", line 412, in train
    test_data=lightwood_test_ds
  File "/MindsDB/lightwood/lightwood/api/predictor.py", line 137, in learn
    self._mixer.fit(train_ds=train_ds, test_ds=test_ds)
  File "/MindsDB/lightwood/lightwood/mixers/base_mixer.py", line 37, in fit
    self._fit(train_ds, test_ds, **kwargs)
  File "/MindsDB/lightwood/lightwood/mixers/nn.py", line 146, in _fit
    input_sample, output_sample = train_ds[0]
  File "/MindsDB/lightwood/lightwood/api/data_source.py", line 225, in __getitem__
    sample[feature_set][col_name] = self.get_encoded_column_data(col_name, custom_data=custom_data)[0]
  File "/MindsDB/lightwood/lightwood/api/data_source.py", line 433, in get_encoded_column_data
    sublist = custom_data[col]
KeyError: '__mdb_ts_previous_Unemployment'
```




Discussion:

Resolving patch:
diff --git a/lightwood/api/data_source.py b/lightwood/api/data_source.py
--- a/lightwood/api/data_source.py
+++ b/lightwood/api/data_source.py
@@ -229,6 +229,9 @@ def __getitem__(self, idx):
                 elif not self.enable_cache:
                     if col_name in self.data_frame:
                         custom_data = {col_name: [self.data_frame[col_name].iloc[idx]]}
+                        if 'depends_on_column' in col_config:
+                            for col in col_config['depends_on_column']:
+                                custom_data[col] = [self.data_frame[col].iloc[idx]]
                     else:
                         custom_data = {col_name: [None]}
 


Test patch:
Repository: mindsdb/lightwood

Issue:
LightGBM is not deterministic
* Lightwood version: 0.58.0

Currently, our LightGBM mixer is not deterministic, presenting slight performance differences when trained for the same dataset. We should fix the corresponding random seeds so that this is doesn't happen anymore.



Discussion:
@paxcema With the latest version, we have a lot of improvements on this. We can close for now, and re-open in case we found additional issues.
Re-opening to keep track of this [PR](https://github.com/optuna/optuna/pull/2431) in Optuna. Once merged we can solve this issue.
As an update, Optuna#2431 has been merged and will be released in 2.8.0, which should close this issue

Resolving patch:
diff --git a/lightwood/mixers/lightgbm.py b/lightwood/mixers/lightgbm.py
--- a/lightwood/mixers/lightgbm.py
+++ b/lightwood/mixers/lightgbm.py
@@ -123,6 +123,7 @@ def _fit(self, train_ds, test_ds=None):
                 if max_itt >= num_iterations and seconds_for_one_iteration < 10:
                     self.grid_search = True
                     kwargs['time_budget'] = self.stop_training_after_seconds
+                    kwargs['optuna_seed'] = 0  # for reproducible grid search results
 
             train_data = lightgbm.Dataset(data['train']['data'], label=data['train']['label_data'][col_name])
             validate_data = lightgbm.Dataset(data['test']['data'], label=data['test']['label_data'][col_name])


Test patch:
Repository: mindsdb/lightwood

Issue:
Escape weird column names in json-ai and code
Characters and sequences such as `\n` , `'`, `"`, `"""` and so on could cause issue in the json-ai or when generating the code if they are part of column names. We should figure ways that are both json and python code friendly of escpaing them.


Discussion:
In addition to this we should add a "sanity check" phase of sort (maybe) where we throw errors for things like dumb column names that will break things.

Resolving patch:
diff --git a/lightwood/api/json_ai.py b/lightwood/api/json_ai.py
--- a/lightwood/api/json_ai.py
+++ b/lightwood/api/json_ai.py
@@ -459,7 +459,9 @@ def code_from_json_ai(json_ai: JsonAI) -> str:
         )
 
     ignored_cols = json_ai.problem_definition.ignore_features
-    input_cols = ','.join([f"""'{name}'""" for name in json_ai.features if name not in ignored_cols])
+    input_cols = [x.replace("'", "\\'").replace('"', '\\"')
+                  for x in json_ai.features]
+    input_cols = ','.join([f"""'{name}'""" for name in input_cols if name not in ignored_cols])
 
     ts_transform_code = ''
     ts_analyze_code = ''
diff --git a/lightwood/helpers/templating.py b/lightwood/helpers/templating.py
--- a/lightwood/helpers/templating.py
+++ b/lightwood/helpers/templating.py
@@ -84,6 +84,7 @@ def call(entity: dict, json_ai: JsonAI) -> str:
 def inline_dict(obj: dict) -> str:
     arr = []
     for k, v in obj.items():
+        k = k.replace("'", "\\'").replace('"', '\\"')
         arr.append(f"""'{k}': {v}""")
 
     dict_code = '{\n' + ',\n'.join(arr) + '\n}'


Test patch:
diff --git a/tests/integration/basic/test_boston_housing.py b/tests/integration/basic/test_boston_housing.py
--- a/tests/integration/basic/test_boston_housing.py
+++ b/tests/integration/basic/test_boston_housing.py
@@ -10,6 +10,10 @@ def test_0_predict_file_flow(self):
         from lightwood.api.high_level import predictor_from_problem
 
         df = pd.read_csv('tests/data/boston.csv')[:500]
+        # Mess with the names to also test if lightwood can deal /w weird names
+        df = df.rename(columns={df.columns[1]: f'\'{df.columns[1]}\''})
+        df = df.rename(columns={df.columns[2]: f'\'{df.columns[2]}}}'})
+        df = df.rename(columns={df.columns[3]: f'{{{df.columns[3]}\"'})
         target = 'MEDV'
 
         predictor = predictor_from_problem(df, ProblemDefinition.from_dict({'target': target, 'time_aim': 200}))
Repository: mindsdb/lightwood

Issue:
`sktime` missing dependency
In requirements.txt
`sktime[arima] >= 0.5.0`
bat 
`(mindsdb) C:\Users\User>pip install sktime[arima]>=0.5.0
WARNING: sktime 0.7.0 does not provide the extra 'arima'`
install
`(mindsdb) C:\Users\User>pip install sktime[arima]==0.5.0`
`sktime` missing dependency
In requirements.txt
`sktime[arima] >= 0.5.0`
bat 
`(mindsdb) C:\Users\User>pip install sktime[arima]>=0.5.0
WARNING: sktime 0.7.0 does not provide the extra 'arima'`
install
`(mindsdb) C:\Users\User>pip install sktime[arima]==0.5.0`


Discussion:
Hi @VladPerervenko thanks for reporting this. We will check if we still need to install sktime with `arima`. Did this stop the lightwood installation, I guess you can count it as working for now.
Hi @VladPerervenko thanks for reporting this. We will check if we still need to install sktime with `arima`. Did this stop the lightwood installation, I guess you can count it as working for now.

Resolving patch:
diff --git a/lightwood/model/sktime.py b/lightwood/model/sktime.py
--- a/lightwood/model/sktime.py
+++ b/lightwood/model/sktime.py
@@ -42,7 +42,8 @@ def fit(self, ds_arr: List[EncodedDs]) -> None:
                                for gcol in self.grouped_by} if self.ts_analysis['tss'].group_by else {}}
 
         for group in self.ts_analysis['group_combinations']:
-            self.models[group] = self.model_class()
+            # many warnings might be thrown inside of statsmodels during stepwise procedure
+            self.models[group] = self.model_class(suppress_warnings=True)
 
             if self.grouped_by == ['__default']:
                 series_idxs = data['data'].index


Test patch:
Repository: mindsdb/lightwood

Issue:
Can't query model without ignored column (Alpha)
Example model query:

```sql
CREATE PREDICTOR Medical_Cost_model2 FROM MySQL(SELECT * from test_data.Medical_Cost) as Medical_Cost_test2 PREDICT charges USING {"ignore_columns": "bmi"};
```
If the user try to query the model without providing the `bmi` value(should be ignored column) in the SELECT as:

```sql
SELECT * from mindsdb.Medical_Cost_model2 WHERE when_data='{"age":19, "sex":"female","children":0, "smoker": "yes", "region":"southwest"}';
```

ERROR 1149 (HY000): "['bmi'] not found in axis" will be returned. Maybe related to mindsdb/mindsdb#1493 




Discussion:
Error raising when we call .predict() function:
```
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/maxs/dev/mdb/venv38/lib/python3.8/site-packages/pandas/util/_decorators.py", line 311, in wrapper
    return func(*args, **kwargs)
  File "/home/maxs/dev/mdb/venv38/lib/python3.8/site-packages/pandas/core/frame.py", line 4906, in drop
    return super().drop(
  File "/home/maxs/dev/mdb/venv38/lib/python3.8/site-packages/pandas/core/generic.py", line 4150, in drop
    obj = obj._drop_axis(labels, axis, level=level, errors=errors)
  File "/home/maxs/dev/mdb/venv38/lib/python3.8/site-packages/pandas/core/generic.py", line 4185, in _drop_axis
    new_axis = axis.drop(labels, errors=errors)
  File "/home/maxs/dev/mdb/venv38/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 6017, in drop
    raise KeyError(f"{labels[mask]} not found in axis")
KeyError: "['sqft'] not found in axis"
```

Resolving patch:
diff --git a/lightwood/api/json_ai.py b/lightwood/api/json_ai.py
--- a/lightwood/api/json_ai.py
+++ b/lightwood/api/json_ai.py
@@ -1050,7 +1050,7 @@ def code_from_json_ai(json_ai: JsonAI) -> str:
     predict_body = f"""
 # Remove columns that user specifies to ignore
 log.info(f'Dropping features: {{self.problem_definition.ignore_features}}')
-data = data.drop(columns=self.problem_definition.ignore_features)
+data = data.drop(columns=self.problem_definition.ignore_features, errors='ignore')
 for col in self.input_cols:
     if col not in data.columns:
         data[col] = [None] * len(data)
@@ -1135,6 +1135,8 @@ def analyze_ensemble(self, enc_data: Dict[str, pd.DataFrame]) -> None:
 {analyze_ensemble}
 
     def learn(self, data: pd.DataFrame) -> None:
+        log.info(f'Dropping features: {{self.problem_definition.ignore_features}}')
+        data = data.drop(columns=self.problem_definition.ignore_features, errors='ignore')
 {learn_body}
 
     def adjust(self, new_data: Dict[str, pd.DataFrame]) -> None:


Test patch:
Repository: mindsdb/lightwood

Issue:
:wrench: Add default logging level environment variable
## Task

Add a `LIGHTWOOD_LOG` environment variable that controls the default logging level for lightwood. It should be possible to set values for it so that `DEBUG`, `INFO`, `WARNING`, `ERROR` and `CRITICAL` are all possible options. The logger lightwood uses is declared and exported [here](https://github.com/mindsdb/lightwood/blob/stable/lightwood/helpers/log.py).

## Steps :male_detective: :female_detective: 

- Fork the Lightwood repository, checkout the `staging` branch and from it create a new one.
- Implement the necessary changes.
- Check that only the appropriate logs are getting through. For this, you can run any of the integration tests, like [`test_boston_housing`](https://github.com/mindsdb/lightwood/blob/stable/tests/integration/basic/test_boston_housing.py), and analyze the output.
- Make the PR and address any comments that reviewers might make.

## Additional rewards :1st_place_medal: 

Each documentation PR brings :one:  point for entry into the draw for a :computer:  Deep Learning Laptop powered by the NVIDIA RTX 3080 Max-Q GPU or other swag :shirt: :bear: .  For more info check out https://mindsdb.com/hacktoberfest/


Discussion:

Resolving patch:
diff --git a/lightwood/helpers/log.py b/lightwood/helpers/log.py
--- a/lightwood/helpers/log.py
+++ b/lightwood/helpers/log.py
@@ -6,7 +6,8 @@ def initialize_log():
     pid = os.getpid()
     logging.basicConfig()
     log = logging.getLogger(f'lightwood-{pid}')
-    log.setLevel(logging.DEBUG)
+    log_level = os.environ.get('LIGHTWOOD_LOG', 'DEBUG')
+    log.setLevel(log_level)
     return log
 
 


Test patch: