internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: boto/boto

Issue:
EMR ListSteps Doesn't Properly Filter Based on cluster_states
I believe this issue is related to Boto as I tried verifying in Java SDK and it works as I expect: I will only see steps that match the cluster state filter.

The following will return all steps for each cluster:

``` python
region='us-east-1'

import boto.emr
>>> emr=boto.emr.connect_to_region(region)
>>> c=[]
>>> for i in emr.list_clusters(cluster_states=['WAITING']).clusters:
...     c.append(i.id)

>>> f=[]
>>> for k in c:
...     f.append(emr.list_steps(cluster_id=k,step_states=["PENDING","RUNNING"]))
... 
>>> for j in f:
...     j.steps
[]
[<boto.emr.emrobject.StepSummary object at 0x1079a77d0>, <boto.emr.emrobject.StepSummary object at 0x1079a7850>, <boto.emr.emrobject.StepSummary object at 0x1079a7090>, <boto.emr.emrobject.StepSummary object at 0x1079a7510>, <boto.emr.emrobject.StepSummary object at 0x107928990>, <boto.emr.emrobject.StepSummary object at 0x107928b50>, <boto.emr.emrobject.StepSummary object at 0x107928890>, <boto.emr.emrobject.StepSummary object at 0x107928e10>, <boto.emr.emrobject.StepSummary object at 0x1079282d0>, <boto.emr.emrobject.StepSummary object at 0x107928490>, <boto.emr.emrobject.StepSummary object at 0x107928410>, <boto.emr.emrobject.StepSummary object at 0x1079287d0>, <boto.emr.emrobject.StepSummary object at 0x1078d81d0>]
[<boto.emr.emrobject.StepSummary object at 0x1078d8450>, <boto.emr.emrobject.StepSummary object at 0x1078d8550>]
[<boto.emr.emrobject.StepSummary object at 0x1078d8650>, <boto.emr.emrobject.StepSummary object at 0x1078d8910>, <boto.emr.emrobject.StepSummary object at 0x1078d8b10>, <boto.emr.emrobject.StepSummary object at 0x1078d8d50>, <boto.emr.emrobject.StepSummary object at 0x1078d4050>]

#check j
>>> j
<boto.emr.emrobject.StepSummaryList object at 0x1079a7a50>
>>> j.steps[-1].status.state
u'COMPLETED'
```



Discussion:

Resolving patch:
diff --git a/boto/emr/connection.py b/boto/emr/connection.py
--- a/boto/emr/connection.py
+++ b/boto/emr/connection.py
@@ -265,7 +265,7 @@ def list_steps(self, cluster_id, step_states=None, marker=None):
             params['Marker'] = marker
 
         if step_states:
-            self.build_list_params(params, step_states, 'StepStateList.member')
+            self.build_list_params(params, step_states, 'StepStates.member')
 
         return self.get_object('ListSteps', params, StepSummaryList)
 


Test patch:
diff --git a/tests/unit/emr/test_connection.py b/tests/unit/emr/test_connection.py
--- a/tests/unit/emr/test_connection.py
+++ b/tests/unit/emr/test_connection.py
@@ -500,8 +500,8 @@ def test_list_steps_with_states(self):
         self.assert_request_parameters({
             'Action': 'ListSteps',
             'ClusterId': 'j-123',
-            'StepStateList.member.1': 'COMPLETED',
-            'StepStateList.member.2': 'FAILED',
+            'StepStates.member.1': 'COMPLETED',
+            'StepStates.member.2': 'FAILED',
             'Version': '2009-03-31'
         })
         self.assertTrue(isinstance(response, StepSummaryList))
Repository: boto/boto

Issue:
Misnamed attribute in `Stack`: `stack_name_reason` and `stack_status_reason`
I think these two attributes have been mixed up:
- https://github.com/boto/boto/blob/d14d5ac260a811aff3e1e1f1b1801b1bae1920bb/boto/cloudformation/stack.py#L20
- https://github.com/boto/boto/blob/d14d5ac260a811aff3e1e1f1b1801b1bae1920bb/boto/cloudformation/stack.py#L61

It doesn't make sense for the stack name to have a "reason"; it does make sense for the stack status, though. Also, if there is no status reason, `self.stack_status_reason` never gets defined, which means that code which needs to do some handling of `Stack` instances needs to be very careful, or else one will get errors like `'Stack' object has no attribute 'stack_status_reason'` (whereas it should just be `None`).

I think the solution is to replace `self.stack_name_reason` with `self.stack_status_reason` in the constructor. I'll throw up a pull request with these changes.



Discussion:

Resolving patch:
diff --git a/boto/cloudformation/stack.py b/boto/cloudformation/stack.py
--- a/boto/cloudformation/stack.py
+++ b/boto/cloudformation/stack.py
@@ -16,10 +16,18 @@ def __init__(self, connection=None):
         self.tags = []
         self.stack_id = None
         self.stack_status = None
+        self.stack_status_reason = None
         self.stack_name = None
-        self.stack_name_reason = None
         self.timeout_in_minutes = None
 
+    @property
+    def stack_name_reason(self):
+        return self.stack_status_reason
+
+    @stack_name_reason.setter
+    def stack_name_reason(self, value):
+        self.stack_status_reason = value
+
     def startElement(self, name, attrs, connection):
         if name == "Parameters":
             self.parameters = ResultSet([('member', Parameter)])


Test patch:
diff --git a/tests/unit/cloudformation/test_connection.py b/tests/unit/cloudformation/test_connection.py
--- a/tests/unit/cloudformation/test_connection.py
+++ b/tests/unit/cloudformation/test_connection.py
@@ -391,8 +391,8 @@ def default_body(self):
                 <member>
                   <StackId>arn:aws:cfn:us-east-1:1:stack</StackId>
                   <StackStatus>CREATE_COMPLETE</StackStatus>
+                  <StackStatusReason>REASON</StackStatusReason>
                   <StackName>MyStack</StackName>
-                  <StackStatusReason/>
                   <Description>My Description</Description>
                   <CreationTime>2012-05-16T22:55:31Z</CreationTime>
                   <Capabilities>
@@ -444,7 +444,8 @@ def test_describe_stacks(self):
         self.assertEqual(stack.stack_id, 'arn:aws:cfn:us-east-1:1:stack')
         self.assertEqual(stack.stack_status, 'CREATE_COMPLETE')
         self.assertEqual(stack.stack_name, 'MyStack')
-        self.assertEqual(stack.stack_name_reason, None)
+        self.assertEqual(stack.stack_name_reason, 'REASON')
+        self.assertEqual(stack.stack_status_reason, 'REASON')
         self.assertEqual(stack.timeout_in_minutes, None)
 
         self.assertEqual(len(stack.outputs), 1)
Repository: linz/geostore

Issue:
Use Python 3.8 across the board
We've already encountered some issues with Python 3.6 (cattrs no longer supporting it and less support for type annotations), and we have no reason to support multiple Python versions in production.

To do:

- [x] Set Python version in `.python-version`.
- [ ] Use a single version in CI, removing the need for the strategy matrix.
- [ ] Use the `PYTHON_3_8` runtime for AWS Lambda jobs.
- [ ] Change the version in `pyproject.toml` to `^3.8,<3.9`.


Discussion:

Resolving patch:
diff --git a/infra/datalake/api_stack.py b/infra/datalake/api_stack.py
--- a/infra/datalake/api_stack.py
+++ b/infra/datalake/api_stack.py
@@ -23,12 +23,12 @@ def __init__(self, scope: core.Construct, stack_id: str, datasets_table, **kwarg
                 f"{endpoint}-endpoint-function",
                 function_name=f"{endpoint}-endpoint",
                 handler=f"endpoints.{endpoint}.entrypoint.lambda_handler",
-                runtime=aws_lambda.Runtime.PYTHON_3_6,
+                runtime=aws_lambda.Runtime.PYTHON_3_8,
                 code=aws_lambda.Code.from_asset(
                     path="..",
                     bundling=core.BundlingOptions(
                         # pylint:disable=no-member
-                        image=aws_lambda.Runtime.PYTHON_3_6.bundling_docker_image,
+                        image=aws_lambda.Runtime.PYTHON_3_8.bundling_docker_image,
                         command=["backend/bundle.bash", f"endpoints/{endpoint}"],
                     ),
                 ),
diff --git a/infra/datalake/processing_stack.py b/infra/datalake/processing_stack.py
--- a/infra/datalake/processing_stack.py
+++ b/infra/datalake/processing_stack.py
@@ -168,12 +168,12 @@ def __init__(self, scope: core.Construct, stack_id: str, deploy_env, vpc, **kwar
                     self,
                     f"{task_name}-function",
                     handler=f"processing.{task_name}.task.lambda_handler",
-                    runtime=aws_lambda.Runtime.PYTHON_3_6,
+                    runtime=aws_lambda.Runtime.PYTHON_3_8,
                     code=aws_lambda.Code.from_asset(
                         path="..",
                         bundling=core.BundlingOptions(
                             # pylint:disable=no-member
-                            image=aws_lambda.Runtime.PYTHON_3_6.bundling_docker_image,
+                            image=aws_lambda.Runtime.PYTHON_3_8.bundling_docker_image,
                             command=["backend/bundle.bash", f"processing/{task_name}"],
                         ),
                     ),


Test patch:
Repository: linz/geostore

Issue:
Improve metadata description for catalog metadata
### User Story

<!-- A user story to describe why a user wants to do something, who the user is and what they want to do -->

So that I can understand more about the catalog metadata for the Geostore, as an external data analyst or developer, I want a good description and other relevant metadata in the catalog.json files.

<!-- optional: Instead of [existing behaviour] -->

#### Acceptance Criteria

<!-- Required artifacts to accept this feature as completed. -->

- [x] Given the bucket is deployed, when a dataset is created, then the title should read "LINZ Geostore"
- [x] Given the bucket is deployed, when a dataset is created, then the description should read:
`The Geospatial Data Store (Geostore) contains all the important geospatial data and metadata held by Land Information New Zealand (LINZ).<br/>Please browse this catalog to find and access our data.<br/>For more information see: TODO - add URL to help page.`
- [ ] Given a dataset, when a new version is created, the description of the dataset 'catalog.json' should be updated from the collection.json description
- [ ] Given a dataset, when a new version is created, the title in the dataset 'catalog.json' should be updated to the title from the collection.json

#### Additional context

Note: see old data lake root catalog for ideas. See other nicely described catalogs on the web for ideas (AWS Open Data Registry for example).

<!-- Add any other context or mocked CLI commands or screenshots about the feature request here.-->

#### Tasks

<!-- Tasks needed to complete this enabler -->

- [ ] Possibly want to change dataset endpoint change 'title' to 'directory-name' (or something), update documentation (Confluence as well as Github) as well
- [ ] 

#### Definition of Ready

- [ ] This story is **ready** to work on, according to the
      [team's definition](https://confluence.linz.govt.nz/pages/viewpage.action?pageId=87930423)

#### Definition of Done

- [ ] This story is **done**, according to the
      [team's definition](https://confluence.linz.govt.nz/pages/viewpage.action?pageId=87930423)



Discussion:

Resolving patch:
diff --git a/backend/populate_catalog/task.py b/backend/populate_catalog/task.py
--- a/backend/populate_catalog/task.py
+++ b/backend/populate_catalog/task.py
@@ -32,8 +32,12 @@
 S3_CLIENT: S3Client = boto3.client("s3")
 
 ROOT_CATALOG_ID = "root_catalog"
-ROOT_CATALOG_TITLE = "Geostore Root Catalog"
-ROOT_CATALOG_DESCRIPTION = "The root catalog which links to all dataset catalogues in Geostore"
+ROOT_CATALOG_TITLE = "LINZ Geostore"
+ROOT_CATALOG_DESCRIPTION = (
+    "The LINZ Geospatial Data Store (Geostore) contains all the important "
+    "geospatial data held by Land Information New Zealand (LINZ).<br/>"
+    "Please browse this catalog to find and access our data."
+)
 CATALOG_KEY = "catalog.json"
 CONTENTS_KEY = "Contents"
 RECORDS_KEY = "Records"


Test patch:
Repository: plotly/plotly.py

Issue:
jsonschema.SchemaError when  a figure  is validated
Here is a minimal example that reproduces the bug: http://nbviewer.jupyter.org/gist/empet/cf922d7c7f4269d6f63432ec67a5d020

The notebook runs OK (with plotly 2.0.1) when I call `plot(fig)`. `iplot(fig)` generates the plot too, but an error box pops up whenever Jupyter tries to save the notebook.  The box has the following content:

_The save operation succeeded, but the notebook does not appear to be valid. The validation error was:
Notebook Validation failed_:
`u'data': [{u'colorscale': u'Viridis', u'z': [[2, 27, 105, 100], [87, 14, 121, 102], [26, 121, 73, 34], [44, 105, 111, 127]], u'type': u'heatmap', u'zsmooth': u'best'}], u'layout': {u'width': 400, u'height': 400}}` _is not valid under any of the given schemas_:

`{
 "data": [
  {
   "colorscale": "Viridis",
   "z": [
    [
     2,
     27,
     105,
     100
    ],
    [
     87,
     14,
     121,
     102
    ],
    [
     26,
     121,
     73,
     34
    ],
    [
     44,
     105,
     111,
     127
    ]
   ],
   "type": "heatmap",
   "zsmooth": "best"
  }
 ],
 "layout": {
  "width": 400,
  "height": 400
 }
}`

Initially I formulated this issue only for heatmaps, but meanwhile I realized that this behaviour manifests for any type of  plot.


Discussion:
cc @chriddyp 

![screen shot 2017-02-20 at 2 27 11 pm](https://cloud.githubusercontent.com/assets/2789078/23139176/1bc38600-f779-11e6-8a60-69e617075019.png)

yikes, I suspect that this was introduced with adding the Nteract plugin. @theengineear @rgbkrk , does that sound right? Any thoughts on how to get around this issue?
Upgrade `nbformat`, either with `pip install --upgrade nbformat` or `conda update nbformat`.

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -45,5 +45,9 @@ def readme():
                 'plotly/matplotlylib/mplexporter',
                 'plotly/matplotlylib/mplexporter/renderers'],
       package_data={'plotly': ['package_data/*']},
-      install_requires=['decorator', 'requests', 'six', 'pytz'],
+      install_requires=['decorator',
+                        'nbformat>=4.2',
+                        'pytz',
+                        'requests',
+                        'six'],
       zip_safe=False)


Test patch:
Repository: plotly/plotly.py

Issue:
Include `frames` in IPython display bundle (`application/vnd.plotly.v1+json`)
`frames` are not sent as JSON in display bundle. It needs to be added..

fyi .. @Kully , @scjody ..


Discussion:

Resolving patch:
diff --git a/plotly/offline/offline.py b/plotly/offline/offline.py
--- a/plotly/offline/offline.py
+++ b/plotly/offline/offline.py
@@ -351,8 +351,15 @@ def iplot(figure_or_data, show_link=True, link_text='Export to plot.ly',
                                    cls=plotly.utils.PlotlyJSONEncoder))
     layout = _json.loads(_json.dumps(figure.get('layout', {}),
                                      cls=plotly.utils.PlotlyJSONEncoder))
+    frames = _json.loads(_json.dumps(figure.get('frames', None),
+                                     cls=plotly.utils.PlotlyJSONEncoder))
+
+    fig = {'data': data, 'layout': layout}
+    if frames:
+        fig['frames'] = frames
+
     display_bundle = {
-        'application/vnd.plotly.v1+json': {'data': data, 'layout': layout},
+        'application/vnd.plotly.v1+json': fig,
         'text/html': plot_html,
         'text/vnd.plotly.v1+html': plot_html
     }
diff --git a/plotly/version.py b/plotly/version.py
--- a/plotly/version.py
+++ b/plotly/version.py
@@ -1 +1 @@
-__version__ = '2.0.7'
+__version__ = '2.0.8'


Test patch:
Repository: plotly/plotly.py

Issue:
Not all dependecies are listed
Would it be possible to add pandas (and numpy) to the list of dependencies? They're both imported in various places but not listed. This occasionally causes problems for packages importing plotly, since `pip` may not install all dependencies.

Xref: https://github.com/deeptools/deepTools/issues/669
Not all dependecies are listed
Would it be possible to add pandas (and numpy) to the list of dependencies? They're both imported in various places but not listed. This occasionally causes problems for packages importing plotly, since `pip` may not install all dependencies.

Xref: https://github.com/deeptools/deepTools/issues/669


Discussion:
More specifically, one can't `import plotly.figure_factory` without pandas in newer versions, due to it being used by `plotly/figure_factory/_county_choropleth.py`.
More specifically, one can't `import plotly.figure_factory` without pandas in newer versions, due to it being used by `plotly/figure_factory/_county_choropleth.py`.

Resolving patch:
diff --git a/plotly/figure_factory/__init__.py b/plotly/figure_factory/__init__.py
--- a/plotly/figure_factory/__init__.py
+++ b/plotly/figure_factory/__init__.py
@@ -1,5 +1,7 @@
 from __future__ import absolute_import
 
+from plotly import optional_imports
+
 # Require that numpy exists for figure_factory
 import numpy
 
@@ -18,4 +20,5 @@
 from plotly.figure_factory._table import create_table
 from plotly.figure_factory._trisurf import create_trisurf
 from plotly.figure_factory._violin import create_violin
-from plotly.figure_factory._county_choropleth import create_choropleth
\ No newline at end of file
+if optional_imports.get_module('pandas') is not None:
+    from plotly.figure_factory._county_choropleth import create_choropleth


Test patch:
Repository: plotly/plotly.py

Issue:
v2 API Retry on 429 status_code if users are throttled?
Right now we [Only retry on request errors with 5XX status code and use exponential backoff](https://github.com/plotly/plotly.py/pull/1146). But should we also retry for 429 status codes which users receive if they go over Plot.ly's throttling limit? Especially since calls to the image servers are now limited to 1/sec.


Discussion:
Sure, that sounds reasonable.

Resolving patch:
diff --git a/plotly/api/v2/utils.py b/plotly/api/v2/utils.py
--- a/plotly/api/v2/utils.py
+++ b/plotly/api/v2/utils.py
@@ -114,8 +114,8 @@ def get_headers():
 def should_retry(exception):
     if isinstance(exception, exceptions.PlotlyRequestError):
         if (isinstance(exception.status_code, int) and
-                500 <= exception.status_code < 600):
-            # Retry on 5XX errors.
+                (500 <= exception.status_code < 600 or exception.status_code == 429)):
+            # Retry on 5XX and 429 (image export throttling) errors.
             return True
         elif 'Uh oh, an error occurred' in exception.message:
             return True


Test patch:
Repository: plotly/plotly.py

Issue:
Chaining fig.update()
Based on previous conversations, this would be a nice improvement :)


Discussion:
Sure, we can make `fig.update` and `graph_obj.update` return `self` rather than `None`.

Resolving patch:
diff --git a/plotly/basedatatypes.py b/plotly/basedatatypes.py
--- a/plotly/basedatatypes.py
+++ b/plotly/basedatatypes.py
@@ -3023,6 +3023,8 @@ def update(self, dict1=None, **kwargs):
             BaseFigure._perform_update(self, dict1)
             BaseFigure._perform_update(self, kwargs)
 
+        return self
+
     @property
     def _in_batch_mode(self):
         """


Test patch:
diff --git a/plotly/tests/test_core/test_graph_objs/test_update.py b/plotly/tests/test_core/test_graph_objs/test_update.py
--- a/plotly/tests/test_core/test_graph_objs/test_update.py
+++ b/plotly/tests/test_core/test_graph_objs/test_update.py
@@ -15,10 +15,12 @@ def setUp(self):
     def test_update_dict(self):
         title = 'this'
         fig = Figure()
-        fig.update(layout=Layout(title=title))
+        update_res1 = fig.update(layout=Layout(title=title))
         assert fig == Figure(layout=Layout(title=title))
-        fig['layout'].update(xaxis=XAxis())
+        update_res2 = fig['layout'].update(xaxis=XAxis())
         assert fig == Figure(layout=Layout(title=title, xaxis=XAxis()))
+        assert update_res1 is fig
+        assert update_res2 is fig.layout
 
 
     def test_update_list(self):
@@ -26,24 +28,27 @@ def test_update_list(self):
         trace2 = Scatter(x=[1, 2, 3], y=[3, 2, 1])
         fig = Figure([trace1, trace2])
         update = dict(x=[2, 3, 4], y=[1, 2, 3])
-        fig.data[0].update(update)
-        fig.data[1].update(update)
+        update_res1 = fig.data[0].update(update)
+        update_res2 = fig.data[1].update(update)
 
         d1, d2 = strip_dict_params(fig.data[0], Scatter(x=[2, 3, 4], y=[1, 2, 3]))
         assert d1 == d2
         d1, d2 = strip_dict_params(fig.data[1], Scatter(x=[2, 3, 4], y=[1, 2, 3]))
         assert d1 == d2
+        assert update_res1 is fig.data[0]
+        assert update_res2 is fig.data[1]
 
 
     def test_update_dict_empty(self):
         trace1 = Scatter(x=[1, 2, 3], y=[2, 1, 2])
         trace2 = Scatter(x=[1, 2, 3], y=[3, 2, 1])
         fig = Figure([trace1, trace2])
-        fig.update({})
+        update_res = fig.update({})
         d1, d2 = strip_dict_params(fig.data[0], Scatter(x=[1, 2, 3], y=[2, 1, 2]))
         assert d1 == d2
         d1, d2 = strip_dict_params(fig.data[1], Scatter(x=[1, 2, 3], y=[3, 2, 1]))
         assert d1 == d2
+        assert update_res is fig
 
 
     def test_update_list_empty(self):
Repository: plotly/plotly.py

Issue:
numpy.asscalar is deperecated
Per https://docs.scipy.org/doc/numpy/reference/generated/numpy.asscalar.html

np.asscalar is deprecated

https://github.com/plotly/plotly.py/blob/bc4fb4697bca78cd5732fdd3f86fa8b7eb53909c/_plotly_utils/basevalidators.py#L57



Discussion:
Thanks @yurzo.  Do you happen to know how far back `numpy.ndarray.item()` is supported?

cc @malmaud
@jonmmease looks like since 1.6 (https://docs.scipy.org/doc/numpy-1.6.0/reference/generated/numpy.ndarray.item.html)
I'll take a look in a few days.

On Wed, Feb 6, 2019 at 6:46 AM Ivan <notifications@github.com> wrote:

> @jonmmease <https://github.com/jonmmease> looks like since 1.6 (
> https://docs.scipy.org/doc/numpy-1.6.0/reference/generated/numpy.ndarray.item.html
> )
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/plotly/plotly.py/issues/1427#issuecomment-460995173>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AA8SvWKz4yjIyk_bOqlV1N14bjoYQdsUks5vKsCqgaJpZM4akh2y>
> .
>


Resolving patch:
diff --git a/_plotly_utils/basevalidators.py b/_plotly_utils/basevalidators.py
--- a/_plotly_utils/basevalidators.py
+++ b/_plotly_utils/basevalidators.py
@@ -49,12 +49,9 @@ def to_scalar_or_list(v):
     # 0-dimensional array, but we instead want it converted to a
     # Python native scalar type ('float' in the example above).
     # We explicitly check if is has the 'item' method, which conventionally
-    # converts these types to native scalars. This guards against 'v' already being
-    # a Python native scalar type since  `numpy.isscalar` would return
-    # True but `numpy.asscalar` will (oddly) raise an error is called with a
-    # a native Python scalar object.
+    # converts these types to native scalars.
     if np and np.isscalar(v) and hasattr(v, 'item'):
-        return np.asscalar(v)
+        return v.item()
     if isinstance(v, (list, tuple)):
         return [to_scalar_or_list(e) for e in v]
     elif np and isinstance(v, np.ndarray):


Test patch: