internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: googleapis/google-api-python-client

Issue:
Deprecation warning due to invalid escape sequences
Deprecation warnings are raised due to invalid escape sequences. This can be fixed by using raw strings or escaping the literals. pyupgrade also helps in automatic conversion : https://github.com/asottile/pyupgrade/

https://github.com/googleapis/google-api-python-client/blob/cd4e8f429422232dd82ef7e9bc685061d5df94a1/describe.py#L229

```
python -Wall -m py_compile describe.py
describe.py:203: DeprecationWarning: invalid escape sequence \s
  m = re.search('^\s+([a-zA-Z0-9_]+): (.*)', line)
```


Discussion:

Resolving patch:
diff --git a/describe.py b/describe.py
--- a/describe.py
+++ b/describe.py
@@ -226,7 +226,7 @@ def add_param(pname, desc):
             parameters.append(pname)
 
         for line in args:
-            m = re.search("^\s+([a-zA-Z0-9_]+): (.*)", line)
+            m = re.search(r"^\s+([a-zA-Z0-9_]+): (.*)", line)
             if m is None:
                 desc += line
                 continue
@@ -284,7 +284,7 @@ def breadcrumbs(path, root_discovery):
         display = p
         if i == 0:
             display = root_discovery.get("title", display)
-        crumbs.append('<a href="%s.html">%s</a>' % (prefix + p, display))
+        crumbs.append('<a href="{}.html">{}</a>'.format(prefix + p, display))
         accumulated.append(p)
 
     return " . ".join(crumbs)
@@ -407,7 +407,7 @@ def document_api(name, version, uri):
     version = safe_version(version)
 
     document_collection_recursive(
-        service, "%s_%s." % (name, version), discovery, discovery
+        service, "{}_{}.".format(name, version), discovery, discovery
     )
 
 
@@ -427,7 +427,7 @@ def document_api_from_discovery_document(uri):
     version = safe_version(discovery["version"])
 
     document_collection_recursive(
-        service, "%s_%s." % (name, version), discovery, discovery
+        service, "{}_{}.".format(name, version), discovery, discovery
     )
 
 


Test patch:
Repository: googleapis/google-api-python-client

Issue:
googleapiclient.discovery.build fails with module 'six.moves' has no attribute 'collections_abc' since version 1.12.0
#### Environment details

  - OS type and version: 18.04.1-Ubuntu
  - Python version: Python 3.6.9
  - pip version: `pip --version` pip 9.0.1
  - `google-api-python-client` version: `pip show google-api-python-client`: Version: 1.12.0

#### Code example
googleapiclient.discovery.build() fails with message: module 'six.moves' has no attribute 'collections_abc'

We only see this problem with google-api-python-client 1.12.0. 1.11.0 is fine.




Discussion:
This affects my team also, and now all of our Travis builds are failing.

NB: pinning our dependencies to 1.11.0 for now.
Our (Cloud Profiler team) probers failed, which is how we stumbled up on this. I am also curious why this was not caught in unit tests before it got merged and released.

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -42,7 +42,7 @@
     "google-auth>=1.16.0",
     "google-auth-httplib2>=0.0.3",
     "google-api-core>=1.21.0,<2dev",
-    "six>=1.6.1,<2dev",
+    "six>=1.13.0,<2dev",
     "uritemplate>=3.0.0,<4dev",
 ]
 


Test patch:
Repository: googleapis/google-api-python-client

Issue:
Create and populate folder for static discovery documents 
`googleapiclient` downloads the discovery docs for a Google API service at runtime and then constructs a client. Issue #1063 captures the need to move to a static package where the discovery docs are not fetched at run time. An incremental step towards satisfying #1063 would be to create a folder with static Google API discovery documents in this repo. The [discoveries folder](https://github.com/googleapis/discovery-artifact-manager/tree/master/discoveries) in the googleapis/discovery-artifact-manager repo has an up to date copy of the discovery documents. One approach would be to modify [synth.py](https://github.com/googleapis/google-api-python-client/blob/master/synth.py) to copy the discovery documents from discovery-artifact-manager similar to how it is done [here](https://github.com/googleapis/python-api-common-protos/blob/master/synth.py#L38).


Discussion:

Resolving patch:
diff --git a/synth.py b/synth.py
--- a/synth.py
+++ b/synth.py
@@ -14,6 +14,9 @@
 
 import synthtool as s
 from synthtool import gcp
+from synthtool.sources import git
+
+DISCOVERY_ARTIFACT_MANAGER_REPO = "googleapis/discovery-artifact-manager"
 
 common = gcp.CommonTemplates()
 
@@ -29,4 +32,20 @@
 # Also move issue templates
 s.move(templated_files / '.github')
 
-s.shell.run(["nox", "-s", "docs"], hide_output=False)
\ No newline at end of file
+# ----------------------------------------------------------------------------
+# Copy discoveries folder from discovery-artifact-manager repo
+# ----------------------------------------------------------------------------
+
+discovery_artifact_manager_url = git.make_repo_clone_url(DISCOVERY_ARTIFACT_MANAGER_REPO)
+discovery_artifacts = git.clone(discovery_artifact_manager_url) / "discoveries"
+
+excludes = [
+    "**/BUILD.bazel",
+]
+s.copy(discovery_artifacts,
+       "googleapiclient/discovery_cache/documents", excludes=excludes)
+
+# ----------------------------------------------------------------------------
+# Generate docs
+# ----------------------------------------------------------------------------
+s.shell.run(["nox", "-s", "docs"], hide_output=False)


Test patch:
Repository: googleapis/google-api-python-client

Issue:
Incorrect order for positional arguments in docs generation 
PR #1135 reduced the noise in the docs generation by sorting args in alphabetical order however a regression was introduced in the docs where positional arguments were also sorted. It should be a quick fix to exclude positional arguments from being sorted.


Discussion:

Resolving patch:
diff --git a/describe.py b/describe.py
--- a/describe.py
+++ b/describe.py
@@ -215,6 +215,7 @@ def method_params(doc):
             args = doclines[begin + 1 :]
 
         parameters = []
+        sorted_parameters = []
         pname = None
         desc = ""
 
@@ -223,7 +224,11 @@ def add_param(pname, desc):
                 return
             if "(required)" not in desc:
                 pname = pname + "=None"
-            parameters.append(pname)
+                parameters.append(pname)
+            else:
+                # required params should be put straight into sorted_parameters
+                # to maintain order for positional args
+                sorted_parameters.append(pname)
 
         for line in args:
             m = re.search(r"^\s+([a-zA-Z0-9_]+): (.*)", line)
@@ -234,10 +239,11 @@ def add_param(pname, desc):
             pname = m.group(1)
             desc = m.group(2)
         add_param(pname, desc)
-        parameters = ", ".join(sorted(parameters))
+        sorted_parameters.extend(sorted(parameters))
+        sorted_parameters = ", ".join(sorted_parameters)
     else:
-        parameters = ""
-    return parameters
+        sorted_parameters = ""
+    return sorted_parameters
 
 
 def method(name, doc):


Test patch:
Repository: googleapis/google-api-python-client

Issue:
HttpError error_details isn't populated unless __repr__ is called first
I was trying to handle an HttpError by looking at the contents of the `error_details` attribute. I noticed the attribute is a null-string unless I trigger the `__repr__` function first. For example, this does not work as the error_details is always `""`. I made a simple test that demonstrates the error:

```
from googleapiclient import discovery, errors

client = discovery.build(
    "discovery", "v1"
)

req = client.apis().getRest(api='fake_api', version='v1')

try:
    resp = req.execute()
except errors.HttpError as err:
    print(f'Error details are currently: "{err.error_details}"')
    print(f'Exception string representation is: "{err}"')
    print(f'Error details are currently: "{err.error_details}"')
```

The output of the above code:

```
Error details are currently: ""
Exception string representation is: "<HttpError 404 when requesting https://www.googleapis.com/discovery/v1/apis/fake_api/v1/rest?alt=json returned "Requested entity was not found.". Details: "Requested entity was not found.">"
Error details are currently: "Requested entity was not found."
```

I tested and the behavior is the same on both `google-api-python-client-1.12.8` and `google-api-python-client-2.0.2`


Discussion:

Resolving patch:
diff --git a/googleapiclient/errors.py b/googleapiclient/errors.py
--- a/googleapiclient/errors.py
+++ b/googleapiclient/errors.py
@@ -43,6 +43,12 @@ def __init__(self, resp, content, uri=None):
         self.content = content
         self.uri = uri
         self.error_details = ""
+        self._get_reason()
+
+    @property
+    def status_code(self):
+        """Return the HTTP status code from the response content."""
+        return self.resp.status
 
     def _get_reason(self):
         """Calculate the reason for the error from the response content."""


Test patch:
diff --git a/tests/test_errors.py b/tests/test_errors.py
--- a/tests/test_errors.py
+++ b/tests/test_errors.py
@@ -83,6 +83,8 @@ def test_json_body(self):
             reason="Failed",
         )
         error = HttpError(resp, content, uri="http://example.org")
+        self.assertEqual(error.error_details, "error details")
+        self.assertEqual(error.status_code, 400)
         self.assertEqual(
             str(error),
             '<HttpError 400 when requesting http://example.org returned "country is required". Details: "error details">',
Repository: googleapis/google-api-python-client

Issue:
Google Auth test_with_scopes_scoped test failing with google-auth==1.25.0 or greater
The GoogleAuth unittest [test_with_scopes_scoped](https://github.com/googleapis/google-api-python-client/blob/master/tests/test__auth.py#L79) is failing at [this line](https://github.com/googleapis/google-api-python-client/blob/master/tests/test__auth.py#L92). 

With `google-auth==1.25.0`, the test fails.
```
tests/test__auth.py::TestAuthWithGoogleAuth::test_with_scopes_scoped FAILED  
```

With `google-auth==1.24.0` , the same test passes.
```
tests/test__auth.py::TestAuthWithGoogleAuth::test_with_scopes_scoped PASSED      
```



Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -39,6 +39,8 @@
     "google-auth>=1.16.0",
     "google-auth-httplib2>=0.0.3",
     "google-api-core>=1.21.0,<2dev",
+    # rsa version 4.5 is the last version that is compatible with Python 2.7
+    "rsa==4.5;python_version<'3'",
     "six>=1.13.0,<2dev",
     "uritemplate>=3.0.0,<4dev",
 ]


Test patch:
diff --git a/tests/test__auth.py b/tests/test__auth.py
--- a/tests/test__auth.py
+++ b/tests/test__auth.py
@@ -18,6 +18,7 @@
 import google_auth_httplib2
 import httplib2
 import oauth2client.client
+import pkg_resources
 import unittest2 as unittest
 
 from googleapiclient import _auth
@@ -82,6 +83,8 @@ class CredentialsWithScopes(
         ):
             pass
 
+        google_auth_version = pkg_resources.get_distribution("google-auth").parsed_version
+
         credentials = mock.Mock(spec=CredentialsWithScopes)
         credentials.requires_scopes = True
 
@@ -89,7 +92,12 @@ class CredentialsWithScopes(
 
         self.assertNotEqual(credentials, returned)
         self.assertEqual(returned, credentials.with_scopes.return_value)
-        credentials.with_scopes.assert_called_once_with(mock.sentinel.scopes)
+
+        # The `default_scopes` argument was added in google-auth==1.25.0
+        if google_auth_version >= pkg_resources.parse_version("1.25.0"):
+            credentials.with_scopes.assert_called_once_with(mock.sentinel.scopes, default_scopes=None)
+        else:
+            credentials.with_scopes.assert_called_once_with(mock.sentinel.scopes)
 
     def test_authorized_http(self):
         credentials = mock.Mock(spec=google.auth.credentials.Credentials)
Repository: googleapis/google-api-python-client

Issue:
Published package is missing discovery files in discovery_cache
Many thanks to the @wyk9787 for noticing this and reaching out.

All calls to `discovery.build()` using `2.0.0` fail with "unknown api name or version".

```python
from googleapiclient import discovery

client = discovery.build("cloudprofiler", "v2")
```


This is because the published package has no `discovery_cache/documents` directory. 

1. `python3 -m venv env`
2. `source env/bin/activate`
3. `python3 -m pip install google-api-python-client`
4. `ls env/lib/python*/site-packages/googleapiclient/discovery_cache`

```
busunkim@busunkim:~/github$ ls env/lib/python*/site-packages/googleapiclient/discovery_cache
appengine_memcache.py  base.py  file_cache.py  __init__.py  __pycache__
```




Discussion:
Version 2.0.1 also has the same issue. The issue is resolved by changing this [line](https://github.com/googleapis/google-api-python-client/blob/master/setup.py#L63) from 
```
    package_data={},
```
to
```
package_data={"googleapiclient": ["discovery_cache/documents/*.json"]},
```




Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -60,7 +60,7 @@
     install_requires=install_requires,
     python_requires=">=3.6",
     packages=packages,
-    package_data={},
+    package_data={"googleapiclient": ["discovery_cache/documents/*.json"]},
     license="Apache 2.0",
     keywords="google api client",
     classifiers=[


Test patch:
Repository: googleapis/google-api-python-client

Issue:
Remove duplicate docs generation
In `synth.py` we have a `nox` session to generate the docs [here](https://github.com/googleapis/google-api-python-client/blob/master/synth.py#L36). The same python script is running as part of the Github action in #1187, so we should remove the `docs` session from `synth.py` and `noxfile.py`.


Discussion:

Resolving patch:
diff --git a/noxfile.py b/noxfile.py
--- a/noxfile.py
+++ b/noxfile.py
@@ -75,9 +75,3 @@ def unit(session, oauth2client):
         "tests",
         *session.posargs,
     )
-
-
-@nox.session(python="3.6")
-def docs(session):
-    session.install('.')
-    session.run("python", "describe.py")
\ No newline at end of file
diff --git a/synth.py b/synth.py
--- a/synth.py
+++ b/synth.py
@@ -29,8 +29,3 @@
 
 # Also move issue templates
 s.move(templated_files / '.github')
-
-# ----------------------------------------------------------------------------
-# Generate docs
-# ----------------------------------------------------------------------------
-s.shell.run(["nox", "-s", "docs"], hide_output=False)


Test patch:
Repository: googleapis/google-api-python-client

Issue:
Modify noxfile to build and test the package
Versions `2.0.0` and `2.0.1` were yanked from PyPI last week due to an issue where discovery documents were not included in the published package causing `discovery.build()` to fail(#1214).  A basic check could be added to verify the package works correctly using the steps in #1214. Ideally it should be done on every PR and push to master so the issue can be caught before the package is published. 

Use these steps from #1214 to re-produce the issue with version `2.0.0` and `2.0.1`:
1. Start with a clean clone of `google-api-python-client`
2. Checkout version `2.0.0` or `2.0.1`, using `git checkout 2.0.0`
3. Run `python setup.py sdist`
4. Run `pip install dist/google-api-python-client-<version>.tar.gz`
5. Run 
```
$ python3
Python 3.8.7 (default, Jan 27 2021, 18:44:05) 
[GCC 10.2.1 20201224] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from googleapiclient import discovery
>>> client = discovery.build("cloudprofiler", "v2")
...
```

Before closing this issue, we should ensure that we have checks in place so that a PR will fail if `package_data` [here](https://github.com/googleapis/google-api-python-client/blob/master/setup.py#L78) is empty.



Discussion:
Bumping to p1/bug to comply with postmortem requirements. 

Resolving patch:
diff --git a/noxfile.py b/noxfile.py
--- a/noxfile.py
+++ b/noxfile.py
@@ -16,6 +16,8 @@
 import sys
 
 import nox
+import os
+import shutil
 
 test_dependencies = [
     "django>=2.0.0",
@@ -58,9 +60,22 @@ def lint(session):
     ],
 )
 def unit(session, oauth2client):
+    # Clean up dist and build folders
+    shutil.rmtree('dist', ignore_errors=True)
+    shutil.rmtree('build', ignore_errors=True)
+
     session.install(*test_dependencies)
     session.install(oauth2client)
-    session.install('.')
+
+    # Create and install wheels
+    session.run('python3', 'setup.py', 'bdist_wheel')
+    session.install(os.path.join('dist', os.listdir('dist').pop()))
+
+    # Run tests from a different directory to test the package artifacts
+    root_dir = os.path.dirname(os.path.realpath(__file__))
+    temp_dir = session.create_tmp()
+    session.chdir(temp_dir)
+    shutil.copytree(os.path.join(root_dir, 'tests'), 'tests')
 
     # Run py.test against the unit tests.
     session.run(


Test patch:
Repository: googleapis/google-api-python-client

Issue:
Original V2_DISCOVERY_URI might be accessed when discoveryServiceUrl is specified
When service provided at discoveryServiceUrl returns NOT_FOUND, V2_DISCOVERY_URI is contacted, and there is no way to override it.

Expected behavior: if discoveryServiceUrl is provided, original googleapis URLs should not be contacted.


Discussion:

Resolving patch:
diff --git a/googleapiclient/discovery.py b/googleapiclient/discovery.py
--- a/googleapiclient/discovery.py
+++ b/googleapiclient/discovery.py
@@ -274,9 +274,6 @@ def build(
         else:
             static_discovery = False
 
-    if discoveryServiceUrl is None:
-        discoveryServiceUrl = DISCOVERY_URI
-
     if http is None:
         discovery_http = build_http()
     else:
@@ -343,14 +340,16 @@ def _discovery_service_uri_options(discoveryServiceUrl, version):
       A list of URIs to be tried for the Service Discovery, in order.
     """
 
-    urls = [discoveryServiceUrl, V2_DISCOVERY_URI]
-    # V1 Discovery won't work if the requested version is None
-    if discoveryServiceUrl == V1_DISCOVERY_URI and version is None:
+    if discoveryServiceUrl is not None:
+        return [discoveryServiceUrl]
+    if version is None:
+        # V1 Discovery won't work if the requested version is None
         logger.warning(
             "Discovery V1 does not support empty versions. Defaulting to V2..."
         )
-        urls.pop(0)
-    return list(OrderedDict.fromkeys(urls))
+        return [V2_DISCOVERY_URI]
+    else:
+        return [DISCOVERY_URI, V2_DISCOVERY_URI]
 
 
 def _retrieve_discovery_doc(


Test patch:
Repository: googleapis/google-api-python-client

Issue:
Certain discovery artifacts are not being updated.
For example, `adexchangebuyer` version `v1.4` hasn't been updated in 6 months.

https://github.com/googleapis/google-api-python-client/blob/master/googleapiclient/discovery_cache/documents/adexchangebuyer.v1.4.json

This is related to #1330


Discussion:

Resolving patch:
diff --git a/describe.py b/describe.py
--- a/describe.py
+++ b/describe.py
@@ -403,9 +403,7 @@ def document_api(name, version, uri, doc_destination_dir):
     if resp.status == 200:
         discovery = json.loads(content)
         service = build_from_document(discovery)
-        version = safe_version(version)
-        doc_name = "{}.{}.json".format(name, version.replace("_", ""))
-
+        doc_name = "{}.{}.json".format(name, version)
         discovery_file_path = DISCOVERY_DOC_DIR / doc_name
         revision = None
 
@@ -435,7 +433,7 @@ def document_api(name, version, uri, doc_destination_dir):
         return
 
     document_collection_recursive(
-        service, "{}_{}.".format(name, version), discovery, discovery, doc_destination_dir
+        service, "{}_{}.".format(name, safe_version(version)), discovery, discovery, doc_destination_dir
     )
 
 


Test patch:
Repository: googleapis/google-api-python-client

Issue:
_next method creates a HTTP request with wrong content-length header
Using google-api-python-client 2.2.0 and the GCP monitoring API, I received an error when retrieving the second page of a [projects.timeSeries.query](https://cloud.google.com/monitoring/api/ref_v3/rest/v3/projects.timeSeries/query) call:

```
can't fetch data, reason: Invalid JSON payload received. Unexpected end of string. Expected an object key or }.
```

Turns out that the content-length header is wrong and doesn't match with the body with the additional pageToken field.

The problem is in https://github.com/googleapis/google-api-python-client/blob/master/googleapiclient/discovery.py#L1285: the body is replaced, but the content-length stays the same because it is only updated at Request object creation time. I will prepare a PR for this.


Discussion:

Resolving patch:
diff --git a/googleapiclient/discovery.py b/googleapiclient/discovery.py
--- a/googleapiclient/discovery.py
+++ b/googleapiclient/discovery.py
@@ -1283,6 +1283,9 @@ def methodNext(self, previous_request, previous_response):
             body = model.deserialize(request.body)
             body[pageTokenName] = nextPageToken
             request.body = model.serialize(body)
+            request.body_size = len(request.body)
+            if "content-length" in request.headers:
+              del request.headers["content-length"]
             logger.debug("Next page request body: %s %s" % (methodName, body))
 
         return request


Test patch:
diff --git a/tests/test_discovery.py b/tests/test_discovery.py
--- a/tests/test_discovery.py
+++ b/tests/test_discovery.py
@@ -2172,6 +2172,9 @@ def test_next_successful_with_next_page_token_in_body(self):
         next_request = logging.entries().list_next(request, {"nextPageToken": "123abc"})
         body = JsonModel().deserialize(next_request.body)
         self.assertEqual(body["pageToken"], "123abc")
+        # The body is changed, make sure that body_length is changed too (see
+        # github #1403)
+        self.assertEqual(next_request.body_size, len(next_request.body))
 
     def test_next_with_method_with_no_properties(self):
         self.http = HttpMock(datafile("latitude.json"), {"status": "200"})