Repository: buildbot/buildbot

Issue:
BitbucketPullrequestPoller fails all requests in python3
`_getChanges` and `_processChanges` all perform `client.getPage` with str arguments, which fail because `client.getPage` expect to receive `bytes` for the `url` parameter.

I worked around the issue locally by replacing all the `client.getPage` calls by `self.getPage` which encodes the url to bytes when necessary, and returns `client.getPage`, but there may be a cleaner way to handle this.


Discussion:
```
2018-06-30 19:36:10+0530 [-] BitbucketPullrequestPoller: polling Bitbucket repository Leadics_Products/querybuilder, branch: feature
2018-06-30 19:36:10+0530 [-] while polling for changes
	Traceback (most recent call last):
	  File "/home/rajat/anaconda3/envs/querybuilder/lib/python3.6/site-packages/twisted/internet/defer.py", line 321, in addCallback
	    callbackKeywords=kw)
	  File "/home/rajat/anaconda3/envs/querybuilder/lib/python3.6/site-packages/twisted/internet/defer.py", line 310, in addCallbacks
	    self._runCallbacks()
	  File "/home/rajat/anaconda3/envs/querybuilder/lib/python3.6/site-packages/twisted/internet/defer.py", line 653, in _runCallbacks
	    current.result = callback(current.result, *args, **kw)
	  File "/home/rajat/anaconda3/envs/querybuilder/lib/python3.6/site-packages/twisted/internet/defer.py", line 1571, in execute
	    d = maybeDeferred(f, *args, **kwargs)
	--- <exception caught here> ---
	  File "/home/rajat/anaconda3/envs/querybuilder/lib/python3.6/site-packages/twisted/internet/defer.py", line 150, in maybeDeferred
	    result = f(*args, **kw)
	  File "/home/rajat/anaconda3/envs/querybuilder/lib/python3.6/site-packages/buildbot/changes/bitbucket.py", line 81, in poll
	    d = self._getChanges()
	  File "/home/rajat/anaconda3/envs/querybuilder/lib/python3.6/site-packages/buildbot/changes/bitbucket.py", line 92, in _getChanges
	    return client.getPage(url, timeout=self.pollInterval)
	  File "/home/rajat/anaconda3/envs/querybuilder/lib/python3.6/site-packages/twisted/python/deprecate.py", line 301, in deprecatedFunction
	    return function(*args, **kwargs)
	  File "/home/rajat/anaconda3/envs/querybuilder/lib/python3.6/site-packages/twisted/web/client.py", line 788, in getPage
	    *args, **kwargs).deferred
	  File "/home/rajat/anaconda3/envs/querybuilder/lib/python3.6/site-packages/twisted/web/client.py", line 735, in _makeGetterFactory
	    uri = URI.fromBytes(url)
	  File "/home/rajat/anaconda3/envs/querybuilder/lib/python3.6/site-packages/twisted/web/client.py", line 638, in fromBytes
	    scheme, netloc, path, params, query, fragment = http.urlparse(uri)
	  File "/home/rajat/anaconda3/envs/querybuilder/lib/python3.6/site-packages/twisted/web/http.py", line 181, in urlparse
	    raise TypeError("url must be bytes, not unicode")
	builtins.TypeError: url must be bytes, not unicode
```
/home//anaconda3/envs//lib/python3.6/site-packages/buildbot/changes/bitbucket.py

in line 92 replaced

`return client.getPage(url, timeout=self.pollInterval)`
with

`return client.getPage(str.encode(url), timeout=self.pollInterval)`

2018-07-02 12:08:10+0530 [HTTPPageGetter (TLSMemoryBIOProtocol),client] BitbucketPullrequestPoller: json api poll failed

2018-07-02 12:08:10+0530 [HTTPPageGetter (TLSMemoryBIOProtocol),client] Unhandled Error
	Traceback (most recent call last):
	Failure: twisted.web.error.Error: 403 Forbidden

Any Work around ?

This code should be converted to HTTPClientService which handles all that correctly, and is testable:

http://docs.buildbot.net/current/developer/utils.html#buildbot-util-httpclientservice

The workaround is fine, but if anybody wants to step up and fix it correctly, this would be great.

@devrajatverma : looks like you did pass wrong credentials
@tardyp 
I didn't passed any
don't know where to pass them, in BitbucketPullrequestPoller or in steps.Git
in docs there is no kwarg naming username and password mentioned for BitbucketPullrequestPoller
Indeed, looks like this contributed code only supports public projects :-/

Feel free to enhance it!
Keep in mind that I will only accept enhancement to this code if it is first moved to httpclientservice
Also there is the bitbucket webhook, which is a much better solution for event based CI (instead of polling based CI)
In case someone stumbles here until a fix is contributed, here is the version I made/used: https://gist.github.com/veloutin/419514e0ffecc52ce6da7738ca770ac6

It does add authentication and fixes this bug.

My priorities shifted away from this a while back, but I suppose a gist is better than nothing, even if it isn't a proper contribution :)

hi @veloutin Thanks for sharing a fix. We need someone from the community to include this fix in a proper pullrequest and update the unit tests.

Thanks you!
If I update bitbucketcloud.py and its unit tests to match the [latest json format](https://confluence.atlassian.com/bitbucket/event-payloads-740262817.html),
 will it have a chance of being pulled?
Hi, what's the status with this bug. Is anyone actively looking at it?

I see that @p12tic has done quite a bit of refactoring since the original issues was reported by @veloutin but this bug still exists.  @p12tic , how are you using this module if this bug remains?  I might be missing something.
@tardyp is this something you have any visibility of or has it just dropped off the radar?  I've been trying to get it working from behind a proxy but the Twisted ProxyAgent class doesn't seem to work or I don't know how to drive it.  Curl, wget, Python requests all work without issue.  If someone can provide some pointers I'd be happy to have a look at fixing this.
@sjalloq    from what I can tell, buildbot is no longer being actively maintained.  You can only fork and fix at this point I think.


@jerrylogansquare ouch, ok, then it might be easier to move to something else.
@sjalloq

> I see that @p12tic has done quite a bit of refactoring since the original issues was reported by @veloutin but this bug still exists. @p12tic , how are you using this module if this bug remains? I might be missing something.

I personally don't use Bitbucket, so like in any open source project the support for less popular features like Bitbucket depends on whether anyone has time to submit at least a test pull request. So far there has been none, so it seems that proper Bitbucket support upstream is not really needed by anyone. If someone could add a failing test case and fix it with needed code changes that would be a great start.
@p12tic 

I've had a look at the unit tests and they all seem to monkey patch `client.getPage` which is why no failures have been seen.  What's the solution here?  To just update the fake `client.getPage`?

Moving forward, I also need to update the class to accept a proxy.  Do any of the available http classes use the http[s]_proxy environment variables?  Twisted seems to want you to use `ProxyAgent` which doesn't fit well with how `BitbucketPullRequestPoller` has been written.

This is what I have so far.  It incorporates veloutin's fix and updates the tests:  https://github.com/sjalloq/buildbot/commit/5e5092e1e41e32c196b1a4f1189a6fb8be27a90b
@sjalloq 

> I've had a look at the unit tests and they all seem to monkey patch client.getPage which is why no failures have been seen. What's the solution here? To just update the fake client.getPage?

Yes, the intention is that we have a set of known good data to test against. As a result we don't use network when running tests. On the bad side, we can't detect when upstream API changes.

If currently the only problem is what's listed in https://github.com/buildbot/buildbot/issues/4153#issuecomment-401697608 , then the best course of action is this:

 - you apply the fix on your buildbot instance and check if it actually works with real Bitbucket (seems to be done on https://github.com/sjalloq/buildbot/commit/5e5092e1e41e32c196b1a4f1189a6fb8be27a90b)
 - you submit a PR with this change
 - I review and merge the PR
 - I replace the current implementation with httpclientservice with corresponding tests
 - you test whether the httpclientservice version still works with real Bitbucket
 - we look into how to add proxy support into httpclientservice so it can be used for all HTTP requests across Buildbot.

Does that sound right?
 

@p12tic 

That all sounds amazing, thanks.  I'll test what I have first thing on Monday.  ~~As a side note, we're already using the GitPoller which seem to work with our proxy so however that is implemented could be copied I guess.~~

Edit: ignore that comment, I now see the git poller doesn't do http requests as I assumed.
@p12tic okay, I give up, how do you install the full bundle from a Github branch.  I've tried adding all the packages to a requirements.txt but am just finding hundreds of ways for pip install to error out.  I couldn't see anything in the documentation about this.

Edit: finally got there.  There really isn't any good documentation on pip errors when you don't specify the the correct subdirectory to find the setup.py file.

Anyway, the fix seems to be working in at least the master doesn't just output errors but the job doesn't seem to do anything.  I'm just starting to debug but any pointers would be welcome. 

Secondly, I can't work out how to get the pull requests properties into the builder.  This doesn't seem to be documented.  I did find some example online where someone had used `util.Properties('repository')` but confirmation would be good.
@p12tic Finally got there.  PR created.  It seems to be working on my local dev machine.  I'll try it in our cluster with the proxy next.  I did notice that client.getPage already calls httpclientservice.

Resolving patch:
diff --git a/master/buildbot/changes/bitbucket.py b/master/buildbot/changes/bitbucket.py
--- a/master/buildbot/changes/bitbucket.py
+++ b/master/buildbot/changes/bitbucket.py
@@ -14,14 +14,14 @@
 # Copyright Buildbot Team Members
 
 
-import base64
 import json
 import time
 from datetime import datetime
 
+from txrequests import Session
+
 from twisted.internet import defer
 from twisted.python import log
-from twisted.web import client
 
 from buildbot.changes import base
 from buildbot.util import bytes2unicode
@@ -73,16 +73,15 @@ def reconfigService(self, owner, slug,
         self.lastChange = time.time()
         self.lastPoll = time.time()
         self.useTimestamps = useTimestamps
-        self.category = category if callable(
-            category) else bytes2unicode(category)
+        self.category = category if callable(category) else bytes2unicode(category)
         self.project = bytes2unicode(project)
         self.external_property_whitelist = bitbucket_property_whitelist
 
         if auth is not None:
-            encoded_credentials = base64.b64encode(":".join(auth).encode())
-            self.headers = {b"Authorization": b"Basic " + encoded_credentials}
+            # Tuple(user,pass)
+            self.auth = auth
         else:
-            self.headers = None
+            self.auth = None
 
         yield super().reconfigService(self.build_name(owner, slug),
                                       pollInterval=pollInterval, pollAtLaunch=pollAtLaunch)
@@ -97,16 +96,28 @@ def describe(self):
     @deferredLocked('initLock')
     @defer.inlineCallbacks
     def poll(self):
-        page = yield self._getChanges()
-        yield self._processChanges(page)
+        response = yield self._getChanges()
+        yield self._processChanges(response.content)
+
+    def getPage(self, url, **kwargs):
+        """ Retrieve HTML page.
+
+        Use txrequests instead of twisted.web.client.getPage() which is deprecated.
+        Returns a deferred object that is a Response type.  The binary response body
+        is available via response.content.
+
+        """
+        with Session() as session:
+            response = session.get(url, auth=self.auth, **kwargs)
+        return response
 
     def _getChanges(self):
         self.lastPoll = time.time()
         log.msg("BitbucketPullrequestPoller: polling "
                 "Bitbucket repository {}/{}, branch: {}".format(self.owner, self.slug, self.branch))
-        url = "https://bitbucket.org/api/2.0/repositories/{}/{}/pullrequests".format(self.owner,
+        url = "https://api.bitbucket.org/2.0/repositories/{}/{}/pullrequests".format(self.owner,
                                                                                      self.slug)
-        return client.getPage(url, timeout=self.pollInterval, headers=self.headers)
+        return self.getPage(url, timeout=self.pollInterval)
 
     @defer.inlineCallbacks
     def _processChanges(self, page):
@@ -126,9 +137,10 @@ def _processChanges(self, page):
                 # compare _short_ hashes to check if the PR has been updated
                 if not current or current[0:12] != revision[0:12]:
                     # parse pull request api page (required for the filter)
-                    page = yield client.getPage(str(pr['links']['self']['href']),
-                                                headers=self.headers)
-                    pr_json = json.loads(page)
+                    response = yield self.getPage(
+                        str(pr['links']['self']['href'])
+                    )
+                    pr_json = json.loads(response.content)
 
                     # filter pull requests by user function
                     if not self.pullrequest_filter(pr_json):
@@ -138,6 +150,7 @@ def _processChanges(self, page):
                     # access additional information
                     author = pr['author']['display_name']
                     prlink = pr['links']['html']['href']
+
                     # Get time updated time. Note that the timezone offset is
                     # ignored.
                     if self.useTimestamps:
@@ -147,25 +160,27 @@ def _processChanges(self, page):
                     else:
                         updated = epoch2datetime(self.master.reactor.seconds())
                     title = pr['title']
+
                     # parse commit api page
-                    page = yield client.getPage(
-                        str(pr['source']['commit']['links']['self']['href']),
-                        headers=self.headers,
+                    response = yield self.getPage(
+                        str(pr['source']['commit']['links']['self']['href'])
                     )
-                    commit_json = json.loads(page)
+                    commit_json = json.loads(response.content)
+
                     # use the full-length hash from now on
                     revision = commit_json['hash']
                     revlink = commit_json['links']['html']['href']
+
                     # parse repo api page
-                    page = yield client.getPage(
-                        str(pr['source']['repository']['links']['self']['href']),
-                        headers=self.headers,
+                    response = yield self.getPage(
+                        str(pr['source']['repository']['links']['self']['href'])
                     )
-                    repo_json = json.loads(page)
+                    repo_json = json.loads(response.content)
                     repo = repo_json['links']['html']['href']
 
                     # update database
                     yield self._setCurrentRev(nr, revision)
+
                     # emit the change
                     yield self.master.data.updates.addChange(
                         author=bytes2unicode(author),


Test patch:
diff --git a/master/buildbot/test/unit/changes/test_bitbucket.py b/master/buildbot/test/unit/changes/test_bitbucket.py
--- a/master/buildbot/test/unit/changes/test_bitbucket.py
+++ b/master/buildbot/test/unit/changes/test_bitbucket.py
@@ -18,7 +18,6 @@
 
 from twisted.internet import defer
 from twisted.trial import unittest
-from twisted.web import client
 from twisted.web.error import Error
 
 from buildbot.changes.bitbucket import BitbucketPullrequestPoller
@@ -26,8 +25,13 @@
 from buildbot.test.util import changesource
 
 
+class FakeResponse:
+    def __init__(self, content):
+        self.content = content
+
+
 class SourceRest():
-    """https://bitbucket.org/!api/2.0/repositories/{owner}/{slug}"""
+    """https://api.bitbucket.org/2.0/repositories/{owner}/{slug}"""
     template = """\
 {
 
@@ -40,7 +44,7 @@ class SourceRest():
     "repository": {
         "links": {
             "self": {
-                "href": "https://bitbucket.org/!api/2.0/repositories/%(owner)s/%(slug)s"
+                "href": "https://api.bitbucket.org/2.0/repositories/%(owner)s/%(slug)s"
             }
         }
     },
@@ -65,23 +69,23 @@ def __init__(self, owner, slug, hash, date):
         self.date = date
 
     def request(self):
-        return self.template % {
+        return FakeResponse(self.template % {
             "owner": self.owner,
             "slug": self.slug,
             "hash": self.hash,
             "short_hash": self.hash[0:12],
             "date": self.date,
-        }
+        })
 
     def repo_request(self):
-        return self.repo_template % {
+        return FakeResponse(self.repo_template % {
             "owner": self.owner,
             "slug": self.slug,
-        }
+        })
 
 
 class PullRequestRest():
-    """https://bitbucket.org/!api/2.0/repositories/{owner}/{slug}/pullrequests/{pull_request_id}"""
+    """https://api.bitbucket.org/2.0/repositories/{owner}/{slug}/pullrequests/{pull_request_id}"""
     template = """\
 {
 
@@ -92,7 +96,7 @@ class PullRequestRest():
             "hash": "%(hash)s",
             "links": {
                 "self": {
-                    "href": "https://bitbucket.org/!api/2.0/repositories/%(owner)s/%(slug)s/commit/%(hash)s"
+                    "href": "https://api.bitbucket.org/2.0/repositories/%(owner)s/%(slug)s/commit/%(hash)s"
                 }
             }
         }
@@ -124,7 +128,7 @@ def __init__(self, nr, title, description, display_name, source, created_on, upd
             self.updated_on = self.created_on
 
     def request(self):
-        return self.template % {
+        return FakeResponse(self.template % {
             "description": self.description,
             "title": self.title,
             "hash": self.source.hash,
@@ -135,17 +139,17 @@ def request(self):
             "created_on": self.created_on,
             "updated_on": self.updated_on,
             "id": self.nr,
-        }
+        })
 
 
 class PullRequestListRest():
-    """https://bitbucket.org/api/2.0/repositories/{owner}/{slug}/pullrequests"""
+    """https://api.bitbucket.org/2.0/repositories/{owner}/{slug}/pullrequests"""
     template = """\
         {
             "description": "%(description)s",
             "links": {
                 "self": {
-                    "href": "https://bitbucket.org/!api/2.0/repositories/%(owner)s/%(slug)s/pullrequests/%(id)d"
+                    "href": "https://api.bitbucket.org/2.0/repositories/%(owner)s/%(slug)s/pullrequests/%(id)d"
                 },
                 "html": {
                     "href": "https://bitbucket.org/%(owner)s/%(slug)s/pull-request/%(id)d"
@@ -160,14 +164,14 @@ class PullRequestListRest():
                     "hash": "%(short_hash)s",
                     "links": {
                         "self": {
-                            "href": "https://bitbucket.org/!api/2.0/repositories/%(src_owner)s/%(src_slug)s/commit/%(short_hash)s"
+                            "href": "https://api.bitbucket.org/2.0/repositories/%(src_owner)s/%(src_slug)s/commit/%(short_hash)s"
                         }
                     }
                 },
                 "repository": {
                     "links": {
                         "self": {
-                            "href": "https://bitbucket.org/!api/2.0/repositories/%(src_owner)s/%(src_slug)s"
+                            "href": "https://api.bitbucket.org/2.0/repositories/%(src_owner)s/%(src_slug)s"
                         }
                     }
                 },
@@ -212,7 +216,7 @@ def request(self):
                 "updated_on": pr.updated_on,
                 "id": pr.nr,
             }
-        return """\
+        return FakeResponse("""\
 {
 
     "pagelen": 10,
@@ -221,19 +225,19 @@ def request(self):
     "page": 1
 
 }
-""" % s
+""" % s)
 
     def getPage(self, url, timeout=None, headers=None):
         list_url_re = re.compile(
-            r"https://bitbucket.org/api/2.0/repositories/{}/{}/pullrequests".format(self.owner,
+            r"https://api.bitbucket.org/2.0/repositories/{}/{}/pullrequests".format(self.owner,
                                                                                     self.slug))
         pr_url_re = re.compile(
-            r"https://bitbucket.org/!api/2.0/repositories/{}/{}/pullrequests/(?P<id>\d+)".format(
+            r"https://api.bitbucket.org/2.0/repositories/{}/{}/pullrequests/(?P<id>\d+)".format(
                     self.owner, self.slug))
         source_commit_url_re = re.compile(
-            r"https://bitbucket.org/!api/2.0/repositories/(?P<src_owner>.*)/(?P<src_slug>.*)/commit/(?P<hash>\d+)")  # noqa pylint: disable=line-too-long
+            r"https://api.bitbucket.org/2.0/repositories/(?P<src_owner>.*)/(?P<src_slug>.*)/commit/(?P<hash>\d+)")  # noqa pylint: disable=line-too-long
         source_url_re = re.compile(
-            r"https://bitbucket.org/!api/2.0/repositories/(?P<src_owner>.*)/(?P<src_slug>.*)")
+            r"https://api.bitbucket.org/2.0/repositories/(?P<src_owner>.*)/(?P<src_slug>.*)")
 
         if list_url_re.match(url):
             return defer.succeed(self.request())
@@ -252,6 +256,7 @@ def getPage(self, url, timeout=None, headers=None):
             return self.src_by_url["{}/{}".format(m.group("src_owner"),
                                                   m.group("src_slug"))].repo_request()
 
+        print('ZZZ:', url)
         raise Error(code=404)
 
 
@@ -318,20 +323,20 @@ def _fakeGetPage(self, result):
         def fake(url, timeout=None, headers=None):
             self.getPage_got_url = url
             return defer.succeed(result)
-        self.patch(client, "getPage", fake)
+        self.patch(self.changesource, "getPage", fake)
 
     def _fakeGetPage403(self, expected_headers):
 
         def fail_unauthorized(url, timeout=None, headers=None):
             if headers != expected_headers:
                 raise Error(code=403)
-        self.patch(client, "getPage", fail_unauthorized)
+        self.patch(self.changesource, "getPage", fail_unauthorized)
 
     def _fakeGetPage404(self):
 
         def fail(url, timeout=None, headers=None):
             raise Error(code=404)
-        self.patch(client, "getPage", fail)
+        self.patch(self.changesource, "getPage", fail)
 
     def attachDefaultChangeSource(self):
         return self.attachChangeSource(BitbucketPullrequestPoller(
@@ -368,22 +373,6 @@ def test_poll_unauthorized_failure(self):
         except Exception as e:
             self.assertEqual(str(e), '403 Forbidden')
 
-    @defer.inlineCallbacks
-    def test_poll_authorized_success(self):
-        auth = ('user', '1234')
-        expected_headers = {b'Authorization': b'Basic dXNlcjoxMjM0'}
-        yield self.attachChangeSource(BitbucketPullrequestPoller(
-            owner='owner',
-            slug='slug',
-            auth=auth,
-        ))
-        # Polling with authorization should success
-        self._fakeGetPage403(expected_headers)
-        try:
-            yield self.changesource.poll()
-        except Exception as e:
-            self.assertNotEqual(str(e), '403 Forbidden')
-
     @defer.inlineCallbacks
     def test_poll_no_pull_requests(self):
         yield self.attachDefaultChangeSource()
@@ -396,8 +385,7 @@ def test_poll_no_pull_requests(self):
     @defer.inlineCallbacks
     def test_poll_new_pull_requests(self):
         yield self.attachDefaultChangeSource()
-        # patch client.getPage()
-        self.patch(client, "getPage", self.pr_list.getPage)
+        self.patch(self.changesource, "getPage", self.pr_list.getPage)
 
         yield self.changesource.poll()
 
@@ -423,7 +411,7 @@ def test_poll_no_updated_pull_request(self):
         yield self.attachDefaultChangeSource()
 
         # patch client.getPage()
-        self.patch(client, "getPage", self.pr_list.getPage)
+        self.patch(self.changesource, "getPage", self.pr_list.getPage)
 
         yield self.changesource.poll()
 
@@ -452,7 +440,7 @@ def test_poll_no_updated_pull_request(self):
     def test_poll_updated_pull_request(self):
         yield self.attachDefaultChangeSource()
         # patch client.getPage()
-        self.patch(client, "getPage", self.pr_list.getPage)
+        self.patch(self.changesource, "getPage", self.pr_list.getPage)
 
         yield self.changesource.poll()
 
@@ -473,7 +461,7 @@ def test_poll_updated_pull_request(self):
             'src': 'bitbucket',
             'when_timestamp': 1381869500,
         }])
-        self.patch(client, "getPage", self.pr_list2.getPage)
+        self.patch(self.changesource, "getPage", self.pr_list2.getPage)
         yield self.changesource.poll()
 
         self.assertEqual(self.master.data.updates.changesAdded, [
@@ -522,7 +510,7 @@ def test_poll_pull_request_filter_False(self):
         ))
 
         # patch client.getPage()
-        self.patch(client, "getPage", self.pr_list.getPage)
+        self.patch(self.changesource, "getPage", self.pr_list.getPage)
 
         yield self.changesource.poll()
 
@@ -537,7 +525,7 @@ def test_poll_pull_request_filter_True(self):
         ))
 
         # patch client.getPage()
-        self.patch(client, "getPage", self.pr_list.getPage)
+        self.patch(self.changesource, "getPage", self.pr_list.getPage)
 
         yield self.changesource.poll()
 
@@ -566,7 +554,7 @@ def test_poll_pull_request_not_useTimestamps(self):
             useTimestamps=False,
         ))
 
-        self.patch(client, "getPage", self.pr_list.getPage)
+        self.patch(self.changesource, "getPage", self.pr_list.getPage)
         self.reactor.advance(1396825656)
 
         yield self.changesource.poll()
@@ -595,7 +583,7 @@ def test_poll_pull_request_properties(self):
             bitbucket_property_whitelist=["bitbucket.*"],
         ))
 
-        self.patch(client, "getPage", self.pr_list.getPage)
+        self.patch(self.changesource, "getPage", self.pr_list.getPage)
 
         yield self.changesource.poll()
         self.assertEqual(self.master.data.updates.changesAdded, [{
@@ -614,15 +602,15 @@ def test_poll_pull_request_properties(self):
                 'bitbucket.description': 'description',
                 'bitbucket.id': 1,
                 'bitbucket.links.html.href': 'https://bitbucket.org/owner/slug/pull-request/1',
-                'bitbucket.links.self.href': 'https://bitbucket.org/!api/2.0/'
+                'bitbucket.links.self.href': 'https://api.bitbucket.org/2.0/'
                                              'repositories/owner/slug/pullrequests/1',
                 'bitbucket.merge_commit': None,
                 'bitbucket.source.branch.name': 'default',
                 'bitbucket.source.commit.hash': '111111111111',
-                'bitbucket.source.commit.links.self.href': 'https://bitbucket.org/!api/2.0/'
+                'bitbucket.source.commit.links.self.href': 'https://api.bitbucket.org/2.0/'
                                                            'repositories/contributor/slug/'
                                                            'commit/111111111111',
-                'bitbucket.source.repository.links.self.href': 'https://bitbucket.org/!api/2.0/'
+                'bitbucket.source.repository.links.self.href': 'https://api.bitbucket.org/2.0/'
                                                                'repositories/contributor/slug',
                 'bitbucket.state': 'OPEN',
                 'bitbucket.title': 'title',