Repository: googleapis/google-cloud-python

Issue:
v1beta3 and emulator use
The docs in this link do not make it clear on how to use the library against the local emulator, with out any creds needed. https://cloud.google.com/datastore/docs/datastore-api-tutorial

The service object can be instantiated without creds for emulator use:
c = datastore.Client(project='test', http=httplib2.Http())

The lib I assume requires the latest version of the emulator to work, it should be noted somewhere in the docs too, I did a "gcloud components install gcd-emulator" to get the latest version.

Lastly, with all the above in place, the lib does not work with the emulator, it calls a url that the emulator does not recognise and therefore complains. I had to make this adjustment to have the lib and emulator working together:

Change  https://github.com/GoogleCloudPlatform/gcloud-python/blob/master/gcloud/datastore/connection.py line 49 
from:
API_URL_TEMPLATE = ('{api_base}/{api_version}/projects'
to:
API_URL_TEMPLATE = ('{api_base}/datastore/{api_version}/projects'

With this change, the library and latest emulator worked together without creds, as expected.

I am not sure if this is a general issue with the lib, or if I am using it incorrectly to need this, or if the emulator is the issue and it should be fixed there?



Discussion:
Thanks for the heads up. I'm looking into this right now.

/cc @pcostell 

You should be able to use the existence of the `DATASTORE_EMULATOR_HOST` variable to indicate that the emulator should be used. However the emulator still is (a little) broken because of the extra `datastore` required in the path name. I'm hoping to fix this in a future emulator release, but you may want to address it in the library faster.

I can just add a hack to put it [on the base URL](https://github.com/GoogleCloudPlatform/gcloud-python/blob/8da951d4b693bf227f1b29c9ca57a09beeb4bb4b/gcloud/datastore/connection.py#L59-L60) when GCD is detected. @pcostell do you think it's worth it? How long will it be before that fix makes it into `gcd.sh` (seems like it could be a one-liner fix?).

I'm not sure we want to fix it in the existing gcd.sh tool because it is wrapped together with the `v1beta2` emulator. I'd like to fix it in the `grpc`-based emulator which I'm working on adding proto-over-http  and JSON support to now.

That won't help us until we support gRPC. I'll send the PR to modify the path and assign you to review.

Sounds good (note it will help once proto-over-http support is added to the new emulator as you'll then be able to use the new emulator in the current setup OR via gRPC).


Resolving patch:
diff --git a/gcloud/datastore/connection.py b/gcloud/datastore/connection.py
--- a/gcloud/datastore/connection.py
+++ b/gcloud/datastore/connection.py
@@ -56,8 +56,12 @@ class Connection(connection.Connection):
     def __init__(self, credentials=None, http=None, api_base_url=None):
         super(Connection, self).__init__(credentials=credentials, http=http)
         if api_base_url is None:
-            api_base_url = os.getenv(GCD_HOST,
-                                     self.__class__.API_BASE_URL)
+            try:
+                # gcd.sh has /datastore/ in the path still since it supports
+                # v1beta2 and v1beta3 simultaneously.
+                api_base_url = '%s/datastore' % (os.environ[GCD_HOST],)
+            except KeyError:
+                api_base_url = self.__class__.API_BASE_URL
         self.api_base_url = api_base_url
 
     def _request(self, project, method, data):


Test patch:
diff --git a/gcloud/datastore/test_connection.py b/gcloud/datastore/test_connection.py
--- a/gcloud/datastore/test_connection.py
+++ b/gcloud/datastore/test_connection.py
@@ -57,14 +57,14 @@ def test_custom_url_from_env(self):
         from gcloud.connection import API_BASE_URL
         from gcloud.environment_vars import GCD_HOST
 
-        HOST = object()
+        HOST = 'CURR_HOST'
         fake_environ = {GCD_HOST: HOST}
 
-        with _Monkey(os, getenv=fake_environ.get):
+        with _Monkey(os, environ=fake_environ):
             conn = self._makeOne()
 
         self.assertNotEqual(conn.api_base_url, API_BASE_URL)
-        self.assertEqual(conn.api_base_url, HOST)
+        self.assertEqual(conn.api_base_url, HOST + '/datastore')
 
     def test_custom_url_from_constructor(self):
         from gcloud.connection import API_BASE_URL
@@ -84,7 +84,7 @@ def test_custom_url_constructor_and_env(self):
         HOST2 = object()
         fake_environ = {GCD_HOST: HOST1}
 
-        with _Monkey(os, getenv=fake_environ.get):
+        with _Monkey(os, environ=fake_environ):
             conn = self._makeOne(api_base_url=HOST2)
 
         self.assertNotEqual(conn.api_base_url, API_BASE_URL)