Repository: geopy/geopy
Pull request #304 (2018-06-12 21:52:09)

Add HERE geocoder
This is a wrapper around [HERE's geocoder API](https://developer.here.com/documentation/geocoder/) (incl. reverse geocoding) which can also be used to retrieve various kinds of polygons (one test is included). Additional tests might follow later.

deeplook: Thanks for giving very detailed comments so quickly! I'll address them in-place, but in general, 10-15 days should be more than enough time! :) 
deeplook: @KostyaEsmukov Because you asked me for changed within 10-15 days: I had kind of no time to work on this, until now, but I'll give it a try until/including the weekend to come.
KostyaEsmukov: Thank you! I'm off my schedule as well: hopefully I'll deal with the rest of the 1.15 blockers by the middle of next week.
deeplook: Making some progress with updates to my PR, I wonder if it is such a good idea to keep geopy's `exactly_one` parameter *and* HERE's `maxresults` one? The former is part of the official plugin interface, I guess, but the latter is more general. I can have both, but it needs some choice of precedence and explanation.
KostyaEsmukov: > I wonder if it is such a good idea to keep geopy's `exactly_one` parameter *and* HERE's `maxresults` one?

Yep, both should be present. See this comment for rationale and precedence explanation: https://github.com/geopy/geopy/pull/281#discussion_r180079906

> In fact, for `maxresults` to make most sense I would need to have a `pageinformation` param for implementing some simple paging mechanism, too. But I can't find anything on "paging" in this package. Has really no other plugin implemented paging?

I'd suggest to not bother about pagination. Looping through result pages is very rarely (if ever) needed for geocoding results. Usually blindly selecting the first result (i.e. with `exactly_one=True` by default) is good enough.

`maxresults` is perfectly viable even without `pageinformation`. For example, to limit the default page size (what is it, ~20?) to 5, which might be reasonable for live-search suggestions.
KostyaEsmukov: Hey @deeplook , how is it going? I'd like to cut a release this weekend. Is there any chance you could push the changes soon? If it's inconvenient or stressful for you, we can simply move this feature to the next version. Thanks anyway!
deeplook: I can push what I have today, but it won't include a few more complicated tests I'd like to provide, too. From my point of view that should be still ok, and I would rather see the code released now and provide these tests, later. What do you think?
deeplook: @KostyaEsmukov I've pushed what I have. Looks like travis isn't happy yet (env. vars?), but I can't do much more now, as I need to pack for a vacation. If you can finalize this, that would be awesome...
KostyaEsmukov: Don’t worry, I’ll do that for you. Thanks and have a great vacation!

> On 14 Jul 2018, at 17:45, deeplook <notifications@github.com> wrote:
> 
> @KostyaEsmukov I've pushed what I have. Looks like travis isn't happy yet (env. vars?), but I can't do much more now, as I need to pack for a vacation. If you can finalize this, that would be awesome...


Patch:
diff --git a/README.rst b/README.rst
index 018cd4d..f507a38 100644
--- a/README.rst
+++ b/README.rst
@@ -26,7 +26,7 @@ geopy includes geocoder classes for the `OpenStreetMap Nominatim`_,
 `Bing Maps API`_, `Yandex`_, `IGN France`_, `GeoNames`_,
 `Pelias`_, `geocode.earth`_,
 `OpenMapQuest`_, `PickPoint`_, `What3Words`_, `OpenCage`_,
-`SmartyStreets`_, and `GeocodeFarm`_ geocoder services.
+`SmartyStreets`_, `GeocodeFarm`_, and `Here`_ geocoder services.
 The various geocoder classes are located in `geopy.geocoders`_.
 
 .. _OpenStreetMap Nominatim: https://wiki.openstreetmap.org/wiki/Nominatim
@@ -45,6 +45,7 @@ The various geocoder classes are located in `geopy.geocoders`_.
 .. _OpenCage: https://geocoder.opencagedata.com/
 .. _SmartyStreets: https://smartystreets.com/products/liveaddress-api
 .. _GeocodeFarm: https://www.geocodefarm.com/
+.. _Here: https://developer.here.com/documentation/geocoder/
 .. _geopy.geocoders: https://github.com/geopy/geopy/tree/master/geopy/geocoders
 
 geopy is tested against CPython (versions 2.7, 3.4, 3.5, 3.6, 3.7), PyPy, and
diff --git a/docs/index.rst b/docs/index.rst
index 595397b..958fc2b 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -112,6 +112,14 @@ GoogleV3
 
    .. automethod:: __init__
 
+HERE
+----
+
+.. autoclass:: geopy.geocoders.Here
+   :members:
+
+   .. automethod:: __init__
+
 IGNFrance
 ---------
 
diff --git a/geopy/geocoders/__init__.py b/geopy/geocoders/__init__.py
index 99fc410..02a7504 100644
--- a/geopy/geocoders/__init__.py
+++ b/geopy/geocoders/__init__.py
@@ -96,6 +96,7 @@ __all__ = (
     "GeocodeFarm",
     "GeoNames",
     "GoogleV3",
+    "Here",
     "IGNFrance",
     "OpenCage",
     "OpenMapQuest",
@@ -119,6 +120,7 @@ from geopy.geocoders.geocodeearth import GeocodeEarth
 from geopy.geocoders.geocodefarm import GeocodeFarm
 from geopy.geocoders.geonames import GeoNames
 from geopy.geocoders.googlev3 import GoogleV3
+from geopy.geocoders.here import Here
 from geopy.geocoders.ignfrance import IGNFrance
 from geopy.geocoders.opencage import OpenCage
 from geopy.geocoders.openmapquest import OpenMapQuest
@@ -146,6 +148,7 @@ SERVICE_TO_GEOCODER = {
     "geonames": GeoNames,
     "google": GoogleV3,
     "googlev3": GoogleV3,
+    "here": Here,
     "ignfrance": IGNFrance,
     "opencage": OpenCage,
     "openmapquest": OpenMapQuest,
diff --git a/geopy/geocoders/here.py b/geopy/geocoders/here.py
new file mode 100644
index 0000000..db7a7e3
--- /dev/null
+++ b/geopy/geocoders/here.py
@@ -0,0 +1,329 @@
+"""
+:class:`.Here` geocoder.
+"""
+
+from geopy.compat import urlencode
+from geopy.exc import (
+    GeocoderAuthenticationFailure,
+    GeocoderInsufficientPrivileges,
+    GeocoderQuotaExceeded,
+    GeocoderServiceError,
+    GeocoderUnavailable,
+)
+from geopy.geocoders.base import DEFAULT_SENTINEL, Geocoder
+from geopy.location import Location
+from geopy.util import join_filter, logger
+
+__all__ = ("Here", )
+
+
+class Here(Geocoder):
+    """Geocoder using the HERE Geocoder API.
+
+    Documentation at:
+        https://developer.here.com/documentation/geocoder/
+    """
+
+    structured_query_params = {
+        'city',
+        'county',
+        'district',
+        'country',
+        'state',
+        'street',
+        'housenumber',
+        'postalcode',
+    }
+
+    def __init__(
+            self,
+            app_id,
+            app_code,
+            format_string=None,
+            scheme=None,
+            timeout=DEFAULT_SENTINEL,
+            proxies=DEFAULT_SENTINEL,
+            user_agent=None,
+            ssl_context=DEFAULT_SENTINEL,
+    ):
+        """
+
+        :param str app_id: Should be a valid HERE Maps APP ID
+            See https://developer.here.com/authenticationpage.
+
+        :param str app_code: Should be a valid HERE Maps APP CODE
+            See https://developer.here.com/authenticationpage.
+
+        :param str format_string:
+            See :attr:`geopy.geocoders.options.default_format_string`.
+
+        :param str scheme:
+            See :attr:`geopy.geocoders.options.default_scheme`.
+
+        :param int timeout:
+            See :attr:`geopy.geocoders.options.default_timeout`.
+
+        :param dict proxies:
+            See :attr:`geopy.geocoders.options.default_proxies`.
+
+        :param str user_agent:
+            See :attr:`geopy.geocoders.options.default_user_agent`.
+
+        :type ssl_context: :class:`ssl.SSLContext`
+        :param ssl_context:
+            See :attr:`geopy.geocoders.options.default_ssl_context`.
+        """
+        super(Here, self).__init__(
+            format_string=format_string,
+            scheme=scheme,
+            timeout=timeout,
+            proxies=proxies,
+            user_agent=user_agent,
+            ssl_context=ssl_context,
+        )
+        self.app_id = app_id
+        self.app_code = app_code
+        self.api = "%s://geocoder.api.here.com/6.2/geocode.json" % self.scheme
+        self.reverse_api = "%s://reverse.geocoder.api.here.com/6.2/reversegeocode.json" \
+            % self.scheme
+
+    def geocode(
+            self,
+            query,
+            bbox=None,
+            mapview=None,
+            exactly_one=True,
+            maxresults=None,
+            pageinformation=None,
+            language=None,
+            additional_data=False,
+            timeout=DEFAULT_SENTINEL
+    ):
+        """
+        Return a location point by address.
+
+        This implementation supports only a subset of all available parameters.
+        A list of all parameters of the pure REST API is available here:
+        https://developer.here.com/documentation/geocoder/topics/resource-geocode.html
+
+        :param str query: The address or query you wish to geocode.
+
+            For a structured query, provide a dictionary whose keys
+            are one of: `city`, `county`, `district`, `country`, `state`,
+            `street`, `housenumber`, or `postalcode`.
+
+        :param bbox: A type of spatial filter, limits the search for any other attributes
+            in the request. Specified by two coordinate (lat/lon) pairs, top-left and
+            bottom-right, respectively.
+        :type bbox: list or tuple of :class:`geopy.point.Point`, list or tuple
+            of ``(latitude, longitude)``, or string as ``"%(latitude)s, %(longitude)s"``.
+
+        :param mapview: The app's viewport, given as two coordinate pairs, specified
+            by two lat/lon pairs, top-left and bottom-right of the bounding box,
+            respectively. Matches from within the set map view plus an extended area
+            are ranked highest. Relevant global results are also returned.
+        :type mapview: list or tuple of :class:`geopy.point.Point`, list or tuple
+            of ``(latitude, longitude)``, or string as ``"%(latitude)s, %(longitude)s"``.
+
+        :param bool exactly_one: Return one result or a list of results, if
+            available.
+
+        :param int maxresults: Defines the maximum number of items in the
+            response structure. If not provided and there are multiple results
+            the HERE API will return 10 results by default. This will be reset
+            to one if ``exactly_one`` is True.
+
+        :param int pageinformation: A key which identifies the page to be returned
+            when the response is separated into multiple pages. Only useful when
+            ``maxresults`` is also provided.
+
+        :param str language: Affects the language of the response,
+            must be a RFC 4647 language code, e.g. 'en-US'.
+
+        :param str additional_data: A string with key-value pairs as described on
+            https://developer.here.com/documentation/geocoder/topics/resource-params-additional.html.
+            These will be added as one query parameter to the URL.
+
+        :param int timeout: Time, in seconds, to wait for the geocoding service
+            to respond before raising a :class:`geopy.exc.GeocoderTimedOut`
+            exception. Set this only if you wish to override, on this call
+            only, the value set during the geocoder's initialization.
+
+        :rtype: ``None``, :class:`geopy.location.Location` or a list of them, if
+            ``exactly_one=False``.
+        """
+        if isinstance(query, dict):
+            params = {
+                key: val
+                for key, val
+                in query.items()
+                if key in self.structured_query_params
+            }
+            params['app_id'] = self.app_id
+            params['app_code'] = self.app_code
+        else:
+            params = {
+                'searchtext': self.format_string % query,
+                'app_id': self.app_id,
+                'app_code': self.app_code
+            }
+        if bbox:
+            # untested
+            top_left = self._coerce_point_to_string(mapview[0])
+            bottom_right = self._coerce_point_to_string(mapview[1])
+            params['bbox'] = '%s;%s' % (top_left, bottom_right)
+        if mapview:
+            # untested
+            top_left = self._coerce_point_to_string(mapview[0])
+            bottom_right = self._coerce_point_to_string(mapview[1])
+            params['mapview'] = '%s;%s' % (top_left, bottom_right)
+        if pageinformation:
+            params['pageinformation'] = pageinformation
+        if maxresults:
+            params['maxresults'] = maxresults
+        if exactly_one:
+            params['maxresults'] = 1
+        if language:
+            params['language'] = language
+        if additional_data:
+            params['additionaldata'] = additional_data
+
+        url = "?".join((self.api, urlencode(params)))
+        print(url)
+        logger.debug("%s.geocode: %s", self.__class__.__name__, url)
+        return self._parse_json(
+            self._call_geocoder(url, timeout=timeout),
+            exactly_one
+        )
+
+    def reverse(
+            self,
+            query,
+            radius=None,
+            exactly_one=True,
+            maxresults=None,
+            pageinformation=None,
+            language=None,
+            mode='retrieveAddresses',
+            timeout=DEFAULT_SENTINEL
+    ):
+        """
+        Return an address by location point.
+
+        This implementation supports only a subset of all available parameters.
+        A list of all parameters of the pure REST API is available here:
+        https://developer.here.com/documentation/geocoder/topics/resource-reverse-geocode.html
+
+        :param query: The coordinates for which you wish to obtain the
+            closest human-readable addresses.
+        :type query: :class:`geopy.point.Point`, list or tuple of ``(latitude,
+            longitude)``, or string as ``"%(latitude)s, %(longitude)s"``.
+
+        :param float radius: Proximity radius in meters.
+
+        :param bool exactly_one: Return one result or a list of results, if
+            available.
+
+        :param int maxresults: Defines the maximum number of items in the
+            response structure. If not provided and there are multiple results
+            the HERE API will return 10 results by default. This will be reset
+            to one if ``exactly_one`` is True.
+
+        :param int pageinformation: A key which identifies the page to be returned
+            when the response is separated into multiple pages. Only useful when
+            ``maxresults`` is also provided.
+
+        :param str language: Affects the language of the response,
+            must be a RFC 4647 language code, e.g. 'en-US'.
+
+        :param str mode: Affects the type of returned response items, must be
+            one of: 'retrieveAddresses' (default), 'retrieveAreas', 'retrieveLandmarks',
+            'retrieveAll', or 'trackPosition'. See online documentation for more
+            information.
+
+        :param int timeout: Time, in seconds, to wait for the geocoding service
+            to respond before raising a :class:`geopy.exc.GeocoderTimedOut`
+            exception. Set this only if you wish to override, on this call
+            only, the value set during the geocoder's initialization.
+
+        :rtype: ``None``, :class:`geopy.location.Location` or a list of them, if
+            ``exactly_one=False``.
+        """
+        point = self._coerce_point_to_string(query)
+        params = {
+            'app_id': self.app_id,
+            'app_code': self.app_code,
+            'mode': mode,
+            'prox': point,
+        }
+        if radius and float(radius) >= 0:
+            params['prox'] = params['prox'] + ',' + str(radius)
+        if pageinformation:
+            params['pageinformation'] = pageinformation
+        if maxresults:
+            params['maxresults'] = maxresults
+        if exactly_one:
+            params['maxresults'] = 1
+        if language:
+            params['language'] = language
+        url = "%s?%s" % (
+            self.reverse_api, urlencode(params))
+        logger.debug("%s.reverse: %s", self.__class__.__name__, url)
+        return self._parse_json(
+            self._call_geocoder(url, timeout=timeout),
+            exactly_one
+        )
+
+    @staticmethod
+    def _parse_json(doc, exactly_one=True):
+        """
+        Parse a location name, latitude, and longitude from an JSON response.
+        """
+        status_code = doc.get("statusCode", 200)
+        if status_code != 200:
+            err = doc.get("errorDetails", "")
+            if status_code == 401:
+                raise GeocoderAuthenticationFailure(err)
+            elif status_code == 403:
+                raise GeocoderInsufficientPrivileges(err)
+            elif status_code == 429:
+                raise GeocoderQuotaExceeded(err)
+            elif status_code == 503:
+                raise GeocoderUnavailable(err)
+            else:
+                raise GeocoderServiceError(err)
+
+        try:
+            resources = doc['Response']['View'][0]['Result']
+        except IndexError:
+            resources = None
+        if not resources or not len(resources):
+            return None
+
+        def parse_resource(resource):
+            """
+            Parse each return object.
+            """
+            stripchars = ", \n"
+            addr = resource['Location']['Address']
+
+            address = addr.get('Label', '').strip(stripchars)
+            city = addr.get('City', '').strip(stripchars)
+            state = addr.get('State', '').strip(stripchars)
+            zipcode = addr.get('PostalCode', '').strip(stripchars)
+            country = addr.get('Country', '').strip(stripchars)
+
+            city_state = join_filter(", ", [city, state])
+            place = join_filter(" ", [city_state, zipcode])
+            location = join_filter(", ", [address, place, country])
+
+            display_pos = resource['Location']['DisplayPosition']
+            latitude = float(display_pos['Latitude'])
+            longitude = float(display_pos['Longitude'])
+
+            return Location(location, (latitude, longitude), resource)
+
+        if exactly_one:
+            return parse_resource(resources[0])
+        else:
+            return [parse_resource(resource) for resource in resources]


Test patch:
diff --git a/test/geocoders/here.py b/test/geocoders/here.py
new file mode 100644
index 0000000..084a7ad
--- /dev/null
+++ b/test/geocoders/here.py
@@ -0,0 +1,217 @@
+import unittest
+
+from geopy.compat import u
+from geopy.geocoders import Here
+from geopy.point import Point
+from test.geocoders.util import GeocoderTestBase, env
+
+
+class HereTestCaseUnitTest(GeocoderTestBase):
+
+    def test_user_agent_custom(self):
+        geocoder = Here(
+            app_id='DUMMYID1234',
+            app_code='DUMMYCODE1234',
+            user_agent='my_user_agent/1.0'
+        )
+        self.assertEqual(geocoder.headers['User-Agent'], 'my_user_agent/1.0')
+
+
+@unittest.skipUnless(
+    bool(env.get('HERE_APP_ID')),
+    "No HERE_APP_ID env variable set"
+)
+@unittest.skipUnless(
+    bool(env.get('HERE_APP_CODE')),
+    "No HERE_APP_CODE env variable set"
+)
+class HereTestCase(GeocoderTestBase):
+
+    @classmethod
+    def setUpClass(cls):
+        cls.geocoder = Here(
+            format_string='%s',
+            scheme='http',
+            app_id=env['HERE_APP_ID'],
+            app_code=env['HERE_APP_CODE'],
+            timeout=10
+        )
+
+    def test_geocode_empty_result(self):
+        """
+        Here.geocode empty results should be graciously handled.
+        """
+        self.geocode_run(
+            {"query": "xqj37"},
+            {},
+            expect_failure=True
+        )
+
+    def test_geocode(self):
+        """
+        Here.geocode
+        """
+        self.geocode_run(
+            {"query": "435 north michigan ave, chicago il 60611 usa"},
+            {"latitude": 41.890, "longitude": -87.624}
+        )
+
+    def test_geocode_structured(self):
+        """
+        Here.geocode
+        """
+        query = {
+            "street": "north michigan ave",
+            "housenumber": "435",
+            "city": "chicago",
+            "state": "il",
+            "postalcode": 60611,
+            "country": "usa"
+        }
+        self.geocode_run(
+            {"query": query},
+            {"latitude": 41.890, "longitude": -87.624}
+        )
+
+    def test_geocode_unicode_name(self):
+        """
+        Here.geocode unicode in Japanese for Paris. (POIs not included.)
+        """
+        self.geocode_run(
+            {"query": u("\u30d1\u30ea")},
+            {"latitude": 48.85718, "longitude": 2.34141}
+        )
+
+    def test_geocode_shapes(self):
+        """
+        Here.geocode using additional data parameter (postal code shapes)
+        """
+        address_string = "435 north michigan ave, chicago il 60611 usa"
+        res = self.geocode_run(
+            {"query": address_string, "additional_data": "IncludeShapeLevel,postalCode"},
+            {"latitude": 41.89035, "longitude": -87.62333},
+        )
+        shape_value = res.raw['Location']['Shape']['Value']
+        self.assertTrue(shape_value.startswith('MULTIPOLYGON ((('))
+
+    def test_geocode_with_language_de(self):
+        """
+        Here.geocode using language parameter to get a non-English response
+        """
+        address_string = "435 north michigan ave, chicago il 60611 usa"
+        res = self.geocode_run(
+            {"query": address_string, "language": "de-DE"},
+            {}
+        )
+        self.assertIn("Vereinigte Staaten", res.address)
+
+    def test_geocode_with_language_en(self):
+        """
+        Here.geocode using language parameter to get an English response
+        """
+        address_string = "435 north michigan ave, chicago il 60611 usa"
+        res = self.geocode_run(
+            {"query": address_string, "language": "en-US"},
+            {}
+        )
+        self.assertIn("United States", res.address)
+
+    def test_geocode_with_paging(self):
+        """
+        Here.geocode using simple paging "ouside" geopy
+        """
+        address_string = "Hauptstr., Berlin, Germany"
+        input = {"query": address_string, "maxresults": 12, "exactly_one": False}
+        res = self.geocode_run(input, {})
+        self.assertEqual(len(res), 12)
+
+        input["pageinformation"] = 2
+        res = self.geocode_run(input, {})
+        self.assertEqual(len(res), 3)
+
+        input["pageinformation"] = 3
+        res = self.geocode_run(input, {}, expect_failure=True)
+
+    def test_reverse_string(self):
+        """
+        Here.reverse string
+        """
+        self.reverse_run(
+            {"query": "40.753898, -73.985071"},
+            {"latitude": 40.753898, "longitude": -73.985071}
+        )
+
+    def test_reverse_point(self):
+        """
+        Here.reverse Point
+        """
+        self.reverse_run(
+            {"query": Point(40.753898, -73.985071)},
+            {"latitude": 40.753898, "longitude": -73.985071}
+        )
+
+    def test_reverse_point_radius_1000(self):
+        """
+        Here.reverse Point with radius
+        """
+        # needs more testing
+        res = self.reverse_run(
+            {"query": Point(40.753898, -73.985071), "radius": 1000, "exactly_one": False},
+            {"latitude": 40.753898, "longitude": -73.985071}
+        )
+        self.assertEqual(len(res), 10)
+
+    def test_reverse_point_radius_10(self):
+        """
+        Here.reverse Point with radius
+        """
+        # needs more testing
+        res = self.reverse_run(
+            {"query": Point(40.753898, -73.985071), "radius": 10, "exactly_one": False},
+            {"latitude": 40.753898, "longitude": -73.985071}
+        )
+        self.assertEqual(len(res), 10)
+
+    def test_reverse_with_language_de(self):
+        """
+        Here.reverse using point and language parameter to get a non-English response
+        """
+        res = self.reverse_run(
+            {"query": Point(40.753898, -73.985071), "language": "de-DE"},
+            {}
+        )
+        self.assertIn("Vereinigte Staaten", res.address)
+
+    def test_reverse_with_language_en(self):
+        """
+        Here.reverse using point and language parameter to get an English response
+        """
+        res = self.reverse_run(
+            {"query": Point(40.753898, -73.985071), "language": "en-US"},
+            {}
+        )
+        self.assertIn("United States", res.address)
+
+    def test_reverse_with_mode_areas(self):
+        """
+        Here.reverse using mode parameter 'retrieveAreas'.
+        """
+        res = self.reverse_run(
+            {"query": Point(40.753898, -73.985071), "mode": "retrieveAreas"},
+            {}
+        )
+        self.assertIn("Theater District-Times Square", res.address)
+
+    def test_reverse_with_maxresults_5(self):
+        """
+        Here.reverse using maxresults parameter 5.
+        """
+        res = self.reverse_run(
+            {
+                "query": Point(40.753898, -73.985071),
+                "maxresults": 5,
+                "exactly_one": False
+            },
+            {}
+        )
+        self.assertEqual(len(res), 5)
diff --git a/test/geocoders/util.py b/test/geocoders/util.py
index f403b9d..bd789c2 100644
--- a/test/geocoders/util.py
+++ b/test/geocoders/util.py
@@ -22,6 +22,8 @@ except IOError:
         'LIVESTREETS_AUTH_TOKEN',
         'GEOCODEEARTH_KEY',
         'GEOCODEFARM_KEY',
+        'HERE_APP_ID',
+        'HERE_APP_CODE',
         'GEOCODEFARM_SKIP',
         'BAIDU_KEY',
         'BAIDU_KEY_REQUIRES_SK',