Repository: opendatacube/datacube-core

Issue:
No index driver found for 'default'
Greetings!

Using the latest Datacube version and libraries, indexing and ingestion runs well except when I try to access the database on Jupyter I get the "No index driver found for 'default': 0 available: " error.

> RuntimeError                              Traceback (most recent call last)
> ipython-input-2-f074b535470c> in <module>()
>       1 
>       2 import datacube
> ----> 3 dc = datacube.Datacube(config='E:\Datacubes\datacube-core-develop\datacube.conf')
>       4 dc
> 
> E:\Datacubes\datacube-core-develop\datacube\api\core.py in __init__(self, index, config, app, env, validate_connection)
>     116             index = index_connect(normalise_config(config),
>     117                                   application_name=app,
> --> 118                                   validate_connection=validate_connection)
>     119 
>     120         self.index = index
> 
> E:\Datacubes\datacube-core-develop\datacube\index\_api.py in index_connect(local_config, application_name, validate_connection)
>      37         raise RuntimeError(
>      38             "No index driver found for %r. %s available: %s" % (
> ---> 39                 driver_name, len(index_drivers()), ', '.join(index_drivers())
>      40             )
>      41         )
> 
> RuntimeError: No index driver found for 'default'. 0 available: 

I've reinstalled PostgreSQL and all libraries, to no avail.
All suggestions are welcome.

**edit:** While ingestion I get this warning

>  datacube.drivers.driver_cache WARNING Failed to resolve driver datacube.plugins.io.write::s3aio
>  datacube.drivers.driver_cache WARNING Failed to resolve driver datacube.plugins.io.write::s3aio_test

where do I update the driver from? Is it because of this plugin?


Discussion:
Hi Amruth,

It's now necessary to install the datacube code, instead of just using it from the source directory. Installing registers the drivers used for accessing the database and reading data. Do this by running from within the `datacube-core` directory:
```bash
    pip install -e .
```
The `-e` gives you an editable install, which means you can continue to update the code from git to receive the latest updates.

I'll leave this ticket open until someone updates the error message to be a bit more helpful.
I think that default index driver should be available even if entry points are messed up or missing. This is the behaviour in the reader plugin.
I have followed the instructions as per the official and latest documentation http://datacube-core.readthedocs.io/en/latest/ops/windows.html during all my previous runs and have never come across this issue. 
@omad  I've obtained the .zip archive of the latest release and installed via the python setup file ` python setup.py install` as per the documentation. Although that also means any update on the Git wont be reflected on my local system.

I updated Conda and all its libraries and I believe something there has corrupted my install/drivers.

Testing on another machine to verify this.
@omad  I tried the command `pip install -e {path}` as you suggested, but to no avail. Attaching screenshots,
![git](https://user-images.githubusercontent.com/17108758/37244331-69c302c4-24ad-11e8-8780-6ad3dbfdce24.png)

Reinstalled DC, all the libraries, PostgresSQL also just to verify. Also `datacube system check` returns that the  index drivers available are "default".
![git2](https://user-images.githubusercontent.com/17108758/37244401-8d57831c-24ae-11e8-87d8-97e0454b6b06.PNG)

Accessing on Jupyter throws me this error.
![git](https://user-images.githubusercontent.com/17108758/37244407-a383c010-24ae-11e8-9bcf-6513810e7383.PNG)

Why do I feel like I'm missing something simple here, Please update with any suggestions and/or correct any mistakes I've made.

**Edit: Closed by accident.**
Hi Amruth, the error you see is a warning and will not prevent you from using the system. This particular driver is not available on Windows, we will update installation script to exclude it when installing on windows, in the mean time if the warning bothers you, you can edit setup.py and remove 

https://github.com/opendatacube/datacube-core/blob/develop/setup.py#L113-L114

You will then have to run pip install again for changes to take effect 
**Update**
Installation as described in the documentation would yield the warning on Jupyter Notebook while using the `load` function and I wasn't able to go ahead in my code from here. (Maybe as @Kirill888 said, it should not prevent me from using the system, so maybe some changes in my script would do the trick).

**FIX**
As suggested by @omad  using the command `pip install -e {path_of _archive}` after installation from source directory and consequently indexing and ingesting data, I was able to view on Jupyter Notebook, albeit with a harmless warning message ( as pointed out by @Kirill888 ).

Sometimes a full reinstall of all libraries and virtual environment came in handy to debug most errors. I'm sure beginners to this field such as myself would be benefited, if this be added to the documentation.

Thanks to @omad  and @Kirill888  for your valuable suggestions.


Resolving patch:
diff --git a/datacube/drivers/driver_cache.py b/datacube/drivers/driver_cache.py
--- a/datacube/drivers/driver_cache.py
+++ b/datacube/drivers/driver_cache.py
@@ -2,7 +2,7 @@
 
 import logging
 
-from pkg_resources import iter_entry_points
+from pkg_resources import iter_entry_points, DistributionNotFound
 
 _LOG = logging.getLogger(__name__)
 
@@ -27,7 +27,11 @@ def load_drivers(group):
     def safe_load(ep):
         # pylint: disable=bare-except
         try:
-            driver_init = ep.resolve()
+            driver_init = ep.load()
+        except DistributionNotFound:
+            # This happens when entry points were marked with extra features,
+            # but extra feature were not requested for installation
+            return None
         except:
             _LOG.warning('Failed to resolve driver %s::%s', group, ep.name)
             return None
diff --git a/datacube/drivers/indexes.py b/datacube/drivers/indexes.py
--- a/datacube/drivers/indexes.py
+++ b/datacube/drivers/indexes.py
@@ -7,6 +7,10 @@ class IndexDriverCache(object):
     def __init__(self, group):
         self._drivers = load_drivers(group)
 
+        if len(self._drivers) == 0:
+            from datacube.index.index import index_driver_init
+            self._drivers = dict(default=index_driver_init())
+
         for driver in list(self._drivers.values()):
             if hasattr(driver, 'aliases'):
                 for alias in driver.aliases:
@@ -17,8 +21,7 @@ def __call__(self, name):
         :returns: None if driver with a given name is not found
 
         :param str name: Driver name
-        :param str fmt: Dataset format
-        :return: Returns WriterDriver
+        :return: Returns IndexDriver
         """
         return self._drivers.get(name, None)
 
@@ -29,7 +32,7 @@ def drivers(self):
 
 
 def index_cache():
-    """ Singleton for WriterDriverCache
+    """ Singleton for IndexDriverCache
     """
     # pylint: disable=protected-access
     if not hasattr(index_cache, '_instance'):
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -2,6 +2,7 @@
 
 import versioneer
 from setuptools import setup, find_packages
+import os
 
 tests_require = [
     'compliance-checker',
@@ -29,6 +30,22 @@
 # An 'all' option, following ipython naming conventions.
 extras_require['all'] = sorted(set(sum(extras_require.values(), [])))
 
+extra_plugins = dict(read=[], write=[], index=[])
+
+if os.name != 'nt':
+    extra_plugins['read'].extend([
+        's3aio = datacube.drivers.s3.driver:reader_driver_init [s3]',
+        's3aio_test = datacube.drivers.s3.driver:reader_test_driver_init [s3]',
+    ])
+    extra_plugins['write'].extend([
+        's3aio = datacube.drivers.s3.driver:writer_driver_init [s3]',
+        's3aio_test = datacube.drivers.s3.driver:writer_test_driver_init [s3]',
+    ])
+
+    extra_plugins['index'].extend([
+        's3aio_index = datacube.drivers.s3aio_index:index_driver_init [s3]',
+    ])
+
 setup(
     name='datacube',
     version=versioneer.get_version(),
@@ -110,17 +127,15 @@
         ],
         'datacube.plugins.io.read': [
             'netcdf = datacube.drivers.netcdf.driver:reader_driver_init',
-            's3aio = datacube.drivers.s3.driver:reader_driver_init',
-            's3aio_test = datacube.drivers.s3.driver:reader_test_driver_init'
+            *extra_plugins['read'],
         ],
         'datacube.plugins.io.write': [
             'netcdf = datacube.drivers.netcdf.driver:writer_driver_init',
-            's3aio = datacube.drivers.s3.driver:writer_driver_init',
-            's3aio_test = datacube.drivers.s3.driver:writer_test_driver_init',
+            *extra_plugins['write'],
         ],
         'datacube.plugins.index': [
             'default = datacube.index.index:index_driver_init',
-            's3aio_index = datacube.drivers.s3aio_index:index_driver_init',
+            *extra_plugins['index'],
         ],
     },
 )


Test patch:
diff --git a/tests/test_driver.py b/tests/test_driver.py
--- a/tests/test_driver.py
+++ b/tests/test_driver.py
@@ -4,6 +4,7 @@
 from collections import namedtuple
 
 from datacube.drivers import new_datasource, reader_drivers, writer_drivers, index_drivers
+from datacube.drivers.indexes import IndexDriverCache
 from datacube.storage.storage import RasterDatasetDataSource
 from .util import mk_sample_dataset
 
@@ -62,6 +63,11 @@ def test_index_drivers():
     assert 's3aio_index' in available_drivers
 
 
+def test_default_injection():
+    cache = IndexDriverCache('datacube.plugins.index-no-such-prefix')
+    assert cache.drivers() == ['default']
+
+
 def test_netcdf_driver_import():
     try:
         import datacube.drivers.netcdf.driver