Repository: modin-project/modin

Issue:
read_hdf fails reading from HDF5 file with more than one stored object.
### System information
- **OS Platform and Distribution**: Linux Lubuntu 19.04
- **Modin installed from (source or binary)**: binary
- **Modin version**: 0.5.0
- **Python version**: 3.7.3
- **Exact command to reproduce**:
```
import modin.pandas as pd
import numpy as np

hdf_file = '/tmp/test_read_hdf.hdf5'
with pd.HDFStore(hdf_file, mode='w') as store:
  store.append("data/df1", pd.DataFrame(np.random.randn(5, 5)))
  store.append("data/df2", pd.DataFrame(np.random.randn(4, 4)))

df = pd.read_hdf(hdf_file, key="data/df1", mode='r')
```

### Describe the problem
read_hdf fails with 
> ValueError: key must be provided when HDF5 file contains multiple datasets.

read_hdf succeeds with regular pandas, or if only one object is appended to the store.

### Source code / logs
2019-05-24 11:25:49,602 WARNING worker.py:1337 -- WARNING: Not updating worker name since `setproctitle` is not installed. Install this with `pip install setproctitle` (or ray[debug]) to enable monitoring of worker processes.
2019-05-24 11:25:49,602 INFO node.py:469 -- Process STDOUT and STDERR is being redirected to /tmp/ray/session_2019-05-24_11-25-49_401/logs.
2019-05-24 11:25:49,707 INFO services.py:407 -- Waiting for redis server at 127.0.0.1:26176 to respond...
2019-05-24 11:25:49,820 INFO services.py:407 -- Waiting for redis server at 127.0.0.1:61604 to respond...
2019-05-24 11:25:49,822 INFO services.py:804 -- Starting Redis shard with 10.0 GB max memory.
2019-05-24 11:25:49,834 INFO node.py:483 -- Process STDOUT and STDERR is being redirected to /tmp/ray/session_2019-05-24_11-25-49_401/logs.
2019-05-24 11:25:49,835 WARNING services.py:1304 -- WARNING: The object store is using /tmp instead of /dev/shm because /dev/shm has only 33006481408 bytes available. This may slow down performance! You may be able to free up space by deleting files in /dev/shm or terminating any running plasma_store_server processes. If you are inside a Docker container, you may need to pass an argument with the flag '--shm-size' to 'docker run'.
2019-05-24 11:25:49,835 INFO services.py:1427 -- Starting the Plasma object store with 40.0 GB memory using /tmp.
UserWarning: Distributing <class 'list'> object. This may take some time.
UserWarning: `open` defaulting to pandas implementation.
To request implementation, send an email to feature_requests@modin.org.
UserWarning: `append` defaulting to pandas implementation.
UserWarning: `get_node` defaulting to pandas implementation.
UserWarning: `close` defaulting to pandas implementation.
Traceback (most recent call last):
  File "./test_modin.py", line 26, in <module>
    df_read = pandas.read_hdf(hdf_file, "data/df1", mode='r')
  File "/home/ted/miniconda/envs/python3/lib/python3.7/site-packages/modin/pandas/io.py", line 238, in read_hdf
    return DataFrame(query_compiler=BaseFactory.read_hdf(**kwargs))
  File "/home/ted/miniconda/envs/python3/lib/python3.7/site-packages/modin/data_management/factories.py", line 107, in read_hdf
    return cls._determine_engine()._read_hdf(**kwargs)
  File "/home/ted/miniconda/envs/python3/lib/python3.7/site-packages/modin/data_management/factories.py", line 111, in _read_hdf
    return cls.io_cls.read_hdf(**kwargs)
  File "/home/ted/miniconda/envs/python3/lib/python3.7/site-packages/modin/engines/ray/generic/io.py", line 590, in read_hdf
    empty_pd_df = pandas.read_hdf(path_or_buf, start=0, stop=0)
  File "/home/ted/miniconda/envs/python3/lib/python3.7/site-packages/pandas/io/pytables.py", line 386, in read_hdf
    raise ValueError('key must be provided when HDF5 file '
ValueError: key must be provided when HDF5 file contains multiple datasets.



Discussion:
Hi @TedBrookings, thanks for the report!

This erroneous error is occurring during the error checking, and it should be a quick fix. 

[This line](https://github.com/modin-project/modin/blob/master/modin/engines/ray/generic/io.py#L592) needs to be updated with the rest of the `kwargs` (outside of `start` and `stop`). Thanks for the report, great find!

Resolving patch:
diff --git a/modin/engines/ray/generic/io.py b/modin/engines/ray/generic/io.py
--- a/modin/engines/ray/generic/io.py
+++ b/modin/engines/ray/generic/io.py
@@ -589,7 +589,11 @@ def read_hdf(cls, path_or_buf, **kwargs):
 
         columns = kwargs.get("columns", None)
         if not columns:
-            empty_pd_df = pandas.read_hdf(path_or_buf, start=0, stop=0)
+            start = kwargs.pop("start", None)
+            stop = kwargs.pop("stop", None)
+            empty_pd_df = pandas.read_hdf(path_or_buf, start=0, stop=0, **kwargs)
+            kwargs["start"] = start
+            kwargs["stop"] = stop
             columns = empty_pd_df.columns
 
         num_partitions = cls.frame_mgr_cls._compute_num_partitions()


Test patch:
diff --git a/modin/pandas/test/test_io.py b/modin/pandas/test/test_io.py
--- a/modin/pandas/test/test_io.py
+++ b/modin/pandas/test/test_io.py
@@ -1086,6 +1086,15 @@ def test_HDFStore():
 
     assert isinstance(modin_store, pd.HDFStore)
 
+    hdf_file = "/tmp/test_read_hdf.hdf5"
+    with pd.HDFStore(hdf_file, mode="w") as store:
+        store.append("data/df1", pd.DataFrame(np.random.randn(5, 5)))
+        store.append("data/df2", pd.DataFrame(np.random.randn(4, 4)))
+
+    modin_df = pd.read_hdf(hdf_file, key="data/df1", mode="r")
+    pandas_df = pandas.read_hdf(hdf_file, key="data/df1", mode="r")
+    df_equals(modin_df, pandas_df)
+
 
 def test_ExcelFile():
     setup_excel_file(SMALL_ROW_SIZE)