Repository: pandas-dev/pandas

Issue:
HDFStore - raising an exception when complevel > 0 and complib is None
#### Code Sample, a copy-pastable example if possible

```python
HDFStore = pd.HDFStore('mydata.ph5', mode='w', complevel=9)
```
#### Problem description
Hi,

It seems when `complib is None`, `pd.HDFStore` does not use any compression even though a user supplies `complevel > 0`. If there is no good default option for `complib`, then in my opinion, `HDFStore` should raise an exception when `(complib is None) and (complevel > 0)` instead. This is because clearly the user wants compression but it cannot be done - I think explicitly raising an exception is better than silently ignoring the option.



#### Output of ``pd.show_versions()``

<details>
pd.show_versions()

INSTALLED VERSIONS
------------------
commit: None
python: 3.5.2.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 62 Stepping 4, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None
LOCALE: None.None

pandas: 0.19.2
nose: 1.3.7
pip: 9.0.1
setuptools: 27.2.0
Cython: 0.24.1
numpy: 1.11.3
scipy: 0.18.1
statsmodels: 0.6.1
xarray: 0.9.1
IPython: 5.1.0
sphinx: 1.4.6
patsy: 0.4.1
dateutil: 2.5.3
pytz: 2016.6.1
blosc: None
bottleneck: 1.1.0
tables: 3.2.2
numexpr: 2.6.1
matplotlib: 1.5.3
openpyxl: 2.3.2
xlrd: 1.0.0
xlwt: 1.1.2
xlsxwriter: 0.9.3
lxml: 3.6.4
bs4: 4.5.1
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: 1.0.13
pymysql: None
psycopg2: None
jinja2: 2.8
boto: 2.42.0
pandas_datareader: None

</details>



Discussion:
xref to #14478 
for sure would take bug fixes around this (and related issues)!. please make sure to have tests!
I see - thanks for the pointers!
If no-one is working on this, I can give it a try?

I looked at the `__init__` in `class HDFStore` and #14478. Can the default in `default_complib` in pytables `tables.filters` module be used as a default value here too?  If this is the way to go, it is perhaps best to fix #14478 first (avoiding the possible issue of using a default that is not otherwise an option)?

Maybe an exception or some pandas defined default value is the way to go instead?

Where would the tests for this belong?
this is orthogonal to #14478 though easy enough to fix both. tests are in ``pandas/tests/io/test_pytables.py``



Resolving patch:
diff --git a/pandas/core/generic.py b/pandas/core/generic.py
--- a/pandas/core/generic.py
+++ b/pandas/core/generic.py
@@ -1278,10 +1278,10 @@ def to_hdf(self, path_or_buf, key, **kwargs):
             <http://pandas.pydata.org/pandas-docs/stable/io.html#query-via-data-columns>`__.
 
             Applicable only to format='table'.
-        complevel : int, 0-9, default 0
+        complevel : int, 0-9, default None
             Specifies a compression level for data.
             A value of 0 disables compression.
-        complib : {'zlib', 'lzo', 'bzip2', 'blosc', None}, default None
+        complib : {'zlib', 'lzo', 'bzip2', 'blosc'}, default 'zlib'
             Specifies the compression library to be used.
             As of v0.20.2 these additional compressors for Blosc are supported
             (default if no compressor specified: 'blosc:blosclz'):
diff --git a/pandas/io/pytables.py b/pandas/io/pytables.py
--- a/pandas/io/pytables.py
+++ b/pandas/io/pytables.py
@@ -411,10 +411,10 @@ class HDFStore(StringMixin):
             and if the file does not exist it is created.
         ``'r+'``
             It is similar to ``'a'``, but the file must already exist.
-    complevel : int, 0-9, default 0
+    complevel : int, 0-9, default None
             Specifies a compression level for data.
             A value of 0 disables compression.
-    complib : {'zlib', 'lzo', 'bzip2', 'blosc', None}, default None
+    complib : {'zlib', 'lzo', 'bzip2', 'blosc'}, default 'zlib'
             Specifies the compression library to be used.
             As of v0.20.2 these additional compressors for Blosc are supported
             (default if no compressor specified: 'blosc:blosclz'):
@@ -449,12 +449,15 @@ def __init__(self, path, mode=None, complevel=None, complib=None,
                 "complib only supports {libs} compression.".format(
                     libs=tables.filters.all_complibs))
 
+        if complib is None and complevel is not None:
+            complib = tables.filters.default_complib
+
         self._path = _stringify_path(path)
         if mode is None:
             mode = 'a'
         self._mode = mode
         self._handle = None
-        self._complevel = complevel
+        self._complevel = complevel if complevel else 0
         self._complib = complib
         self._fletcher32 = fletcher32
         self._filters = None
@@ -566,11 +569,8 @@ def open(self, mode='a', **kwargs):
         if self.is_open:
             self.close()
 
-        if self._complib is not None:
-            if self._complevel is None:
-                self._complevel = 9
-            self._filters = _tables().Filters(self._complevel,
-                                              self._complib,
+        if self._complevel and self._complevel > 0:
+            self._filters = _tables().Filters(self._complevel, self._complib,
                                               fletcher32=self._fletcher32)
 
         try:


Test patch:
diff --git a/pandas/tests/io/test_pytables.py b/pandas/tests/io/test_pytables.py
--- a/pandas/tests/io/test_pytables.py
+++ b/pandas/tests/io/test_pytables.py
@@ -736,6 +736,59 @@ def test_put_compression_blosc(self):
             store.put('c', df, format='table', complib='blosc')
             tm.assert_frame_equal(store['c'], df)
 
+    def test_complibs_default_settings(self):
+        # GH15943
+        df = tm.makeDataFrame()
+
+        # Set complevel and check if complib is automatically set to
+        # default value
+        with ensure_clean_path(self.path) as tmpfile:
+            df.to_hdf(tmpfile, 'df', complevel=9)
+            result = pd.read_hdf(tmpfile, 'df')
+            tm.assert_frame_equal(result, df)
+
+            with tables.open_file(tmpfile, mode='r') as h5file:
+                for node in h5file.walk_nodes(where='/df', classname='Leaf'):
+                    assert node.filters.complevel == 9
+                    assert node.filters.complib == 'zlib'
+
+        # Set complib and check to see if compression is disabled
+        with ensure_clean_path(self.path) as tmpfile:
+            df.to_hdf(tmpfile, 'df', complib='zlib')
+            result = pd.read_hdf(tmpfile, 'df')
+            tm.assert_frame_equal(result, df)
+
+            with tables.open_file(tmpfile, mode='r') as h5file:
+                for node in h5file.walk_nodes(where='/df', classname='Leaf'):
+                    assert node.filters.complevel == 0
+                    assert node.filters.complib is None
+
+        # Check if not setting complib or complevel results in no compression
+        with ensure_clean_path(self.path) as tmpfile:
+            df.to_hdf(tmpfile, 'df')
+            result = pd.read_hdf(tmpfile, 'df')
+            tm.assert_frame_equal(result, df)
+
+            with tables.open_file(tmpfile, mode='r') as h5file:
+                for node in h5file.walk_nodes(where='/df', classname='Leaf'):
+                    assert node.filters.complevel == 0
+                    assert node.filters.complib is None
+
+        # Check if file-defaults can be overridden on a per table basis
+        with ensure_clean_path(self.path) as tmpfile:
+            store = pd.HDFStore(tmpfile)
+            store.append('dfc', df, complevel=9, complib='blosc')
+            store.append('df', df)
+            store.close()
+
+            with tables.open_file(tmpfile, mode='r') as h5file:
+                for node in h5file.walk_nodes(where='/df', classname='Leaf'):
+                    assert node.filters.complevel == 0
+                    assert node.filters.complib is None
+                for node in h5file.walk_nodes(where='/dfc', classname='Leaf'):
+                    assert node.filters.complevel == 9
+                    assert node.filters.complib == 'blosc'
+
     def test_complibs(self):
         # GH14478
         df = tm.makeDataFrame()