Repository: vaexio/vaex

Issue:
[BUG-REPORT] (Major) vaex hdf5 incompatible with number 1.22.4
Thank you for reaching out and helping us improve Vaex!

Before you submit a new Issue, please read through the [documentation](https://docs.vaex.io/en/latest/). Also, make sure you search through the Open and Closed Issues - your problem may already be discussed or addressed.

**Description**
Vaex cannot export hdf5 files with numpy 1.22.4 arrays

**Software information**
 - Vaex version (`import vaex; vaex.__version__)`: `{'vaex-core': '4.9.1', 'vaex-hdf5': '0.12.1'}`
 - Vaex was installed via: pip / conda-forge / from source  pip
 - OS: Mac/linux

**Additional information**
The following works with numpy 1.22.3 but fails with numpy 1.22.4
```
import vaex

print(vaex.__version__)  # {'vaex-core': '4.9.1', 'vaex-hdf5': '0.12.1'}

df =vaex.example()
df.export("file.hdf5")
```


```

In [3]: df.export("file.hdf5")
---------------------------------------------------------------------------
BufferError                               Traceback (most recent call last)
<ipython-input-3-c4ce35d9bf95> in <module>
----> 1 df.export("file.hdf5")

~/.pyenv/versions/3.8.11/lib/python3.8/site-packages/vaex/dataframe.py in export(self, path, progress, chunk_size, parallel, fs_options, fs)
   6694             self.export_feather(path, parallel=parallel, fs_options=fs_options)
   6695         elif naked_path.endswith('.hdf5'):
-> 6696             self.export_hdf5(path, progress=progress, parallel=parallel)
   6697         elif naked_path.endswith('.fits'):
   6698             self.export_fits(path, progress=progress)

~/.pyenv/versions/3.8.11/lib/python3.8/site-packages/vaex/dataframe.py in export_hdf5(self, path, byteorder, progress, chunk_size, parallel, column_count, writer_threads, group, mode)
   6910             with Writer(path=path, group=group, mode=mode, byteorder=byteorder) as writer:
   6911                 writer.layout(self, progress=progressbar_layout)
-> 6912                 writer.write(
   6913                     self,
   6914                     chunk_size=chunk_size,

~/.pyenv/versions/3.8.11/lib/python3.8/site-packages/vaex/hdf5/writer.py in __exit__(self, *args)
     40
     41     def __exit__(self, *args):
---> 42         self.close()
     43
     44     def layout(self, df, progress=None):

~/.pyenv/versions/3.8.11/lib/python3.8/site-packages/vaex/hdf5/writer.py in close(self)
     32     def close(self):
     33         if self.mmap is not None:
---> 34             self.mmap.close()
     35             self.file.close()
     36         self.h5.close()

BufferError: cannot close exported pointers exist
```


Discussion:
I also get this error on the same vaex and numpy 1.22.4 setup, but it works on numpy 1.22.3!
Same issue as well. Seems to be related to https://github.com/numpy/numpy/pull/21446.

Resolving patch:
diff --git a/packages/vaex-hdf5/vaex/hdf5/writer.py b/packages/vaex-hdf5/vaex/hdf5/writer.py
--- a/packages/vaex-hdf5/vaex/hdf5/writer.py
+++ b/packages/vaex-hdf5/vaex/hdf5/writer.py
@@ -30,6 +30,8 @@ def __init__(self, path, group="/table", mode="w", byteorder="="):
         self._layout_called = False
 
     def close(self):
+        # make sure we don't have references to the numpy arrays any more
+        self.column_writers = {}
         if self.mmap is not None:
             self.mmap.close()
             self.file.close()


Test patch: