Repository: vaexio/vaex

Issue:
Window OS bug, can't import fcntl

Hi,

**OS: Windows 10.**   

I have encountered a bug.  When I open a hdf file, using vaex.open('myfile.hdf'), I get the error message below.  When I comment out the line 'import fcntl' in dataset_mmap.py , the issue goes away with no noticeable effect.   The fcntl module does not work on Windows, see https://stackoverflow.com/questions/1422368/fcntl-substitute-on-windows.




ERROR:MainThread:vaex:error opening 'C:\\myfile.hdf'
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-140-b984d6ab4edc> in <module>()
----> 1 dataVX = vx.open(VXfiles_df.iloc[num]['path'])

~\AppData\Local\Continuum\anaconda3\lib\site-packages\vaex\__init__.py in open(path, convert, shuffle, copy_index, *args, **kwargs)
    157             return vaex.distributed.open(path, *args, **kwargs)
    158         else:
--> 159             import vaex.file
    160             import glob
    161             # sort to get predicatable behaviour (useful for testing)

~\AppData\Local\Continuum\anaconda3\lib\site-packages\vaex\file\__init__.py in <module>()
      3 logger = logging.getLogger("vaex.file")
      4 
----> 5 import vaex.file.other
      6 try:
      7         import vaex.hdf5 as hdf5

~\AppData\Local\Continuum\anaconda3\lib\site-packages\vaex\file\other.py in <module>()
     24 from vaex.expression import Expression
     25 
---> 26 from vaex.dataset_mmap import DatasetMemoryMapped
     27 
     28 

~\AppData\Local\Continuum\anaconda3\lib\site-packages\vaex\dataset_mmap.py in <module>()
     21 from vaex.expression import Expression
     22 import struct
---> 23 import fcntl
     24 
     25 logger = logging.getLogger("vaex.file")

ModuleNotFoundError: No module named 'fcntl'

Window OS bug, can't import fcntl

Hi,

**OS: Windows 10.**   

I have encountered a bug.  When I open a hdf file, using vaex.open('myfile.hdf'), I get the error message below.  When I comment out the line 'import fcntl' in dataset_mmap.py , the issue goes away with no noticeable effect.   The fcntl module does not work on Windows, see https://stackoverflow.com/questions/1422368/fcntl-substitute-on-windows.




ERROR:MainThread:vaex:error opening 'C:\\myfile.hdf'
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-140-b984d6ab4edc> in <module>()
----> 1 dataVX = vx.open(VXfiles_df.iloc[num]['path'])

~\AppData\Local\Continuum\anaconda3\lib\site-packages\vaex\__init__.py in open(path, convert, shuffle, copy_index, *args, **kwargs)
    157             return vaex.distributed.open(path, *args, **kwargs)
    158         else:
--> 159             import vaex.file
    160             import glob
    161             # sort to get predicatable behaviour (useful for testing)

~\AppData\Local\Continuum\anaconda3\lib\site-packages\vaex\file\__init__.py in <module>()
      3 logger = logging.getLogger("vaex.file")
      4 
----> 5 import vaex.file.other
      6 try:
      7         import vaex.hdf5 as hdf5

~\AppData\Local\Continuum\anaconda3\lib\site-packages\vaex\file\other.py in <module>()
     24 from vaex.expression import Expression
     25 
---> 26 from vaex.dataset_mmap import DatasetMemoryMapped
     27 
     28 

~\AppData\Local\Continuum\anaconda3\lib\site-packages\vaex\dataset_mmap.py in <module>()
     21 from vaex.expression import Expression
     22 import struct
---> 23 import fcntl
     24 
     25 logger = logging.getLogger("vaex.file")

ModuleNotFoundError: No module named 'fcntl'



Discussion:
I have the same problem in Windows10 related to not having fcntl.
The error: ModuleNotFoundError: No module named 'fcntl'
occurs where I have tried using the example dataset with

ds = vaex.example() 

or trying with a pandas dataframe loaded previously:

ds = vaex.from_pandas(df)

I would add that I originally tried inside a conda virtualenv, then later reverted to installing via pip and retrying, the error happens in either case.

Thanks for letting me know. This should be fixed actually.
I'd say, try removing all vaex packages, and try with a clean slate, and maybe with a specific version number, like:
```
pip install -c vaex-core=0.6 vaex-hdf5 vaex-viz
```
Let me know if this helps (for conda install the same)
I have the same problem in Windows10 related to not having fcntl.
The error: ModuleNotFoundError: No module named 'fcntl'
occurs where I have tried using the example dataset with

ds = vaex.example() 

or trying with a pandas dataframe loaded previously:

ds = vaex.from_pandas(df)

I would add that I originally tried inside a conda virtualenv, then later reverted to installing via pip and retrying, the error happens in either case.

Thanks for letting me know. This should be fixed actually.
I'd say, try removing all vaex packages, and try with a clean slate, and maybe with a specific version number, like:
```
pip install -c vaex-core=0.6 vaex-hdf5 vaex-viz
```
Let me know if this helps (for conda install the same)

Resolving patch:
diff --git a/packages/vaex-core/vaex/dataset_mmap.py b/packages/vaex-core/vaex/dataset_mmap.py
--- a/packages/vaex-core/vaex/dataset_mmap.py
+++ b/packages/vaex-core/vaex/dataset_mmap.py
@@ -20,7 +20,6 @@
 import vaex.file
 from vaex.expression import Expression
 import struct
-import fcntl
 
 logger = logging.getLogger("vaex.file")
 
@@ -45,6 +44,7 @@
 
     from cachetools import LRUCache
     import threading
+    import fcntl
     GB = 1024**3
     def getsizeof(ar):
         return ar.nbytes


Test patch: