Repository: apache/tvm

Issue:
Reconsider dynamic library loading strategy
Here is the current state of affairs in TVM:

* The cmake build dumps compiled libraries in the `lib` directory of the *source* project. This is achieved by explicitly setting `CMAKE_LIBRARY_OUTPUT_DIRECTORY`
* The configure/make build builds libraries into the `lib` directory
* `python/tvm/_ffi/libinfo.py` is responsible for figuring out where the dynamic library lives. It looks in the following locations, preferring the first that it finds:
  * `dirname(__file__)` aka `curr_path` aka the directory which contains this source file. This could either be the actual `_ffi` source directory (in the case you are running inplace or with `python setup.py develop`), or the install directory of TVM
  * `$curr_path/../` aka `root_path` aka the root directory of TVM Python modules, e.g., `python/tvm` in the source checkout
  * `$curr_path/../../../lib`. When `libinfo.py` lives in a source checkout, this points to the top-level `lib` directory of the source project, aka, where cmake/make dump their build products. When `libinfo.py` was installed, curr_path will typically look something like `$prefix/lib/python3.6/site-packages/tvm/_ffi` which means that this point will point to `$prefix/lib/python3.6/lib` which generally doesn't exist
  * `$curr_path/../../../build/Release/'. Once again, in a source checkout, this will point to the Release directory of cmake (but NOTE that due to the manual configuration in cmake, this folder will generally never be populated)
  * Some Windows-only paths for VS in `build` and `windows`
  * If the OS is POSIX-based (linux or osx), all of the entries in `LD_LIBRARY_PATH`

Here are the problems with this strategy:

* It's easy to accidentally get a stale copy of an so file, and be unable to tell why updates are not being reflected. This happened to me in #280, where a stale `libtvm.so` living in my `lib` directory overrode an installed copy, and the ABI didn't match up at all (leading to strange errors). I'm not altogether certain why later invocations of `make` did not update the dynamic library, but I know empirically that this was the case.

* It means that TVM will behave differently re install lookup depending on whether or not it is installed with `python setup.py install` or `python setup.py develop`, because the source file location changes. Most notably, the search strategy doesn't work at all if you actually install the files.

* In general, while this search strategy makes TVM very friendly for in-tree development (since it is able to find in tree build products), it works very poorly when TVM is actually installed somewhere, since there isn't any way to say, "please use exactly this library."

I can see two primary ways of solving the problem.

First, is to configure the location of the TVM library at `python setup.py` time. The usual way to do this is to define a C extension module, and then declare the TVM library as a dependency. Then the standard mechanisms to declare where libraries are to be found.

Second is to tweak the search path strategy a bit. Here are my recommendations:

* The precedence of `LD_LIBRARY_PATH` should be over the hardcoded search paths is wrong. `LD_LIBRARY_PATH` should always OVERRIDE any of the built-in search paths, because it's the thing someone (like me) will use to try to force a particular dynamic library to be used.
* cmake should NOT dump build products outside of the `build` directory, this runs counter to the cmake philosophy where all build products live in `build`. A concrete case where this is bad is it means you cannot run parallel debug and regular builds in the same directory. However, I understand there is a countervailing desire, which is for Python to be able to automatically pick up on an inplace built binary; but with this change, there may be two places the build product may live. With this I have two orthogonal suggestions:
  * We should get rid of the Makefile based build system. There isn't any good reason to two build systems at the same time: it just means more opportunity for things to go wrong. I don't know if there are particular things we are relying on in the Makefile, but they should get ported to cmake if they are. This has an added bonus whereby there is now only one place an inplace build product may be stored.
  * We should require a user to *install* their build products before they are picked up by Python. The default install location could be to somewhere in the source directory (it could even be `lib`, as it is today). This means that if you have multiple build trees, you can simply "install" one to make it be picked up by the Python library. In this case, we should also print a debug message notifying the user that they're working with an inplace library
* I don't know of a known-working way to record a setup-time provided library path durably so that the search algorithm could look in the location. But we could still use the `__file__` trick to guess where the library might be installed at install time.

So here is the new search algorithm:

* Check LD_LIBRARY_PATH
* Check the in-place install location (emitting a notice if it is)
* Check the real install location

Do people agree with this? If so, I'll write a patch.


Discussion:

Resolving patch:
diff --git a/python/tvm/_ffi/base.py b/python/tvm/_ffi/base.py
--- a/python/tvm/_ffi/base.py
+++ b/python/tvm/_ffi/base.py
@@ -13,13 +13,13 @@
 # library loading
 #----------------------------
 if sys.version_info[0] == 3:
-    string_types = str,
+    string_types = (str,)
     numeric_types = (float, int, np.float32, np.int32)
     # this function is needed for python3
     # to convert ctypes.char_p .value back to python str
     py_str = lambda x: x.decode('utf-8')
 else:
-    string_types = basestring,
+    string_types = (basestring,)
     numeric_types = (float, int, long, np.float32, np.int32)
     py_str = lambda x: x
 
diff --git a/python/tvm/_ffi/libinfo.py b/python/tvm/_ffi/libinfo.py
--- a/python/tvm/_ffi/libinfo.py
+++ b/python/tvm/_ffi/libinfo.py
@@ -2,7 +2,6 @@
 from __future__ import absolute_import
 import sys
 import os
-import platform
 
 
 def find_lib_path(name=None, search_path=None):
@@ -19,21 +18,34 @@ def find_lib_path(name=None, search_path=None):
         List of all found path to the libraries
     """
     use_runtime = os.environ.get("TVM_USE_RUNTIME_LIB", False)
-    curr_path = os.path.dirname(os.path.abspath(os.path.expanduser(__file__)))
-    root_path = os.path.join(curr_path, '../')
-    api_path = os.path.join(curr_path, '../../../lib/')
-    cmake_build_path = os.path.join(curr_path, '../../../build/Release/')
-    dll_path = [curr_path, root_path, api_path, cmake_build_path]
-    if os.name == 'nt':
-        vs_configuration = 'Release'
-        if platform.architecture()[0] == '64bit':
-            dll_path.append(os.path.join(curr_path, '../../../build', vs_configuration))
-            dll_path.append(os.path.join(curr_path, '../../../windows/x64', vs_configuration))
-        else:
-            dll_path.append(os.path.join(curr_path, '../../../build', vs_configuration))
-            dll_path.append(os.path.join(curr_path, '../../../windows', vs_configuration))
-    elif os.name == "posix" and os.environ.get('LD_LIBRARY_PATH', None):
+
+    # See https://github.com/dmlc/tvm/issues/281 for some background.
+
+    # NB: This will either be the source directory (if TVM is run
+    # inplace) or the install directory (if TVM is installed).
+    # An installed TVM's curr_path will look something like:
+    #   $PREFIX/lib/python3.6/site-packages/tvm/_ffi
+    ffi_dir = os.path.dirname(os.path.abspath(os.path.expanduser(__file__)))
+    source_dir = os.path.join(ffi_dir, "..", "..", "..")
+    install_lib_dir = os.path.join(ffi_dir, "..", "..", "..", "..")
+
+    dll_path = []
+
+    if os.environ.get('TVM_LIBRARY_PATH', None):
+        dll_path.append(os.environ['TVM_LIBRARY_PATH'])
+
+    if sys.platform.startswith('linux') and os.environ.get('LD_LIBRARY_PATH', None):
         dll_path.extend([p.strip() for p in os.environ['LD_LIBRARY_PATH'].split(":")])
+    elif sys.platform.startswith('darwin') and os.environ.get('DYLD_LIBRARY_PATH', None):
+        dll_path.extend([p.strip() for p in os.environ['DYLD_LIBRARY_PATH'].split(":")])
+
+    # Default cmake build directory
+    dll_path.append(os.path.join(source_dir, "build"))
+    # Default mkae build directory
+    dll_path.append(os.path.join(source_dir, "lib"))
+
+    dll_path.append(install_lib_dir)
+
     dll_path = [os.path.abspath(x) for x in dll_path]
     if search_path is not None:
         if search_path is list:
@@ -44,9 +56,12 @@ def find_lib_path(name=None, search_path=None):
         lib_dll_path = [os.path.join(p, name) for p in dll_path]
         runtime_dll_path = []
     else:
-        if os.name == 'nt':
+        if sys.platform.startswith('win32'):
             lib_dll_path = [os.path.join(p, 'libtvm.dll') for p in dll_path]
             runtime_dll_path = [os.path.join(p, 'libtvm_runtime.dll') for p in dll_path]
+        elif sys.platform.startswith('darwin'):
+            lib_dll_path = [os.path.join(p, 'libtvm.dylib') for p in dll_path]
+            runtime_dll_path = [os.path.join(p, 'libtvm_runtime.dylib') for p in dll_path]
         else:
             lib_dll_path = [os.path.join(p, 'libtvm.so') for p in dll_path]
             runtime_dll_path = [os.path.join(p, 'libtvm_runtime.so') for p in dll_path]


Test patch: