Package: tfdeploy
|Build Status| |Documentation Status| |Package Status|Deploy `tensorflow <https://www.tensorflow.org>`__ graphs for *fast*evaluation and export to *tensorflow-less* environments running`numpy <http://www.numpy.org>`__.**Now with tensorflow 1.0 support.**Evaluation usage''''''''''''''''.. code:: pythonimport tfdeploy as tdimport numpy as npmodel = td.Model("/path/to/model.pkl")inp, outp = model.get("input", "output")batch = np.random.rand(10000, 784)result = outp.eval({inp: batch})Installation and dependencies'''''''''''''''''''''''''''''Via `pip <https://pypi.python.org/pypi/tfdeploy>`__.. code:: bashpip install tfdeployor by simply copying the file into your project.Numpy ≥ 1.10 should be installed on your system.`Scipy <http://www.scipy.org/>`__ is optional. See`optimization <#optimization>`__ for more info on optional packages.By design, tensorflow is required when creating a model. All versions ≥1.0.1 are supported.Content~~~~~~~-  `Why? <#why>`__-  `How? <#how>`__-  `Convert your graph <#convert-your-graph>`__-  `Load the model and evaluate <#load-the-model-and-evaluate>`__-  `Write your own operation <#write-your-own-operation>`__-  `Ensembles <#ensembles>`__-  `Optimization <#optimization>`__-  `Performance <#performance>`__-  `Contributing <#contributing>`__-  `Development <#development>`__-  `Authors <#authors>`__-  `License <#license>`__Why?----Working with tensorflow is awesome. Model definition and training issimple yet powerful, and the range of built-in features is juststriking.However, when it comes down to model deployment and evaluation, thingsget a bit more cumbersome than they should be. You either export yourgraph to a new file *and* `save your trainedvariables <https://www.tensorflow.org/versions/master/how_tos/variables/index.html#saving-variables>`__in a separate file, or you make use of tensorflow's `servingsystem <https://www.tensorflow.org/versions/master/tutorials/tfserve/index.html>`__.Wouldn't it be great if you could just export your model to a simplenumpy-based callable? Of course it would. And this is exactly whattfdeploy does for you.To boil it down, tfdeploy-  is lightweight. A single file with < 150 lines of core code. Justcopy it to your project.-  `faster <#performance>`__ than using tensorflow's ``Tensor.eval``.-  **does not need tensorflow** during evaluation.-  only depends on numpy.-  can load one or more models from a single file.-  does not support GPUs (maybe`gnumpy <http://www.cs.toronto.edu/~tijmen/gnumpy.html>`__ is worth atry here).How?----The central class is ``tfdeploy.Model``. The following two examplesdemonstrate how a model can be created from a tensorflow graph, saved toand loaded from disk, and eventually evaluated.Convert your graph''''''''''''''''''.. code:: pythonimport tensorflow as tfimport tfdeploy as td# setup tfdeploy (only when creating models)td.setup(tf)# build your graphsess = tf.Session()# use names for input and output layersx = tf.placeholder("float", shape=[None, 784], name="input")W = tf.Variable(tf.truncated_normal([784, 100], stddev=0.05))b = tf.Variable(tf.zeros([100]))y = tf.nn.softmax(tf.matmul(x, W) + b, name="output")sess.run(tf.global_variables_initializer())# ... training ...# create a tfdeploy model and save it to diskmodel = td.Model()model.add(y, sess) # y and all its ops and related tensors are added recursivelymodel.save("model.pkl")Load the model and evaluate'''''''''''''''''''''''''''.. code:: pythonimport numpy as npimport tfdeploy as tdmodel = td.Model("model.pkl")# shorthand to x and yx, y = model.get("input", "output")# evaluatebatch = np.random.rand(10000, 784)result = y.eval({x: batch})Write your own ``Operation``''''''''''''''''''''''''''''tfdeploy supports most of the ``Operation``'s `implemented intensorflow <https://www.tensorflow.org/versions/master/api_docs/python/math_ops.html>`__.However, if you miss one (in that case, submit a PR or an issue ;) ) orif you're using custom ops, you might want to extend tfdeploy bydefining a new class op that inherits from ``tfdeploy.Operation``:.. code:: pythonimport tensorflow as tfimport tfdeploy as tdimport numpy as np# setup tfdeploy (only when creating models)td.setup(tf)# ... write you model here ...# let's assume your final tensor "y" relies on an op of type "InvertedSoftmax"# before creating the td.Model, you should add that op to tfdeployclass InvertedSoftmax(td.Operation):@staticmethoddef func(a):e = np.exp(-a)# ops should return a tuplereturn np.divide(e, np.sum(e, axis=-1, keepdims=True)),# this is equivalent to# @td.Operation.factory# def InvertedSoftmax(a):#     e = np.exp(-a)#     return np.divide(e, np.sum(e, axis=-1, keepdims=True)),# now we're good to gomodel = td.Model()model.add(y, sess)model.save("model.pkl")When writing new ops, three things are important:-  Try to avoid loops, prefer numpy vectorization.-  Return a tuple.-  Don't change incoming tensors/arrays in-place, always work on andreturn copies.Ensembles---------tfdeploy provides a helper class to evaluate an ensemble of models:``Ensemble``. It can load multiple models, evaluate them and combinetheir output values using different methods... code:: python# create the ensembleensemble = td.Ensemble(["model1.pkl", "model2.pkl", ...], method=td.METHOD_MEAN)# get input and output tensors (which actually are TensorEnsemble instances)input, output = ensemble.get("input", "output")# evaluate the ensemble just like a normal modelbatch = ...value = output.eval({input: batch})The return value of ``get()`` is a ``TensorEnsemble`` istance. It isbasically a wrapper around multiple tensors and should be used as keysin the ``feed_dict`` of the ``eval()`` call.You can choose between ``METHOD_MEAN`` (the default), ``METHOD_MAX`` and``METHOD_MIN``. If you want to use a custom ensembling method, use``METHOD_CUSTOM`` and overwrite the static ``func_custom()`` method ofthe ``TensorEnsemble`` instance.Optimization------------Most ops are written using pure numpy. However, multiple implementationsof the same op are allowed that may use additional third-party Pythonpackages providing even faster functionality for some situations.For example, numpy does not provide a vectorized *lgamma* function.Thus, the standard ``tfdeploy.Lgamma`` op uses ``math.lgamma`` that waspreviously vectorized using ``numpy.vectorize``. For these situations,additional implementations of the same op are possible (the *lgamma*example is quite academic, but this definitely makes sense for moresophisticated ops like pooling). We can simply tell the op to use itsscipy implementation instead:.. code:: pythontd.Lgamma.use_impl(td.IMPL_SCIPY)Currently, allowed implementation types are numpy (``IMPL_NUMPY``, thedefault) and scipy (``IMPL_SCIPY``).Adding additional implementations'''''''''''''''''''''''''''''''''Additional implementations can be added by setting the ``impl``attribute of the op factory or by using the ``add_impl`` decorator ofexisting operations. The first registered implementation will be thedefault one... code:: python# create the default lgamma op with numpy implementationlgamma_vec = np.vectorize(math.lgamma)@td.Operation.factory# equivalent to# @td.Operation.factory(impl=td.IMPL_NUMPY)def Lgamma(a):return lgamma_vec(a),# add a scipy-based implementation@Lgamma.add_impl(td.IMPL_SCIPY)def Lgamma(a):return sp.special.gammaln(a),Auto-optimization'''''''''''''''''If scipy is available on your system, it is reasonable to use all ops intheir scipy implementation (if it exists, of course). This should beconfigured before you create any model from tensorflow objects using thesecond argument of the ``setup`` function:.. code:: pythontd.setup(tf, td.IMPL_SCIPY)Ops that do not implement ``IMPL_SCIPY`` stick with the numpy version(``IMPL_NUMPY``).Performance-----------tfdeploy is lightweight (1 file, < 150 lines of core code) and fast.Internal evaluation calls have only very few overhead and tensoroperations use numpy vectorization. The actual performance depends onthe ops in your graph. While most of the tensorflow ops have a numpyequivalent or can be constructed from numpy functions, a few ops requireadditional Python-based loops (e.g. ``BatchMatMul``). But in many casesit's potentially faster than using tensorflow's ``Tensor.eval``.This is a comparison for a basic graph where all ops are vectorized(basically ``Add``, ``MatMul`` and ``Softmax``):.. code:: bash> ipython -i tests/perf/simple.pyIn [1]: %timeit -n 100 test_tf()100 loops, best of 3: 109 ms per loopIn [2]: %timeit -n 100 test_td()100 loops, best of 3: 60.5 ms per loopContributing------------If you want to contribute with new ops and features, I'm happy toreceive pull requests. Just make sure to add a new test case to``tests/core.py`` or ``tests/ops.py`` and run them via:.. code:: bash> python -m unittest testsTest grid'''''''''In general, tests should be run for different environments:+----------------------+-------------+| Variation            | Values      |+======================+=============+| tensorflow version   | ``1.0.1``   |+----------------------+-------------+| python version       | 2, 3        |+----------------------+-------------+| ``TD_TEST_SCIPY``    | 0, 1        |+----------------------+-------------+| ``TD_TEST_GPU``      | 0, 1        |+----------------------+-------------+Docker''''''For testing purposes, it is convenient to use docker. Fortunately, theofficial `tensorflowimages <https://hub.docker.com/r/tensorflow/tensorflow/>`__ contain allwe need:.. code:: bashgit clone https://github.com/riga/tfdeploy.gitcd tfdeploydocker run --rm -v `pwd`:/root/tfdeploy -w /root/tfdeploy -e "TD_TEST_SCIPY=1" tensorflow/tensorflow:1.0.1 python -m unittest testsDevelopment------------  Source hosted at `GitHub <https://github.com/riga/tfdeploy>`__-  Report issues, questions, feature requests on `GitHubIssues <https://github.com/riga/tfdeploy/issues>`__Authors--------  `Marcel R. <https://github.com/riga>`__-  `Benjamin F. <https://github.com/bfis>`__License-------The MIT License (MIT)Copyright (c) 2017 Marcel R.Permission is hereby granted, free of charge, to any person obtaining acopy of this software and associated documentation files (the"Software"), to deal in the Software without restriction, includingwithout limitation the rights to use, copy, modify, merge, publish,distribute, sublicense, and/or sell copies of the Software, and topermit persons to whom the Software is furnished to do so, subject tothe following conditions:The above copyright notice and this permission notice shall be includedin all copies or substantial portions of the Software.THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESSOR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OFMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANYCLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THESOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE... |Build Status| image:: https://travis-ci.org/riga/tfdeploy.svg?branch=master:target: https://travis-ci.org/riga/tfdeploy.. |Documentation Status| image:: https://readthedocs.org/projects/tfdeploy/badge/?version=latest:target: http://tfdeploy.readthedocs.org/en/latest/?badge=latest.. |Package Status| image:: https://badge.fury.io/py/tfdeploy.svg:target: https://badge.fury.io/py/tfdeploy