Package: tensorspacejs
TensorSpace ConverterEnglish|中文About TensorSpace 🤔:TensorSpace GithubTensorSpace-Converter is a tool used to generate a TensorSpace compatible model from a pre-trained model built by TensorFlow, Keras and TensorFlow.js. TensorSpace-Converter includes the functions of: extracting information from hidden layers, matching intermediate data based on the configurations and exporting preprocessed TensorSpace compatible model. TensorSpace simplifies the preprocess and helps developers to focus on the development of model visualization.Table of ContentMotivationGetting StartedInstallUsageRunning with DockerConverter APIConverter Usage ExamplesTensorFlowKerasTensorFlow.jsDevelopmentContributorsContactLicenseMotivationTensorSpaceis a JavaScript framework used to 3D visualize deep learning models built by TensorFlow, Keras and TensorFlow.js. Before applying TensorSpace to the pre-trained model, there is an important pipeline - TensorSpace model preprocessing ( Checkout thisarticlefor more information about TensorSpace preprocessing ). TensorSpace-Converter is designed to simplify the model preprocessing and generate a TensorSpace compatible model easily and quickly.Without TensorSpace-Converter, the developer needs to be expert on the pre-trained model and machine learning library the model used. For example, if the developer has an LeNet pre-trained model built by tf.keras, it is required to know the structure of the LeNet network as well as how to implement a new multi-output model by tf.keras. Now, with TensorSpace-Converter, it only needs some commands to complete the preprocessing process. For example, the developer only needs to use thecommandsto preprocess a tf.keras pre-trained model.As a component of TensorSpace ecosystem, TensorSpace-Converter simplifies the TensorSpace preprocess, release the workloads from learning how to generate TensorSpace compatible model. As a development tool, TensorSpace-Converter helps to separate the work ofmodel trainingandmodel visualization.Fig. 1- TensorSpace-Converter UsageGetting StartedInstallInstall thetensorspacejspip package:$pipinstalltensorspacejsIftensorspacejsis installed successfully, you can check the TensorSpace-Converter version by using the command:$tensorspacejs_converter-vTheninitTensorSpace Converter (important step):$tensorspacejs_converter-initNoteTensorSpace-Converter requires to run under Python 3.6, Node 11.3+, NPM 6.5+. If you have other pre-installed Python version in your local environment, we suggest you to create a new fresh virtual environment. For example, thecondacommands is like:$condacreate-nenvnamepython=3.6
$sourceactivateenvname
$pipinstalltensorspacejsUsageThe following part introduces the usage and workflow on:How to use TensorSpace-Converter to convert a pre-trained model;How to apply TensorSpace to the converted model for model visualization.An MNIST-digit tf.keras model is used as an example in the tutorial. The sample files used in the tutorial includespre-trained tf.keras model,TensorSpace-Converter scriptandTensorSpace visualization code.Fig. 2- TensorSpace-Converter WorkflowStep 1: Use TensorSpace-Converter to preprocess pre-trained modelTensorSpace-Converter will convert an input model into a multi-output model, checkout thisarticlefor more information about multi-output model and model preprocessing.$tensorspacejs_converter\--input_model_from="tensorflow"\--input_model_format="tf_keras"\--output_layer_names="conv_1,maxpool_1,conv_2,maxpool_2,dense_1,dense_2,softmax"\./PATH/TO/MODEL/tf_keras_model.h5\./PATH/TO/SAVE/DIRFig. 3- converted multi-output modelStep 2: Apply TensorSpace for model visualizationmodel.load({type:"tensorflow",url:"/PATH/TO/MODEL/model.json"});Fig. 4- LeNet VisualizationRunning with DockerEstablishingTensorSpace-Converterruntime environment is a tedious topic? Dockerize it!Here is a TensorSpace-ConverterDockerfile, you can use it to build a out-of-box TensorSpace-Converterimage. We also provide some easy to use scripts to init (init_docker_converter.sh) and run (run_docker_converter.sh)tensorspacejsdocker image.To inittensorspacejsDocker image (make sure start Docker daemon before init the image):cd./docker
bashinit_docker_converter.shTo run docker imagePut TensorSpace-Converter script and model assets in aWORK_DIR, and executerun_docker_converter.shto runtensorspacejsimage:cd./docker
bashrun_docker_converter.sh--work_dirPATH/TO/WORK_DIRCheckout thisDocker Examplefor more practical usage of running TensorSpace-Converter with Docker.Converter APISample TensorSpace-Converter script:$tensorspacejs_converter\--input_model_from="XXX"\--input_model_format="YYY"\--output_layer_names="EEE1,EEE2,EEE3"\input_path\output_pathArguments explanation:Positional ArgumentsDescriptioninput_pathPath for model input artifacts. CheckoutUsage Examplefor how to set this attribute for different kinds of models.output_pathFolder for all output artifacts.OptionsDescription--input_model_fromConfigure the training library for pre-trained model, use:tensorflowfor TensorFlow,kerasfor Keras,tfjsfor TensorFlow.js.--input_model_formatThe format of input model, checkoutUsage Examplefor how to set this attribute for different kinds of models.--output_layer_namesThe names of the layer which will be visualized in TensorSpace, separated by comma ",".Converter Usage ExamplesThis section introduces the usage of TensorSpace-Converter for different types of pre-trained model from TensorFlow, Keras, TensorFlow.js.TensorFlowA pre-trained model built by TensorFlow can be saved as saved model, frozen model, combined HDF5 model or separated HDF5 model. Use different TensorSpace-Converter commands for different kinds of TensorFlow model formats. TensorSpace-Converter collects the data fromtensor, then use the outputs as the inputs oflayerof TensorSpace visualization. The developer can collect all necessary tensor names and set the name list asoutput_layer_names.For a combined HDF5 model, topology and weights are saved in a combined HDF5 filexxx.h5. Setinput_model_formatto betf_keras. The sample command script should be like:$tensorspacejs_converter\--input_model_from="tensorflow"\--input_model_format="tf_keras"\--output_layer_names="layer1Name,layer2Name,layer3Name"\./PATH/TO/MODEL/xxx.h5\./PATH/TO/SAVE/DIRFor a separated HDF5 model, topology and weights are saved in separate files, topology filexxx.jsonand weights filexxx.h5. Setinput_model_formatto betf_keras_separated. In this case, the model have two input files, merge two file's paths and separate them with comma (.json first, .h5 last), and then set the combined path to positional argumentinput_path. The sample command script should be like:$tensorspacejs_converter\--input_model_from="tensorflow"\--input_model_format="tf_keras_separated"\--output_layer_names="layer1Name,layer2Name,layer3Name"\./PATH/TO/MODEL/xxx.json,./PATH/TO/MODEL/eee.h5\./PATH/TO/SAVE/DIRFor a TensorFlow saved model. Setinput_model_formatto betf_saved. The sample command script should be like:$tensorspacejs_converter\--input_model_from="tensorflow"\--input_model_format="tf_saved"\--output_layer_names="layer1Name,layer2Name,layer3Name"\./PATH/TO/SAVED/MODEL/FOLDER\./PATH/TO/SAVE/DIRFor a TensorFlow frozen model. Setinput_model_formatto betf_frozen. The sample command script should be like:$tensorspacejs_converter\--input_model_from="tensorflow"\--input_model_format="tf_frozen"\--output_layer_names="layer1Name,layer2Name,layer3Name"\./PATH/TO/MODEL/xxx.pb\./PATH/TO/SAVE/DIRCheckout thisTensorFlow Tutorialfor more practical usage of TensorSpace-Converter for TensorFlow models.KerasA pre-trained model built by Keras, may have two formats: topology and weights are saved in a single HDF5 file, or topology and weights are saved in separated files. Use different TensorSpace-Converter commands for these two saved Keras models.For a Keras model, topology and weights are saved in a single HDF5 file, i.e.xxx.h5. Setinput_model_formatto betopology_weights_combined. The sample command script should be like:$tensorspacejs_converter\--input_model_from="keras"\--input_model_format="topology_weights_combined"\--output_layer_names="layer1Name,layer2Name,layer3Name"\./PATH/TO/MODEL/xxx.h5\./PATH/TO/SAVE/DIRFor a Keras model, topology and weights are saved in separated files, i.e. a topology filexxx.jsonand a weights filexxx.h5. Setinput_model_formatto betopology_weights_separated. In this case, the model have two input files, merge two file's paths and separate them with comma (.json first, .h5 last), and then set the combined path to positional argumentinput_path. The sample command script should be like:$tensorspacejs_converter\--input_model_from="keras"\--input_model_format="topology_weights_separated"\--output_layer_names="layer1Name,layer2Name,layer3Name"\./PATH/TO/MODEL/xxx.json,./PATH/TO/MODEL/eee.h5\./PATH/TO/SAVE/DIRCheckout thisKeras Tutorialfor more practical usage of TensorSpace-Converter for Keras models.TensorFlow.jsA pre-trained model built by TensorFlow.js, may have a topology filexxx.jsonand a weights filexxx.weight.bin. To converter the model with TensorSpace-Converter, the two files should be put in the same folder and set topology file's path toinput_path. The sample command script should be like:$tensorspacejs_converter\--input_model_from="tfjs"\--output_layer_names="layer1Name,layer2Name,layer3Name"\./PATH/TO/MODEL/xxx.json\./PATH/TO/SAVE/DIRCheckout thisTensorFlow.js tutorialfor more practical usage of TensorSpace-Converter for TensorFlow.js models.DevelopmentEnsure to have a freshpython=3.6,node>=11.3,npm>=6.5,tensorflowjs=0.8.0environment.SetupTo setup a TensorSpace-Converter development environment:gitclonehttps://github.com/tensorspace-team/tensorspace-converter.gitcdtensorspace-converter
bashinit-converter-dev.sh
npminstallBuildTo build TensorSpace-Converter pip package (Build files can be find indistfolder):bashbuild-pip-package.shTo install local build files:pipinstalldist/tensorspacejs-VERSION-py3-none-any.whl
tensorspacejs_converter-vTestGrand permissions to test scripts before running test scripts:bash./test/grandPermission.shTo run end-to-end test (tests shall be run in pre-set environment):npmruntestContributorsThanks goes to these wonderful people (emoji key):Chenhua Zhu💻🎨📖💡syt123450💻🎨📖💡Qi(Nora)🎨BoTime💻💡YaoXing Liu📖🎨ContactIf you have any issue or doubt, feel free to contact us by:Email:tensorspaceteam@gmail.comGitHub Issues:create issueSlack:#questionsGitter:#LobbyLicenseApache License 2.0