Package: tensorflow-transform
TensorFlow TransformTensorFlow Transformis a library for preprocessing data with TensorFlow.tf.Transformis useful for data that requires a full-pass, such as:Normalize an input value by mean and standard deviation.Convert strings to integers by generating a vocabulary over all input values.Convert floats to integers by assigning them to buckets based on the observed
data distribution.TensorFlow has built-in support for manipulations on a single example or a batch
of examples.tf.Transformextends these capabilities to support full-passes
over the example data.The output oftf.Transformis exported as aTensorFlow graphto use for training and serving.
Using the same graph for both training and serving can prevent skew since the
same transformations are applied in both stages.For an introduction totf.Transform, see thetf.Transformsection of the
TFX Dev Summit talk on TFX
(link).InstallationThetensorflow-transformPyPI packageis the
recommended way to installtf.Transform:pipinstalltensorflow-transformBuild TFT from sourceTo build from source follow the following steps:
Create a virtual environment by running the commandspython3 -m venv <virtualenv_name>
source <virtualenv_name>/bin/activate
pip3 install setuptools wheel
git clone https://github.com/tensorflow/transform.git
cd transform
python3 setup.py bdist_wheelThis will build the TFT wheel in the dist directory. To install the wheel from
dist directory run the commandscd dist
pip3 install tensorflow_transform-<version>-py3-none-any.whlNightly PackagesTFT also hosts nightly packages athttps://pypi-nightly.tensorflow.orgon
Google Cloud. To install the latest nightly package, please use the following
command:pipinstall--extra-index-urlhttps://pypi-nightly.tensorflow.org/simpletensorflow-transformThis will install the nightly packages for the major dependencies of TFT such
as TensorFlow Metadata (TFMD), TFX Basic Shared Libraries (TFX-BSL).Notable DependenciesTensorFlow is required.Apache Beamis required; it's the way that efficient
distributed computation is supported. By default, Apache Beam runs in local
mode but can also run in distributed mode usingGoogle Cloud Dataflowand other Apache
Beamrunners.Apache Arrowis also required. TFT uses Arrow to
represent data internally in order to make use of vectorized numpy functions.Compatible versionsThe following table is thetf.Transformpackage versions that are
compatible with each other. This is determined by our testing framework, but
otheruntestedcombinations may also work.tensorflow-transformapache-beam[gcp]pyarrowtensorflowtensorflow-metadatatfx-bslGitHub master2.47.010.0.0nightly (2.x)1.14.01.14.01.14.02.47.010.0.02.131.14.01.14.01.13.02.41.06.0.02.121.13.11.13.01.12.02.41.06.0.02.111.12.01.12.01.11.02.41.06.0.01.15.5 / 2.101.11.01.11.01.10.02.40.06.0.01.15.5 / 2.91.10.01.10.01.9.02.38.05.0.01.15.5 / 2.91.9.01.9.01.8.02.38.05.0.01.15.5 / 2.81.8.01.8.01.7.02.36.05.0.01.15.5 / 2.81.7.01.7.01.6.12.35.05.0.01.15.5 / 2.81.6.01.6.01.6.02.35.05.0.01.15.5 / 2.71.6.01.6.01.5.02.34.05.0.01.15.2 / 2.71.5.01.5.01.4.12.33.04.0.11.15.2 / 2.61.4.01.4.01.4.02.33.04.0.11.15.2 / 2.61.4.01.4.01.3.02.31.02.0.01.15.2 / 2.61.2.01.3.01.2.02.31.02.0.01.15.2 / 2.51.2.01.2.01.1.12.29.02.0.01.15.2 / 2.51.1.01.1.11.1.02.29.02.0.01.15.2 / 2.51.1.01.1.01.0.02.29.02.0.01.15 / 2.51.0.01.0.00.30.02.28.02.0.01.15 / 2.40.30.00.30.00.29.02.28.02.0.01.15 / 2.40.29.00.29.00.28.02.28.02.0.01.15 / 2.40.28.00.28.10.27.02.27.02.0.01.15 / 2.40.27.00.27.00.26.02.25.00.17.01.15 / 2.30.26.00.26.00.25.02.25.00.17.01.15 / 2.30.25.00.25.00.24.12.24.00.17.01.15 / 2.30.24.00.24.10.24.02.23.00.17.01.15 / 2.30.24.00.24.00.23.02.23.00.17.01.15 / 2.30.23.00.23.00.22.02.20.00.16.01.15 / 2.20.22.00.22.00.21.22.17.00.15.01.15 / 2.10.21.00.21.30.21.02.17.00.15.01.15 / 2.10.21.00.21.00.15.02.16.00.14.01.15 / 2.00.15.00.15.00.14.02.14.00.14.01.140.14.0n/a0.13.02.11.0n/a1.130.12.1n/a0.12.02.10.0n/a1.120.12.0n/a0.11.02.8.0n/a1.110.9.0n/a0.9.02.6.0n/a1.90.9.0n/a0.8.02.5.0n/a1.8n/an/a0.6.02.4.0n/a1.6n/an/a0.5.02.3.0n/a1.5n/an/a0.4.02.2.0n/a1.4n/an/a0.3.12.1.1n/a1.3n/an/a0.3.02.1.1n/a1.3n/an/a0.1.102.0.0n/a1.0n/an/aQuestionsPlease direct any questions about working withtf.TransformtoStack Overflowusing thetensorflow-transformtag.