Package: trifacta
Python SDK for TrifactaIntegrate your Python-based environment with Trifacta to rapidly transform your datasets. Please complete the following steps to install and set up the Python SDK for Trifacta.AvailabilityAlpharelease - software and supported capabilities may be changed without notice. Do not deploy in product environments.Available for the following product editions:Trifacta EnterpriseTrifacta ProfessionalTrifacta PremiumLimitationsSome Wrangle functions and transformations are not supported by Python Pandas. Known limitations:NUMFORMAT functionString comparison functionsTransformations that use Array or Map data types are not supported for Python Pandas generation.Uploaded files must be in CSV file format.Pre-RequisitesAssumptionsListed commands are for Mac OSX.Examples below assume that you are using Jupyter Notebooks for Python flow development.Trifacta RequirementsA valid account to a project or workspace for one of the above product editions.A valid access token to the project or workspace. Instructions are provided below.To export your Trifacta recipe as Python code, a workspace administrator must enable theWrangle to Python Conversionfeature in the application. For more information, please visitWorkspace Settings Page.Python RequirementsPython 3.7, Python 3.8For version requirements of specific Python components, please seerequirements.txtin this package.InstallInstalltrifactausing pip:pip install trifactaConfigureEnable access to your Trifacta workspaceLogin to your Trifacta workspace.In the left menu, selectUser menu > Preferences > Access tokens.To create a new access token, clickGenerate new token. Copy the token to the clipboard. You cannot retrieve a token after you exit the modal.Paste this token into a text file. Instructions for using it with the SDK are provided later.Configure Trifacta packageBefore you can use it to interact with your Trifacta environment, the Python SDK for Trifacta requires the following configuration:In your home directory, create the following configuration file:.trifacta.py.conf.Open the file in a text editor, and insert following configuration. Replace values as needed:[CONFIGURATION]
username = <username_for_trifacta_account>  # example: test-user@gmail.com
endpoint = <uri_for_your_trifacta_worskapce>  # example: https://test-workspace.example.com
token = <copied_token_from_steps_above>Save the file.UseFor this release, you can use the Python SDK to upload a CSV file for transformation in a new file. Additional file formats and workflows may be supported in future releases.Upload to new flowCreate a new python3 notebook and import thetrifactamodule:import trifacta as tftfis your handler for interacting with your Trifacta workspace.Insert the following code, which uploads a specified CSV for transformation in Trifacta:import pandas as pd
df = pd.read_csv(<path_to_csv_dataset>)
wf = tf.wrangle(df)Thewranglefunction lets you upload a dataset to Trifacta and create a flow for it. This flow is then available through the Trifacta application, where you can transform the dataset through the user interface.wfis returned as a handle for the created flow with which
you can perform other operations on your dataset.Run the notebook.Launch Trifacta in browserAfter the upload completes, execute the following to open Trifacta in a browser window.wf.open()In the Trifacta window, navigate to the flow that was created. This flow is likely to be namedUntitledand to be listed in the Flows page at the top when sorted by timestamp.In the created flow, create a recipe connected to your imported dataset.Edit the recipe.In the Transformer page, you can identify issues with your dataset and add transformations to your recipe through a point-and-click interface. ClickAddto add the corresponding transformation step to your recipe. For more information on using Trifacta, please visitTrifacta Documentation.When you have finished defining your recipe steps, return to your Python notebook window.Generate Pandas codeIn the Python SDK, you use theget_pandas()method to export the Wrangle recipe steps to Python code.NOTE:Wrangle to Python Conversionsetting must be enabled in Trifacta by your workspace administrator. See above.Use the following to get pandas code for the recipe that you created in Trifacta. This code can be applied to transform
yourPandas DataFrame.wf.get_pandas(add_to_next_cell=True,recipe_name='<my_recipe>')get_pandastranslates Trifacta's transform recipe into pandas code.add_to_next_cellset toTrueensures that the generated code is added to the next cell of notebook.<recipe_name>can be specified to generate pandas code for a specific recipe. If not specified, code is generated for the default recipe. To retrieve a list of available recipes, usewf.recipe_names().Execute the generated code.In a new cell perform the following actions to transform the dataframe using
above generated Pandas code.wrangled_df = run_transforms(df)
wrangled_dfAbove returns the output of your cleansed/transformed pandas dataframe.ExamplesWrangle multiple datasetsThe following example describes how to wrangle multiple datasets. In this example,violationsandviolations_actionsare reference names forviolations_dfandviolations_actions_dfrespectively. This mapping enables users to correctly map Pandas DataFrames to arguments/variables in generated Pandas code for the Wrangle recipe.import pandas as pd
import trifacta as tf
violations_df = pd.read_csv('../test/data/violations.csv')
violations_actions_df = pd.read_csv('../test/data/violations_actions.csv')
wrangle_flow = tf.wrangle((violations_df, 'violations'), (violations_actions_df, 'violations_actions'), flow_name='Example Flow')Wrangle existing flowFrom your notebook, you can also begin wrangling an existing flow. The following example launches the Trifacta application in a flow whose internal identifier (flow_id) is13.import trifacta as tf
import pandas as pd
flow_id = 13
wf = tf.wrangle_existing(flow_id)
# Following call opens the flow corresponding to 'flow_id' in Flow View. If a 'recipe_name' arg is provided, the recipe is opened in the Transformer page. 
wf.open()For additional examples, please see thenotebooksdirectory in this package.Wrangle function referenceThe following wrangling functions are available through the SDK.Trifacta module functionstfis an alias to the Trifacta module.MethodDescriptionArgumentstf.wrangle(*datasets)Upload one ore more datasets to the Trifacta application and create a flow for it.   This flow is then available through the Trifacta application, where you can transform the dataset through the user interface.*datasets: Pandas DataFrames to be wrangled.   It could also be a tuple, where the first element in the tuple is a Pandas DataFrame, and second element is the reference name (string) for the DataFrame.WrangleFlow module functionsAll of the below functions are available for theWrangleFlowobject in your Python environment. So, you must call them using aWrangleFlowobject.wfis a reference to theWrangleFlowobject.MethodDescriptionArgumentswf.add_datasets(*datasets)Add Pandas DataFrames to a flow, wheredatasetsis a list of DataFrames.*datasets: Pandas DataFrames to be added to a flow.    It could also be a tuple, where the first element in the tuple is a Pandas DataFrame, and second element is the reference name (string) for the DataFrame.get_pandas(add_to_next_cell=False, recipe_name=None)Generates Python Pandas code for your Wrangle recipe.add_to_next_cell:Set it to True, if you're using Jupyter Notebook and would like to add the generated Pandas code to be added to next cell. If False, the Pandas code is returned as string.recipe_name:Recipe for which you want to get the Pandas code. If not specified, the default recipe is used. Usewf.recipe_names()to retrieve available recipes.wf.run_job(pbar=None, execution='photon', recipe_name=None)Run a job for a specified recipe.pbar: can be ignored.execution: Running environment in the Trifacta platform where you want to execute the job. Possible values:photonoremrSpark.recipe_name: Recipe for which you want to execute the job. If set toNone, input is the default recipe.wf.profile(recipe_name=None)Generate a profile for a specified recipe.recipe_name: Recipe for which you want to generate profile. If set toNone, input is the default recipe.wf.recipe_names()Lists the recipe names for the recipe present in the Trifacta application.N/Awf.open_profile(recipe_name=None)Open a profile that you have previously generated for the specified recipe.recipe_name: Recipe for which you want to open the profile. If set toNone, input is the default recipe.Data profiling functionsThe SDK enables generation of data profiles based on the output of your Trifacta recipe:MethodDescriptionArgumentssummary(recipe_name=None)Returns a table of summary statistics per columnrecipe_name: Recipe name for which you want to generate the summary. If set toNone, input is the default recipe.dq_bars(show_types=True, recipe_name=None)Returns the valid/invalid/missing ratio per columnshow_types: Show column types information along with data quality bars for the column.recipe_name: Recipe name for which you want to generate the data quality bar. If set toNone, input is the default recipe.col_types(recipe_name=None)Lists the inferred data type for each columnrecipe_name: Recipe name for which you want to infer data types for each column. If set toNone, input is the default recipe.bars_df_list(recipe_name)Returns a list of dataframes, one per column, representing a bar-chart for that columnN/Apdf_profile(filename=None, recipe_name=None)Returns a snazzy PDF report with all the statisticsfilename: Name of the file to which PDF profile results are written. If set toNone, results are returned back from the function.recipe_name: Recipe for which you want to generate PDF profile results. If set toNone, results are generated for the default recipe.