Package: tableautransformer
TableauTransformerETL tooling for preparing tableau seed dataDescriptionThis library was built with the intentions of enhancing the experience of data wrangling for tableau.
Tableau can be very particular about the data it reads from.
In addition preparing data to fit the shape for different graphs can be time consuming.
TableauTransformer can be used to hurdle over these two barriers.Github Repohttps://github.com/jteixcppib/tableautransformerDependenciespip, python 3.6, pandas, numpyGetting Startedpip install tableautransformerimport tableautransformer.datawrangling as tbttbt is a collection of functions, not a collection of methods, so all calls are "tbt.function_name()"Function DocsHere you can find a list of all functions within the library, a description of what they do, and their inputs.Basic_Table()basic_table(read_path, read_type='csv', sheet_name=None, columns_to_keep=None, columns_rename=None, 
                filters=None, group_by=None, aggregate_columns=None, pre_agg_math_columns=None, 
                post_agg_math_columns=None, remove_NAN=True, remove_NAN_col='all')Descriptionbasic_table is the basis for the tbt library as it refactors ~20 lines of commonly repeated code down to one input heavy function. The function reads in a dataframe, cleans up the data, and performs commonly used table operations.Inputsread_path:stringThe path to the file you wish to read. Theonly mandatoryinput.read_type:'csv' or 'excel'Default is'csv', if type is excel then sheet_name must have a value.sheet_name:stringThe name of the tab you wish to read in.columns_to_keep:list of strings['colA', 'colB', 'colC']Default retains all columns, this function runs immediately after reading in the data, any column mentioned in the list will remain in the dataframe, all others are dropped.columns_rename:list of strings['colA', 'colB', 'colC']Default does not alter the names. The renaming process occurs after the file is read in and columns_to_keep have been selected. All other column related inputs should use the new name dictated by the rename process if used.filters:list of 3-element tuples[('col_name', 'operand', 'value')]The input can be multiple filters, each filter is a 3-element tuple where the first element is the column name, the second is the operand, and the third is the value. The column name and operand must be strings while the value can be numeric (or a string if the operand is '==').group_by:list of strings['colA', 'colB', 'colC']groups dataframe by specified columns, aggregate_columns must not be None in order to work.aggregate_columns:dicitonary{'colA': 'agg_func'}when group_by is in effect it selects which columns to perform an agg function on, example 'sum' will do np.sum.pre_agg_math_columns:dictionary{'new_col': 'colA + colB'}Create a new column labeled as the key (from dictionary) and the scalar value as the value (from dictionary). Creates before group_by function is read. Caveat: columns in the value (from dictionary) must contain no spaces.post_agg_math_columns:dictionary{'new_col': 'colA + colB'}Create a new column labeled as the key (from dictionary) and the scalar value as the value (from dictionary). Creates after group_by function is read. Caveat: columns in the value (from dictionary) must contain no spaces.remove_NAN:True or False booleanDefault isTrue, removes any rows where a scalar value in the target column is NaN.remove_NAN_col:'all' or list of stirngs['colA', 'colB', 'colC'](default is'all'), selects the targets for remove_NAN function.Bucket()bucket(df, column, bucket_col_name, intervals)DescriptionCreates a new column with values assigned depending on discrete intervals of another columns value.Inputsdf:DataFrameThe dataframe you want to bucketcolumn:stringThe column which the values are used to determine what the descrete interval is assignedbucket_col_name:stringThe name of the new column which contains the discrete intervalsintervals:dictionary where value is a 2-element numeric tuple[{'low': (10,200)}]this would assign the bucketed discrete value of 'low' to any index's target column inbetween or equal to 20 and 200.Caveat: the operand is fixed tolower_value<=col_value<=upper_value.Export()export(file_name, tables)DescriptionExports selected dataframes into an Excel file where each tab is its own dataframe.Inputsfile_name:stringThe path you want to write the file tootables:dictionaryThe key is the tab name, the value is the dfFind_Dtypes()find_dtypes(df)DescriptionFor each column in the DataFrame print the dtypes present inside the series.Inputsdf:dataframeThe DataFrame you want to loop throughFind_Dups()find_dups(df, name)DescriptionFind duplicate records and print them out inside a given df, duplicates must be duplicates accross all columns.Inputsdf:dataframeThe DataFrame you want to loop throughname:stringThe name that will be printed for reference in the logForce_String()force_string(df)DescriptionForce all scalar data to be strings in a given dataframe, mitigates float percision error for database upload.Inputsdf:dataframeThe DataFrame you want to loop throughForce_Object()force_object(df)DescriptionForce all series to be of type object, mitigates merge fail.Inputsdf:dataframeThe DataFrame you want to loop throughIs_In()is_in(df, target_col, isin_list)DescriptionFilters for only things that are inside the list.Inputsdf:DataFrameThe dataframe you want to filtertarget_col:stringThe column you want to apply the logic too for the filter.isin_list:listThe list that the target_col value must equal at least one element.Cast()cast(df, target_col, value)DescriptionAssign value to entire column, creates new column if column does not exist.Caveatdo not use to cast a value which requires mathInputsdf:DataFrameThe dataframe you want to casttarget_col:stringThe column you want to apply the logic too for the cast.value:any primary typeThe scalar value for the entire column.Date_Format()date_format(df, target_col, date_format)DescriptionTurns a columns value into a specific pandas date_time typeInputsdf:DataFrameThe dataframe you want to work withtarget_col:stringThe column you want to alterdate_format:strftime Format Codethe strftime format code which you want to represent your date. see more:https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behaviorAuthorsContributors names and contact infoJosh Teixeira  |jteixeira@cppib.comVersion History0.0.31columns_to_keep default is all, fixed columns_rename bug0.0.27export function added0.0.26date_format function fixed0.0.24README basic documentation complete0.0.17README documentation enhanced0.0.16bucket function added0.0.1Initial beta releaseLicenseThis project is licensed under the MIT License - see the LICENSE.md file for details