Package: rdt-identity
This repository is part ofThe Synthetic Data Vault Project, a project fromDataCebo.OverviewRDT (Reversible Data Transforms) is a Python library that transforms raw data into fully numerical
data, ready for data science. The transforms are reversible, allowing you to convert from numerical
data back into your original format.InstallInstallRDTusingpiporconda. We recommend using a virtual environment to avoid
conflicts with other software on your device.pipinstallrdtcondainstall-cconda-forgerdtFor more information about using reversible data transformations, visit theRDT Documentation.QuickstartIn this short series of tutorials we will guide you through a series of steps that will
help you getting started usingRDTto transform columns, tables and datasets.Load the demo dataAfter you have installed RDT, you can get started using the demo dataset.fromrdtimportget_democustomers=get_demo()This dataset contains some randomly generated values that describe the customers of an online
marketplace.last_login email_optin credit_card  age  dollars_spent
0 2021-06-26       False        VISA   29          99.99
1 2021-02-10       False        VISA   18            NaN
2        NaT       False        AMEX   21           2.50
3 2020-09-26        True         NaN   45          25.00
4 2020-12-22         NaN    DISCOVER   32          19.99Let's transform this data so that each column is converted to full, numerical data ready for data
science.Creating the HyperTransformer & configTheHyperTransformeris capable of transforming multi-column datasets.fromrdtimportHyperTransformerht=HyperTransformer()TheHyperTransformerneeds to know about the columns in your dataset and which transformers to
apply to each. These are described by a config. We can ask theHyperTransformerto automatically
detect it based on the data we plan to use.ht.detect_initial_config(data=customers)This will create and set the config.Config:
{
    "sdtypes": {
        "last_login": "datetime",
        "email_optin": "boolean",
        "credit_card": "categorical",
        "age": "numerical",
        "dollars_spent": "numerical"
    },
    "transformers": {
        "last_login": "UnixTimestampEncoder()",
        "email_optin": "BinaryEncoder()",
        "credit_card": "FrequencyEncoder()",
        "age": "FloatFormatter()",
        "dollars_spent": "FloatFormatter()"
    }
}Thesdtypesdictionary describes the semantic data types of each of your columns and thetransformersdictionary describes which transformer to use for each column. You can customize the
transformers and their settings. (See theTransformers Glossaryfor more information).Fitting & using the HyperTransformerTheHyperTransformerreferences the config while learning the data during thefitstage.ht.fit(customers)Once the transformer is fit, it's ready to use. Use the transform method to transform all columns
of your dataset at once.transformed_data=ht.transform(customers)last_login.value  email_optin.value  credit_card.value  age.value  dollars_spent.value
0      1.624666e+18                0.0                0.2         29                99.99
1      1.612915e+18                0.0                0.2         18                36.87
2      1.611814e+18                0.0                0.5         21                 2.50
3      1.601078e+18                1.0                0.7         45                25.00
4      1.608595e+18                0.0                0.9         32                19.99TheHyperTransformerapplied the assigned transformer to each individual column. Each column
now contains fully numerical data that you can use for your project!When you're done with your project, you can also transform the data back to the original format
using thereverse_transformmethod.original_format_data=ht.reverse_transform(transformed_data)last_login email_optin credit_card  age  dollars_spent
0        NaT       False        VISA   29          99.99
1 2021-02-10       False        VISA   18            NaN
2        NaT       False        AMEX   21            NaN
3 2020-09-26        True         NaN   45          25.00
4 2020-12-22       False    DISCOVER   32          19.99What's Next?To learn more about reversible data transformations, visit theRDT Documentation.The Synthetic Data Vault Projectwas first created at MIT'sData to AI Labin 2016. After 4 years of research and traction with enterprise, we
createdDataCeboin 2020 with the goal of growing the project.
Today, DataCebo is the proud developer of SDV, the largest ecosystem for
synthetic data generation & evaluation. It is home to multiple libraries that support synthetic
data, including:🔄 Data discovery & transformation. Reverse the transforms to reproduce realistic data.🧠 Multiple machine learning models -- ranging from Copulas to Deep Learning -- to create tabular,
multi table and time series data.📊 Measuring quality and privacy of synthetic data, and comparing different synthetic data
generation models.Get started using the SDV package-- a fully
integrated solution and your one-stop shop for synthetic data. Or, use the standalone libraries
for specific needs.