Package: tablefaker
Table Fakertablefaker is a versatile Python package that empowers you to effortlessly create realistic but synthetic table data for a wide range of applications. If you need to generate test data for software development, this tool simplifies the process with an intuitive schema definition in YAML format.Key FeaturesSchema Definition:Define your target schema using a simple YAML file. Specify the structure of your tables, column names, fake data generation code, and relationships. You can define multiple tables in a yaml file.Faker and Randomization:Leverage the power of the Faker library and random data generation to create authentic-looking fake data that mimics real-world scenarios.Multiple Output Formats:Generate fake data in various formats to suit your needsPandas DataframeSql insert scriptCSV FileParquet FileJSON FileExcel FileInstallationpipinstalltablefakerSample Yaml Fileversion: 1
config:
  locale: en_US
tables:
  - table_name: person
    row_count: 10
    columns:
      - column_name: id
        data: row_id
      - column_name: first_name
        data: fake.first_name()
      - column_name: last_name
        data: fake.last_name()
      - column_name: age
        data: fake.random_int(18, 90)
      - column_name: dob
        data: fake.date_of_birth()
        null_percentage: 0.20
      - column_name: salary
        data: None                # NULL
      - column_name: height
        data: "\"170 cm\""        # string
      - column_name: weight
        data: 150                 # number
  - table_name: employee
    row_count: 5
    columns:
      - column_name: id
        data: row_id
      - column_name: person_id
        data: fake.random_int(1, 10)
      - column_name: hire_date
        data: fake.date_between()
      - column_name: school
        data: fake.school_name()  # custom providerfull yml exampleSample Codeimporttablefaker# exports to current folder in csv formattablefaker.to_csv("test_table.yaml")# exports to sql insert into scripts to insert to your databasetablefaker.to_sql("test_table.yaml")# exports all tables in json formattablefaker.to_json("test_table.yaml","./target_folder")# exports all tables in parquet formattablefaker.to_parquet("test_table.yaml","./target_folder")# exports only the first table in excel formattablefaker.to_excel("test_table.yaml","./target_folder/target_file.xlsx")# you can use customer faker providerfromfaker_educationimportSchoolProvidertablefaker.to_csv("test_table.yaml","./target_folder",fake_provider=SchoolProvider)# multiple custom provider in list also worksSample CLI CommandYou can use tablefaker in your terminal for adhoc needs or shell script to automate fake data generation.Faker custom providers and custom functions are not supported in CLI.# exports to current folder in csv formattablefaker--configtest_table.yaml# exports as sql insert into script filestablefaker--configtest_table.yaml--file_typesql# exports to current folder in excel formattablefaker--configtest_table.yaml--file_typeexcel# exports all tables in json formattablefaker--configtest_table.yaml--file_typejson--target./target_folder# exports only the first tabletablefaker--configtest_table.yaml--file_typeparquet--target./target_folder/target_file.parquetSample CSV Outputid,first_name,last_name,age,dob,salary,height,weight
1,John,Smith,35,1992-01-11,,170 cm,150
2,Charles,Shepherd,27,1987-01-02,,170 cm,150
3,Troy,Johnson,42,,170 cm,150
4,Joshua,Hill,86,1985-07-11,,170 cm,150
5,Matthew,Johnson,31,1940-03-31,,170 cm,150Sample Sql OutputINSERTINTOemployee(id,person_id,hire_date,title,salary,height,weight,school,level)VALUES(1,4,'2020-10-09','principal engineer',NULL,'170 cm',150,'ISLIP HIGH SCHOOL','level 2'),(2,9,'2002-12-20','principal engineer',NULL,'170 cm',150,'GUY-PERKINS HIGH SCHOOL','level 1'),(3,2,'1996-01-06','principal engineer',NULL,'170 cm',150,'SPRINGLAKE-EARTH ELEM/MIDDLE SCHOOL','level 3');Custom FunctionsWith Table Faker, you have the flexibility to provide your own custom functions to generate column data. This advanced feature empowers developers to create custom fake data generation logic that can pull data from a database, API, file, or any other source as needed. You can also supply multiple functions in a list, allowing for even more versatility. The custom function you provide should return a single value, giving you full control over your synthetic data generation.fromtablefakerimporttablefakerfromfakerimportFakerfake=Faker()defget_level():returnf"level{fake.random_int(1,5)}"tablefaker.to_csv("test_table.yaml","./target_folder",custom_function=get_level)Add get_level function to your yaml fileversion: 1
config:
  locale: en_US
tables:
  - table_name: employee
    row_count: 5
    columns:
      - column_name: id
        data: row_id
      - column_name: person_id
        data: fake.random_int(1, 10)
      - column_name: hire_date
        data: fake.date_between()
      - column_name: level
        data: get_level() # custom functionFaker Functions Listhttps://faker.readthedocs.io/en/master/providers.html#Bug Report & New Feature Requesthttps://github.com/necatiarslan/table-faker/issues/newTODOForeign keyParquet Column TypesNice To HaveInline schema definitionJson schema filePyarrow tableUse Logging packageException ManagementFollow me on linkedin to get latest newshttps://www.linkedin.com/in/necati-arslan/Thanks,Necati ARSLANnecatia@gmail.com