Package: datamaker
DatamakerRapidly build tabular datasets in the following formats:SQL Tables (Generate code for SQL CREATE and INSERT queries)CSVJSONSupports: PostgreSQL, MySQL, MS SQL, SQLiteGetting StartedDatamaker uses core python 3 and has been extensively tested on python 3.6+.PrerequisitesPython 3.6+Installingpip install datamakerBuild a datasetIn this example we build a SQL table with 50 rows and two columns: id, name. The create and
insert queries will be written to the file example.sql.from datamaker.SQLTableGenerator import SQLTableGen


table_name = "Example"
num_rows = 50
column_defs = {
    "id": {"col_type": "int", "is_primary_key": True},
    "name": {"col_type": "varchar", "is_name": True}
}
output_path = "example.sql"

gen = SQLTableGen(table_name, num_rows, column_defs, output_path)

gen.build_sql_doc()DocumentationInput parameterstable_name- This is the name of the table to be creatednum_rows- The number of rows to insert into the tablecolumn_defs- A dictionary object with key value pairs.path- The path to the directory and file name where you want to export the sql
queries or dataset for the specified format.column_defsis a dictionary object of the form:{column_name: {"col_type": type, ...}}Note that col_type is always specified for each column. The definitions of the parameters forcolum_defsis given
below.col_type: int, float, varchar, boolvarchar_length: int (required for MySQL and MS SQL)is_primary_key: bool - True if this field is the primary keyis_name: bool - True if values for this column should come from the names collectionis_email: bool - True if this column is an email. Note that you must have at least one column with is_name = True
to use is_email.is_date: bool - True if this column is a date. Default format is YYYY-mm-ddis_timestamp: bool - True if this column is a timestamp. Default format is YYYY-mm-dd HH:MM:SSis_country: bool - True of this column is a country code. Codes are ISO Alpha-2.is_state: bool - True if this column is a US state abbreviation.is_address: bool - True if this column is a street address.bounds: tuple(lowerbnd, upperbnd) - Forces lowerbnd <= column values <= upperbndfixed_collection: tuple(elem1, elem2, ..., elemn) - Forces column values to be elements of this tuplefixed_map: tuple(column_name, {key: value}) - Forces values for this column to be associated to the value of another
columnis_nullable: bool - True if this column can have null valuesproportion_null: float - The approximate proportion of expected null values in this column. Note that is_nullable
must be true to use proportion_nullNote that using is_primary_key assumes the column you're defining is an integer column starting at 1 and incrementing
by 1 for each row.If you're planning on running queries on a MySQL or MS SQL Server machine then you
need to specify the length of each varchar field using the varchar_length parameter.CSV and JSON generatorsDatamaker can also be used to generate CSV and JSON files.To create a CSV file add the following import statement to your script.from datamaker.CSVGenerator import CSVGenAnd to create a JSON file add the following import to your script.from datamaker.JSONGenerator import JSONGenThe input parameters are no different for CSVGen and JSONGen are the same as given above. If we want to create theExampletable given in the example code above as a CSV or JSON document we only to change the import statement and
instantiate the CSVGen or JSONGen object using the same column_defs dictionary. Then call the appropriate build method
for the given class. The CSVGen class usesbuild_csv_docand the JSONGen class usesbuild_json_doc.from datamaker.CSVGenerator import CSVGen


table_name = "Example"
num_rows = 50
column_defs = {
    "id": {"col_type": "int", "is_primary_key": True},
    "name": {"col_type": "varchar", "is_name": True}
}
output_path = "example.csv"

gen = CSVGen(table_name, num_rows, column_defs, output_path)

gen.build_csv_doc()Here's the sameExampletable built as a JSON document.from datamaker.JSONGenerator import JSONGen


table_name = "Example"
num_rows = 50
column_defs = {
    "id": {"col_type": "int", "is_primary_key": True},
    "name": {"col_type": "varchar", "is_name": True}
}
output_path = "example.json"

gen = JSONGen(table_name, num_rows, column_defs, output_path)

gen.build_json_doc()A more complex exampleIn this example we're generating SQL code to create a MySQL table which tracks orders placed for an imaginary online
store. We'll create a table with the following columns:id - the primary key for this tablename - the name associated to the user who placed this orderorder_ts - the timestamp associated with this orderproduct_id - the id for the product orderedprice - the cost of the product being ordered. Since we want every row with a given product_id to have the same cost we
use a fixed map to implement this columnNote that in this example we're defining the output path as a valid directory instead of a path to a file. In this
case datamaker will automatically generate the name of the file as:table_name.current_date.sqlWheretable_nameis the name of the table andcurrent_dateis today's date.from datamaker.SQLTableGenerator import SQLTableGen


table_name = "Orders"
num_rows = 500
column_defs = {
    "id": {"col_type": "int", "is_primary_key": True},
    "name": {"col_type": "varchar", "varchar_length": 50, "is_name": True},
    "order_ts": {"col_type": "varchar", "varchar_length": 26, "is_timestamp": True},
    "product_id": {"col_type": "int", "bounds": (1, 5)},
    "price": {"col_type": "float", "fixed_map": ("product_id", {1: 5.99, 2: 10.99, 3: 0.99, 4: 99.99, 5: 20})},
}
output_path = "."

gen = SQLTableGen(table_name, num_rows, column_defs, output_path)

gen.build_sql_doc()Future improvements:Add support for sequential fieldsAdd support for column type inferenceAdd support for tsv generation