Package: randomdataset
RandomDatasetGenerates random datasets for testing and fun.This repository contains a simple library for generating random tabular datasets of virtually any size. It also serves
as an example repository for a Python code base with basic CI/CD integration and tools.Install this library from a git clone:$pipinstall.Data is generated from a YAML schema describing the names of tables/datasets and the fields they have. The YAML file
consists of a sequence of dictionaries used to instantiate objects from the library or from other libraries present
in the Python path. This allows custom code to be injected into the generation process.An example schema is used to generate a list of customer records incustomerschema.yaml:-typename:randomdataset.generators.CSVGeneratornum_lines:10dataset:name:customerstypename:randomdataset.Datasetfields:-name:idtypename:randomdataset.UIDFieldGen-name:FirstNametypename:randomdataset.StrFieldGenlmin:6lmax:14-name:LastNametypename:randomdataset.StrFieldGenlmin:6lmax:14This will create a single dataset "customers" stored in a CSV filecustomers.csv. This file is geneated by invoking
the included command:$generate_datasetcustomerschema.yaml.This generates thecustomers.csvfile:id,FirstName,LastName
0,"QDFFgv4XBd5VW","O1Odro"
1,"Gp4mYq","82IPIChjBALg"
2,"LR7KVudB","HcAPBwM"
3,"6FfWGEYS0Q","5NbspSBJk"
4,"si1Tj0xSBB2","eChYKAaW5aa8R"
5,"DYP6OMerUUFOR","pYNXUTNLqdrv"
6,"ltfnhTgrJF","2Rctye"
7,"1tAoaDl57Lo5","xMkVKt6O"
8,"1yJImoqiwf","IJICD8W6B8k"
9,"XkYgS7","8owHyjR"Repository SetupA relatively simple set of features which link into the code are set up on this repo to ensure good coding practice:Automatic documentation generation is done using ReadTheDocs, seeREADME.mdCI/CD implemented as flake8 and unit test execution using Github Actions, seepython-app.ymlCode coverage is displayed using CodecovBoth ReadTheDocs and Codecov are integrated with the repo as webhooks. These can be setup through their respective sites
which require Github credentials to link with repos.This repo mostly followsGitFlowwith amasterbranch
which is always the current release of the code, and adevbranch that is the development version of the code.
Branch protection rules are in place formasterwhich ensure that code can only be committed to the branch through
reviewed PRs:Require pull request reviews before mergingRequire status checks to pass before merging ("build" action selected)Require branches to be up to date before mergingRequire linear historyInclude administratorsPyPI ReleaseWhenever a new release is made this is uploaded automatically to PyPI using the default Github workflow "Publish Python
Package". To upload to PyPIthese stepsexplain the
process. For this repo the basic steps are:Create account on pypi.orgCreate a wheel file withpython setup.py bdist_wheel, this createsdist/RandomDataset-0.1.0-py3-none-any.whlUpload this package manually to PyPI withpython -m twine upload dist/*(assuming you have twine already installed)Get the API token for the new package and set it to the secretPYPI_API_TOKENin the repository's settingsAdd the workflow file.github/workflows/python-publish.ymlfromhere.Commit changes and create a release for the project, this should upload to PyPI automatically