Package: faker-datasets
BuildFakerproviders based on datasetsfaker-datasetsoffers a building block for seeding the data generation
with existing data.You can create simple providers picking a random entry from a tailored dataset or
assemble complex ones where you generate new combinations from more datasets,
all this while keeping an eye on speed and memory consumption.Let's see how to.Crash courseWe'll use the wonderfulCountries State Cities DBmaintained byDarshan Gada. Download thecitiesand thecountriesdatasets.Basic random pickerCitiesgenerates a city by randomly picking an entry in the cities
dataset. Here the dataset is namedcities, the dataset file iscities.json(adjust to the actual path of the file saved earlier)
and the picker, the method to get a random city, is namedcity.How we define it in filecities_provider.py:fromfaker_datasetsimportProvider,add_dataset@add_dataset("cities","cities.json",picker="city")classCities(Provider):passHow we could use it to generate 10 cities:fromfakerimportFakerfromcities_providerimportCitiesfake=Faker()fake.add_provider(Cities)for_inrange(10):# Use of the picker named in @add_datesetcity=fake.city()print("{name}is in{country_name}".format(**city))One of the many possible outputs:Poiana Cristei is in Romania
Codosera La is in Spain
Jeremoabo is in Brazil
Rodrígo M. Quevedo is in Mexico
Cary is in United States
Locking is in United Kingdom
Mezinovskiy is in Russia
Nesoddtangen is in Norway
Zalesnoye is in Ukraine
Cefa is in RomaniaBecause the data generation is a pseudo-random process, every execution outputs
different results. If you want reproducible outputs, you have to seed the Faker
generator as documentedhere.Customize the random pickerCitiesExis functionally identical toCitiesbut shows how to define
the picker by yourself. Herepicker=is gone from the parameters of@add_datasetbut a newcitymethod is defined.fromfaker_datasetsimportProvider,add_dataset,with_datasets@add_dataset("cities","cities.json")classCitiesEx(Provider):@with_datasets("cities")defcity(self,cities):returnself.__pick__(cities)Note how thecitymethod is decorated with@with_datasets("cities")and how, consequently, it receives the said dataset as parameter.
The call to__pick__just selects a random entry fromcities.Matching a criteriumCitiesFromCountryexploits the custom picker to return only cities from a
given country. A first implementation could just discard cities from any
other country, getting slower with increasing bad luck.fromfaker_datasetsimportProvider,add_dataset,with_datasets@add_dataset("cities","cities.json")classCitiesFromCountry(Provider):@with_datasets("cities")defcity(self,cities,country_name):whileTrue:city=self.__pick__(cities)ifcity["country_name"]==country_name:returncityIt's better to limit to the number of attempts though otherwise ifcountry_nameis misspelled the picker would enter in an infinite loop.fromfaker_datasetsimportProvider,add_dataset,with_datasets@add_dataset("cities","cities.json")classCitiesFromCountry(Provider):@with_datasets("cities")defcity(self,cities,country_name,max_attempts=10000):whilemax_attempts:city=self.__pick__(cities)ifcity["country_name"]==country_name:returncitymax_attempts-=1raiseValueError("Run out of attempts")Or, with same results, use thematch=andmax_attempts=parameters of__pick__.fromfaker_datasetsimportProvider,add_dataset,with_datasets@add_dataset("cities","cities.json")classCitiesFromCountry(Provider):@with_datasets("cities")defcity(self,cities,country_name):# match tells to __picker__ whether the city is good or notmatch=lambdacity:city["country_name"]==country_namereturnself.__pick__(cities,match=match,max_attempts=10000)If you know ahead which country you are interested in, say Afghanistan,
you can use the@with_matchpicker decorator. It produces a new index
of only matching entries and the picking speed is again constant and
independent from bad luck.fromfaker_datasetsimportProvider,add_dataset,with_datasets,with_match@add_dataset("cities","cities.json")classCitiesFromCountry(Provider):@with_datasets("cities")@with_match(lambdacity:city["country_name"]=="Afghanistan")defafghan_city(self,cities):returnself.__pick__(cities)At such conditions though it's maybe better to massage your dataset and
leave only the entries matching your criteria.Using multiple datasetsCitiesAndCountriesfuses two datasets for more advanced matches. Note
how@add_datasetmakes multiple datasets available to the provider
and@with_datasetspasses them to the given picker.fromfaker_datasetsimportProvider,add_dataset,with_datasets,with_match@add_dataset("cities","cities.json")@add_dataset("countries","countries.json")classCitiesAndCountries(Provider):@with_datasets("cities","countries")defcity_by_region(self,cities,countries,region):defmatch(city):# Given a city, find its country info in the countries datasetcountry=next(countryforcountryincountriesifcountry["name"]==city["country_name"])# Check that the country is in the region of interestreturncountry["region"]==regionreturnself.__pick__(cities,match=match,max_attempts=10000)The picker performs the data mix and match so that the region request
is satisfied or an error is signaled.SummaryYou use@add_datasetto attach a dataset to your provider, if you specify
apicker=parameter you'll get for free a random picker of entries.
The more datasets you need, the more@add_datasetyou can use.If you have special needs you can define the pickers for yourself, each
using what datasets are most appropriate among those made available with@add_dataset. You can add as many pickers as you need.A picker can usematch=andmax_attempts=to make the generation respect
some useful criteria.