Package: csample
csample: Sampling library for Pythoncsampleprovides pseudo-random sampling methods applicable when the size
of population is unknown:Use hash-based sampling to fix sampling rateUse reservoir sampling to fix sample sizeHash-based samplingHash-based sampling is a filtering method that tries to approximate random
sampling by using a hash function as a selection criterion.Following list describes some features of the method:Since there are no randomness involved at all, the same data set with the
same sampling rate (and also with the same salt value) always yields
exactly the same result.The size of population doesn’t need to be specified beforehand. It means
that the sampling process can be applied to data stream with unknown size
such as system logs.Here are some real and hypothetical applications:[RFC5475] Sampling and Filtering Techniques for IP Packet Selectionis a well-known application.Online streaming algorithm to select 10% of users for A/B testing.
“Consistent” nature of the algorithm guarantees that any user ID selected
once will always be selected again. There’s no need to maintain a list of
selected user IDs.csampleprovides two sampling functions for a convenience.sample_line()accepts iterable type containing strs:data = [
    'alan',
    'brad',
    'cate',
    'david',
]
samples = csample.sample_line(data, 0.5)sample_tuple()expects tuples instead of strs as a content of
iterable. The third argument 0 indicates a column index:data = [
    ('alan', 10, 5),
    ('brad', 53, 7),
    ('cate', 12, 6),
    ('david', 26, 5),
]
samples = csample.sample_tuple(data, 0.5, 0)In both cases, the function returns immediately with sampled iterable.Reservoir samplingReservoir sampling is a family of randomized algorithms for randomly choosing
a sample of k items from a list S containing n items, where n is either a very
large or unknown number.You can specify random seed to perform reproducible sampling.For more information, readWikipediacsampleprovides single function for reservoir sampling:data = [
    'alan',
    'brad',
    'cate',
    'david',
]
samples = csample.reservoir(data, 2)Resultingsamplescontains two elements randomly choosen from givendata.Note that the function doesn’t return a generator but list, and also won’t
finish until it consume the entire input stream.Also note that, by default, reservoir sampling doesn’t preserve order of original
list which means that following assertion holds in general:population = [0, 1, 2, 3, 4, 5]
samples = csample.reservoir(population, 3)
assert sorted(samples) != samplesTo maintain original order, providekeep_order=Trueparameter:population = [0, 1, 2, 3, 4, 5]
samples = csample.reservoir(population, 3, keep_order=True)
assert sorted(samples) == samplesAPI documentationRead thefull API documentation.Command-line interfacecsamplealso provides command-line interface.Following command prints 50% sample from 100 integers:> seq 100 | csample -r 0.5To see more options use--helpcommand-line argument:> csample --helpHash functionsIn order to obtain fairly random/unbiased sample, it is critical to use suitable
hash function.There could be many criteria such asavalanche effect.
For those who are interested, see link below:Empirical Evaluation of Hash Functions for Multipoint MeasurementsHash-based sampling implemented incsamplecurrently supportsxxhashandspooky.InstallationInstalling csample is easy:pip install csampleor download the source and run:python setup.py install