Package: vose
voseThis is a Cython implemention of Michael Vose'sAlias method. It can be used to perform weighted sampling with replacement of integers inO(1)time. It requires a construction phase that runs inO(n)time, withnbeing the number of integers with associated weights. As far as I know, it's faster than any other method available in Python. But I would love to be proven wrong!I wrote this because I had a specific usecase where I needed to repeatidly sample integers with a weight associated to each integer. I stumbled on Keith Schwarz'sDarts, Dice, and Coins: Sampling from a Discrete Distribution, which is very well written, and decided to use the Alias method. Alas,numpydoesn't seem to have it available, and neither does therandommodule from Python's standard library. There is, however, thevose_samplerpackage, but it is written in pure Python and isn't fast enough for my purposes. I therefore decided to write it in Cython and shamelessly adapted Keith Schmarz'sJava implementation.InstallationMy feeling is that this package is too small to justify putting it on PyPI. Therefore, you have to install it from GitHub. If ever the scope grows larger I might reconsider this.pipinstallgit+https://github.com/MaxHalford/voseThis will installcythonas well asnumpyif you do not already have them installed.UsageYou first have to initialize a sampler with an array of weights. These weights are not required to sum up to 1.>>>importnumpyasnp>>>importvose>>>weights=np.array([.1,.3,.4,.2])>>>sampler=vose.Sampler(weights,seed=42)You can then call the.sample()method to sample a random integer in range[0, n - 1], wherenis the number of weights that were passed.>>>sampler.sample()2You can set thekparameter in order to produce multiple samples.>>>sampler.sample(k=10)array([1,1,2,1,2,1,0,1,3,3])By default, a copy of the weights is made. You can disable this behavior in order to save a few microseconds, but this will modify the provided array.>>>sampler=vose.Sampler(weights,seed=42,copy=False)Note thatvose.Samplerexpects to be provided with amemoryview. In order to pass a list, you have to convert it yourself to a numpy array.>>>weights=[.2,.3,.5]>>>sampler=vose.Sampler(np.array(weights))You can also usevose.Samplerfrom within your own Cython.pyxfile:importnumpyasnpcimportvosecimportnumpyasnpcdefnp.float_t[:]weights=np.array([.2,.3,.5],dtype=float)cdefvose.Samplersamplersampler=vose.Sampler(weights)cdefintsample=sampler.sample_1()cdefnp.int_t[:]samples=sampler.sample_k(10)Note that the latter requires having to include thenumpyheaders in the extension definition of yoursetup.py:fromsetuptoolsimportExtensionfromsetuptoolsimportsetupfromCython.Buildimportcythonizeimportnumpyasnpextension=Extension('*',['your_file.pyx'],include_dirs=[np.get_include()],define_macros=[('NPY_NO_DEPRECATED_API','NPY_1_7_API_VERSION')])setup(ext_modules=cythonize([extension]))Is it reliable?It seems to be working correctly; at least according to the followingchi-squared tests:>>>importnumpyasnp>>>fromscipyimportstats>>>rng=np.random.RandomState(42)>>>k=1000>>>forninrange(3,20):...weights=rng.dirichlet(np.arange(1,n))...sampler=vose.Sampler(weights,seed=42)...samples=sampler.sample(k)...chi2=stats.chisquare(f_obs=np.bincount(samples),f_exp=weights*k)...assertchi2.pvalue>.05Is it fast?Hell yeah. The following graph shows the average time taken to sample one integer for different amounts of weights:As you can see,vose.Samplertakes less than a nanosecond to produce a random integer. Here is the construction time:vose.Sampleris also fast enough to compete withnumpyandrandom, even when including the construction time. The following table summarizes a comparison I made on my laptop, withnbeing the number of weights andkthe number of integers to sample:nknp.random.choicerandom.choicesvose.Samplervose_sampler.VoseAlias3126ns2ns4ns11ns3226ns3ns7ns13ns3326ns3ns7ns14ns31026ns6ns7ns27ns310028ns47ns8ns198ns3100046ns461ns19ns1μs, 887ns30127ns6ns4ns69ns30226ns7ns7ns73ns30327ns7ns8ns72ns301027ns14ns7ns88ns3010031ns63ns8ns256ns30100067ns580ns19ns1μs, 935ns300129ns47ns6ns661ns300229ns47ns9ns659ns300329ns49ns9ns685ns3001029ns54ns9ns685ns30010036ns112ns10ns877ns300100096ns717ns20ns2μs, 599ns3000152ns416ns18ns6μs, 988ns3000250ns420ns21ns7μs, 39ns3000351ns439ns21ns7μs, 102ns30001051ns420ns21ns7μs, 332ns300010059ns496ns23ns7μs, 349ns30001000137ns1μs, 213ns35ns10μs, 190nsIn summary, you probably don't need to be usingvose.Samplerif you only need to sample once, regardless of the number of integers you wish to sample. You want to usevose.Samplerwhen you need to sample in a sequential manner, because at that point the construction time will be amortized. Indeed, this will bring you two orders of magnitude improved speed, when compared to callingnp.random.choiceorrandom.choicesmultiple times.Developmentgitclonehttps://github.com/MaxHalford/vosecdvose
pythonsetup.pybuild_ext--inplace
pytestFurther workThe weights assigned to each integer cannot be modified. Asearch treecan be used as a data structure that supports modifications. This allows modifying weights inO(log(n))time, but means sampling also happens inO(log(n))time. More informationhere.I'm not 100% the memory allocation of the memoryviews is optimal.Initializing avose.Samplerfrom another Cython.pyxfile seems to require some Python interaction; there's probably a way to avoid this.