Package: distrel
DistrelDistrel (distribution relations) is a package used to approximate the relationship between dependent distributions when only the properties of the distribution are known.Example use caseSuppose we have 3 parameters with known distributions, such that: $a \sim \mathcal{N}(\mu_a, \sigma_a)$, $b \sim \mathcal{N}(\mu_b, \sigma_b)$ and $c \sim \mathcal{N}(\mu_c, \sigma_c)$ and distributions are related by $c = a \times b$.Distrel attempts to solve the question, how do we correctly pseudo-randomly sample from $a$ and $b$ such then $c$ follows the correct distribution.How it worksDistrel is a wrapper around a simple neural network written inpytorchthat finds the function $f_{\mu}(a)$ and $f_{\sigma}(a)$ from:$b \sim \mathcal{N}(f_{\mu}(a), f_{\sigma}(a))$such that:$a \times b \sim \mathcal{N}(\mu_c, \sigma_c)$That is, Distrel assumes that $a$ and $b$ are weakly related and hence samples $b$ from a normal distribution defined by a particular value of $a$.Below is a diagram of the distrel neural network architecture:InstallationWithpoetrypoetryadddistrelor withpippipinstalldistrelUsageTo make things interesting, let's consider the relationship $a \times b = c$ but each distribution is a unique non-central t-distribution.
The key thing to note is that non-central t-distributions are defined by their degrees of freedom and a non-centrality parameter.
We could use any distribution but currently distributions can only be characterised by their mean, variance, skewness and kurtosis - if more properties are required then please do open an issue.fromscipy.statsimportnctimportdistrel# Defines the degrees of freedom (df) and non-centrality parameter (nc) for each distributiona_df,a_nc=9,52b_df,b_nc=4,30c_df,c_nc=143,90The distribution relation network (drn) needs three things:
- A generator function to generate the input data.
- A calculator function to obtain the 3rd distribution from the output of the generator and the output of the neural network.
- The properties of each of the 3 distributions.# We need to define a generator function that can generate samples of A.# It must take N (the number of samples as it's only argument.# It doesn't matter that gen_a doesn't return a torch tensor, this is accounted for internally.defgen_a(N):returnnct.rvs(a_df,a_nc,size=N)# We also need to define a function that calculates $c$ from $a$ and $b$.# That is, the neural network will take input $a$, generate $b$ and then calculate $c$ from $a$ and# $b$.defcalc_c(a,b):returna*b# Now we define the distribution relation networkdrn=distrel.drn(gen=gen_a,calc=calc_c,seed=42)# We need to define the distribution properties with the following:# nct.stats returns mean, var, skew, kurta_mean,a_var,a_skew,a_kurt=nct.stats(a_df,a_nc,moments='mvsk')b_mean,b_var,b_skew,b_kurt=nct.stats(b_df,b_nc,moments='mvsk')c_mean,c_var,c_skew,c_kurt=nct.stats(c_df,c_nc,moments='mvsk')drn.set_properties({"mean":a_mean,"var":a_var,"skew":a_skew,"kurt":a_kurt},{"mean":b_mean,"var":b_var,"skew":b_skew,"kurt":b_kurt},{"mean":c_mean,"var":c_var,"skew":c_skew,"kurt":c_kurt},)# We could have alternatively initialised the drn with the following# drn = distrel.drn(#    gen=gen_a,#    calc=calc_c,#    properties=[#        {"mean": a_mean, "var": a_var},#        {"mean": b_mean, "var": b_var},#        {"mean": c_mean, "var": c_var},#    ],#)# This also shows that we don't /have/ to test for all four metrics.# Currently, only mean, variance, skewness and kurtosis are supported.Now the drn is ready to train, a tolerance can be provided to allow for early stopping if ALL of
the distribution properties are within a percentage tolerance.
The tolerance defaults to zero which means no early stopping occurs.
Additionally, the maximum number of epochs can be set (this defaults to 100) and then drn will
stop training upon reaching this number regardless of performance.The training starts off with gradient free optimisation before switching to a gradient based
optimiser.
To disable the gradient free optimiser, set budget to 1.You can set the number of samples generated by the generator with N.For reproducible results, you can also set the seed which sets the seed for pytorch, numpy and python's random.drn.trainreturns a boolean, True if the training exited via to early stopping due to
convergence, and False otherwise.converge=drn.train(max_epochs=1000,tol=1e-2,progress_bar=False,lr=1e-3budget=1000,N=1000,seed=42,)If you want more fine grain controlled over the optimiser - you can do that!fromtorch.optimimportLBFGSconverge=drn.train(max_epochs=max_epochs,tol=tol,lr=lr,progress_bar=True,require_closure=True,optim=LBFGS,optim_kwargs={'line_search_fn':'strong_wolfe'},)print(f"Converged?:\t{converge}")Theoptim_kwargsare unpacked when calling the optimiser.
When the optimiser is called internally, setrequire_closureto True if you'd like to call:optim.step(closure)orrequire_closureto False if you'd like to call:optim.step()# To view the network waits simply print the drnprint(drn)Which returns something like:_l2_ mu
in _l1_/
       \_l3_ sigma
out ~ N(mu, sigma)
---

l1:
Linear(in_features=1, out_features=1, bias=True)
Bias: -3.3627309799194336
Weight: 0.027783172205090523

l2:
Linear(in_features=1, out_features=1, bias=True)
Bias: 0.14515426754951477
Weight: 3.0655031204223633

l3:
Linear(in_features=1, out_features=1, bias=True)
Bias: -13.486244201660156
Weight: 2.5820298194885254# To use the drn as a predictor, call the predict method, a seed is optional but recommend for.# reproducible results.new_a=37new_b=drn.predict(37,seed=42)new_c=drn.calc(new_a,new_b)# Wrapps around the calc_c functionContributingPlease feel free to submit an issue for something you'd like to see fixed or implemented.
Alternatively merge requests are always welcome!CopyingThis project is licenced under the GNU GPL version 3 license - to read the license seeLICENSE.