bayes-torch 0.0.4
A light weight bayes inference framework based on pytorch.

        # Bayes-torch: A light weight bayes inference framework

        

        Though there're a lot of bayes inference modeling lib/language

        such as [stan](https://github.com/stan-dev/stan), 

        [edward](https://github.com/blei-lab/edward) (tensorflow) 

        [pymc](https://github.com/pymc-devs/pymc3) (theano), 

        [pyro](https://github.com/uber/pyro) (pytorch) the relation between

        their computation ground and absract high API is awkward.

        

        So the project is found to implment stan-like API on the flexible

        autograd library, pytorch. It's a light-weight framework, you will

        directly write joint likelihood function to run inference instead of

        fake sampling statment in stan, pymc or ugly style in Edward, 

        weired namebinding in pyro.

        

        ## Example

        

        We can implement following stan model as such:

        

        ```

        data {

            int<lower=1> N;

            real y[N];

        }

        parameters {

            real mu;

        }

        model {

            y ~ normal(mu, 1);

        }

        ```

        

        torch-bayes model code:

        

        ```python

        mu = Parameter(0.0) # optimizing/vb/sampling init value

        sigma = Data(1.0)

        X = Data(_X)

        

        def target():

            target = Normal(mu,sigma).log_prob(X).sum(0)

            return target

        ```

        

        Full code comparing two framework:

        

        ```python

        from bayestorch import Parameter,Data,optimizing,vb,sampling,reset

        import torch

        from torch.distributions import Normal

        

        _X = torch.arange(10)

        print(_X.mean())

        

        # torch-bayes model

        

        mu = Parameter(0.0)

        sigma = Data(1.0)

        X = Data(_X)

        

        def target():

            target = Normal(mu,sigma).log_prob(X).sum(0)

            return target

        

        optimizing(target)

        print(f'optimizing: mu={mu.data}')

        

        res = vb(target)

        q_mu = res.params['mu']

        print(f'vb mu={q_mu["loc"]} omega={q_mu["omega"]} sigma={torch.exp(q_mu["omega"])}')

        

        reset()

        

        mu = Parameter(0.0)

        

        res = vb(target, q_size = 10, n_epoch=200)

        q_mu = res.params['mu']

        print(f'vb mu={q_mu["loc"]} omega={q_mu["omega"]} sigma={torch.exp(q_mu["omega"])}')

        

        trace = sampling(target,trace_length=300)

        mu_trace = torch.tensor([t['mu'].item() for t in trace])

        print('sampling: mu={} sigma={}'.format(torch.mean(mu_trace), torch.std(mu_trace)))

        

        

        # stan model

        

        

        import numpy as np

        import pystan

        stan_code = '''

        data {

            int<lower=1> N;

            real y[N];

        }

        parameters {

            real mu;

        }

        model {

            y ~ normal(mu, 1);

        }

        '''

        sm = pystan.StanModel(model_code = stan_code)

        

        _X = _X.numpy()

        res2 = sm.optimizing(data = dict(N = len(_X), y = _X))

        print(f'optimizing(stan): mu={res2["mu"]}')

        res3 = sm.vb(data = dict(N = len(_X), y = _X))

        res3a=np.array(res3['sampler_params'])

        print(f'vb(stan): mu={res3a[0,:].mean()} sigma={res3a[0,:].std()}')

        

        ```

        

        Enemy location detecting example:

        

        ```python

        # model

        friend = Data(friend_point)

        battle = Data(battle_point)

        enemy = Parameter(enemy_point) # set real value as init value, though maybe a randomed init is more proper

        

        logPC = Data(_logPC)

        

        conflict_threshold = 0.2

        distance_threshold = 1.0

        tense = 10.0

        alpha = 5.0

        prior_threshold = 5.0

        prior_tense = 5.0

        

        def target():

            friend_enemy = torch.cat((friend, enemy),0)

            

            distance = cdist(battle, friend_enemy).min(dim=1)[0]

            

            

            mu = torch.stack([friend.mean(0),enemy.mean(0)],0)

            sd = torch.stack([friend.std(0),enemy.std(0)],0)

            

            conflict = torch.exp(norm_naive_bayes_predict(battle, mu, sd, logPC)).prod(1)

            p = soft_cut_ge(conflict,conflict_threshold, tense = tense) *  \

                soft_cut_le(distance, distance_threshold, tense = tense)

            

            target= torch.log(p).sum(0)

            return target

        

        def target2():

            target1 = target()

            # location prior

            target2 = target1 + enemy.sum(0).sum(0)

            return target2

        ```

        

        <img src="images/example.png">

        

        ## Basic principle

        

        bayes-torch(BT) use `target_f.__globals__` to access and change variables labeled `Parameter` or `Data`. 

        It implies some limit to way to introduce parameters, for example you can't define a list of `Parameter`

        and expect BT can find it.

        

        In `optimizing`, BT run standard SGD algorithm. 

        In `sampling`, BT will frequently replace variable with another(same shape) using HMC. 

        In `vb`, BT will map a variable to a normal variational distribution object, that contain variational

        parameters mu and omega (`omega = log(sigma)`). The last dimention will be used by `vb` innerly.

        So code such as `sum(-1)` or `sum()`(reduce to scaler) will raise error.

        

        However, the thin implementation(`core.py`) only consists of 300- lines, 

        you can always check origin code to figure out what happen anyway.

        

        ## Reference

        

        Automatic Differentiation Variational Inference

        

        Kucukelbir, Alp and Tran, Dustin and Ranganath, Rajesh and Gelman, Andrew and Blei, David M

        

        https://arxiv.org/abs/1603.00788

        

        Fully automatic variational inference of differentiable probability models

        

        Kucukelbir, Alp and Ranganath, Rajesh and Gelman, Andrew and Blei, David

        

        http://andrewgelman.com/wp-content/uploads/2014/12/pp_workshop_nips2014.pdf

        

        The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo.

        

        Hoffman, Matthew D and Gelman, Andrew

        

        https://arxiv.org/abs/1111.4246
Keywords: bayes statistic scientific
Platform: UNKNOWN
Description-Content-Type: text/markdown