Package: aboleth
A bare-bonesTensorFlowframework forBayesiandeep learning and Gaussian process approximation[1]with
stochastic gradient variational Bayes inference[2].FeaturesSome of the features of Aboleth:Bayesian fully-connected, embedding and convolutional layers using SGVB[2]for inference.Random Fourier and arc-cosine features for approximate Gaussian processes.
Optional variational optimisation of these feature weights as per[1].Imputation layers with parameters that are learned as part of a model.Noise Contrastive Priors[3]for better out-of-domain uncertainty
estimation.Very flexible construction of networks, e.g. multiple inputs, ResNets etc.Compatible and interoperable with other neural net frameworks such asKeras(see thedemosfor more
information).Why?The purpose of Aboleth is to provide a set of high performance and light weight
components for building Bayesian neural nets and approximate (deep) Gaussian
process computational graphs. We aim forminimalabstraction over pure
TensorFlow, so you can still assign parts of the computational graph to
different hardware, use your own data feeds/queues, and manage your own
sessions etc.Here is an example of building a simple Bayesian neural net classifier with one
hidden layer and Normal prior/posterior distributions on the network weights:importtensorflowastfimportabolethasab# Define the network, ">>" implements function composition,# the InputLayer gives a kwarg for this network, and# allows us to specify the number of samples for stochastic# gradient variational Bayes.net=(ab.InputLayer(name="X",n_samples=5)>>ab.DenseVariational(output_dim=100)>>ab.Activation(tf.nn.relu)>>ab.DenseVariational(output_dim=1))X_=tf.placeholder(tf.float,shape=(None,D))Y_=tf.placeholder(tf.float,shape=(None,1))# Build the network, nn, and the parameter regularization, klnn,kl=net(X=X_)# Define the likelihood modellikelihood=tf.distributions.Bernoulli(logits=nn).log_prob(Y_)# Build the final loss function to use with TensorFlow trainloss=ab.elbo(likelihood,kl,N)# Now your TensorFlow training code here!...At the moment the focus of Aboleth is on supervised tasks, however this is
subject to change in subsequent releases if there is interest in this
capability.InstallationNOTE: Aboleth is aPython 3library only. Some of the functionality
within it depends on features only found in python 3. Sorry.To get up and running quickly you can use pip and get the Aboleth package fromPyPI:$ pip install abolethFor the best performance on your architecture, we recommend installingTensorFlow from sources.Or, to install additional dependencies required by thedemos:$ pip install aboleth[demos]To install in develop mode with packages required for development we recommend
you clone the repository from GitHub:$ git clone git@github.com:data61/aboleth.gitThen in the directory that you cloned into, issue the following:$ pip install -e .[dev]Getting StartedSee thequick start guideto get started, and
for more in depth guide, have a look at ourtutorials.
Also see thedemosfolder for more
examples of creating and training algorithms with Aboleth.The full project documentation can be found onreadthedocs.References[1](1,2)Cutajar, K. Bonilla, E. Michiardi, P. Filippone, M. Random Feature
Expansions for Deep Gaussian Processes. In ICML, 2017.[2](1,2)Kingma, D. P. and Welling, M. Auto-encoding variational Bayes. In ICLR,
2014.[3]Hafner, D., Tran, D., Irpan, A., Lillicrap, T. and Davidson, J., 2018.
Reliable Uncertainty Estimates in Deep Neural Networks using Noise
Contrastive Priors. arXiv preprint arXiv:1807.09289.LicenseCopyright 2017 CSIRO (Data61)Licensed under the Apache License, Version 2.0 (the “License”);
you may not use this file except in compliance with the License.
You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an “AS IS” BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.