Package: maxent-infer
Maximum Entropy InferenceThis provides a Keras implementation of maximum entropy simulation based inference. The point of this package is to reweight outcomes from a simulator to agree with observations, rather than trying to optimize your simulators input parameters. The simulator must necessarily give multiple outcomes - either because you're trying multiple sets of input parameters or it has intrinsic noise. The assumption of this model is that your simulator is approximately correct. The observations being fit could have come the distribution of outcomes of your simulator.About maximum entropyMaximum entropy reweighting is a straightforward black box method that can be applied to arbitrary simulators with few observations. Its runtime is independent of the number of parameters used by the simulator, and it has been shown analytically to minimally change the prior to agree with observations. This method fills a niche in the small-data, high-complexity regime of SBI parameter inference, because it accurately and minimally biases a prior to match observations and does not scale in runtime with the number of model parameters.Installationpipinstallmaxent-inferQuick StartA Pandas Data FrameConsider a data frame representing outcomes from our prior model/simulator. We would like to
regress these outcomes to data.importpandasaspdimportnumpyasnpimportmaxentdata=pd.read_csv('data.csv')Perhaps we have a single observation we would like to match. We can define it with a restraint. Let's say
the observation corresponds to the values in column 3.defobserve(single_row):returnsingle_row[3]r=maxent.Restraint(observe,target=1.5)Now we'll fit using MaxEntmodel=maxent.MaxentModel(r)model.compile()model.fit(data.values)We now have a set of weights -- one per row -- that we can use to compute other expressions.
For example, here is the most likely outcome (mode)i=np.argmax(model.traj_weights)mode=data.iloc[i,:]Here are the new column averagescol_avg=np.sum(data.values*model.traj_weights[:,np.newaxis],axis=0)A simulatorHere we show how to take a random walk simulator and usemaxentto have reweight the random walk so that the average end is at x = 2, y= 1.# simulatedefrandom_walk_simulator(T=10):x=[0,0]traj=np.empty((T,2))foriinrange(T):traj[i]=xx+=np.random.normal(size=2)returntrajN=500trajs=[random_walk_simulator()for_inrange(N)]# now have N x T x 2 tensortrajs=np.array(trajs)# here is a plot of these trajectories# we want the random walk to have average end of 2,1rx=maxent.Restraint(lambdatraj:traj[-1,0],target=2)ry=maxent.Restraint(lambdatraj:traj[-1,1],target=1)# create model by passing in restraintsmodel=maxent.MaxentModel([rx,ry])# convert model to be differentiable/GPU (if available)model.compile()# fit to datah=model.fit(trajs)# can now compute other averages properties# with new weightsmodel.traj_weights# plot showing weights of trajectories:Further ExamplesYou can find the examples used in the manuscript, including comparisons with competing methods:here. These examples use the latest package versions, so the figures will not exactly match those in the manuscript. If you would like to reproduce the manuscript exactly, install the packages inpaper/requirements.txtand execute the notebooks inpaper(this is the output from thepaperworkflow above).CitationSee preprintand the citation:@article{barrett2022simulation,title={Simulation-Based Inference with Approximately Correct Parameters via Maximum Entropy},author={Barrett, Rainier and Ansari, Mehrad and Ghoshal, Gourab and White, Andrew D},journal={Machine Learning: Science and Technology},year={2022}}License