Package: asbe
Automatic Stopping for Batch-mode ExperimentationCreated with nbdev by Zoltan PuhaInstallpython -m pip install  git+https://github.com/puhazoli/asbeHow to useASBE builds on the functional views of modAL, where an AL algorithm can
be run by putting together pieces. You need the following ingredients: -
an ITE estimator (ITEEstimator()), - an acquisition function, - and an
assignment function. - Additionaly, you can add a stopping criteria to
your model. If all the above are defined, you can construct anASLearner, which will help you in the active learning process.fromasbe.baseimport*fromasbe.modelsimport*fromsklearn.linear_modelimportLogisticRegressionfromsklearn.ensembleimportRandomForestClassifierimportnumpyasnpN=1000X=np.random.normal(size=N*2).reshape((-1,2))t=np.random.binomial(n=1,p=0.5,size=N)y=np.random.binomial(n=1,p=1/(1+np.exp(X[:,1]*2+t*3)))ite=1/(1+np.exp(X[:,1]*2+t*3))-1/(1+np.exp(X[:,1]*2))a=BaseITEEstimator(LogisticRegression(solver="lbfgs"))a.fit(X_training=X,t_training=t,y_training=y)Learning activelySimilarly, you can create anBaseActiveLearner,
for which you will initialize the dataset and set the preferred modeling
options. Let’s see how it works: - we will use XBART to model the
treatment effect with a one-model approach - we will use expected model
change maximization - for that, we need an approximate model, we will
use theSGDRegressorYou can call.fit()on theBaseActiveLearner,
which will by default fit the training data supplied. To select new
units from the pool, you just need to call thequery()method, which
will return the selectedXand thequery_ixof these units.BaseActiveLearnerexpects then2argument, which tells how many units are queried at
once. For sequential AL, we can set this to 1. Additionally, some query
strategies can require different treatment effect estimates - EMCM needs
uncertainty around the ITE. We can explicitly tell the theBaseITEEstimatorto return all the predicted treatment effects. Then, we can teach the
newly acquired units to the learner, by calling theteachfunction.
Thescorefunction provides an evaluation of the given learner.fromsklearn.model_selectionimporttrain_test_splitfromsklearn.linear_modelimportSGDRegressorfromcopyimportdeepcopyimportpandasaspdX_train,X_test,t_train,t_test,y_train,y_test,ite_train,ite_test=train_test_split(X,t,y,ite,test_size=0.8,random_state=1005)ds={"X_training":X_train,"y_training":y_train,"t_training":t_train,"ite_training":np.zeros_like(y_train),"X_pool":deepcopy(X_test),"y_pool":deepcopy(y_test),"t_pool":deepcopy(t_test),"ite_pool":np.zeros_like(y_test),"X_test":X_test,"y_test":y_test,"t_test":t_test,"ite_test":ite_test}asl=BaseActiveLearner(estimator=BaseITEEstimator(model=RandomForestClassifier(),two_model=False),acquisition_function=BaseAcquisitionFunction(),assignment_function=BaseAssignmentFunction(),stopping_function=None,dataset=ds)asl.fit()X_new,query_idx=asl.query(no_query=10)asl.teach(query_idx)preds=asl.predict(asl.dataset["X_test"])asl.score()0.34842037641629464asl=BaseActiveLearner(estimator=BaseITEEstimator(model=RandomForestClassifier(),two_model=True),acquisition_function=[BaseAcquisitionFunction(),BaseAcquisitionFunction(no_query=20)],assignment_function=BaseAssignmentFunction(),stopping_function=None,dataset=ds,al_steps=3)resd=pd.DataFrame(asl.simulate(metric="decision"))resd.plot()<AxesSubplot:>