Package: lkauto
LensKit-AutoLensKit-Auto is built as a wrapper around the PythonLensKitrecommender-system library. It automates algorithm selection and hyper parameter optimization an can build ensemble
models based on the LensKit models.ResourcesDocumentation:LensKit-Auto DocumenatationRecSys23 Demo:RecSys23 DemoRecSys23 Demo Video:RecSys23 Demo VideoInstallPip Install:pipinstalllkautoConda InstallCreate conda environmentInstall pipFollow the Pip Install subchapter to install lenskit-Auto in your conda environmentGetting StartedLensKit-Auto is built as a wrapper around the PythonLensKitrecommender-system library. It automates algorithm selection and hyper parameter optimization an can build ensemble
models based on LensKit models.Standard use-caseIn the standard use-case you just need to call a single function to get the best performing model for your dataset. It
is eitherfromlkauto.lkautoimportget_best_recommender_modelget_best_recommender_model()(train=train_split)for the recommendation use-case orfromlkauto.lkautoimportget_best_prediction_modelget_best_prediction_model(train=train_split)for the prediction use-caseExamples and Advanced Use-CasesLensKit-Auto allows three application scenarios:Note: All application scenarios apply to Top-N ranking prediction and rating prediction use-cases.Scenario 1:LensKit-Auto performs combined algorithm selection and hyperparameter optimization for a given
dataset.Scenario 2:LensKit-Auto performs hyperparameter optimization on a single algorithm for a given dataset.Scenario 3:LensKit-Auto performs combined algorithm selection and hyperparameter optimization for a specified set
of algorithms and/or different hyperparameter ranges for the provided dataset.In order to take advantage of LensKit-Auto, a developer needs to read in a dataset.fromlenskit.datasetsimportML100Kml100k=ML100K('path_to_file')ratings=ml100k.ratingsratings.name='ml_100k'Furthermore, it is suggested, that we take advantage of the Filer to control the LensKit-Auto outputfromlkauto.utils.filerimportFilerfiler=Filer('output/')Top-N ranking predictionFirst, we need to split the data in a train and test split to evaluate our model. The train-test splits can be performed
based on data rows or user data. For the rating prediction example we are splitting the data based on user data.importlenskit.crossfoldasxffromlenskit.batchimportrecommendfromlenskitimporttopnfromlkauto.lkautoimportget_best_recommender_model# User based data-splitfori,tpinenumerate(xf.partition_users(ratings,1,xf.SampleN(5))):train_split=tp.train.copy()test_split=tp.test.copy()# Fixme: INSERT SCENARIO CODE HERE# fitmodel.fit(train_split)# recommendrecs=recommend(algo=model,users=test_split['user'].unique(),n=5,n_jobs=1)# initialize RecListAnalysisrla=topn.RecListAnalysis()# add precision metricrla.add_metric(topn.ndcg)# compute scoresscores=rla.compute(recs,test_split,include_missing=True)Rating PredictionFirst, we need to split the data in a train and test split to evaluate our model. The train-test splits can be performed
based on data rows or user data. For the rating prediction example we are splitting the data based on the data rows. The
Top-N ranking predicion example showcases the data-split based on user data.fromlenskit.metrics.predictimportrmsefromlenskit.crossfoldimportsample_rowsfromlkauto.lkautoimportget_best_prediction_modeltrain_split,test_split=sample_rows(ratings,None,25000)# Fixme: INSERT SCENARIO CODE HEREmodel.fit(train_split)predictions=model.predict(test_split)root_mean_square_error=rmse(predictions,test_split['rating'])Scenario 1Scenario 1 describes the fully automated combined algorithm selection and hyperparameter optimization (CASH problem).
This scenario is recommended for inexperienced developers who have no or little experience in model selection.LensKit-Auto performs the combined algorithm selection and hyperparameter optimization with a single function call.model,config=get_best_recommender_model(train=train_split,filer=filer)Note: As described above, theget_best_recommender_model()is used for Top-N ranking prediction. If you want to find a
predictor instead of a recommender, replace the function call withget_best_prediction_model()Theget_best_recommender_model()orget_best_prediction_model()function call will return the best performing model,
with tuned hyperparameters and a configuration dictionary that contains all information about the model. In the Scenario
1 use-case the model is chosen out of all LensKit algorithms with hyperparameters within the LensKit-Auto defaulthyperparameter range. We can use the model in the exact same way like a regular LensKit model.Scenario 2In Senario 2 we are going to perform hyperparameter optimization on a single algorithm. First we need to define our
custom configuration space with just a single algorithm included.fromConfigSpaceimportConstantfromlkauto.algorithms.item_knnimportItemItem# initialize ItemItem ConfigurationSpacecs=ItemItem.get_default_configspace()cs.add_hyperparameters([Constant("algo","ItemItem")])# set a random seed for reproducible resultscs.seed(42)# Provide the ItemItem ConfigurationSpace to the get_best_recommender_model function.model,config=get_best_recommender_model(train=train_split,filer=filer,cs=cs)Note: As described above, theget_best_recommender_model()is used for Top-N ranking prediction. If you want to find a
predictor instead of a recommender, replace the function call withget_best_prediction_model()Theget_best_recommender_model()orget_best_prediction_model()function call will return the best performing
ItemItem model. Besides the model, theget_best_recommender_model()function returns a configuration dictionary with
all information about the model.Scenario 3Scenario 3 describes the automated combined algorithm selection and hyperparameter optimization of a custom
configuration space. A developer that wants to use Scenario 3 needs to provide hyperparameter ranges for the
hyperparameter optimization process.First, a parent-ConfigurationSpace needs to be initialized. All algorithm names need to be added to the
parent-ConfigurationSpace categoricalalgohyperparameter.fromConfigSpaceimportConfigurationSpace,Categorical# initialize ItemItem ConfigurationSpaceparent_cs=ConfigurationSpace()# set a random seed for reproducible resultsparent_cs.seed(42)# add algorithm names as a constantparent_cs.add_hyperparameters([Categorical("algo",["ItemItem","UserUser"])])Afterward, we need to build theItemItemandUserUsersub-ConfigurationSpace.We can use the default sub-ConfigurationSpace from LensKit-Auto and add it to the parent-ConfigurationSpace:fromlkauto.algorithms.item_knnimportItemItem# get default ItemItem ConfigurationSpaceitem_item_cs=ItemItem.get_default_configspace()# Add sub-ConfigurationSpace to parent-ConfigurationSpaceparent_cs.add_configuration_space(prefix="ItemItem",delimiter=":",configuration_space=item_item_cs,parent_hyperparameter={"parent":parent_cs["algo"],"value":"ItemItem"},)Or we can build our own ConfigurationSpace for a specific algorithm.fromConfigSpaceimportConfigurationSpacefromConfigSpaceimportInteger,Float,Constant# first we initialize hyperparameter objects for all hyperparameters that we want to optimizennbrs=Constant('nnbrs',1000)min_nbrs=Integer('min_nbrs',bounds=(1,50),default=1)min_sim=Float('min_sim',bounds=(0,0.1),default=0)# Then, we initialize the sub-ConfigurationSpace and add the hyperparameters to ituser_user_cs=ConfigurationSpace()user_user_cs.add_hyperparameters([nnbrs,min_nbrs,min_sim])# Last, we add the user_user_cs to the parent-ConfigurationSpaceparent_cs.add_configuration_space(prefix="UserUser",delimiter=":",configuration_space=user_user_cs,parent_hyperparameter={"parent":parent_cs["algo"],"value":"UserUser"},)After creating the parent-ConfigurationSpace, we can use it in the same way like Scenario 2# Provide the parent-ConfigurationSpace to the get_best_recommender_model function.model,config=get_best_recommender_model(train=train_split,filer=filer,cs=parent_cs)Note: As described above, theget_best_recommender_model()is used for Top-N ranking prediction. If you want to find a
predictor instead of a recommender, replace the function call withget_best_prediction_model()Experiments on Default ConfigurationsThe experiments to gather some hyperparameters for LensKit-Autos default configuration are described here:Experiments