Package: cf-step
CF STEP - Incremental Collaborative FilteringIncremental learning for recommender systemsCF STEP is an open-source library, written in python, that enables fast implementation of incremental learning recommender systems. The library is a by-product of the research projectCloudDBAppliance.InstallRunpip install cf-stepto install the library in your environment.How to useFor this example, we will use the popularmovielensdataset. The dataset has collected and made available rating data sets from theMovieLensweb site. The data sets were collected over various periods of time, depending on the size of the set.First let us load the data in a pandasDataFrame. We assume that the reader has downloaded the 1m movielense dataset and have unziped it in the/tmpfolder.To avoid creating a user and movie vocabularies we turn each user and movie to a categorical feature and use the pandas convenient cat attribute to get the codes# local# load the datacol_names=['user_id','movie_id','rating','timestamp']ratings_df=pd.read_csv('/tmp/ratings.dat',delimiter='::',names=col_names,engine='python')# transform users and movies to categorical featuresratings_df['user_id']=ratings_df['user_id'].astype('category')ratings_df['movie_id']=ratings_df['movie_id'].astype('category')# use the codes to avoid creating separate vocabulariesratings_df['user_code']=ratings_df['user_id'].cat.codes.astype(int)ratings_df['movie_code']=ratings_df['movie_id'].cat.codes.astype(int)ratings_df.head()<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}</style>user_idmovie_idratingtimestampuser_codemovie_code01119359783007600110411661397830210906392191439783019680853313408497830027503177412355597882429102162Using the codes we can see how many users and movies are in the dataset.# localn_users=ratings_df['user_code'].max()+1n_movies=ratings_df['movie_code'].max()+1print(f'There are{n_users}unique users and{n_movies}unique movies in the movielens dataset.')There are 6040 unique users and 3706 unique movies in the movielens dataset.We will sort the data by Timestamp so as to simulate streaming events.# localdata_df=ratings_df.sort_values(by='timestamp')TheStepmodel supports only positive feedback. Thus, we will consider a rating of 5 as positive feedback and discard any other. We want to identify likes with1and dislikes with0.# local# more than 4 -> 1, less than 5 -> 0data_df['preference']=np.where(data_df['rating']>4,1,0)# keep only ones and discard the othersdata_df_cleaned=data_df.loc[data_df['preference']==1]data_df_cleaned.head()<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}</style>user_idmovie_idratingtimestampuser_codemovie_codepreference999873604059359567039546039579110001926040201959567039776039183919999206040213595670405660392071999967604031115956704056603928951999971604025035956704191603923091Following, let us initialize our model.# localnet=SimpleCF(n_users,n_movies,factors=128,mean=0.,std=.1)objective=lambdapred,targ:targ-predoptimizer=SGD(net.parameters(),lr=0.06)device='cuda'iftorch.cuda.is_available()else'cpu'model=Step(net,objective,optimizer,device=device)Finally, let us get 20% of the data to fit the model for bootstrapping and create the Pytorch Dataset that we will use.# localpct=int(data_df_cleaned.shape[0]*.2)bootstrapping_data=data_df_cleaned[:pct]We will create a dataset from our Dataframe. We extract four elements:The user codeThe movie codeThe ratingThe preference# localfeatures=['user_code','movie_code','rating']target=['preference']data_set=TensorDataset(torch.tensor(bootstrapping_data[features].values),torch.tensor(bootstrapping_data[target].values))Create the Pytorch DataLoader that we will use. Batch size should always be1for online training.# localdata_loader=DataLoader(data_set,batch_size=512,shuffle=False)Let us now use thebatch_fit()method of theSteptrainer to bootstrap our model.# localmodel.batch_fit(data_loader)100%|██████████| 89/89 [00:01<00:00, 81.00it/s]Then, to simulate streaming we get the remaining data and create a different data set.# localdata_df_step=data_df_cleaned.drop(bootstrapping_data.index)data_df_step=data_df_step.reset_index(drop=True)data_df_step.head()# create the DataLoaderstream_data_set=TensorDataset(torch.tensor(data_df_step[features].values),torch.tensor(data_df_step[target].values))stream_data_loader=DataLoader(stream_data_set,batch_size=1,shuffle=False)Simulate the stream...# localk=10# we keep only the top 10 recommendationsrecalls=[]known_users=[]withtqdm(total=len(stream_data_loader))aspbar:foridx,(features,preferences)inenumerate(stream_data_loader):itr=idx+1user=features[:,0]item=features[:,1]rtng=features[:,2]pref=preferencesifuser.item()inknown_users:predictions=model.predict(user,k)recall=recall_at_k(predictions.tolist(),item.tolist(),k)recalls.append(recall)model.step(user,item,rtng,pref)else:model.step(user,item,rtng,pref)known_users.append(user.item())pbar.update(1)100%|██████████| 181048/181048 [15:23<00:00, 195.94it/s]Last but not least, we visualize the results of the recall@10 metric, using a moving average window of 5k elements.# localavgs=moving_avg(recalls,5000)plt.title('Recall@10')plt.xlabel('Iterations')plt.ylabel('Metric')plt.ylim(0.,.1)plt.plot(avgs)plt.show()Finally, save the model's weights.# localmodel.save(os.path.join('artefacts','positive_step.pt'))ReferencesVinagre, J., Jorge, A. M., & Gama, J. (2014, July). Fast incremental matrix factorization for recommendation with positive-only feedback. In International Conference on User Modeling, Adaptation, and Personalization (pp. 459-470). Springer, Cham.Hu, Y., Koren, Y., & Volinsky, C. (2008, December). Collaborative filtering for implicit feedback datasets. In 2008 Eighth IEEE International Conference on Data Mining (pp. 263-272). Ieee.