Package: cachingalgo
cachingalgoThis project contains implementation different caching algorithms and request generation module to analyze the caching algorithmsInstallationRun the following to install the package:pipinstallcachingalgoor Run the following to install in the Google Colab:# Clone the repo.!gitclonehttps://github.com/SuryaKrishna02/caching-algorithms.git# Change the working directory to the repo root.%cdcachingalgo# Add the repo root to the Python path.importsys,ossys.path.append(os.getcwd())Github Repository NotebooksFull Observation Single Cache:LFU, WLFU, LFULite, CountSketchFull Observation Multiple Cache:LRU, fLRU, LRUmPartial Observation Single Cache:CBMPS, CBSI, CBSILiteTandem Model Static Data:LFU-LFU, LFULite-LFULiteTandem Model Dynamic Data:LU2, LU2-LFU, LU2-LFULiteDynamic Data:LU, LU-LFU, LU-LFULite[C or 2C or 3C], LU-LFULite(V)[C or 2C or 3C]Cost Minimization:CMSR, CMDR, CMDRM, CMDRPRequest Generationfromcachingalgo.request_generation.continuousimportszipf,dzipf,youtube,netflix# To generate requests which follows Static Zipf distribution.# 50,000 requests with library size = 1,61,085 and Zipf shape parameter = 1szdata=szipf(count=50_000,a=1,L=1_61_085)# Probabilty distributionszdataprob=szdata['prob']# Array of requestsszdatareq=szdata['req']# To generate requests which follows Static Zipf distribution but with changing popularity.# 50,000 requests with library size = 1,61,085 and popularity changes for every 5,000 requests# by shifting probabilities of the top 500 items by 50.dzdata=dzipf(count=50_000,a=1,L=1_61_085,req_step=5_000,window=500,top=50,cache_size=10)# Array of requestsdzdatareq=dzdata['req']# Array of size (count//req_step)*cache_size i.e optimal cache for each req_step requestsoptimalcache=dzdata['optim_cache']# To generate requests from the YouTube data.# 50,000 requests with the library size = 1,61,085ytdata=youtube(count=50_000)# Probability distributionytdataprob=ytdata['prob']# Array of requestsytdatareq=ytdata['req']# To generate requests from the Netflix data.# 50,000 requests with the library size = 17,770netdata=netflix(count=50_000)# Probability distributionnetdataprob=netdata['prob']# Array of requestsnetdatareq=netdata['req']We can use any of the four types of requests for analyzing the caching algorithms. In this documentation, we will use requests generated from the YouTube data.Usage of LFU algorithmfromcachingalgo.full_observation.single_cacheimportLFU# Initialising the algorithmalg=LFU(L=100,cache_size=5)# Total no. of requeststotalreq=ytdatareq.shape[0]foriinrange(totalreq):request=ytdatareq[i]# Updates the counteralg.update(request)# Returns the cache at current instantcurrcache=alg.currcache()Usage of WLFU algorithmfromcachingalgo.full_observation.single_cacheimportWLFU# Initialising the algorithmalg=WLFU(L=100,cache_size=5)# Total no. of requeststotalreq=ytdatareq.shape[0]foriinrange(totalreq):request=ytdatareq[i]# Updates the counteralg.update(request)# Returns the cache at current instantcurrcache=alg.currcache()Usage of LFU-Lite algorithmfromcachingalgo.full_observation.single_cacheimportLFULite# Initialising the algorithmalg=LFULite(L=100,cache_size=5)# Total no. of requeststotalreq=ytdatareq.shape[0]foriinrange(totalreq):request=ytdatareq[i]# Updates the counteralg.update(request,i)# Returns the cache at current instantcurrcache=alg.currcache(i)Usage of LRU algorithmfromcachingalgo.full_observation.single_cacheimportLRU# Initialising the algorithmalg=LRU(L=100,cache_size=5)# Total no. of requeststotalreq=ytdatareq.shape[0]foriinrange(totalreq):request=ytdatareq[i]# Updates the counteralg.update(request)# To check whether a request is present in the cachepresent=requestinalgUsage of Count-Sketch algorithmfromcachingalgo.full_observation.single_cacheimportCountSketch# Initialising the algorithmalg=CountSketch(l=6,b=10,L=100)# Total no. of requeststotalreq=ytdatareq.shape[0]foriinrange(totalreq):request=ytdatareq[i]# Updates the counteralg.update(request)# Returns the cache at current instantcurrcache=alg.currcache(request)Usage of f-LRU algorithmfromcachingalgo.full_observation.multiple_cacheimportfLRU# Initialising the algorithmalg=fLRU(L=100,size=5,f=2)# Total no. of requeststotalreq=ytdatareq.shape[0]foriinrange(totalreq):request=ytdatareq[i]# Updates the counteralg.update(request)# To check whether a request is present in the cachepresent=requestinalgUsage of LRU(m) algorithmfromcachingalgo.full_observation.multiple_cacheimportLRUm# Initialising the algorithmalg=LRUm(L=100,size=5,f=2,v=1)# Total no. of requeststotalreq=ytdatareq.shape[0]foriinrange(totalreq):request=ytdatareq[i]# Updates the counteralg.update(request)# To check whether a request is present in the cachepresent=requestinalgUsage of CB-MPS algorithmfromcachingalgo.partial_observation.single_cacheimportCBMPS# Initialising the algorithmalg=CBMPS(L=100,cache_size=5)# Total no. of requeststotalreq=ytdatareq.shape[0]foriinrange(totalreq):request=ytdatareq[i]# Returns the cache at current instantcurrcache=alg.currcache(request)# Updates the counteralg.update(request)Usage of CB-SI algorithmfromcachingalgo.partial_observation.single_cacheimportCBSI,calculate_delta# Calculate the parameter which are to be provided for the CB-SI algorithmsdelta,mu_c=calculate_delta(ytdataprob,cache_size=10)# Initialising the algorithmalg=CBSI(L=100,cache_size=5,delta=delta,mu_c=mu_c)# Total no. of requeststotalreq=ytdatareq.shape[0]foriinrange(totalreq):request=ytdatareq[i]# Returns the cache at current instantcurrcache=alg.currcache(request)# Updates the counteralg.update(request)Usage of CB-SILite algorithmfromcachingalgo.partial_observation.single_cacheimportCBSILite,calculate_delta# Calculate the parameter which are to be provided for the CB-SI algorithmsdelta,mu_c=calculate_delta(ytdataprob,cache_size=10)# Initialising the algorithmalg=CBSILite(L=100,cache_size=5,delta=delta,mu_c=mu_c)# Total no. of requeststotalreq=ytdatareq.shape[0]foriinrange(totalreq):request=ytdatareq[i]# update the counterbankalg.counterbank_update(request)# Returns the cache at current instantcurrcache=alg.currcache(request)# Updates the counteralg.update(request)Usage of LU algorithmfromcachingalgo.full_observation.single_cacheimportLU# Defining the parametersL=100cache_size=5# Freshness Constraint arrayF=np.ones((L,),dtype=int)# Initialising the algorithmalg=LU(L=L,F=F,cache_size=cache_size,arr=ytdataprob)# Total no. of requeststotalreq=ytdatareq.shape[0]foriinrange(totalreq):request=ytdatareq[i]# Returns the cache and hit or miss information at current instantcurrcache=alg.currcache(req=request,time=i)Usage of LU-LFU algorithmfromcachingalgo.full_observation.single_cacheimportLU# Defining the parametersL=100cache_size=5# Freshness Constraint arrayF=np.ones((L,),dtype=int)# Initialising the algorithmalg=LU(L=L,F=F,cache_size=cache_size,method='lfu')# Total no. of requeststotalreq=ytdatareq.shape[0]foriinrange(totalreq):request=ytdatareq[i]# Updates the counteralg.update(request)# Returns the cache and hit or miss information at current instantcurrcache=alg.currcache(req=request,time=i)Usage of LU-LFULite[C or 2C or 3C] algorithmfromcachingalgo.full_observation.single_cacheimportLUimportmath# Defining the parametersL=100cache_size=5window=int((cache_size^2)*math.log(L))# Freshness Constraint arrayF=np.ones((L,),dtype=int)# Initialising the algorithmalg=LU(L=L,F=F,cache_size=cache_size,method='lfulite',window=window,freqtop=cache_size)# In the above initialisation changing freqtop, one can get# freqtop = 2*cache_size - LU-LFULite[2C] algorithm# freqtop = 3*cache_size - LU-LFULite[3C] algorithm# Total no. of requeststotalreq=ytdatareq.shape[0]foriinrange(totalreq):request=ytdatareq[i]# Updates the counteralg.update(request,ithreq=i)# Returns the cache and hit or miss information at current instantcurrcache=alg.currcache(req=request,time=i,ithreq=i)Usage of LU-LFULite(V)[C or 2C or 3C] algorithmfromcachingalgo.full_observation.single_cacheimportLUimportmath# Defining the parametersL=100cache_size=5window=int((cache_size^2)*math.log(L))# Freshness Constraint arrayF=np.ones((L,),dtype=int)# Initialising the algorithmalg=LU(L=L,F=F,cache_size=cache_size,method='lfulite',window=window,freqtop=cache_size,useF=True)# In the above initialisation changing freqtop, one can get# freqtop = 2*cache_size - LU-LFULite(V)[2C] algorithm# freqtop = 3*cache_size - LU-LFULite(V)[3C] algorithm# Total no. of requeststotalreq=ytdatareq.shape[0]foriinrange(totalreq):request=ytdatareq[i]# Updates the counteralg.update(request,ithreq=i)# Returns the cache and hit or miss information at current instantcurrcache=alg.currcache(req=request,time=i,ithreq=i)Usage of LU2 algorithmfromcachingalgo.tandem_model.dynamicimportLU2# Defining the parametersL=100# Freshness Constraint arrayF=np.ones((L,),dtype=int)# Initialising the algorithmalg=LU2(L=L,F=F,cache_sizes=[5,10],arr=ytdataprob)# Total no. of requeststotalreq=ytdatareq.shape[0]foriinrange(totalreq):request=ytdatareq[i]# Returns the cache and hit or miss information at current instantcurrcache=alg.currcache(req=request,time=i)Usage of LU2-LFU algorithmfromcachingalgo.tandem_model.dynamicimportLU2# Defining the parametersL=100# Freshness Constraint arrayF=np.ones((L,),dtype=int)# Initialising the algorithmalg=LU2(L=L,F=F,cache_sizes=[5,10],method='lfu')# Total no. of requeststotalreq=ytdatareq.shape[0]foriinrange(totalreq):request=ytdatareq[i]# Updates the counteralg.update(request)# Returns the cache and hit or miss information at current instantcurrcache=alg.currcache(req=request,time=i)Usage of LU2-LFULite(V)[3C] algorithmfromcachingalgo.tandem_model.dynamicimportLU2importmath# Defining the parametersL=100# 5 is the size of cache 1window=int((5^2)*math.log(L))# Freshness Constraint arrayF=np.ones((L,),dtype=int)# Initialising the algorithmalg=LU2(L=L,F=F,cache_sizes=[5,10],method='lfulite',window=window,fretop=3*5,useF=True)# Total no. of requeststotalreq=ytdatareq.shape[0]foriinrange(totalreq):request=ytdatareq[i]# Updates the counteralg.update(request,ithreq=i)# Returns the cache and hit or miss information at current instantcurrcache=alg.currcache(req=request,time=i,ithreq=i)Usage of CMSR algorithmfromcachingalgo.full_observation.cost_minimportCMSR# Initialising the algorithmalg=CMSR(cache_size=5,L=100,beta=10,z=1.5,lambda_param=5,Cost=[1,0.1,0.05,0.025],prob=ytdataprob,seed=7)# To retrieve the cachealgcache=alg.currcache()# To calculate the average system costavg_sys_cost=alg.avg_sys_cost(alg.mu_hat)Usage of CMDR algorithmfromcachingalgo.full_observation.cost_minimportCMDR# Defining the parametercache_size=5# Initialising the algorithmalg=CMDR(cache_size=cache_size,L=100,beta=10,z=1.5,lambda_param=5,Cost=[1,0.1,0.05,0.025],prob=ytdataprob,C_hat=np.arange(cache_size,dtype='int'),seed=7)# To retrieve the cachealgcache=alg.currcache()# To calculate the average system costavg_sys_cost=alg.avg_sys_cost(alg.mu_hat)Usage of CMDRM algorithmfromcachingalgo.full_observation.cost_minimportCMDR# Defining the parametercache_size=5# Initialising the algorithmalg=CMDR(cache_size=cache_size,L=100,beta=10,z=1.5,lambda_param=5,Cost=[1,0.1,0.05,0.025],cache_update='marg',seed=7)# Total no. of requeststotalreq=ytdatareq.shape[0]foriinrange(totalreq):request=ytdatareq[i]# To retrieve the cachealgcache=alg.currcache(request)# To calculate the average system costavg_sys_cost=alg.avg_sys_cost(alg.mu_hat)Usage of CMDRP algorithmfromcachingalgo.full_observation.cost_minimportCMDR# Defining the parametercache_size=5# Initialising the algorithmalg=CMDR(cache_size=cache_size,L=100,beta=10,z=1.5,lambda_param=5,Cost=[1,0.1,0.05,0.025],cache_update='pop',seed=7)# Total no. of requeststotalreq=ytdatareq.shape[0]foriinrange(totalreq):request=ytdatareq[i]# To retrieve the cachealgcache=alg.currcache(request)# To calculate the average system costavg_sys_cost=alg.avg_sys_cost(alg.mu_hat)